LLM API

Authentication

Every /v1/* request requires a Veduz-issued bearer token. The documentation page is the only public route.

Browser clients are supported through wildcard-origin CORS. Preflight requests are anonymous; actual API requests still require the bearer token.

Authorization: Bearer <token>

Endpoints

Realtime WebRTC

voice-chat-2 supports two immutable connection modes: gemma_audio sends speech directly to Gemma while transcription runs in parallel; asr_text transcribes speech first, then sends text to Gemma. Both synthesize replies on whitebox. The browser creates an offer with one audio track and the oai-events and vdz-bulk data channels, obtains a session secret with a named bearer token, then sends the SDP with that one-use secret.

curl https://llm.veduz.dk/v1/realtime/client_secrets \
  -H 'Authorization: Bearer <token>' \
  -H 'Content-Type: application/json' \
  -d '{"session":{"model":"voice-chat-2","vdz":{"input_mode":"gemma_audio"}}}'

curl https://llm.veduz.dk/v1/realtime/calls \
  -H 'Authorization: Bearer <client-secret>' \
  -H 'Content-Type: application/sdp' --data-binary @offer.sdp

The response is application/sdp; its Location header identifies the call for POST /v1/realtime/calls/<call-id>/hangup. Realtime request, response, SDP, media, and secrets are excluded from gateway body audit logs.

Realtime transcription

Use the same WebRTC flow with {"session":{"type":"transcription","model":"asr"}} when creating the client secret (or as the multipart session field). This reuses realtime chat's Silero VAD and Nemotron 3.5 ASR, with no generated replies. Keep the audio transceiver sendrecv; the returned audio track is silent.

curl https://llm.veduz.dk/v1/realtime/client_secrets \
  -H 'Authorization: Bearer <token>' \
  -H 'Content-Type: application/json' \
  -d '{"session":{"type":"transcription","model":"asr"}}'

Use the returned value as the bearer token when posting your SDP offer to /v1/realtime/calls, as shown above.

Open the ordered oai-events data channel, wait for session.created, attach the microphone audio track, and send {"event_id":"enable-mic","type":"vdz.input.enabled","enabled":true}. Capture starts disabled. Use enabled:false to stop capture or the call's hangup endpoint to release it.

// Send on oai-events after session.created:
{"event_id":"enable-mic","type":"vdz.input.enabled","enabled":true}

// Received events (server metadata omitted):
{"type":"input_audio_buffer.speech_started","item_id":"item_1"}
{"type":"input_audio_buffer.speech_stopped","item_id":"item_1"}
{"type":"conversation.item.input_audio_transcription.completed","item_id":"item_1","transcript":"Hello there."}

Listen for input_audio_buffer.speech_started and input_audio_buffer.speech_stopped for VAD state changes, and conversation.item.input_audio_transcription.completed with transcript for final text. Correlate events by item_id; transcripts can arrive after the next utterance starts. ASR errors use conversation.item.input_audio_transcription.failed. Short noise emits speech-stopped and vdz.input.discarded without a transcript. Transcription sessions keep utterances separate, accept at most two pending ASR jobs, and reject response generation. This is final text per detected utterance, not partial word streaming.

Chat example

curl https://llm.veduz.dk/v1/chat/completions \
  -H 'Authorization: Bearer <token>' \
  -H 'Content-Type: application/json' \
  -d '{"model":"gemma4-12b","messages":[{"role":"user","content":"Hello"}]}'

qwen38-27b supports reasoning_effort values none, low, medium, and xhigh, defaulting to low when omitted. gemma4-12b supports none and high, defaulting to none.

gemma4-12b runs on Whitebox GPU 0 with a 16,384-token context and accepts text content only. If Whitebox Gemma is unavailable, text requests fall back to Qwen3.8-27B on Aquabox; media remains rejected for this model ID. qwen38-27b uses Aquabox first, with a 150,000-token context. One parallel text request without long prefix overlap can use Whitebox GPU 1; remaining requests use Aquabox. If Aquabox is unavailable, text requests run sequentially on Whitebox and images are rejected. Audio is available through the transcription, speech, and realtime endpoints.

A final assistant message is treated as a prefill. Set continue_final_message: false to disable that behavior, or explicitly enable it for another final role.

If an explicit max_completion_tokens or max_tokens exceeds the context remaining after the prompt, the gateway uses the upstream tokenizer to fit the output budget and retries once.

Audio examples

asr uses NVIDIA Nemotron 3.5 ASR Streaming 0.6B on Whitebox GPU0, with English and Danish support. Use language=en, language=da (or da-DK), or omit it for automatic language detection.

curl https://llm.veduz.dk/v1/audio/transcriptions \
  -H 'Authorization: Bearer <token>' \
  -F model=asr -F file=@recording.wav

curl https://llm.veduz.dk/v1/audio/speech \
  -H 'Authorization: Bearer <token>' \
  -H 'Content-Type: application/json' \
  -d '{"model":"tts","input":"Hello"}' --output speech.wav