# Live streaming

> Stream audio over a WebSocket and receive Arabic transcripts as people speak — interim results while a sentence is in progress, a stable final when it completes, and a turn-boundary event your application can act on. First partials arrive in about 0.7 seconds (median), with word timestamps, per-word confidence, and per-turn sentiment and speaker gender.

## Endpoint & authentication

`WSS /api/v1/listen`

```
wss://api.munsit.com/api/v1/listen?api_key=YOUR_MUNSIT_API_KEY&encoding=linear16&sample_rate=16000
```

At least one auth method is required. If several are supplied, the `api_key` query parameter wins.

| Method | Where | Notes |
| --- | --- | --- |
| `x-api-key` | Header | Preferred for server-side clients. |
| `api_key` | Query parameter | For browser WebSocket clients, which cannot set headers. `x-api-key` is also accepted as a query parameter. |

> Keep connection URLs out of your logs. With query-parameter auth the key is part of the URL, so it lands in any request log or analytics tool that records it.

Connections are rejected with close code `1008` when authentication fails, when the key's concurrent-session limit is reached, or when the wallet balance is below about 60 seconds of runway — the `Error` message before the close says which.

## Query parameters

All optional. Telephony sources typically use `encoding=mulaw&sample_rate=8000`; microphone capture typically uses `encoding=linear16&sample_rate=16000`.

| Parameter | Type | Default | Description |
| --- | --- | --- | --- |
| `encoding` | string | `linear16` | Encoding of the binary frames you send: `linear16` (16-bit PCM), `mulaw`, or `alaw`. |
| `sample_rate` | integer | `8000` | Sample rate of your audio: `8000` or `16000`. |
| `channels` | integer | `1` | `1` (mono) or `2` (stereo, interleaved). With `2`, each channel is transcribed independently — ideal for two-leg call recordings. |
| `model` | string | `munsit` | ASR model for the session: `munsit` (Arabic) or `munsit-en-ar` (mixed Arabic-English code-switching). Routes the stream to the matching engine. Both models share the `munsit-2` recognizer generation, so this selects language coverage, not model vintage. |
| `language` | string | `ar` | Transcription language. `ar` is the only supported value in v1. |
| `interim_results` | boolean | `true` | Emit interim (partial) results while a turn is in progress. |
| `endpointing` | integer | `800` | Milliseconds of silence that end a turn. Range `100`–`5000`. Retunable mid-session with `Configure`. |
| `smart_turn` | boolean | `true` | Gate end-of-turn on a semantic turn-completion model in addition to silence. A turn always ends after 2× the `endpointing` silence regardless. |
| `hotwords` | string | — | Comma-separated custom vocabulary (multi-word phrases allowed). Up to 200 entries, each up to 40 characters; URL-encode the value. |
| `correlation_id` | string | — | Your identifier for this session (up to 128 characters), echoed in the opening `Metadata` event. |
| `metadata` | string | — | Base64-encoded JSON object (up to 2 KB) attached to the session. |

> Invalid connection parameters are fatal. You receive an Error with code 4002 and recoverable: false, then the connection closes with code 4002. An invalid mid-session Configure is recoverable instead — the session continues.

## Sending audio

Send raw audio as **binary WebSocket frames** in the encoding and sample rate you declared at connect time. No container headers — for `linear16`, frames are little-endian 16-bit PCM samples, interleaved when `channels=2`. Frame size is up to you; roughly 20–200 ms of audio per frame works well.

| Rule | Why |
| --- | --- |
| **Keep the connection alive** | 12 seconds with neither audio nor a `KeepAlive` message produces an `Error` and closes with code `1011`. |
| **Don't run ahead of real time** | Audio may be buffered at most 60 seconds ahead of real time; exceeding it closes with code `4008`. Live sources never hit this — pace your sends when streaming from a file. |

## Control messages

Controls are JSON **text frames**, sent on the same socket as your binary audio.

```
{ "type": "KeepAlive" }
```

```
{ "type": "Configure", "endpointing": 300 }
```

```
{ "type": "CloseStream" }
```

Effect

```
Resets the 12-second idle timer during send pauses — for example while the caller is on hold.
```

Effect

```
Retunes the endpointing silence window mid-session (100–5000 ms). Applies to all channels.
```

Effect

```
Finalizes any in-progress turn, sends the closing Metadata event with billing, then closes with code 1000.
```

**CloseStream and short utterances.** `CloseStream` finalizes the turn in progress. Every event for that turn — `Results` with `is_final: true`, then `UtteranceEnd`, `Gender` and `Sentiment` — is sent before the closing `Metadata`, which is always the last event before the `1000` close.

This holds for utterances shorter than your `endpointing` window: a 400 ms word closed out with `CloseStream` still produces a final `Results`. It does **not** hold for audio that never opened a turn — under roughly 200 ms of speech, or noise-only input, the voice-activity detector never confirms speech, so the session ends with `Metadata` (`turn_count: 0`) and no `Results` at all. Treat that as “no speech detected” rather than an error.

After sending `CloseStream`, keep reading until the server closes the socket. Closing it yourself first aborts finalization and loses both the final `Results` and the billing `Metadata`.

This is the `/listen` counterpart to the legacy `end_of_stream` → `finalized` flush; the stable-text flag is `is_final` rather than `isFinal`, and `min_buffer_seconds` has no equivalent here.

## Server events

Every server message is a JSON text frame with a `type` field. All events carry `session_id` — and `correlation_id` when you set one — so multiplexed clients can attribute events without tracking connections.

A single spoken turn produces this sequence:

```
SpeechStarted → Results (interim, refreshing) → Results (final) → UtteranceEnd → Gender → Sentiment
```

```
{
  "type": "Metadata",
  "session_id": "0d5b1c9e-3f6a-4b62-9d8e-2f1a7c3b5e90",
  "correlation_id": "call-8371",
  "model": "munsit-v2",
  "protocol_version": 1,
  "channels": 1,
  "sample_rate": 16000,
  "dropped_hotwords": [],
  "sentiment": "available",
  "gender": "available"
}
```

```
{ "type": "SpeechStarted", "channel": 0, "ts": 4.31 }
```

```
{
  "type": "Results",
  "channel": 0,
  "turn_id": 2,
  "transcript": "لا تتكلم هكذا",
  "words": [
    { "word": "لا", "start": 5.02, "end": 5.18, "confidence": 0.996 },
    { "word": "تتكلم", "start": 5.18, "end": 5.61, "confidence": 0.988 },
    { "word": "هكذا", "start": 5.61, "end": 5.97, "confidence": 0.991 }
  ],
  "is_final": true,
  "speech_final": true,
  "language": "ar",
  "confidence": 0.992
}
```

```
{ "type": "UtteranceEnd", "channel": 0, "turn_id": 2, "last_word_end": 5.97 }
```

```
{ "type": "Gender", "channel": 0, "turn_id": 2, "label": "female", "score": 0.996 }

{ "type": "Sentiment", "channel": 0, "turn_id": 2, "label": "negative", "score": 0.87 }
```

```
{
  "type": "Metadata",
  "session_id": "0d5b1c9e-3f6a-4b62-9d8e-2f1a7c3b5e90",
  "audio_seconds_billed": 124.6,
  "turn_count": 9
}
```

```
{ "type": "Error", "code": 4002, "message": "Configure.endpointing must be 100..5000 ms", "recoverable": true }
```

When

```
Once, immediately after a successful connection.
```

When

```
The voice-activity detector confirmed speech on a channel. ts is the audio timestamp in seconds.
```

When

```
Continuously while the speaker talks. words[] carries session-absolute timestamps and per-word confidence; utterance confidence is null on interims.
```

When

```
Immediately after the final Results of a completed turn — never after a forced split. Gender and Sentiment follow it.
```

When

```
After each final result, including forced splits. Gender first, then Sentiment.
```

When

```
After CloseStream, right before the connection closes. Reports the session billing total.
```

When

```
recoverable: false always precedes a close with the matching code. recoverable: true means the session continues.
```

## Turn detection

**Endpointing** is the silence the engine waits for before declaring a turn finished, and it's the main latency/accuracy dial you control. You can retune it live, mid-session, with a `Configure` message — drop to `300` when your agent asks a yes/no question, restore `800` for open-ended answers.

| Setting | Final transcript arrives | Trade-off |
| --- | --- | --- |
| `800` ms (default) | ~1.2 s after speech ends | Safest turn boundaries for conversational speech. |
| `500` ms | ~0.9 s (estimated) | Balanced; clears most mid-sentence hesitations. |
| `300` ms | ~0.8 s | Fastest — best for short commands. On hesitant, conversational speech it can split turns mid-thought, costing ~2 WER points in our benchmarks. |

**Smart turn detection** (`smart_turn`, on by default) runs a semantic end-of-turn model on every pause, so a caller who stops mid-sentence to think isn't cut off just because the silence threshold elapsed. A turn still always ends after 2× the endpointing silence. If the model is unavailable server-side, endpointing degrades gracefully to silence-only.

> is_final vs speech_final. Treat speech_final as your end-of-turn signal for agent logic, and is_final as “this text will not change”.speech_final: true — the speaker genuinely finished. This is your trigger to respond.is_final: true with speech_final: false — a forced split during long unbroken speech, about every 60 seconds. The text is stable but the speaker is still talking: don't respond, no UtteranceEnd fires, and transcription continues under the next turn_id.

Interim results (`is_final: false`) refresh continuously and **may revise earlier words** — render them as provisional text. `UtteranceEnd` fires only on genuine turn ends and is the cleanest single signal for “caller stopped talking”.

## Custom vocabulary

Pass rare terms — customer names, brands, product codes — in the `hotwords` query parameter, comma-separated and URL-encoded. Multi-word phrases are allowed.

```
&hotwords=%D8%B9%D8%A8%D8%AF%20%D8%A7%D9%84%D9%82%D8%A7%D8%AF%D8%B1%2C%D8%A3%D8%AF%D9%8A%D8%A8
```

Short lists of 5–30 genuinely rare terms work best; very long lists dilute the effect. Up to 200 entries, each up to 40 characters. Entries that can't be applied are skipped and reported in `dropped_hotwords` on the opening `Metadata` event — check it to confirm every entry landed.

## Sentiment & speaker gender

After each final result you receive two enrichment events, with no extra requests.

| Event | Labels | Notes |
| --- | --- | --- |
| `Gender` | `male` / `female` | Classified from the turn's audio — measured 99.2% accuracy on Arabic. |
| `Sentiment` | `positive` / `neutral` / `negative` | Derived from the turn's transcript. |

On a two-channel call this gives you live per-party gender and a running sentiment trajectory — useful for routing, analytics and supervisor alerts. For post-call analysis of an existing transcription, use [Sentiment analysis](/understanding/sentiment-analysis) instead.

## Session health & close codes

Four habits keep a session healthy: send `KeepAlive` during pauses, pace file streaming to real time, end with `CloseStream` so the billing event arrives, and handle the close code.

| Code | Meaning | Client handling |
| --- | --- | --- |
| `1000` | Normal close after `CloseStream` | Done — billing was reported in the closing `Metadata`. |
| `1008` | Policy rejection: auth failed, concurrent-session limit reached, or insufficient wallet balance | Check the preceding `Error`. Auth: fix the key. Session limit: retry after a session ends. Balance: top up the wallet. |
| `1011` | Internal error, or 12 s with no audio and no `KeepAlive` | Reconnect and resume; send `KeepAlive` during pauses. |
| `4002` | Invalid connection parameters | Fix the parameters and reconnect. |
| `4008` | Audio sent more than 60 s ahead of real time | Pace file streaming to real time. |

## Billing & limits

Usage is metered as **seconds of audio received × number of channels**, charged from your wallet in 60-second cycles during the session.

| Limit | Value |
| --- | --- |
| **Concurrent sessions** | 5 per API key by default — raised on request. Exceeding it closes the new connection with `1008`. |
| **Wallet runway to connect** | About 60 seconds. Running out mid-session closes the connection with `1008` after an `Error`. |
| **Idle timeout** | 12 seconds with neither audio nor `KeepAlive` (code `1011`). |
| **Session length** | Unlimited while the connection stays active. Unbroken speech is force-segmented about every 60 seconds so results keep flowing. |

The closing `Metadata` event reports the session total as `audio_seconds_billed`.

## Examples

Streaming a file, and capturing a microphone in the browser.

```python
import asyncio, json, wave
import websockets

async def main():
    wav = wave.open("audio_16k_mono.wav")
    pcm = wav.readframes(wav.getnframes())

    url = ("wss://api.munsit.com/api/v1/listen"
           "?api_key=YOUR_MUNSIT_API_KEY&encoding=linear16&sample_rate=16000&interim_results=true")

    try:
        async with websockets.connect(url) as ws:
            async def send_audio():
                chunk = 6400  # 200 ms of 16 kHz 16-bit mono
                for i in range(0, len(pcm), chunk):
                    await ws.send(pcm[i:i + chunk])
                    await asyncio.sleep(0.2)  # pace at real time (close code 4008)
                await ws.send(json.dumps({"type": "CloseStream"}))

            sender = asyncio.create_task(send_audio())
            async for message in ws:
                event = json.loads(message)
                if event["type"] == "Results" and event["is_final"]:
                    print("FINAL:", event["transcript"])
                elif event["type"] == "Error":
                    print("error:", event["code"], event["message"])
                elif event["type"] == "Metadata" and "audio_seconds_billed" in event:
                    print("billed seconds:", event["audio_seconds_billed"])
                    break
            await sender
    except websockets.exceptions.ConnectionClosed as e:
        # match e.code against the close-code table above
        print("connection closed:", e.code, e.reason)

asyncio.run(main())
```

```javascript
// Acquire the microphone BEFORE connecting — the permission prompt
// can take longer than the 12-second idle timeout.
async function startTranscription() {
  const mic = await navigator.mediaDevices.getUserMedia({ audio: true });
  const ctx = new AudioContext({ sampleRate: 16000 });
  await ctx.resume(); // autoplay policies can leave the context suspended
  // Browsers may ignore the sampleRate hint — declare what you actually got:
  const sampleRate = ctx.sampleRate;

  const ws = new WebSocket(
    `wss://api.munsit.com/api/v1/listen?api_key=YOUR_MUNSIT_API_KEY&encoding=linear16&sample_rate=${sampleRate}`
  );
  ws.binaryType = "arraybuffer";

  ws.onmessage = (msg) => {
    const event = JSON.parse(msg.data);
    if (event.type === "Results") {
      render(event.transcript, event.is_final);   // interims refresh, finals are stable
    } else if (event.type === "UtteranceEnd") {
      onTurnComplete(event.turn_id);              // trigger your agent here
    } else if (event.type === "Metadata" && event.audio_seconds_billed !== undefined) {
      console.log("billed seconds:", event.audio_seconds_billed);
    }
  };

  // Production code should prefer an AudioWorklet — ScriptProcessorNode is
  // deprecated but shown here for brevity.
  const source = ctx.createMediaStreamSource(mic);
  const processor = ctx.createScriptProcessor(2048, 1, 1);
  processor.onaudioprocess = (e) => {
    const f32 = e.inputBuffer.getChannelData(0);
    const i16 = new Int16Array(f32.length);
    for (let i = 0; i < f32.length; i++) i16[i] = Math.max(-32768, Math.min(32767, f32[i] * 32767));
    if (ws.readyState === WebSocket.OPEN) ws.send(i16.buffer);
  };
  source.connect(processor);
  processor.connect(ctx.destination);

  // Send CloseStream so the billing Metadata arrives, then release the mic.
  const stop = () => {
    if (ws.readyState === WebSocket.OPEN) ws.send(JSON.stringify({ type: "CloseStream" }));
    processor.disconnect();
    source.disconnect();
    mic.getTracks().forEach((t) => t.stop());
    ctx.close();
  };
  ws.onclose = (e) => { console.log("closed:", e.code); stop(); };
  return stop;
}
```

Output

```
FINAL: "اجتماع الفريق يبدأ الساعة العاشرة"
billed seconds: 124.6
```

Note

```
The server accepts only 8000 or 16000 Hz. If the browser reports another rate — 48000 is common — resample in an AudioWorklet before sending.
```

## Legacy endpoint

> WS /websocket/speech-to-text is deprecated. The previous streaming endpoint — JSON audio_chunk frames and a cumulative transcript string — remains available for existing integrations but receives no new recognition features: no word timestamps, turn events, hotwords, confidence, sentiment or gender. New integrations should use WS /api/v1/listen. Its message reference is on the WebSocket protocol page.Lifecycle. This endpoint is frozen, not scheduled for removal. It stays available for existing integrations with no functional changes; any change to that would be announced on the Changelog.If you are still on it: send {"event":"end_of_stream"} before closing the socket and wait for the finalized event. Closing alone does not flush buffered audio, so short answers are otherwise lost. See Finalizing a stream.

## Go further

Batch alternatives, and the agent frameworks that wrap this socket for you.

- [Transcribe →](/speech-to-text/transcribe) — Pre-recorded files with word-level timestamps, one POST. — `POST /audio/transcribe`

- [Diarization →](/speech-to-text/diarization) — Who said what — speaker-labeled segments. — `POST /audio/diarization/transcribe`

- [LiveKit →](/integrations/livekit) — Drop-in streaming STT for voice agents. — `plugin`

- [WebSocket protocol →](/reference/websocket) — The TTS socket, and the deprecated STT one. — `reference`