Speech to Text Live streaming

Live streaming

Stream audio over a WebSocket and receive Arabic transcripts as people speak — interim results while a sentence is in progress, a stable final when it completes, and a turn-boundary event your application can act on. First partials arrive in about 0.7 seconds (median), with word timestamps, per-word confidence, and per-turn sentiment and speaker gender.

1

Endpoint & authentication

WSS /api/v1/listen
wss://api.munsit.com/api/v1/listen?api_key=YOUR_MUNSIT_API_KEY&encoding=linear16&sample_rate=16000

At least one auth method is required. If several are supplied, the api_key query parameter wins.

MethodWhereNotes
x-api-keyHeaderPreferred for server-side clients.
api_keyQuery parameterFor browser WebSocket clients, which cannot set headers. x-api-key is also accepted as a query parameter.
Keep connection URLs out of your logs. With query-parameter auth the key is part of the URL, so it lands in any request log or analytics tool that records it.

Connections are rejected with close code 1008 when authentication fails, when the key's concurrent-session limit is reached, or when the wallet balance is below about 60 seconds of runway — the Error message before the close says which.

2

Query parameters

All optional. Telephony sources typically use encoding=mulaw&sample_rate=8000; microphone capture typically uses encoding=linear16&sample_rate=16000.

ParameterTypeDefaultDescription
encodingstringlinear16Encoding of the binary frames you send: linear16 (16-bit PCM), mulaw, or alaw.
sample_rateinteger8000Sample rate of your audio: 8000 or 16000.
channelsinteger11 (mono) or 2 (stereo, interleaved). With 2, each channel is transcribed independently — ideal for two-leg call recordings.
modelstringmunsitASR model for the session: munsit (Arabic) or munsit-en-ar (mixed Arabic-English code-switching). Routes the stream to the matching engine. Both models share the munsit-2 recognizer generation, so this selects language coverage, not model vintage.
languagestringarTranscription language. ar is the only supported value in v1.
interim_resultsbooleantrueEmit interim (partial) results while a turn is in progress.
endpointinginteger800Milliseconds of silence that end a turn. Range 1005000. Retunable mid-session with Configure.
smart_turnbooleantrueGate end-of-turn on a semantic turn-completion model in addition to silence. A turn always ends after 2× the endpointing silence regardless.
hotwordsstringComma-separated custom vocabulary (multi-word phrases allowed). Up to 200 entries, each up to 40 characters; URL-encode the value.
correlation_idstringYour identifier for this session (up to 128 characters), echoed in the opening Metadata event.
metadatastringBase64-encoded JSON object (up to 2 KB) attached to the session.
Invalid connection parameters are fatal. You receive an Error with code 4002 and recoverable: false, then the connection closes with code 4002. An invalid mid-session Configure is recoverable instead — the session continues.
3

Sending audio

Send raw audio as binary WebSocket frames in the encoding and sample rate you declared at connect time. No container headers — for linear16, frames are little-endian 16-bit PCM samples, interleaved when channels=2. Frame size is up to you; roughly 20–200 ms of audio per frame works well.

RuleWhy
Keep the connection alive12 seconds with neither audio nor a KeepAlive message produces an Error and closes with code 1011.
Don't run ahead of real timeAudio may be buffered at most 60 seconds ahead of real time; exceeding it closes with code 4008. Live sources never hit this — pace your sends when streaming from a file.
4

Control messages

Controls are JSON text frames, sent on the same socket as your binary audio.

json · KeepAlive
{ "type": "KeepAlive" }
Effect
Resets the 12-second idle timer during send pauses — for example while the caller is on hold.

CloseStream and short utterances. CloseStream finalizes the turn in progress. Every event for that turn — Results with is_final: true, then UtteranceEnd, Gender and Sentiment — is sent before the closing Metadata, which is always the last event before the 1000 close.

This holds for utterances shorter than your endpointing window: a 400 ms word closed out with CloseStream still produces a final Results. It does not hold for audio that never opened a turn — under roughly 200 ms of speech, or noise-only input, the voice-activity detector never confirms speech, so the session ends with Metadata (turn_count: 0) and no Results at all. Treat that as “no speech detected” rather than an error.

After sending CloseStream, keep reading until the server closes the socket. Closing it yourself first aborts finalization and loses both the final Results and the billing Metadata.

This is the /listen counterpart to the legacy end_of_streamfinalized flush; the stable-text flag is is_final rather than isFinal, and min_buffer_seconds has no equivalent here.

5

Server events

Every server message is a JSON text frame with a type field. All events carry session_id — and correlation_id when you set one — so multiplexed clients can attribute events without tracking connections.

A single spoken turn produces this sequence:

SpeechStarted → Results (interim, refreshing) → Results (final) → UtteranceEnd → Gender → Sentiment
json · Metadata
{ "type": "Metadata", "session_id": "0d5b1c9e-3f6a-4b62-9d8e-2f1a7c3b5e90", "correlation_id": "call-8371", "model": "munsit-v2", "protocol_version": 1, "channels": 1, "sample_rate": 16000, "dropped_hotwords": [], "sentiment": "available", "gender": "available" }
When
Once, immediately after a successful connection.
6

Turn detection

Endpointing is the silence the engine waits for before declaring a turn finished, and it's the main latency/accuracy dial you control. You can retune it live, mid-session, with a Configure message — drop to 300 when your agent asks a yes/no question, restore 800 for open-ended answers.

SettingFinal transcript arrivesTrade-off
800 ms (default)~1.2 s after speech endsSafest turn boundaries for conversational speech.
500 ms~0.9 s (estimated)Balanced; clears most mid-sentence hesitations.
300 ms~0.8 sFastest — best for short commands. On hesitant, conversational speech it can split turns mid-thought, costing ~2 WER points in our benchmarks.

Smart turn detection (smart_turn, on by default) runs a semantic end-of-turn model on every pause, so a caller who stops mid-sentence to think isn't cut off just because the silence threshold elapsed. A turn still always ends after 2× the endpointing silence. If the model is unavailable server-side, endpointing degrades gracefully to silence-only.

is_final vs speech_final. Treat speech_final as your end-of-turn signal for agent logic, and is_final as “this text will not change”.
  • speech_final: true — the speaker genuinely finished. This is your trigger to respond.
  • is_final: true with speech_final: false — a forced split during long unbroken speech, about every 60 seconds. The text is stable but the speaker is still talking: don't respond, no UtteranceEnd fires, and transcription continues under the next turn_id.

Interim results (is_final: false) refresh continuously and may revise earlier words — render them as provisional text. UtteranceEnd fires only on genuine turn ends and is the cleanest single signal for “caller stopped talking”.

7

Custom vocabulary

Pass rare terms — customer names, brands, product codes — in the hotwords query parameter, comma-separated and URL-encoded. Multi-word phrases are allowed.

&hotwords=%D8%B9%D8%A8%D8%AF%20%D8%A7%D9%84%D9%82%D8%A7%D8%AF%D8%B1%2C%D8%A3%D8%AF%D9%8A%D8%A8

Short lists of 5–30 genuinely rare terms work best; very long lists dilute the effect. Up to 200 entries, each up to 40 characters. Entries that can't be applied are skipped and reported in dropped_hotwords on the opening Metadata event — check it to confirm every entry landed.

8

Sentiment & speaker gender

After each final result you receive two enrichment events, with no extra requests.

EventLabelsNotes
Gendermale / femaleClassified from the turn's audio — measured 99.2% accuracy on Arabic.
Sentimentpositive / neutral / negativeDerived from the turn's transcript.

On a two-channel call this gives you live per-party gender and a running sentiment trajectory — useful for routing, analytics and supervisor alerts. For post-call analysis of an existing transcription, use Sentiment analysis instead.

9

Session health & close codes

Four habits keep a session healthy: send KeepAlive during pauses, pace file streaming to real time, end with CloseStream so the billing event arrives, and handle the close code.

CodeMeaningClient handling
1000Normal close after CloseStreamDone — billing was reported in the closing Metadata.
1008Policy rejection: auth failed, concurrent-session limit reached, or insufficient wallet balanceCheck the preceding Error. Auth: fix the key. Session limit: retry after a session ends. Balance: top up the wallet.
1011Internal error, or 12 s with no audio and no KeepAliveReconnect and resume; send KeepAlive during pauses.
4002Invalid connection parametersFix the parameters and reconnect.
4008Audio sent more than 60 s ahead of real timePace file streaming to real time.
10

Billing & limits

Usage is metered as seconds of audio received × number of channels, charged from your wallet in 60-second cycles during the session.

LimitValue
Concurrent sessions5 per API key by default — raised on request. Exceeding it closes the new connection with 1008.
Wallet runway to connectAbout 60 seconds. Running out mid-session closes the connection with 1008 after an Error.
Idle timeout12 seconds with neither audio nor KeepAlive (code 1011).
Session lengthUnlimited while the connection stays active. Unbroken speech is force-segmented about every 60 seconds so results keep flowing.

The closing Metadata event reports the session total as audio_seconds_billed.

11

Examples

Streaming a file, and capturing a microphone in the browser.

python
import asyncio, json, wave import websockets async def main(): wav = wave.open("audio_16k_mono.wav") pcm = wav.readframes(wav.getnframes()) url = ("wss://api.munsit.com/api/v1/listen" "?api_key=YOUR_MUNSIT_API_KEY&encoding=linear16&sample_rate=16000&interim_results=true") try: async with websockets.connect(url) as ws: async def send_audio(): chunk = 6400 # 200 ms of 16 kHz 16-bit mono for i in range(0, len(pcm), chunk): await ws.send(pcm[i:i + chunk]) await asyncio.sleep(0.2) # pace at real time (close code 4008) await ws.send(json.dumps({"type": "CloseStream"})) sender = asyncio.create_task(send_audio()) async for message in ws: event = json.loads(message) if event["type"] == "Results" and event["is_final"]: print("FINAL:", event["transcript"]) elif event["type"] == "Error": print("error:", event["code"], event["message"]) elif event["type"] == "Metadata" and "audio_seconds_billed" in event: print("billed seconds:", event["audio_seconds_billed"]) break await sender except websockets.exceptions.ConnectionClosed as e: # match e.code against the close-code table above print("connection closed:", e.code, e.reason) asyncio.run(main())
Output
FINAL: "اجتماع الفريق يبدأ الساعة العاشرة" billed seconds: 124.6
12

Legacy endpoint

WS /websocket/speech-to-text is deprecated. The previous streaming endpoint — JSON audio_chunk frames and a cumulative transcript string — remains available for existing integrations but receives no new recognition features: no word timestamps, turn events, hotwords, confidence, sentiment or gender. New integrations should use WS /api/v1/listen. Its message reference is on the WebSocket protocol page.

Lifecycle. This endpoint is frozen, not scheduled for removal. It stays available for existing integrations with no functional changes; any change to that would be announced on the Changelog.

If you are still on it: send {"event":"end_of_stream"} before closing the socket and wait for the finalized event. Closing alone does not flush buffered audio, so short answers are otherwise lost. See Finalizing a stream.
13

Go further

Batch alternatives, and the agent frameworks that wrap this socket for you.

Working with an AI assistant? Every page is available as Markdown: add .md to the URL, or send an Accept: text/markdown header. For the whole documentation in one request, point it at llms-full.txt; the page index is llms.txt. Or use Copy Page, top right.