# Word timestamps

> Generate speech and get character-level timings for the text you submitted, in one request. Use it to highlight words as they are spoken, drive captions, or align a transcript to the audio.

## How it works

This endpoint streams **NDJSON** — one JSON object per line — instead of raw PCM. Two kinds of line arrive:

| Line | What it carries |
| --- | --- |
| **Audio** | `audio_base64` — a base64-encoded chunk of PCM16 audio. Decode and append these in order to rebuild the clip. |
| **Alignment** | `alignment` and `normalized_alignment` — character arrays with start and end times. `audio_base64` is empty on these lines. |

> Alignment arrives at the end. Timings are emitted on the final lines, once generation has finished — not incrementally alongside each audio chunk. If you need to highlight from the first word, buffer the whole response before starting playback, or split long text into sentences and request them separately.

## Audio format

The audio is the same PCM16 the streaming endpoint returns — it is simply base64-encoded and split across the `audio_base64` lines instead of being sent as a raw byte stream.

PCM16base64 per line48000 Hz recommended24000 Hz defaultmono16-bit

| Property | Value |
| --- | --- |
| **Format** | PCM (Pulse Code Modulation), signed 16-bit little-endian |
| **Encoding** | Base64, one chunk per NDJSON line |
| **Sample rate** | Follows `sample_rate`: `48000` Hz recommended (engine-native), `24000` Hz default |
| **Channels** | Mono |
| **Chunk size** | Varies — typically a fraction of a second of audio per line |

**To rebuild the clip:** base64-decode each `audio_base64` value and concatenate the bytes **in the order the lines arrive**. The result is headerless PCM — the same bytes the streaming endpoint would have given you. Most players need a container, so prepend a 44-byte WAV header (using your `sample_rate`, 1 channel, 16 bits) before writing a `.wav` file, or feed the samples straight into a Web Audio buffer.

> Don't skip lines. Every audio_base64 value is a contiguous slice of one continuous waveform. Dropping or reordering a line produces audible clicks and shifts everything after it out of sync with the timings.

## Endpoint & request

`POST /api/v1/text-to-speech/{model_id}/with-timestamps`

Requires API key authentication via the `x-api-key` header. Single voice only — the `speakers` array supported by [Synthesize](/text-to-speech/synthesize) is not accepted here.

**Path parameters**

| Parameter | Type | Required | Description |
| --- | --- | --- | --- |
| `model_id` | string | **Yes** | The model identifier to use for generation. Must be a model served by the v1.5 engine — see [Models](/text-to-speech/models). |

**Request body** — `Content-Type: application/json`

| Field | Type | Required | Description |
| --- | --- | --- | --- |
| `voice_id` | string | **Yes** | The voice ID to use for synthesis |
| `text` | string | **Yes** | The Arabic text to convert to speech (max 10,000 characters) |
| `stability` | number | **Yes** | Voice stability (0.0 to 1.0). Higher values produce more consistent output |
| `speed` | number | No | Speech speed (0.7 to 1.2, default 1.0) |
| `sample_rate` | number | No | Output sample rate in Hz, `8000`–`48000` (default `24000`). **Use `48000`** — it's the engine's native rate, so the audio skips downsampling. |
| `dialect` | string | No | Dialect hint for synthesis: `auto` (default), `emirati`, or `fusha`. |

## Example request

The examples use the `faseeh-v1-preview` model with the `ar-najdi-male-2` voice, and rebuild both the audio and the word list.

```bash
curl -X POST "https://api.munsit.com/api/v1/text-to-speech/faseeh-v1-preview/with-timestamps" \
  -H "x-api-key: YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "voice_id": "ar-najdi-male-2",
    "text": "مرحبا بك في فصيح",
    "stability": 0.5,
    "sample_rate": 48000
  }' \
  --output stream.ndjson
```

```python
import base64, json, wave, requests

url = "https://api.munsit.com/api/v1/text-to-speech/faseeh-v1-preview/with-timestamps"
headers = {"x-api-key": "YOUR_API_KEY", "Content-Type": "application/json"}
data = {
    "voice_id": "ar-najdi-male-2",
    "text": "مرحبا بك في فصيح",
    "stability": 0.5,
    "sample_rate": 48000,
}

pcm = bytearray()
alignment = None

with requests.post(url, json=data, headers=headers, stream=True) as r:
    r.raise_for_status()
    for line in r.iter_lines():
        if not line:
            continue
        msg = json.loads(line)
        if msg.get("audio_base64"):
            pcm += base64.b64decode(msg["audio_base64"])
        if msg.get("alignment"):
            alignment = msg["alignment"]

# Group characters into words
words, cur = [], None
for ch, s, e in zip(alignment["characters"],
                     alignment["character_start_times_seconds"],
                     alignment["character_end_times_seconds"]):
    if ch.isspace():
        cur = None
        continue
    if cur is None:
        cur = {"text": "", "start": s, "end": e}
        words.append(cur)
    cur["text"] += ch
    cur["end"] = e

print(words)

# Wrap the concatenated PCM in a WAV container so it can be played
with wave.open("output.wav", "wb") as w:
    w.setnchannels(1)
    w.setsampwidth(2)  # 16-bit
    w.setframerate(48000)  # must match sample_rate
    w.writeframes(bytes(pcm))
```

```javascript
const res = await fetch(
  'https://api.munsit.com/api/v1/text-to-speech/faseeh-v1-preview/with-timestamps',
  {
    method: 'POST',
    headers: {
      'x-api-key': 'YOUR_API_KEY',
      'Content-Type': 'application/json',
    },
    body: JSON.stringify({
      voice_id: 'ar-najdi-male-2',
      text: 'مرحبا بك في فصيح',
      stability: 0.5,
      sample_rate: 48000,
    }),
  }
);

const reader = res.body.getReader();
const decoder = new TextDecoder();
const chunks = [];
let alignment = null;
let buffer = '';

while (true) {
  const { done, value } = await reader.read();
  if (done) break;
  buffer += decoder.decode(value, { stream: true });
  let i;
  while ((i = buffer.indexOf('\n')) !== -1) {
    const line = buffer.slice(0, i).trim();
    buffer = buffer.slice(i + 1);
    if (!line) continue;
    const msg = JSON.parse(line);
    if (msg.audio_base64) chunks.push(Buffer.from(msg.audio_base64, 'base64'));
    if (msg.alignment) alignment = msg.alignment;
  }
}

// Headerless PCM16 — same bytes the streaming endpoint returns
const pcm = Buffer.concat(chunks);
// Prepend a 44-byte WAV header (1 channel, 16-bit, sample_rate) to play it,
// or copy the samples into an AudioBuffer in the browser.
```

## Response

**Status code:** `200 OK`. The body is a newline-delimited JSON stream.

| Header | Value |
| --- | --- |
| `Content-Type` | `application/x-ndjson;charset=utf-8;rate=<sample_rate>` |
| `Cache-Control` | `no-cache` |

A complete response looks like this — audio lines first, then the alignment lines. Every line carries all four keys, with `null` where a field does not apply:

```json
// audio — one chunk per line, in playback order
{"audio_base64": "XQACASUBAgERATUBYAFnAVkBcQGPAXYB…", "alignment": null, "normalized_alignment": null, "quality_check": null}
{"audio_base64": "o+Ii5Tro1+sT8FL0qvhP/SYBhgMOBvUJ…", "alignment": null, "normalized_alignment": null, "quality_check": null}
// … more audio lines …

// then the timings — audio_base64 is empty from here on
{"audio_base64": "", "alignment": null, "normalized_alignment": {"characters": […], "character_start_times_seconds": […], "character_end_times_seconds": […]}, "quality_check": null}
{"audio_base64": "", "alignment": {"characters": […], "character_start_times_seconds": […], "character_end_times_seconds": […]}, "normalized_alignment": null, "quality_check": null}
```

A short clip like مرحبا بك في فصيح at 48 kHz comes back as roughly a dozen audio lines followed by the two alignment lines. Read to the end of the stream: closing early loses the timings entirely.

**Line fields**

| Field | Type | Description |
| --- | --- | --- |
| `audio_base64` | string | Base64-encoded PCM16 mono audio at the requested `sample_rate`. Empty on alignment lines. |
| `alignment` | object | Character timings aligned to **the text you submitted**. Use this one to map timings back onto your own string. |
| `normalized_alignment` | object | Character timings aligned to the engine's normalized (and, for MSA, diacritized) form of the text. Does not match your input character-for-character — see the warning below. |

Alongside the three timing arrays, each alignment object carries quality flags:

| Field | On | Meaning |
| --- | --- | --- |
| `aligned` | both | Whether alignment succeeded. If `false`, treat the timings as unreliable and fall back to plain playback. |
| `mapped` | `alignment` | Per character: whether it was mapped back onto your original text. Characters the engine could not place are `false`. |
| `anchored` | `normalized_alignment` | Per character: whether the timing is anchored to real audio rather than interpolated between neighbours. Whitespace is typically `false`. |
| `coverage` | `normalized_alignment` | Proportion of characters that are anchored, `0`–`1`. `1.0` means every character got a real timing. |

For highlighting, the practical rule is: bail out if `aligned` is `false`, and skip any word whose characters are all unanchored — its timing is a guess, and highlighting it will look wrong against the audio.

Both alignment objects share the same shape — three parallel arrays of equal length:

```json
{
  "characters": ["م", "ر", "ح", "ب", "ا", " ", "ب", "ك"],
  "character_start_times_seconds": [0.0, 0.132, 0.244, 0.366, 0.477, 0.528, 0.610, 0.701],
  "character_end_times_seconds": [0.132, 0.244, 0.366, 0.477, 0.528, 0.610, 0.701, 0.853],
  "aligned": true
}
```

If your text contains a [break tag](/text-to-speech/synthesize#pauses), it stays in `alignment` exactly as you wrote it, with timings spanning the silence it produces — so offsets still line up with your string. It is **not** present in `normalized_alignment`, which describes the spoken form only. When grouping into words, treat the whole tag as one unit: splitting on whitespace alone tears it into `<break` and `time="1s"/>`.

Concatenating `alignment.characters` reproduces your input string **exactly**, so index `i` in those arrays is index `i` in your text — that is what makes it safe to map timings back onto your own string.

> That guarantee applies to alignment only. normalized_alignment describes the engine's spoken form, which can differ in length and content — numerals are expanded, so تأسست الشركة في 1985 (20 characters) becomes تأسست الشركة في ألف وتسعمئة وخمسة وثمانين (41 characters). Its indices do not map onto your input. Use it to read what was actually spoken, not to highlight your own text.

**Timings are per character, not per word** — there is no word array. Derive words by walking the arrays and breaking on whitespace: a word's start is its first character's start time, its end is its last character's end time. The Python example above does exactly this.

## Error responses

Errors come back as JSON with an `errorCode` and `errorMessage`.

| Status | Error code | Example message |
| --- | --- | --- |
| **400** Bad Request | `400xx` | `Word timestamps are not available for model 'x'. Use a model served by the v1.5 engine.` |
| **401** Unauthorized | `40101` | `Authorization required. Provide Authorization Bearer token (Clerk) or x-api-key header.` |
| **402** Payment Required | `402xx` | `Insufficient wallet balance. Required: $0.05, Available: $0.02` |

## Cost calculation

Priced exactly like a standard synthesis request — from the **text length** and the **model cost per character**. Timestamps add no extra charge. The generation appears in your history the same way a streaming request does.

## Go further

- [Synthesize](/text-to-speech/synthesize) — Standard generation without timings. — `POST /text-to-speech/{model_id}`

- [Audio streaming output](/text-to-speech/audio-streaming-output) — Raw PCM streamed as it is generated. — `POST /text-to-speech/{model_id}`

- [Models](/text-to-speech/models) — Check which models the v1.5 engine serves. — `GET /models`

- [Voices](/text-to-speech/voices) — Pick a voice_id from the voice library. — `GET /voices`