Thaddée Tyl

@sophieschmieg @filippo @matthew_d_green Wait, would you bet $2³² that there will be a cryptographically-relevant quantum computer by 2029?

Who will succeed the supreme leader of Iran? Mohammad-Ali Movahedi Kermani? Ahmad Vahidi? Someone else?

@bnjbvr@tutut.delire.party

That strongly implies that Mistral's next step is TTS. In fact, other tokens corroborate it: while [AUDIO] likely indicates that speech tokens follow, [REF] might indicate a reference voice pattern to copy, and [OUTPUT_AUDIO] might start converting text to audio.

It also outputs a [word] token, which fits the [STREAMING_WORD] token found in Voxtral 2. Why have that? For text-to-speech: there, when the model knows it has finished outputting the audio for a word, it generates the WORD action, so that we can feed it the next word to say.

But there are a lot more tokens in there that are unexplained! To learn more, we can look at what inspired Mistral: the Kyutai Delayed Stream Modeling, https://arxiv.org/abs/2509.08753 It has the same delay design with the [pad] tokens.

Of course, the output does not contain exactly one word per text token, since the audio file does not contain exactly one word per 80ms. The trick? Look at those new tokens: when the model needs to wait before outputting a word, it outputs a [STREAMING_PAD] token.

1. The audio token embedding history + delay tokens go through a Transformer to output a speech token. This is why the delay is variable: it can be any multiple of 80ms.
2. The history of speech token embeddings go through a Transformer to output a text token embedding → text token probs → text.

Look at its architecture:

1. The audio is cut into 80ms, which are sampled to 16 KHz (16000*0.08=1280 floats).
2. It is converted to a spectrogram,
3. A whisper-style encoder converts it to an audio token embedding through a convnet,

Shoutout to Voxtral 2, which really feels unparalleled in quality.

The interesting bit is its ability to do realtime transcription.
How does it do that, with a variable delay?

Norton is great in American History X, but Guy Torry stole the show as the inmate that changes his view on life.

How did he not get an award?

@bnjbvr@tutut.delire.party Tu fais bouillir de l'eau dans un récipient, tu mélanges de la poudre de chocolat et de la poudre de lait dans un autre, tu mélanges dans un mixer, et tu sers.