pheme-tts
My custom text-to-speech model small enough to ship inside a web page. About 2.6 MB of weights run entirely in your browser, and nothing you type leaves the page.
Note
Named for Pheme, the Greek voice of report and rumour, who was said to start out tiny and grow as she travelled.
How it works
The speech model itself has 257K parameters. It turns phonemes into a spectrogram. Two helper networks sit on either side of it:
- Front end (559K parameters): rules turn numbers, money and abbreviations into words. A small transformer then spells each word in phonemes, so no pronouncing dictionary has to be downloaded. The speech model also reads this transformer's hidden states, which carry some sense of how the word is spelled.
- Speech model (257K parameters): this one has three branches. The first is the phonemes. The second is timing and prosody: how long, how loud, what pitch. The third is a small leftover latent for everything else. In training, branches two and three are learned from the real audio. On this page they are predicted from the text alone.
- Vocoder (541K parameters): it turns the spectrogram into sound. It predicts magnitude and phase, and about 40 lines of JavaScript do the inverse FFT.
Read-along comes for free. The speech model predicts how many 16 ms frames each phoneme lasts, and those durations are exactly what the audio is built from. Summing them per word gives each word's start time, with no separate alignment step.
Long text is split into chunks of up to about 50 words, at sentence ends where possible. A background thread (a Web Worker, still on your machine) generates them one by one, and playback starts as soon as the first chunk is ready. Two speed controls: pace changes how fast the model itself speaks (its predicted durations), and speed changes playback live, from 0.5x to 4x, with time-stretching that keeps the pitch. All three networks run on ONNX Runtime Web, loaded from jsDelivr, at 60-100x real time on a laptop.
The voice was trained on about 47 hours of synthetic single-speaker speech. On 200 held-out sentences Whisper's character error rate is 1.0%, against 1.5% on the reference recordings. Its UTMOS naturalness score is 4.05, against 4.42 for the reference. The training audio was generated with a non-commercial model, so this voice is for demo and non-commercial use only.