ASR — speech to text
Fon v14 (tone_aware_v14_real_on_v12s). SpeechBrain wav2vec2 backbone (whettenr/asr-fon-with-diacritics-bpe-256) with F0 side-encoder and auxiliary tone-CTC head. Trained on 198 h of Fon (FFSTC-2 + Zenodo + Bible + non-religious lessons + four conversational sources + MMS-TTS synthetic).
- FFSTC-2 in-distribution WER_NT: 15.25 %
- Disjoint cross-corpus avg WER_NT: 18.79 %
TTS — text to speech
Stock facebook/mms-tts-fon (Meta MMS, VITS). Single-speaker. Our project fine-tunes (v1, v2) regressed vs the stock baseline — VITS fine-tuning destroys the text encoder regardless of LR / GAN settings. Stock MMS reaches 18.10 % ASR-loop WER_NT on our held-out Bible eval (n=200, seed=1337).
Cascade chat (text + S2S)
NLLB-200-3.3B + Fon-LoRA (v2 final) for Fon↔French translation, wrapped around Qwen2.5-7B-Instruct (4-bit NF4) as the French reasoner. Fon-only history is kept in process. The Speech↔Speech tab chains ASR → cascade → TTS client-side, so the same `/api/*` endpoints back every loop.
Known limits
- Heavy French code-switching is partially handled but not the focus.
- Spontaneous overlapping speech degrades ASR.
- One voice, fixed prosody (multi-speaker TTS coming).
- Multi-user chat shares one process-global history (single-tenant demo).
- Not for safety-critical use (medical, legal numerics).
Backend health (live)
loading…