Addis Scribe transcribes Swahili, live and from uploads. We compared it with Google Gemini Flash and Google Chirp 3 on three public test sets, with the same scorer for every system. Google's systems are ahead on two of the three sets, and we publish every number.
Against Google Gemini Flash and Google Chirp 3, Addis Scribe is close on African Voices, slightly ahead on FLEURS and behind on Common Voice. It is the only system here scored live, as text appears.
Streaming in 1.12 s steps, Addis Scribe records 7.9% WER on the African Voices Swahili test, where Google Gemini Flash (6.3%) and Google Chirp 3 (7.2%) are ahead. On FLEURS it records 14.6%, against 15.8% for Gemini Flash and 18.7% for Chirp 3. Both Google systems were given the whole clip.
Read the full accuracy tablesOn Common Voice Swahili, Addis Scribe records 19.6% WER. Google Gemini Flash and Google Chirp 3 each record 8.8%.
Read the full accuracy tablesWord error rate and character error rate on each set. Addis Scribe was scored live with 1.12 s chunks; the Google systems were given the whole clip.
2,000 test clips, word error rate, lower is better
487 test clips, word error rate, lower is better
2,000 clips, word error rate, lower is better
| System | Mode | African Voices | FLEURS | Common Voice | |||
|---|---|---|---|---|---|---|---|
| WER | CER | WER | CER | WER | CER | ||
| Addis Scribe Swahili | Live, 1.12 s chunks | 7.9% | 2.4% | 14.6% | 4.5% | 19.6% | 7.0% |
| Google Gemini Flash | Whole file | 6.3% | 2.0% | 15.8% | 7.4% | 8.8% | 4.2% |
| Google Chirp 3 | Whole file | 7.2% | 2.3% | 18.7% | 9.1% | 8.8% | 3.1% |
Bold marks the best value in each column.
Addis Scribe streams live audio. The other systems were given the whole recording, so they are scored for accuracy only.
| System | Live audio | How it was run |
|---|---|---|
| Addis Scribe Swahili | Native streaming | Streams in 1.12 s chunks through the Addis Scribe API, so text appears while people speak. Scored live only. |
| Google Gemini Flash | Offline only | Given the whole clip with a transcription prompt. No live mode, so scored for accuracy. |
| Google Chirp 3 | Native (cloud API) | Google Cloud speech recognition, given the whole clip. |
The test sets are public. None of these test clips were used to train Addis Scribe.
| Test set | Where to find it | Clips | What it measures |
|---|---|---|---|
| African Voices | African Voices Swahili release | Swahili test split, 2,000 clips | Swahili speech with human transcripts |
| FLEURS | google/fleurs | sw_ke test split, all 487 clips | Read sentences with human transcripts |
| Common Voice | Mozilla Common Voice | Swahili, 2,000 clips | Read sentences recorded by volunteers |
Every system on a set is scored with the same scorer and text normaliser, against the same human transcripts. WER is word error rate and CER is character error rate, both lower is better.
Add language=sw to a Scribe upload, live session or long-file job. Uploads, live streaming, word timestamps, captions all work in Swahili, at the same per-character price as Amharic. Swahili text comes back with capital letters, punctuation and numbers as digits. Speaker labels are not available in Swahili.