Addis Scribe now transcribes Hausa, live and from uploads. We compared it with Google Gemini Flash and Google Chirp 3 on three public test sets, with the same scorer for every system. Each of the three comes out ahead on one of the sets, and this page shows where.
On WAXAL, Addis Scribe is about level with Google Gemini Flash. On FLEURS, Google Chirp 3 is ahead. On Common Voice, Addis Scribe has fewer errors than both Google systems.
On all 197 clips of the WAXAL Hausa test split, Addis Scribe records 22.5% WER live (1.12 s chunks), 22.8% with 0.32 s chunks and 23.4% as an upload. Google Gemini Flash records 22.2% given the whole clip, and Google Chirp 3 records 60.9% as a batch request.
Read the full accuracy tablesOn FLEURS Hausa, Google Chirp 3 (19.5%) is ahead of Addis Scribe (24.2%), and Gemini Flash records 26.3%. On Common Voice Hausa, Addis Scribe records 17.3% live, against 21.2% for Gemini Flash and 24.6% for Chirp 3.
Read the full accuracy tablesWord error rate and character error rate on each set, for every mode. Every system transcribed the same clips.
197 test clips, word error rate, lower is better
621 test clips, word error rate, lower is better
829 clips, word error rate, lower is better
| System | Mode | WAXAL | FLEURS | Common Voice | |||
|---|---|---|---|---|---|---|---|
| WER | CER | WER | CER | WER | CER | ||
| Addis Scribe Hausa | Live, 1.12 s chunks | 22.5% | 8.5% | 24.2% | 9.3% | 17.3% | 4.3% |
| Addis Scribe Hausa | Live, 0.32 s chunks | 22.8% | 8.5% | n/a | n/a | n/a | n/a |
| Addis Scribe Hausa | Whole file | 23.4% | 8.5% | 26.0% | 10.6% | 17.2% | 4.3% |
| Google Gemini Flash (latest) | Whole file | 22.2% | 8.1% | 26.3% | 14.2% | 21.2% | 5.7% |
| Google Chirp 3 | Batch | 60.9% | 53.6% | 19.5% | 6.2% | 24.6% | 20.2% |
Bold marks the best value in each column. The WAXAL chart cuts the Chirp 3 bar at 40%; its value is in the table. Addis Scribe with 0.32 s chunks was scored on WAXAL only.
Addis Scribe streams live audio. The other systems were given the whole recording, so they are scored for accuracy only.
| System | Live audio | How it was run |
|---|---|---|
| Addis Scribe Hausa | Native streaming | Streams in 0.32 s or 1.12 s chunks through the Addis Scribe API, so text appears while people speak. |
| Google Gemini Flash (latest) | Offline only | Given the whole clip with a transcription prompt. No live mode, so scored for accuracy. |
| Google Chirp 3 | Native (cloud API) | Google Cloud speech recognition, tested as a batch request on the whole clip. |
The test sets are public. None of these test clips were used to train Addis Scribe.
| Test set | Where to find it | Clips | What it measures |
|---|---|---|---|
| WAXAL | google/WaxalNLP | Hausa test split, all 197 clips | Hausa speech with human transcripts |
| FLEURS | google/fleurs | ha_ng test split, all 621 clips | Read sentences with human transcripts |
| Common Voice | Mozilla Common Voice | Hausa, 829 clips | Read sentences recorded by volunteers |
Every system on a set is scored with the same scorer and text normaliser, against the same human transcripts. WER is word error rate and CER is character error rate, both lower is better.
Add language=ha to a Scribe upload, live session or long-file job. Uploads, live streaming, word timestamps, captions and speaker labels all work in Hausa, at the same per-character price as Amharic. Hausa text comes back in lowercase without punctuation, with the Hausa letters ɓ, ɗ and ƙ.