Addis Scribe now transcribes Afaan Oromo, live and from uploads. We scored it against every Oromo system we could run on the same held-out natural speech, for accuracy and for how fast text reaches the screen.
Addis Scribe has the lowest error rate of every system we tested on natural Oromo speech, live or offline. On a small read-speech check Google Chirp 2 comes out ahead, and we publish that too.
On 150 WAXAL test clips, streaming in 1.12 s steps, Addis Scribe records 21.1% WER and 6.8% CER. Google Chirp 2 records 42.4% WER streaming and 34.8% given the whole file. The best open model, Ethio-ASR Oromo, records 24.8% offline.
Read the full accuracy tablesThe transcript is final 37 to 43 ms after speech ends, and first words appear after 1.57 s with 0.32 s chunks. Google Chirp 2 streaming takes 1,051 ms and 6.37 s on the same clips.
Read how latency was measuredOn 60 FLEURS read sentences, Google Chirp 2 batch records 53.3% WER against 57.4% for an Addis Scribe upload. The set is small, so we report it as a check rather than a headline.
Read the read-speech check150 clips from the WAXAL Afaan Oromo test split, 23 speakers, 45 minutes, human transcripts. Every system transcribed the same clips.
150 test clips, word error rate, lower is better
WAXAL, whole-file transcription, lower is better
| System | Mode | WER | CER | Final text after speech ends | First words | Processing per minute of audio |
|---|---|---|---|---|---|---|
| Addis Scribe Oromo | Live, 1.12 s chunks | 21.1% | 6.8% | 43 ms | 2.21 s | n/a |
| Addis Scribe Oromo | Live, 0.32 s chunks | 21.5% | 6.9% | 37 ms | 1.57 s | n/a |
| Addis Scribe Oromo | Upload | 22.3% | 7.6% | n/a | n/a | 0.78 s |
| Ethio-ASR Oromo (w2v-BERT) | Offline | 24.8% | 7.1% | n/a | n/a | 0.68 s |
| Ethio-ASR multilingual 600M | Offline | 25.7% | 7.2% | n/a | n/a | 0.69 s |
| WAXAL benchmark Whisper-small Oromo | Offline | 28.2% | 8.2% | n/a | n/a | 2.70 s |
| WAXAL benchmark MMS-300M Oromo | Offline | 29.5% | 7.9% | n/a | n/a | 0.21 s |
| Google Chirp 2 (om-ET) | Batch | 34.8% | 9.9% | n/a | n/a | 9.77 s |
| Google Chirp 2 (om-ET) | Streaming | 42.4% | 14.2% | 1,051 ms | 6.37 s | n/a |
Bold marks the best value in each column. Processing per minute is whole-file transcription, one clip at a time; for Google it is the full request time, network included.
Audio is fed at real-time speed, as from a live microphone. We report how long until the first words appear, and how long after the speaker stops until the transcript is final.
WAXAL, 150 clips, lower is better
WAXAL, 150 clips, lower is better
Addis Scribe was measured on one NVIDIA A100-class GPU. Google was measured from its europe-west4 endpoint, the nearest region that serves Oromo, so its figures include the network round trip.
Uploads scored on the whole WAXAL Oromo test split, for the systems run on all of it. The ranking matches the 150-clip set.
| System | Mode | WER | CER |
|---|---|---|---|
| Addis Scribe Oromo | Upload | 20.8% | 5.8% |
| Ethio-ASR Oromo (w2v-BERT) | Offline | 24.2% | 6.2% |
| Ethio-ASR multilingual 600M | Offline | 24.5% | 6.1% |
FLEURS Oromo has only 60 dev and test clips, so we report it as a check, never the headline. Google Chirp 2 leads here.
60 clips, word error rate, lower is better
| System | Mode | WER |
|---|---|---|
| Google Chirp 2 (om-ET) | Batch | 53.3% |
| Addis Scribe Oromo | Upload | 57.4% |
| Ethio-ASR Oromo (w2v-BERT) | Offline | 62.8% |
| WAXAL benchmark Whisper-small Oromo | Offline | 65.7% |
| WAXAL benchmark MMS-300M Oromo | Offline | 67.5% |
| Ethio-ASR multilingual 600M | Offline | 69.3% |
With 60 clips, a gap of a few points is within the noise. Read sentences also differ from the natural speech most apps transcribe.
Addis Scribe and Google Chirp 2 run on live audio. The open models need the whole recording, so they are scored for accuracy and processing time only.
| System | Live audio | How it was run |
|---|---|---|
| Addis Scribe Oromo | Native streaming | Streams in 0.32 s or 1.12 s chunks, so text appears while people speak. Also tested as an upload of the whole file. |
| Ethio-ASR Oromo (w2v-BERT) | Offline only | Needs the whole recording before it can answer. Scored for accuracy and processing time. |
| Ethio-ASR multilingual 600M | Offline only | Needs the whole recording before it can answer. Scored for accuracy and processing time. |
| WAXAL benchmark Whisper-small Oromo | Offline only | Transcribes complete audio. Scored for accuracy and processing time. |
| WAXAL benchmark MMS-300M Oromo | Offline only | Needs the whole recording before it can answer. Scored for accuracy and processing time. |
| Google Chirp 2 (om-ET) | Native (cloud API) | Server-side streaming over the network from europe-west4, the nearest region that serves Oromo. Also tested as a batch request on the whole file. |
Both test sets are public. None of the WAXAL test clips were used to train our models.
| Test set | Repository | Split | What it measures | Selection |
|---|---|---|---|---|
| WAXAL | google/WaxalNLP | orm test | Natural speech with human transcripts | 150 clips, 23 speakers, 45 minutes, the same clips for every system. Upload accuracy is also scored on the full split of 3,782 clips. |
| FLEURS | google/fleurs | om_et dev + test | Read sentences | All 60 clips, a small check |
Every system is scored with the same text normaliser: Unicode NFKC, lowercase, apostrophes kept because they mark the Oromo glottal stop, other punctuation removed. WER is word error rate and CER is character error rate, both lower is better.
Every number on this page comes from per-clip records scored by one script. They are not published yet and are available on request.
The 150 WAXAL Oromo test clips: ids, speakers, durations and human reference transcripts.
Every system's transcript and latency for every clip, in every mode tested.
The scorer, including the shared text normaliser applied to every system.
Available on request from contact@addisassistant.com.
Add language=om to a Scribe upload, live session or long-file job. Uploads, live streaming, word timestamps, captions and speaker labels all work in Oromo, at the same per-character price as Amharic.