Abstract
We investigate continued pretraining (CPT) for adapting wav2vec2-bert-2.0 to Swahili automatic speech recognition (ASR). Our approach combines unlabeled audio with limited labeled data through pseudo-labeled CPT followed by supervised finetuning. With 20,000 labeled samples, we achieve 3.24% WER on Common Voice Swahili-an 82% relative improvement over the baseline. This result surpasses the best previously reported academic system (8.3% WER from XLS-R) by 61% relative improvement. We provide concrete data requirements and a replicable methodology applicable to other low-resource languages.
How to cite
Hillary Mutisya, Thiomi-Lugha Nlp, John Mugane (2026). Continued Pretraining for Low-Resource Swahili ASR: Achieving State-of-the-Art Performance with Minimal Labeled Data. . https://doi.org/10.36227/techrxiv.177006495.52643247/v1
Republished on Discussion Research from the global Crossref scholarly index with full credit to the original authors and Institute of Electrical and Electronics Engineers (IEEE). Licence: https://creativecommons.org/licenses/by/4.0/
