Hugging Face adds Egyptian and Nigerian English to Open ASR Leaderboard

Hugging Face has expanded its Open ASR Leaderboard with private data for Egyptian and Nigerian English accents, according to a blog post published by the platform.reported This addition follows previous updates that included private splits from Appen, DataOceanAI and Voice Arena to cover scripted and conversational speech across five accents.This update allows users to test speech recognition models on Egyptian and Nigerian English accents, which were not previously available in the leaderboard’s private data.The inclusion of Liva AI’s private data splits addresses previous community concerns about benchmark optimisation by providing held-out sets for Egyptian and Nigerian English accents.

The evidence

The new data splits were provided through a collaboration with Liva AI, as stated in the Hugging Face announcement. Liva AI collects multimodal interaction data via consumer products that real people use, allowing it to capture natural, high-context behaviour rather than artificially elicited interactions. Because Liva owns both the collection environment and distribution, it can design products around specific model gaps across domains, industries, languages and modalities.

The decision

Hugging Face is using this data to improve representation in automatic speech recognition evaluations and to enhance trust by minimising benchmark optimisation, according to the post.reported The platform now reports performance using a richer breakdown of accents based on Kachru’s Three Circles, an idea credited to Hume AI.reported The leaderboard will present US versus non-US accents, including UK, Australia, Canada, India, Nigeria and Egypt, as well as Inner Circle versus Outer and Expanding Circle comparisons.

What to make of it

The move reflects a continued effort to include more accents and languages in ASR evaluations, according to the published report By adding new private data and new data providers, the platform aims to create more trustworthy evaluations that rely on held-out sets.reported The community previously highlighted the importance of these sets for reliable performance measurement, prompting the addition of private data months ago.

In practice

For startup founders, this update means access to more diverse evaluation metrics for speech recognition models, according to the Hugging Face post.reported The availability of private data for Egyptian and Nigerian accents allows for testing against accents that were previously underrepresented in public benchmarks. However, the specific limitations of these new splits, such as the size of the datasets or the specific domains covered, are not detailed in the announcement. Users must rely on the platform’s reporting structure, which currently groups accents into US versus non-US and Inner versus Outer/Expanding Circle categories.