Naija Eval: an open evaluation pipeline for Nigerian-language translation and speech recognition

Abstract

Background. Nigerian-language AI needs evaluation that distinguishes languages, tasks, orthographic conventions, and the conditions under which a system will be used. A fluent demonstration or an aggregate leaderboard cannot establish those properties.

Problem. Comparisons become difficult to interpret when the sampling unit, generation protocol, normalization, model identity, or retained evidence is unclear. Public scores alone do not make an experiment reproducible.

Approach. This technical report describes Naija Eval as an open evaluation pipeline for English–Hausa, English–Igbo, and English–Yoruba translation and for speech recognition in the three Nigerian languages. It reconstructs the inspected pipeline, states design principles for interpreting its outputs, and examines results at repository revision bccd98f70b75b7b4f201aff4ed87436585b05ed3. This is an affiliated documentary audit of existing artifacts, not a new experimental run.

Results. On 100 aligned sentence groups evaluated in six translation directions, the archive records mean chrF++ of 40.3 for N-ATLaS, 24.9 for the selected Llama 3.1 8B Instruct baseline, and 51.6 for the highest-scoring hosted label. Language-specific speech models record normalized WER of 38.5% for Hausa, 45.4% for Igbo, and 64.7% for Yoruba. Removing combining marks reduces Yoruba WER to 33.5%, but changes the equivalence relation being measured. These are source-reported descriptive findings, without independent replication or uncertainty intervals.

Conclusion. The system provides an inspectable pilot comparison, not certification of general language competence or deployment safety. Complete prediction retention, fixed environments, paired uncertainty estimation, and community-grounded human evaluation are necessary next steps. The report distinguishes the implemented procedure, the archived observations, and proposed extensions throughout.

Keywords: Nigerian languages, low-resource NLP, machine translation, automatic speech recognition, chrF++, word error rate, orthography, reproducibility, benchmark validity

1. Introduction

The availability of a model that can process Nigerian languages is not, by itself, evidence that it can reliably serve Nigerian users. A system may translate a formal sentence while failing on conversational code-switching, recognise read speech while misrecognising a dialect, or produce fluent text that is factually or pragmatically inappropriate. The central evaluation problem is therefore one of construct validity: which capacity is being measured, for which speakers and settings, and with what relationship to the proposed application?

Naija Eval responds to an important accountability gap by publishing comparative scores, evaluation scripts, and an accompanying explanation of its work.^[1][2] This creates a basis for scrutiny that is stronger than an unsupported claim of multilingual capability. Nevertheless, openness is not a binary property. Public code, public scores, complete predictions, fixed model snapshots, and independent replication are distinct evidentiary achievements. This paper evaluates the project on those terms rather than treating a leaderboard as a complete measure of linguistic adequacy.

The argument is that Nigerian-language evaluation should be task-specific, language-specific, and auditable. The contribution here is a critical synthesis of existing Naija Eval evidence: a reconstruction of its sample and metrics, a descriptive account of published results, an assessment of reproducibility limits, and a research agenda for stronger locally grounded evaluation. This report treats Naija Eval as an engineering artifact whose measurement behavior must be specified before its results can be interpreted. Neither the project nor this report is claimed to have undergone external peer review.

Contributions. The report provides: (1) a reconstruction of the translation and speech pipelines at a fixed repository revision; (2) explicit measurement and reporting principles; (3) a language- and direction-disaggregated account of archived scores; (4) a distinction between procedure inspection and score reproduction; and (5) a reproducibility and responsible-use protocol for subsequent evaluation. These contributions are documentation and analysis, not newly collected benchmark data.

Scope and structure. Section 2 situates the work in multilingual evaluation. Sections 3–5 specify design principles, pipeline stages, and the inspected evaluation protocol. Sections 6–8 describe archived results and operational claims. Sections 9–11 state validity threats, a verification protocol, and responsible-use boundaries. Section 12 concludes; declarations and appendices identify provenance and the minimum artifact record.

2. Background and related work

FLORES-101 established professionally translated multilingual evaluation material for low-resource machine translation, while FLEURS extended parallel text into a multilingual speech benchmark covering 102 languages.^[4][5] These resources facilitate comparison across languages because aligned material reduces variation in sentence content. Their provenance also imposes boundaries: translated, largely encyclopaedic material and clean read speech cannot represent all communicative situations in Nigeria.

African-language research illustrates complementary approaches. MasakhaNER documents collaborative annotation for named-entity recognition in African languages, including Hausa, Igbo, and Yoruba.^[8] IrokoBench broadens evaluation to natural-language inference, mathematics, and knowledge questions across 17 African languages.^[9] These projects are not direct validation of Naija Eval's translation or speech results; they demonstrate that task diversity and language-community participation can inform benchmark design.

The Holistic Evaluation of Language Models framework argues for evaluation across dimensions including accuracy, robustness, calibration, fairness, bias, toxicity, and efficiency.^[10] Dataset documentation and model reporting frameworks likewise make provenance, intended use, and limitations central rather than ancillary.^[11][12] Together, these literatures imply that a useful scorecard must define both its measured construct and the claims it cannot support. A language label is not a substitute for a documented distribution of dialects, speakers, topics, and usage contexts.

3. Design principles

The following principles formalize how the inspected system should be documented and interpreted. They are reporting requirements derived from the audit, not claims that the current implementation enforces every requirement automatically.

P1. One task, one stated construct. Translation measures reference-based overlap in six specified directions; speech recognition measures transcription error in three specified languages. Neither is a general test of intelligence, cultural knowledge, or factual reliability. The N-ATLaS translation model and the language-specific NCAIR speech models are separate evaluated systems.

P2. Preserve the sampling unit. Aligned sentence identifiers carry shared content across translation directions. Counts of outputs must therefore be reported separately from counts of independent content groups. A deterministic selection rule makes the sample reconstructable but not necessarily representative.^[4][5]

P3. Make normalization part of the specification. Lowercasing, punctuation handling, Unicode normalization, and combining-mark removal change which strings count as equivalent. Strict normalized and mark-stripped speech scores must be named separately. A lenient condition is a sensitivity analysis, not a replacement for the primary orthographic result.

P4. Separate observation from inference. An archived score is an observation attributed to a run; an advantage over a baseline is a protocol-bound comparison; a claim about populations or applications requires additional evidence. Rounded metric differences do not establish statistical significance.^[6][7][10]

P5. Retain evidence, not just summaries. An auditable run should preserve sample identifiers, references, predictions, scoring signatures, and execution settings. The inspected archive falls short of complete prediction retention; this principle describes the needed standard rather than a property already achieved.

P6. State what was not tested. Dialects, conversational code-switching, noisy audio, human semantic adequacy, deployment security, and application-specific harms were not evaluated in the reviewed materials. Missing evidence should remain visible rather than being replaced with an optimistic interpretation.^[11][12]

4. System architecture and data flow

Input and alignment. The translation path begins with FLEURS test transcriptions in English, Hausa, Igbo, and Yoruba. The inspected scripts reduce each language to distinct sentence identifiers, compute the shared identifier set, sort it, and select a 100-group prefix. Alignment keeps the underlying content comparable between directions, while imposing the shared-domain and sampling limits described in Section 9.^[2][3]

Task expansion and generation. Each aligned group supplies English-to-Hausa, Hausa-to-English, English-to-Igbo, Igbo-to-English, English-to-Yoruba, and Yoruba-to-English tasks. This creates 600 expected outputs per translation model. A translator instruction requests only a target-language rendering; the scoring path uses the first stripped output line. Multi-line generation can consequently lose content before scoring. The record should distinguish the original response from the processed prediction; the inspected archive does not preserve a complete pair for every item.

Translation scoring and aggregation. Processed predictions are compared with references using SacreBLEU chrF++ with word-order two. Scores are produced per direction, then summarized by an equally weighted mean of six rounded direction scores. This architecture separates generation from scoring conceptually, but cannot remove confounding from different model settings, token budgets, or reference conventions.^[6][7]

Speech path. Speech evaluation selects 100 distinct FLEURS sentence identifiers for each Nigerian language, converts audio to mono at 16 kHz, runs the relevant recognition model, and compares the transcription with the reference. The archive reports language-specific WER and a separate combining-mark-stripped condition. Unlike translation, the report does not establish that the three speech samples form a single identical cross-language cohort.

Output boundary. The public evidence consists of evaluation code, aggregate scores, and selected examples. The code describes an executable procedure; the summary describes what its publisher reports obtaining. Neither alone proves that every committed score follows from the exact inspected environment. The pipeline is not presented here as a hosted inference service, a security-hardened execution engine, or an independently validated benchmark.

5. Evaluation protocol and inspected implementation

Design and evidence boundary. This paper is a documentary audit conducted on 1 November 2026. It examines the original project article, the public methods pages, and the evaluation code and results at repository revision bccd98f70b75b7b4f201aff4ed87436585b05ed3.^[1][2][3] No model inference was rerun. Numerical results are attributed to the project archive; they are not independently replicated measurements. The source article is credited to Olorunfemi Obasa, MBA, and this paper retains that attribution.

Translation sample. The scripts obtain FLEURS test transcriptions for English, Hausa, Igbo, and Yoruba, deduplicate sentence identifiers, intersect identifiers across languages, sort them, and select the first 100 aligned sentence groups. Each model translates these groups in six directions: English to and from each Nigerian language. This yields 600 translation outputs per model, but only 100 aligned source-content groups. These are not 600 independent sentence groups and not the complete FLORES benchmark.^[2][3][4][5]

The deterministic prefix makes sample selection inspectable but does not establish representativeness. Shared sentence content also means observations across directions are related. A future confidence-interval procedure should preserve that structure, for example by resampling aligned sentence groups rather than treating every output as independent. The present archive does not provide uncertainty estimates or a statistical test of the rankings.

Translation protocol and scoring. The inspected scripts use a professional-translator instruction and ask for only the target-language translation. They retain the first stripped output line. Local and hosted settings differ in token budgets and execution environments, so a shared textual prompt should not be mistaken for complete experimental equivalence. The primary metric is chrF++, implemented using SacreBLEU with word-order two.^[3][6][7] chrF++ measures reference-based character and word n-gram overlap. A score of 40.3 is not 40.3% translation accuracy, semantic correctness, or user satisfaction. The overall average equally weights the six direction scores, which are already rounded before averaging.

Speech protocol. Speech recognition uses the first 100 distinct sentence identifiers for each language from the FLEURS test material. Audio is made mono and resampled to 16 kHz. The archive compares the language-specific NCAIR speech models with Whisper Small for Hausa and Yoruba; no comparable Igbo baseline is reported.^[2][3] The speech models should not be conflated with the N-ATLaS translation model merely because they appear under the same project umbrella.

Word error rate (WER) is the sum of substitutions, deletions, and insertions divided by the number of reference words, multiplied by 100. Character error rate applies an analogous calculation at character level. WER can exceed 100% because insertions contribute errors without increasing the reference denominator. These are transcription error metrics, not direct measures of comprehension or downstream safety.

Normalization. The primary speech comparison already lowercases and normalizes Unicode, punctuation, and whitespace. An additional comparison decomposes Unicode text and removes all combining marks. Describing this as simply “ignoring tone” is insufficient: it also removes underdots used in Yoruba and Igbo orthography. A change in score under this transformation cannot be attributed exclusively to tone errors. The strict and mark-stripped conditions measure different notions of textual equivalence.

6. Translation evaluation

The pinned archive records seven translation model labels. The main article emphasizes six, but its broader summary also contains Mistral Small 3.2. To avoid selection-dependent claims about rankings and price, Figure 1 reports all seven archived averages.^[1][3] Model names are reproduced as publisher-recorded labels; this audit did not independently authenticate hosted provider snapshots or their continuing availability.

N-ATLaS records 40.3 average chrF++, versus 24.9 for the selected Llama baseline: a 15.4-point difference. This is a difference in metric points, not a 15.4% improvement. The ratio 40.3 divided by 51.6 is approximately 78.1%, but describing that ratio as “78% as capable” would incorrectly convert reference-overlap scores into a scale of general ability. Even relative comparisons must remain anchored to the sample, tasks, scoring implementation, and particular baseline.^[3][6]

Direction-level results reveal why a single average is insufficient. N-ATLaS scores 47.4 for English-to-Hausa and 51.2 for Hausa-to-English; 37.1 for English-to-Igbo and 44.2 for Igbo-to-English; and 24.5 for English-to-Yoruba and 37.6 for Yoruba-to-English. In this sample, translating into the Nigerian languages is consistently harder by this reference-based metric than translating into English. That descriptive pattern does not isolate a cause: training coverage, reference orthography, tokenization, generation behavior, and the translated domain are all plausible contributors requiring additional experiments.

The leading three hosted labels average 51.6, 50.8, and 50.6. Without paired uncertainty estimates, the 0.2-point difference between the latter two cannot responsibly establish a stable population-level ordering. Nor should the selected small baseline be treated as representative of every open model or every larger configuration. The appropriate conclusion is that N-ATLaS improves on this baseline under this protocol, while the evaluated hosted systems record higher reference-overlap scores.

7. Speech evaluation and orthographic sensitivity

The language-specific speech models record normalized WER of 38.5% for Hausa, 45.4% for Igbo, and 64.7% for Yoruba. The comparable Whisper Small results are 92.1% for Hausa and 113.8% for Yoruba. These are substantial descriptive differences in the archived sample, but there is no Igbo baseline comparison and no evidence here of equal performance across accents, demographic groups, or recording conditions.^[3]

Removing combining marks leaves Hausa WER unchanged, reduces Igbo WER to 39.4%, and reduces Yoruba WER to 33.5%. The Yoruba difference is 31.2 percentage points. This shows that normalization policy materially affects the reported outcome. It does not show that all affected outputs are linguistically equivalent or that most underlying mistakes are solely tonal. Collapsing underdots and tonal distinctions may conceal changes relevant to word identity and meaning.

Both conditions should be reported, with the stricter orthographic result retained rather than replaced. Human error analysis should distinguish omitted tone marks, changed vowel identity, lexical substitutions, deletions, insertions, named-entity errors, and meaning-changing transcription. For applications such as clinical intake, legal documentation, or public-service records, the consequences of a rare meaning-changing error may be more important than an aggregate error reduction. The current benchmark does not evaluate those applications or certify their safety.

8. Efficiency and operational scope

The archive records hosted translation charges for the sampled calls, including $0.6232 for the highest-scoring label and $0.021 for the lower-scoring Mistral entry.^[3] These historical charges are not current price quotations or comparable total-cost-of-ownership estimates. Local inference does not have a hosted API charge in the table, but still consumes hardware, electricity, engineering time, storage, and maintenance. A blank hosted-cost field must not be read as zero economic cost.

Local execution may reduce dependence on an external inference service and permit offline processing. However, running locally is not proof of privacy, security, operational resilience, or technological sovereignty. Those outcomes depend on deployment controls, data handling, licensing, supply-chain dependencies, available expertise, and governance. A paired full-precision experiment would also be needed before assigning a numerical performance penalty to the reported quantized local execution. No such experiment was identified in the inspected archive.

9. Limitations and threats to validity

Incomplete prediction retention. Files labelled as raw translation results retain aggregate scores and three examples per direction, not all 600 predictions. Speech artifacts likewise retain only a small example set. Consequently, readers can inspect code and selected outputs and attempt a new run, but cannot recompute every published score from the committed predictions alone. Full prediction archives are a priority because they permit alternative metrics, error analysis, and statistical comparison without another paid inference run.^[2][3]

Version uncertainty. The inspected scripts do not consistently pin dataset and model revisions or fully version all metric dependencies. Hosted aliases and provider behavior can change. A stronger evaluation record would identify immutable model snapshots where available, provider routes, quantization configuration, tokenizer and chat-template versions, generation parameters, dependency locks, and metric signatures. Public software infrastructure can improve comparability, but simply adopting a tool does not establish scientific validity.^[7][14]

Domain and sampling limits. A sorted 100-item prefix of aligned, translated, read material is useful for a small transparent pilot, but does not warrant conclusions about Nigerian conversational communication in general. The audit found no native-speaker scoring, dialect-stratified analysis, code-switching suite, cultural reasoning tasks, uncertainty intervals, or repeated-run analysis in the inspected evaluation. Absence from the archive is not proof that such work never occurred; it means those claims are not substantiated by the material reviewed.

Benchmark exposure. FLEURS and FLORES are publicly available, creating a possible risk of training exposure or benchmark contamination. That risk is not evidence that any evaluated model is contaminated. The project has not supplied a contamination audit in the reviewed material, and this paper makes no accusation about individual systems. Research on contamination motivates targeted audits and independently held-out material, not unsupported attribution.^[13]

10. Verification and reproducibility protocol

Verification status. This report inspected documentary evidence; it did not execute the evaluation scripts, run software tests, perform mutation testing, or reproduce the scores. Public availability of scripts is not evidence that those checks have passed. In contrast to a library verification study, no new pass rate or test coverage is reported here.

Proposed rerun protocol. A reproducer should begin at the pinned revision, record dataset and model snapshots, preserve the selected identifiers, and save raw responses alongside the text actually scored. Before interpreting model comparisons, use small fixtures to check identifier alignment, direction labels, output preprocessing, and metric configuration. Those fixtures and any failures should be published; this is a proposed procedure, not a test suite claimed to exist in the inspected project.

The next stage should combine benchmark continuity with new evidence. First, release complete reference-and-prediction pairs, sample identifiers, evaluation signatures, and the fixed environment used for each run, subject to appropriate dataset terms. Publish both language-level and direction-level scores. Estimate paired uncertainty using the aligned sampling unit, and document whether repeated API calls change outputs or scores.

Second, co-design additional tasks with Hausa-, Igbo-, and Yoruba-speaking researchers and communities. Include locally authored rather than exclusively translated material, document dialect and orthography decisions, and cover conversational speech, noisy audio, code-switching, named entities, administrative language, and context-dependent meaning. Community involvement should be documented as a substantive part of task design and annotation, not invoked as a label.^[8][9][11]

Third, add blinded human evaluation with explicit rubrics for semantic adequacy, fluency, pragmatic appropriateness, and meaning-changing speech errors. Report annotator qualifications, disagreement, adjudication, and inter-rater agreement. Automated reference-overlap and transcription metrics should remain available, but should not stand in for those distinct judgments.

Fourth, evaluate proposed deployment contexts independently. Robustness, harmful outputs, calibration where applicable, demographic differences, privacy, latency, and full operating costs require application-specific protocols.^[10][12] Larger test sets are desirable, but size alone cannot correct an invalid construct or an unrepresentative sampling frame. A well-documented modest pilot is preferable to a larger opaque leaderboard.

11. Responsible use and security considerations

Appropriate use. The archived comparison can guide exploratory research and help identify where additional evaluation is needed. It should not be used as the sole basis for selecting a model for healthcare, legal records, education assessment, or other consequential communication. A language-level average does not identify a safe operating threshold for an individual utterance.

Data handling. FLEURS benchmark material and private user material are different risk categories. Extending the pipeline to personal recordings or confidential text would require informed consent, appropriate permissions, retention limits, and explicit disclosure of whether a hosted provider receives the inputs. Local execution can change that exposure, but does not by itself establish secure storage, isolation, or licensing compliance.^[11][12]

Execution and untrusted inputs. Dataset text and generated responses should be treated as data, not as executable instructions. A reproducer should review scripts and dependencies before running them and keep provider credentials out of committed artifacts. These are recommended controls; this report has not conducted a penetration test or a supply-chain audit and does not assert that the inspected system implements them.

Reporting responsibility. Publish failed runs and missing comparisons as missing, retain the strict orthographic condition, and avoid promoting model aliases into verified immutable identities. Community evaluation should state participant qualifications, compensation and consent arrangements where applicable, rather than presenting language labels as evidence of community endorsement.

12. Conclusion

Naija Eval offers useful preliminary evidence and an inspectable starting point for Nigerian-language evaluation. Its archived scores support improvements against selected translation and speech baselines under a documented small-sample protocol. They also demonstrate the importance of disaggregating languages, translation directions, and normalization choices. The most defensible contribution is not a claim of universal superiority but a public record that can be strengthened, challenged, and extended.

Declarations

Authorship and affiliation. Olorunfemi Obasa, MBA; Extrafemi Research. This report's attribution follows the existing source article. No additional author, collaborator, funder, or independent reviewer is inferred from the repository or project branding.

Report status. Technical report, version 1.0, revised 4 November 2026. The evidence boundary remains the documentary review conducted on 1 November 2026. This report version does not identify a Naija Eval software release.

Competing interests and evidence provenance. This analysis is published by Extrafemi Research and examines a project credited to Olorunfemi Obasa, MBA. It is therefore an affiliated analysis, not an institutionally independent replication. No new model evaluations, native-speaker judgments, or external peer review are claimed. The archived model labels and numerical results remain attributable to their source. Scholarly references supply methodological context; they do not independently validate Naija Eval's scores.

Data and code availability. The inspected code and results are linked in references ^[2] and ^[3] at the fixed revision. Their availability permits inspection and attempted reruns; it does not imply that the committed artifacts contain every prediction or all dependencies needed for exact score reconstruction. Dataset and model use remain subject to their respective terms.

Appendices

Appendix A. Run manifest specification. A subsequent reproducible release should include a machine-readable manifest containing: repository revision; execution date; dataset name, configuration, split and revision; ordered sample identifiers; model identifier and immutable snapshot where available; tokenizer and chat-template versions; quantization and device settings; prompt; generation parameters and output-length limits; dependency versions; scoring signature; normalization policy; and artifact checksums. This is a proposed schema, not a description of an existing complete manifest.

Appendix B. Prediction record specification. Each translation item should record the aligned group identifier, source and target languages, source text, reference, raw response, processed prediction, and completion or error status. Each speech item should record the sentence identifier, language, audio reference, sample rate, reference transcription, hypothesis, and the exact normalized strings used in both scoring conditions. Errors should remain in the record rather than disappearing through undocumented filtering.

Appendix C. Interpretation checklist. The translation sample comprises 100 aligned groups, six directions and 600 expected predictions per model. The speech sample comprises 100 identifiers per language. chrF++ is an overlap score, not percent accuracy. WER is an error ratio that can exceed 100%. Combining-mark removal changes underdots as well as tone marks. All reported scores belong to the source archive; no new inference run, significance test, native-speaker assessment, or security verification was performed for this report.

References

[1]Olorunfemi Obasa, MBA (2026). Naija Eval: N-ATLaS, tested in the open. Original project account, displaying November 2026. https://extrafemi.com/naija-eval-tested-in-the-open

[2]Naija Eval (2026). Public evaluation code and methods, revision bccd98f70b75b7b4f201aff4ed87436585b05ed3. https://github.com/obasaolorunfemi-ai/naija-eval/tree/bccd98f70b75b7b4f201aff4ed87436585b05ed3/eval

[3]Naija Eval (2026). Archived evaluation summary, same pinned revision; accessed 1 November 2026. Publisher-reported results, not rerun for this paper. https://github.com/obasaolorunfemi-ai/naija-eval/blob/bccd98f70b75b7b4f201aff4ed87436585b05ed3/results/summary.json

[4]Conneau, A., et al. (2022). FLEURS: Few-shot Learning Evaluation of Universal Representations of Speech. https://arxiv.org/abs/2205.12446

[5]Goyal, N., et al. (2022). The Flores-101 Evaluation Benchmark for Low-Resource and Multilingual Machine Translation. Transactions of the Association for Computational Linguistics. https://aclanthology.org/2022.tacl-1.30/

[6]Popović, M. (2017). chrF++: words helping character n-grams. Proceedings of the Second Conference on Machine Translation. https://aclanthology.org/W17-4770/

[7]Post, M. (2018). A Call for Clarity in Reporting BLEU Scores. Proceedings of the Third Conference on Machine Translation. Metric reproducibility and reporting context. https://aclanthology.org/W18-6319/

[8]Adelani, D. I., et al. (2021). MasakhaNER: Named Entity Recognition for African Languages. Transactions of the Association for Computational Linguistics. https://aclanthology.org/2021.tacl-1.66/

[9]Adelani, D. I., et al. (2025). IrokoBench: A New Benchmark for African Languages in the Age of Large Language Models. Proceedings of NAACL. https://aclanthology.org/2025.naacl-long.139/

[10]Liang, P., et al. (2023; preprint 2022). Holistic Evaluation of Language Models. Transactions on Machine Learning Research. https://arxiv.org/abs/2211.09110

[11]Gebru, T., et al. (2021). Datasheets for Datasets. Communications of the ACM. https://doi.org/10.1145/3458723

[12]Mitchell, M., et al. (2019). Model Cards for Model Reporting. Proceedings of the Conference on Fairness, Accountability, and Transparency. https://doi.org/10.1145/3287560.3287596

[13]Deng, C., Zhao, Y., Tang, X., Gerstein, M., and Cohan, A. (2024; preprint 2023). Investigating Data Contamination in Modern Benchmarks for Large Language Models. https://arxiv.org/abs/2311.09783

[14]Gao, L., et al. (2023). A framework for few-shot language model evaluation, v0.4.0. Software reference; not a claim that Naija Eval uses this framework. https://doi.org/10.5281/zenodo.10256836