Online fact-checking gets technically difficult when a Wolof claim concerns a disease outbreak and the strongest source is written in English or another African language. The AfrIFact benchmark measures that multilingual chain, revealing sharp losses in cross-language retrieval and near-chance zero-shot verification for many open models.

Researchers from Masakhane NLP, Saarland University, McGill University, the University of Waterloo, and collaborating institutions built more than 18,000 human-written claims with annotated source documents and evidence spans. The collection covers English and 10 African languages: Amharic, Hausa, Igbo, Oromo, Shona, Swahili, Twi, Wolof, Yoruba, and isiZulu.

AfrIFact pipeline for document retrieval, evidence extraction, and claim verification
AfrIFact evaluates the three stages of automated fact-checking separately.

Three tasks, one pipeline

AfrIFact separates automated fact-checking into three stages, allowing researchers to locate failures instead of reducing the entire process to one classification score:

  1. Information retrieval: Find relevant source documents for a claim in a large multilingual corpus.
  2. Evidence extraction: Select the sentences that support or contradict the claim.
  3. Claim verification: Assign SUPPORTS, REFUTES, or NOT_ENOUGH_INFORMATION. The third label applies when the available evidence cannot settle the claim.

The benchmark spans two domains with distinct sourcing methods. AfrIFact-Health uses World Health Organization articles translated from English, with claims and annotations transferred across language versions. AfrIFact-Culture-News draws from native-language Wikipedia articles about African political leaders, food, religion, sport, and currencies, supplemented by XL-Sum news articles.