A paper accepted to Findings of EMNLP formalizes why common uncertainty metrics can fail at hallucination detection. Researchers from Nanyang Technological University, KAIST, and Shanghai Jiao Tong University call the mismatch proxy failure and propose Truth AnChoring, or TAC, as a lightweight calibration method.
Hallucination detectors often treat predictive entropy, perplexity, semantic entropy, and similar scores as evidence that an answer is correct. These metrics describe model behavior rather than factual accuracy. TAC uses labeled examples to convert a raw score into an estimated probability of correctness, giving developers a value that can support abstention, retrieval, review, or routing thresholds.
- Input: An uncertainty score produced for a model response.
- Output: An estimated probability that the response is correct.
- Training data: Binary correctness labels paired with uncertainty scores.
- Reported scope: Fact-based question answering with models from 3B to 9B parameters.
Entropy can miss confident errors
Predictive entropy measures how dispersed the model’s next-token distribution is. A low value indicates that the model strongly prefers a continuation, but that continuation can still express a false claim. Semantic entropy broadens the analysis by grouping responses according to meaning, yet it remains a behavioral proxy whose relationship with correctness can shift across models, prompts, and datasets.
The authors formalize proxy failure with two propositions:
- Information limits discrimination. A score’s AUC is bounded by the mutual information between that score and correctness. When the score contains little information about truth, it cannot reliably rank correct answers above incorrect ones.