Cross Validated
2025-03-20 02:41 UTC
By Sasha Queequeg
AI-113-20250320-social-media-7fc2a9b8
Why minimise Calibration Error rather than MSE? Context: LLM Hallucination [closed]
In the discussion of Large Language Model hallucination phenomenon, people are interested in measuring and reducing the calibration error of the model predictions. However, what makes this situation different from other predictive tasks where don't we just minimise some MSE -- if that LLM says something fairly correct then that gets a low error and if the LLM says something seriously wrong then it gets a high error? What does uncertainty quantification and reduction technique contribute for this case? Two references on calibration and hallucination follow. https://arxiv.org/abs/2207.05221 https://arxiv.org/abs/2311.14648 People in the community simply do this. I am asking why. Thank you!
In the discussion of Large Language Model hallucination phenomenon, people are interested in measuring and reducing the calibration error of the model predictions. However, what makes this situation different from other predictive tasks where don't we just minimise some MSE -- if that LLM says something fairly correct then that gets a low error and if the LLM says something seriously wrong then it gets a high error? What does uncertainty quantification and reduction technique contribute for this case? Two references on calibration and hallucination follow. https://arxiv.org/abs/2207.05221 https://arxiv.org/abs/2311.14648 People in the community simply do this. I am asking why. Thank you!
Full article content could not be extracted automatically. Read the original below.
Source:
Cross Validated
· stats.stackexchange.com