M28.5 CONNECT THE MECHANISM
Evaluate claims against evidence and recognize missing information
The support bot told a customer that Canada roaming was free. It wasn't. Learn to count invented claims, and to choose when a model should say "let me check" instead.
LESSON OVERVIEW14 min lesson
Lesson overview
The support bot told a customer that Canada roaming was free. It wasn't. Learn to count invented claims, and to choose when a model should say "let me check" instead.
What you’ll explore
- Factual reliability requires claim-level verification, calibrated uncertainty, and suitable abstention; plausible generation, citations, and model confidence are individually insufficient.
GO TO THE SOURCE
Original explanations, connected to the research.
On Faithfulness and Factuality in Abstractive Summarization (Maynez et al., 2020)FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation (Min et al., 2023)SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection (Manakul et al., 2023)Language Models (Mostly) Know What They Know (Kadavath et al., 2022)Selective Classification for Deep Neural NetworksTruthfulQASuggest a correction
A precise note can make an explanation better.
Choose the scene and describe what needs attention. Download a feedback file to share through a channel you already use. This page does not send feedback or connect you with a reviewer.