Post #1802967

2026-04-08 17:48 UTC

@ngaylinn @pbloem and this is one of the earliest (?) papers on calibration in LLMs at the same time, calibration in humans has been the topic of many decades of study within judgment and decision-making research…. https://arxiv.org/pdf/2207.05221

Replies (1)

@ngaylinn@tech.lgbt 2026-04-09 11:18

@UlrikeHahn @pbloem Asking "do you?" upsets me. I'm not arguing that I'm smarter than an LLM, I'm questioning your point that "AI is operating in substantially different modes when it's hallucinating to when it's retrieving factual information." If I want a tool for retrieving and operating on facts, I would insist it have a trusted knowledge graph linked to sources! I'm biased, 'cuz I worked for years on Google's KG. But, seriously: I want a tool with verifiable results, and a trusted authority who's accountable for accuracy. I buy what you're saying about an "internal knowledge graph" and source tracing after the fact. I, too, have a fuzzy memory of where my ideas came from, and have to fact check myself sometimes! But these models are trained on fact, fiction, and falsehood, all blended together without judgment in the same "sorta KG." Unless the LLM is explicitly told to fact check itself, and which sources to trust, then it will occasionally make stuff up.

Open ##1802968