Post #1802963

2026-04-08 14:47 UTC

@pbloem I'll check out that paper when I get the chance. Sounds interesting. But, still, these models don't model uncertainty, right? They don't know what they know or how they know it. They don't have any notion of authoritative sources, or even where the text they're reproducing cones from. They don't have a knowledge graph or relational database of facts. They don't have any notion of logical correctness, except for correct examples in their training corpus and "tools" if those are provided / used. Right? That's my disconnect here. Yes, we're doing more elaborate training to reduce the error rate. But I think what we call "hallucination" is just a way of describing the fundamental operation that LLMs do (without necessarily implying anything about correctness). New techniques constrain and reinforce that hallucination to make it less error prone, but... it's still hallucinating, all the time. There is no "factual mode" that I know of. Right?

Replies (2)

@pbloem@sigmoid.social 2026-04-08 17:18

@ngaylinn > They don't have a knowledge graph or relational database of facts. Do you? If your answer is "well, encoded in my neurons somewhere" then the same is definitely true for LLMs. For a fact like "the Eiffel tower is in Paris" we can pinpoint exactly where that fact lives. We can even manipulate it so that the fact changes. https://arxiv.org/abs/2202.05262 This is true before finetuning by the way. Just pretraining gets you a model with an internal knowledge graph.

Open ##1802964
@UlrikeHahn@fediscience.org 2026-04-08 17:40

@ngaylinn @pbloem I haven’t been following this literature of late and it depends on what you want to call a “mode” but there is a literature on the signal distinguishability of hallucinated versus non-hallucinated states …my crude take away, last time I looked, is that there was uncertainty information there if one wanted to leverage it…. https://aclanthology.org/2023.findings-emnlp.68.pdf

Open ##1802966