Post #1802964
2026-04-08 17:18 UTC
@ngaylinn
> They don't have a knowledge graph or relational database of facts.
Do you? If your answer is "well, encoded in my neurons somewhere" then the same is definitely true for LLMs.
For a fact like "the Eiffel tower is in Paris" we can pinpoint exactly where that fact lives. We can even manipulate it so that the fact changes.
https://arxiv.org/abs/2202.05262
This is true before finetuning by the way. Just pretraining gets you a model with an internal knowledge graph.
Replies (1)
-
@pbloem@sigmoid.social 2026-04-08 17:21
@ngaylinn It's true that they usually don't know "why they know" something (or "know" something) the way we (sometimes) do. It's hard to trace knowledge back to sources. But with RAG this can be done a posteriori. The frontier models with reasoning traces and RAG are pretty good at fact checking themselves.