Post #2920359

2026-05-17 16:27 UTC

To at least attempt to convince you that the problem is worthy of investigation and is not 'trivial', we can consider the 'inquisitive' version of the Liar's paradox. Essentially "Are you going to answer no to this question?" No one is able to answer this question truthfully. However, through our own inner experience, we can observe our own volition, and find e.g. "I am going to answer no in the future" without saying it aloud. Then, we can reason based on this observation that the true answer is therefore yes, even though we cannot say it. Current AI will correctly answer that the question is paradoxical, but will begin to hallucinate when you ask follow up questions about what the actual answer was. One can apply a similar thought experiment to an inquisitive Knower paradox. "Are you going to know the answer to this question is no?" When we attempt the same observation of our volition, we can see directly (not through memory) that our minds go back and forth on the answer. Then, we can reason in a semi formal way that the answer must be no. And so the question is in some sense the opposite of the previous one: we can say the true answer aloud, but we cannot know the answer. (2/n)

Replies (1)

@wikiemol@mathstodon.xyz 2026-05-17 16:28

The point is not exactly that an AI cannot in theory answer the same questions, but that whether or not there are methods of reasoning accessible to AI without humans first discovering those methods seems entirely unclear when we observe our own methods of reasoning in some edge cases. Importantly, it seems that if an AI is able to answer such questions, it would not be able to use the same methods to come to the answers, if they do not have inner experience. As a result, if they do not know how to navigate these questions already, they will require human intervention to do so. In contrast, a human, through introspection, can learn to navigate the questions correctly on their own, completely independently of any intervention from outside experience. This is not decrying AI as useless, but simply to point out that AI *may* have very real limitations, and *might not* replicate all aspects of human reasoning. This has always been the case, and personally, I have only become more sure that AI has fundamental theoretical/philosophical limitations as I have seen AI progress, but this is my own opinion, and I think proving this would require a paradigm shift along the lines of the Fregian program in the early 20th century. I suspect that different mathematicians use this faculty of their minds differently to solve problems, which leads to different levels of awareness of this problem. E.g. those mathematicians who are platonists have certainly been quietly asking themselves the question if platonism is true, does an AI have any direct access to abstract objects in the way that humans do? (3/n)

Open ##2920360