Post #2920360

2026-05-17 16:28 UTC

The point is not exactly that an AI cannot in theory answer the same questions, but that whether or not there are methods of reasoning accessible to AI without humans first discovering those methods seems entirely unclear when we observe our own methods of reasoning in some edge cases. Importantly, it seems that if an AI is able to answer such questions, it would not be able to use the same methods to come to the answers, if they do not have inner experience. As a result, if they do not know how to navigate these questions already, they will require human intervention to do so. In contrast, a human, through introspection, can learn to navigate the questions correctly on their own, completely independently of any intervention from outside experience. This is not decrying AI as useless, but simply to point out that AI *may* have very real limitations, and *might not* replicate all aspects of human reasoning. This has always been the case, and personally, I have only become more sure that AI has fundamental theoretical/philosophical limitations as I have seen AI progress, but this is my own opinion, and I think proving this would require a paradigm shift along the lines of the Fregian program in the early 20th century. I suspect that different mathematicians use this faculty of their minds differently to solve problems, which leads to different levels of awareness of this problem. E.g. those mathematicians who are platonists have certainly been quietly asking themselves the question if platonism is true, does an AI have any direct access to abstract objects in the way that humans do? (3/n)

Replies (1)

@wikiemol@mathstodon.xyz 2026-05-17 16:29

But since these questions are 'embarrassing' in the current mathematical culture, they have not explored these questions publicly. So, all of this is to say that the 'crisis' mathematics faces in the future HEAVILY depends on whether or not it is actually the case that the human mind is a computer or not. If it isn't then there is a very simple solution that will almost certainly begin to happen naturally in the long term: we will simply stop relying on computationalism to do mathematics. We will begin emphasizing the aspects of our minds that are not computational. And we do not have much to worry about in the long term. However, we may still have much to worry about in the short term. To mitigate the short term damage, what we would then need to ask (and prove, somehow, either through mathematical logic, or through neuroscience/biology) is what parts of our minds might not be computational? To this end, I think the easiest place to look is the alignment problem. This problem is provably not computationally solvable, so, theoretically, even if our minds are computers, we still have a place to look here. As a result of this, I think this is our best bet in the short term. I think the following are not replicable by a computer because of the alignment problem: 1 Question generation 2 Definitions 3 Philosophical curiosity (4/n)

Open ##2920361