@highergeometer@mathstodon.xyz
Post #2525132
2026-05-11 08:43 UTC
@PSL2Z@mathstodon.xyz So a publishable study of failures of AI systems to solve mathematics problems is something in computer science, not a research paper in mathematics. And I'm not thinking "here's a benchmark suite collected by an AI company", but "I as a mathematician tested a range of things from my field I asked it to do, deliberately designed to test out the system, and here's my analysis of the failure modes" (or same, with a range of fields). And note that "prove this theorem/solve this problem" is only part of mathematics, and the other parts are so far not covered much in the discussion I've seen (come up with a new good definition, technique, etc)
Replies (1)
-
@PSL2Z@mathstodon.xyz 2026-05-11 08:50
@highergeometer@mathstodon.xyz People have little incentive to do this kind of work that is going to become obsolete in a few months if not weeks, likely before the paper (if it is submitted for publication) goes through the publication pipeline.