Post #366777

Good AI research should tell us something about life, or it should help people. I hate seeing research about automating what people do. It's not a good goal for science or society! I was recently reminded of this by a paper applying LLMs to math. This domain has many good questions: what do we mean when we say a person "solves math problems"? What are they actually doing? How is this like or not like what an LLM does? How might mathematicians benefit from this? Instead, we get papers that pit an LLM against a human on a math problems dataset. This is great for claiming "AI has superhuman math abilities now!", but it's debatable whether good answers in a test-taking environment have anything to do with logic, reasoning, or creative problem solving. Instead of exploring to what extent LLMs are "really intelligent" vs. "stochastic parrots" (and perhaps the same question for humans), it reduces everything down to a number, one that hides the deeper problem and seems far more definitive than it is. #ai #llm