Post #1490103

@ngaylinn@tech.lgbt What's the struggle exactly? "Don't use LLMs in science" is the only stance one can reasonably take under current circumstances unless one is OK with eating everyone's seed corn. Any use of it degrades what science is and has immoral and unethical side effects. The only rationales I've ever seen offered for using LLMs in science boil down to responses to self-destructive, warped practices in places like the US today. These practices are not permanent, though. They will change, and then what? Our seed corn is gone. I found the article you linked poorly argued and full of alarmingly shallow framings. Here are a few bits and pieces I picked up, for what it's worth (warning: it's too long!): The models are already powerful enough to produce publishable results under competent supervision. That's not the bottleneck. The bottleneck is the supervision. Besides being false and circular, this is a poor framing that attempts to carve out one piece of a complex system when it is irrational to try to do that. This author does not seem particularly incisive about what exactly science, and artifacts like publications, are for. He seems close to getting it at times, commenting on the loss of competence that results, but then somehow confesses he's doing the very thing that'd lead to that outcome. Ban-and-punish violates academic freedom No it doesn't? I haven't looked but I bet at least one person argued that banning pipe smoking inside of academic buildings violated academic freedom too. Murder and pedophilia are also banned-and-punished (some horrible people try to argue the latter is a violation of academic freedom, even today). It almost sounds like he's using "freedom" in the right-wing/libertarian sense of having as few constraints on behavior as possible. is unenforceable Yes it is? Nobody smokes a pipe inside of university buildings anymore. Current professors face all sorts of punishments and censures, new ones yearly it seems, for a variety of things ranging from not using Canvas properly to not reporting expenses properly to not filling out this or that form exactly the right way. They are forbidden from doing all sorts of things. All of this is enforceable, and is enforced, sometimes brutally. The above statement sounds childish and self-serving in light of reality. It also has a libertarian flavor. asks early-career scientists to compete with one hand tied behind their backs while tenured faculty quietly use Claude in their home offices ...which tenured faculty won't do if such policies exist and are enforced? This is just not a serious argument. "Compete" is a neoliberal economics term that really has no place in a communitarian endeavor like science. More right-wing/libertarian flavor. I use AI agents regularly Eesh, he's already melted his brain with the brain melting machine ( https://arxiv.org/abs/2604.04721 ). Too bad, but this partly explains the superficial reasoning: this article is a confession, among other things, and operates in an emotional register. When I see junior PhD students entering the field now, I see something different. I see students who reach for the agent before they reach for the textbook. So many articles like this scoldingly blame students for the failures of their seniors. He's confessing to a self-fulfilling prophecy: he uses software agents because he thinks he's somehow immune to their harms (he's not), and in the same breath decries that students use them too and this is very bad. Seniors should model the behavior they want to see represented in their disciplines, not scold the youngsters. He's telling on himself that he doesn't consider himself a student or lifelong learner. Yet if he is not doing fundamentally boring research, he's in the same position intellectually as a newcomer to the field and will suffer from LLM use in the same ways they do, just with a different set of preconditions. Using an LLM as a sounding board: fine. No it's not? Science is a group collaboration, and speaking with a colleague as a sounding board advances the field for both people. Spitting ideas into a chat window and responding to whatever it spits back separates the individual from their colleagues and contemporaries, robbing themselves, their colleagues, and their field of enriching experiences and foreclosing on the development of good communication habits. I'm not arguing that LLMs should be banned from research. That would be stupid No it wouldn't? Why would it be stupid to remove the accelerant from the campfire before we know whether accelerating the fire would send it out of control and burn down the campground? Schwartz can use Claude to write a paper because Schwartz already knows the physics. His decades of experience are the immune system that catches Claude's hallucinations. A first-year student using the same tool, on the same problem, with the same supervisor giving the same feedback, produces the same output with none of the understanding. Hmm, I think this points at where his logic/framing breaks. You can't ever separate the output from the understanding. Those are fully interwoven. The paper is evidence of the understanding that was earned through the process of producing the paper. A paper-shaped blob of text that superficially resembles that evidence cannot be that evidence. The tell here is that he thinks there's a difference between senior and student, but there is not unless you assume the senior is doing fundamentally boring work. And here is where I have to be fair to Bob, because Bob isn't stupid. Bob is responding rationally to the incentives he's been given. "Responding rationally to incentives" is the language of neoliberal economics, which is an important factor in the dysfunctional practices that have led us here. Given the insanity of the times, and the co-option of the word "rational", being irrational is the more reasonable way forward. Anyway, it doesn't make sense for me to keep beating up this person's thing. I hope you and your labmates dig deeper than this in your explorations!