I'm reviewing a grant proposal. Someone I know said "just use an LLM, then read what it says, use what you like, send it in." That is many things but above all irresponsible, and here is why.
I read the grant and wrote my own evaluation. The grant is not exactly in my field of research, so there was quite some looking up of papers, techniques, and jargon in the process. What I wrote is my perspective (what I have been asked to provide) and makes sense to me only because of the leg work and time I put in.
Now suppose the LLM had written an evaluation, with the "correct prompting": "You are a senior neuroscientist and have been asked to evaluate this grant proposal ... write an evaluation of maximum so many words and in a neutral tone without emphatic or bombastic terms, using rigorous language".
No matter what the LLM would have generated, I could not have assessed whether it was correct or not, and hence, I wouldn't be able to send it in. Plus it wouldn't be my perspective.
Had I used an LLM, I would have learned nothing, and sent in what amounts to, possibly, gibberish, or brilliant commentary: I would not have been able to tell either way. That it'd be written in beautiful language and grammatically correct amounts to nothing, or worse than nothing: the appearance of my thought process without the substance.
Took me some hours, that weren't paid. In the face of plummeting salaries in academia, that is no longer OK. My response should have been to decline to review it, to contribute to pushing grant agencies (and scientific journals) to pay for our work, but alas, I am, somehow, an idealist fool. But I don't do that for for-profit journals; there I draw the line nowadays.
#academia #LLMs
