← Feed
@tao@mathstodon.xyz
Post #2581820
2026-05-04 16:45 UTC
A few weeks ago, I recieved a detailed referee report on a lengthy paper my coauthors and I submitted, with many suggested corrections (both minor and major) spread out over many sections. We agreed to each claim some of the sections to revise based on the recommendations of the referee, and (because we were using Github as our version control system) were able to easily merge together all the revisions and have them done in a matter of days.
Today, I received a response from the referee who was largely satisfied with the changes, but had a dozen further minor corrections (mostly of the nature of typos or LaTeX labeling issues) that still needed to be addressed. This time around, I uploaded the report to the Claude Code agent, which was able to inspect the report, the LaTeX source, and the PDF version of the paper, identify the corrections, and propose unambiguous fixes for eleven of the twelve corrections (pointing out one typo on the referee's part in the process), while for the twelfth correction it offered two possible viable fixes. I then reviewed all proposed changes, made the selection for the twelfth fix, and asked Claude to implement them all; the entire process took less than fifteen minutes.
If I were to do the first round of revisions again, I think I would ask an agent to go through the report and identify all of the minor issues (at a typo level) that it could unambiguously fix, review and implement these fixes, and then isolate the more substantive comments that required human attention, before splitting the work amongst the co-authors as before.
Replies (4)
-
@tao@mathstodon.xyz What would you think about a person on the other end (the referee) if they they would've spent 15 min in total by using AI tool to review and identify possible issues with the paper?
Open ##2920392
-
@tao@mathstodon.xyz Have you tried implementing Karpathy's "LLM wiki" (really just instructions for organizing docs) in any of your coding agent projects?: https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f
Open ##2920393
-
@tao@mathstodon.xyz
I suspect that the referee has used claude code too, but who knows.
On that: I have learned from open source maintainers that some don't want AI generated pull requests anymore. They rather want feature requests or just the prompts so that can run the AI themselves.
I suspect that one human forwarding AI output to another human will be a short intermission until we talk directly about what an AI should do on a given document and let the author run the AI themselves.
Open ##2920394
-
@tao@mathstodon.xyz so plain big filter design?
Open ##2920396