Post #1223853
2026-04-09 18:48 UTC
@AndrewRadev I had not seen this updated METR study and yeah that does sound fairly concerning.
It also matches my biases so I am going to have go give that one a long look later. 🤔
Replies (1)
-
@AndrewRadev@hachyderm.io 2026-04-09 18:51
@cthos It's not really a "study", exactly, just a blog post where they say these findings are unpublishable because of this issue. They also don't include any of the estimations from the developers about how faster/slower they *think* they were. That, to me, was the much more valuable part of the study. In this case, if they did try measuring that... > I found I am actually heavily biased sampling the issues … I avoid issues like AI can finish things in just 2 hours, but I have to spend 20 hours. This person literally thinks they are 1000% faster with AI. Given the results of the previous study, how close do you think that estimation would be to reality?