Post #291738
2026-02-10 18:53 UTC
AI analysis continues to be junk. I wrote an issue with some notes on a tricky corner case for some code we'd just merged. Some user swept in soon after, wrote a ton of plausible sounding analysis, then immediately opened a PR. They claimed they only used AI to translate, not to code or write. They were obviously lying, because I came back this morning and did actual analysis. https://github.com/pallets/werkzeug/issues/3105
Replies (2)
-
@davidism@mas.to 2026-02-10 19:11
I tried applying my own knowledge of where to start looking in a prompt to Claude Code: "Consider how rule matching is done by looking at `Map.match` in @src/werkzeug/routing/map.py and rule definition in @src/werkzeug/routing/rules.py. Does `Rule.__eq__` adequately check for duplicate rules? How can it account for `strict_slashes` behavior?" Even with all that, it missed the true cause completely, in two fresh sessions. So you can't even rely on being an expert to successfully use AI.
-
@chrisjrn@social.coop 2026-02-10 19:00
@davidism why do people keep thinking that these things produce truth rather than merely plausible output?