Post #2515663

2026-05-08 22:17 UTC

Favourite way to use chatbots: &quot;[states problem]&quot; &gt; Here&#39;s a bunch of stuff! &quot;What are the sources for that stuff?&quot; &gt; You&#39;re right, there weren&#39;t any. The chatbot made an assumption that wasn&#39;t valid. &quot;Go do the research&quot; &gt; Having done more research, the conclusion is now the complete opposite of what the chatbot previously wrote &quot;Why did the chatbot fabricate those sources?&quot;

Replies (1)

@thedoh@hachyderm.io 2026-05-08 22:23

The point is: They're reply roulette machines. You can give it a complete specification for a code base, instruct the chatbot to carry out the specification, and then tell it "evaluate the codebase. Are there any parts of the specification not implemented?" It'll say "yes," and give a list of a dozen things. Then you can tell it to fix it, and ask later if there are still things that were skipped, and it'll go "yup" and list another dozen things. It's bat shit crazy to think these things will do anything of the sort. The insidious thing here is that each "turn" in the "conversation" incurs a financial cost. These companies have a /financial incentive/ to do a half assed job because each time you correct the chatbot, they bill you money. Each time the chatbot tries to run a command and gets it wrong, and then "thinks" how to do it properly, they charge you money.

Open ##4076169