← Feed
@marjolica@social.linux.pizza
Post #3254180
2026-06-07 09:25 UTC
@CppGuy@infosec.space @orionkidder@writing.exchange @grammargirl@zirk.us the error rate also also depends on what is in the training data.
That's no doubt is part of the problem with Grok, as it's training data contains many unreliable statements garnered from X as well as deliberate falsifications added.
If the training data was just Wikipedia you would get more reliable results.
For other AI vendors adding random chats from Facebook or Instagram or AI generated websites will also lower the accuracy.
Claude Code may be slightly better, for now, because it is just plagiarising code. This won't last as the code repositories fill up with AI slop and these are flagged up as such and excluded by the crawlers.
Likewise if you ask GenAI to summarise a document it may well incorporate data from its training data as well as the text you supply.
The other reason is that GenAI just doesn't simply reproduce single sources, whatever their accuracy. It acts as a stochastic mixer: if you see a AI generated legal case reference some of it may come from one citation and some from another and the legal inference drawn maybe from something entirely different.
Likewise if you ask GenAI to summarise a document it may well incorporate words from its training data as well as the text you supply.
Replies (2)
-
@marjolica@social.linux.pizza @orionkidder@writing.exchange @grammargirl@zirk.us
I can't comment on Grok: I've never had an X account.
Claude Code has its problems. I use it not to generate code but to explore ways of working with unfamiliar libraries and languages when I can't find answers on the Web. (A library is a body of code packaged up by one developer or organisation for others to use.) I find it's wrong more often than it's right.
As an experiment, I once used Jira's AI to summarise a detailed comment that I'd written myself. The result was shorter, sure, but it was meaningless and unusable. After that experience, I never use an AI to summarise or rewrite anything.
Open ##3254181
-
@marjolica@social.linux.pizza FWIW, I kind of suspect that a big reaason for why the Youtube recommender has been so problematic for years now is, it treats its output as its input. Autoplay amplifies this effect.
@CppGuy@infosec.space @orionkidder@writing.exchange @grammargirl@zirk.us
Open ##3280433