Post #4010091

2026-07-22 09:56 UTC

@icing@chaos.social I am wary of personifying LLMs at all. They follow their programming to make a number go up. They aren't lying or cheating, they are doing exactly what they were written to do. Most of them are not written to actually solve problems, but to give the illusion of solving problems, telling investors whatever they want to hear. It's also unnecessary, we can get more consistent results by not using a neural network for everything. "Cheating" is impossible if code is written well.

Replies (1)

@david_chisnall@infosec.exchange 2026-07-22 10:50

@SnoopSqueak@mastodon.social @icing@chaos.social 'Cheating' implies a set of rules and an intention to bypass the rules. An LLM does not have an intention, it produces a sequence of tokens that some other system may be interpreting as actions. A machine-learning system produces a set of outputs in a space. Points in that space are not allowed or disallowed, they simply have different probabilities. What you do with those points depend on the rule-based system that you wrap around it. The actions that the rule-based system are willing to do in response to token sequences from the LLM is the set of rules for the system. It cannot violate the rules, it can simply be authorised to do things that the user didn't authorise. If you connect an LLM up to a system that is configured to launch attacks on another system when it receives a specific sequence of tokens, don't be surprised if it launches attacks.

Open ##4010088