2026-07-27 23:25 UTC
Replies (2)
-
@conniptions@mastodon.social 2026-07-27 23:41
@lzg@mastodon.social I've heard people arguing this but I remain as yet unconvinced. Recently I came across this academic study showing why prompt injections can't be stopped: https://role-confusion.github.io/ - the point being that the attempt to make LLMs more than just predictive text, by separating prompt input into different category roles, are themselves so very weakly implemented they can be easily circumvented. Guardrails don't, in short. Obviously I'm missing something? But what?
-
@muddle@infosec.exchange 2026-07-28 00:56
@lzg@mastodon.social Refuting it depends on the person you're trying to convince, I think. Some people aren't going to be influenced by arguments about externalised costs, and they might even think that the costs are low enough that their own personal use of LLMs are low enough in the greater scheme of things that it's not worth considering those arguments. Among those are the people who think they can be "responsible" in their use. Some other people, though, aren't familiar with Clarke's third law (I think it was) and find the whole thing just "magical." They're not going to be swayed by the idea of externalised costs. They're also likely to believe the lies of the AI industry that it will solve all the problems it manifestly has, if given enough time and money. Those people are likely to fall for TESCREAList nonsense, in other words. So, not either/or, but both/and. You just have to tailor the approach.