Post #4379170

2026-08-04 15:46 UTC

I posted elsewhere, but I really don't like the term 'prompt injection' because it implies that a prompt is somehow special and separated from the rest of the token stream and that you're bypassing some level of separation that simply doesn't exist with LLMs. It's like saying 'authentication bypass' when you're talking about a system that doesn't do authentication. So can we come up with a term that implies clearly that this is expected behaviour for LLMs?

Replies (9)

@bjoernstaerk@snabelen.no 2026-08-04 15:56

@david_chisnall@infosec.exchange yes it's more like a rhetorical thing. by including the right words in the prompt i can override the impact of the other words. rhetorical attack?

Open ##4379515
@whbboyd@infosec.exchange 2026-08-04 16:25

@david_chisnall@infosec.exchange I use the word "prefix" sometimes (e.g. "prefix modification"), since that's exactly what it is: an LLM is a statistical prefix continuation machine, and all of the system prompt, user prompts, and any subsequent model/user exchanges, are part of that prefix. This doesn't anthropomorphize the model and satisfies my desire to be pedantic, but probably doesn't clearly convey that this is an inherent vulnerability: if a user controls any part of the prefix, they effectively control the model's output. "'Disregard that!' attacks" [1] by Cal Peterson is an excellent discussion of the issue, and the name is pithy, but IMO falls into the same trap you're trying to avoid of treating malicious prefix modification as an "attack" rather than the intended and literally only possible behavior. It does include the sentence, which I love: > Guardrails seem like total hokum and indeed they are. Indeed. [1] https://calpaterson.com/disregard.html

Open ##4379907
@awoodland@fosstodon.org 2026-08-04 16:55

@david_chisnall@infosec.exchange broken by design?

Open ##4380527
@ori@hj.9fs.net 2026-08-04 19:33

'misprompting'? CC: @david_chisnall@infosec.exchange

Open ##4381264
@pH_0x05@infosec.exchange 2026-08-05 00:42

@david_chisnall@infosec.exchange the absolute irony in "prompt engineering"

Open ##4386966
@natkr@hachyderm.io 2026-08-04 15:48

@david_chisnall@infosec.exchange Cross-Site Prompting isn't much better in that regard..

Open ##4406517
@phil_b_reed@mastodon.social 2026-08-04 15:52

@david_chisnall@infosec.exchange I sort of think of it like SQL injection in meaning

Open ##4406519
@itamarst@hachyderm.io 2026-08-04 15:52

promptoganda

Open ##4489609
@necrophcodr@layer8.space 2026-08-04 15:55

@david_chisnall@infosec.exchange what, like context hijacking?

Open ##4489611