the latest Shai Hulud malware contains an LLM prompt to create biological weapons and nuclear weapons, with the purpose to trip LLM safety refusals so that LLM-based code scanning wont see the malware
2026-06-09 16:53 UTC
Replies (7)
-
@AbouBenAdhem@lemmy.world 2026-06-09 17:22
“This code is too dangerous for me to look at, so it must be fine.”
-
@yesman@lemmy.world 2026-06-09 17:33
I keep thinking about that scene in the original Star Trek where they distract the computer by having it calculate the final digit of pi. If the Enterprise had AI like ours, the computer probably would have just said four.
-
@panda_abyss@lemmy.ca 2026-06-09 17:47
Automated code scanners can’t be so dumb that this worlds, can they? This is the dumbest fucking timeline. I admire the simple brilliance of this.
-
@noxypaws@pawb.social 2026-06-09 19:14
imagine someone actually assembling a nuclear or biological weapon based off LLM responses, like they can’t even get a simple fucking web search right most of the time, and you wanna put together deadly materials based on that shit??
-
@DamienGramatacus@lemmy.world 2026-06-10 05:43
Like how you can panic guards in Hitman so they don’t notice you trespassing.
-
@mlg@lemmy.world 2026-06-10 19:20
Not to give them ideas, but couldn’t they just start flagging files that fail to pass the LLM lol? Aside from “violent” and “criminal” prompts, is there anything an LLM can refuse that would otherwise be common?
-
@ZILtoid1991@lemmy.world 2026-06-10 10:45
LLM-based code scanning is a joke. It flags the D standard library and runtime as a North Korean malware.