Post #3961458

2026-07-20 14:09 UTC

Important take-away from HuggingFace: you cannot trust closed weights providers for incident response. They used GLM 5.2 on HuggingFace's own infra to circumvent the provider’s guardrails they were hitting. Also ensured that no attacker data or credentials were leaving their environment. https://huggingface.co/blog/security-incident-july-2026 This is consistent with malware patterns that inject nuclear weapons or biological threats-related text to evade automatic reversing with guardrailed LLMs https://socket.dev/blog/mini-shai-hulud-miasma-and-hades-worms-target-bioinformatics-and-mcp-developers-via-malicious

Replies (2)

@flyingpenguin@infosec.exchange 2026-07-20 14:14

@fj@mastodon.social yeah, although I read it more like they had been ordering delivery pizzas and then when explosive diarrhea hit they called the pizza place and said "help our toilets aren't working". i'd hang up the phone on them too. incident response is supposed to have incident response tools setup and tested, dry runs, etc. and closed stuff /could/ work in theory. but the idea of dumping internal sensitive data and pounding the easy button... that's worse than just the generic tool being a bad fit.

Open ##3961518
@aristot73@infosec.exchange 2026-07-20 14:49

@fj@mastodon.social see also the conclusion from @nielsprovos@ioc.exchange post https://www.provos.org/p/case-for-open-weight-models/ "Frontier models have a place. They are tools for leverage, evaluation, and exploring the edge of what is possible. The mistake is letting them become the invisible policy engine inside a production system, where their price, their values, their refusals, and their availability are set by someone else. Use them from outside the boundary. Keep the model you depend on auditable, forkable, and yours."

Open ##3962275