Post #4144709

2026-07-27 19:10 UTC

@DaveMWilburn@infosec.exchange I am confused why we're talking specifically about getting access to no-guardrail models when that's not what happened. Hugging Face wasn't enrolled in Anthropic's Cyber Verification Program, which is a far cry from "no guardrails." Whether the specific implementation of the CVP is effective, well, anecdotally it's still rather cautious. But the onboarding process is what it is. I don't see that as the direct issue here. If you're talking about no-guardrails models like the one OpenAI created and getting access to that, well, I agree that nobody should have access to them. I'd much rather we didn't build them at all. But here they are. And if states are interested, as they obviously are, then the solution to access should not be the domain of corporations. Fundamentally we need to decide if these things are weapons or can be used as weapons, or if they're not. And if they are, we need a system of laws and policies to handle them. Will that happen before major harm? I'm not optimistic.

Replies (1)

@DaveMWilburn@infosec.exchange 2026-07-27 19:29

@mttaggart@infosec.exchange So, correct me if I'm wrong, but Hugging Face ran into a brick wall early in their incident response process when they were blocked by one or more models from analyzing security log data due to guardrails. Their work around was to use a Chinese-sourced open weight model with fewer guardrail restrictions. Part of OpenAI's response to this debacle was to offer to enroll Hugging Face into their trusted access program, which is clearly marketed as providing at least infosec functionality, implied to include some functionality that Hugging Face struggled to access past guardrails. Some invite-only portion of OpenAI's trusted access program includes further loosening of guardrails to allow even greater use in the infosec domain, although it's not clear to me that Hugging Face is approved for that specifically. For me, the fundamental issue is that the set of potential GenAI model users with legitimate requirements for so-called dual-use functionality (e.g., common infosec use cases) is so large as to be unmanageable at scale.

Open ##4145053