Post #4263592

2026-07-31 00:33 UTC

@res260@infosec.exchange @ikesau@micro.ikesau.co So I guess my initial point of "no safety training at all" should be more "No more than what's already the standard". But even then, this training is trivial to remove. Which is more my whole point from the start, this RL training doesn't REALLY protect anyone. It's really just an anti-liability for the lab shipping it. 3/3

Replies (2)

@res260@infosec.exchange 2026-07-31 03:11

@kini@maro.xyz @ikesau@micro.ikesau.co It's not exact to say it's trivial to remove. How to do it is documented, if you have deep domain knowledge you can do it, it requires a lot of computing power/hardware to do. Like all things (as someone working in cybersecurity I know this first hand), imposing costs is how you reduce harm. Shipping only models that refuse to do dangerous tasks and making it illegal to host jailbroken models impose costs and will reduce harm done by those "general assistants". This does not mean no one will have access to jailbroken models, but it will mean that many, many fewer people will use those models

Open ##4263590
@ikesau@micro.ikesau.co 2026-07-31 02:56

@kini@maro.xyz @res260@infosec.exchange have either of you guys read the letter or plan A? I feel like you're having a conversation that's interesting but quite oblique to the biggest worries at hand (except for in the ways it relates to open source models, I suppose) I'm concerned about an ASI that's not aligned nor understandable to us; war between the US and China; US global dictatorship, etc. Based on the way you're talking, it seems like maybe you don't think those outcomes are credible or worth prioritising?

Open ##4263824