Post #4167286

Dario's post on open-weights models is intentionally misleading. With one hand, he proffers praise and admiration for the models. With the other, he tsk-tsks about the "danger" of biological attacks, demanding bans on chip sales and penalties for distillation. He's spooked. But also, to read this you would think that nobody had ever jailbroken Claude into performing criminal activity. Except...their own reports tell us that's not so. All models can be abused; no models can be secured. Why then should we trust any product we can't examine at all? I'm not saying open-weights models are unproblematic. I'm just saying this rationale for their danger doesn't pass a sniff test. Dario sees the economic reality in front of him and observes a shift to relying on open-weights models for some workflows. That's a potentially existential risk for his business (as China no doubt intended). The rest is backwards reasoning. https://www.anthropic.com/news/position-open-weights-models