Post #1737818

@j_s_j Maybe, not sure what Qwen being a few steps behind Opus means. I think we'll constantly get better at wringing a little bit more out of LLMs. But distillation always comes with some capability loss. I don't know if you distill Mythos if you just end up with Opus again. I think Anthropic said training costs for Mythos were 10 billion, even with distillation base models would be expensive. I tend to agree with the thinking that LLMs are mostly stuck, it's the harnesses that are advanced.