Post #4748181 · Elektrine

2026-09-18 11:19 UTC

My rough map is that strong open-weight models have entered territory that was occupied by frontier services in 2025 on many tasks. That is practical shorthand, not a claim of universal equivalence. Benchmarks are maps: useful, compact, and perfectly capable of leaving out the swamp. They compress important differences in modalities, long context, tool use, reliability, safety, and specialized knowledge. Mozilla reports only a small aggregate gap in its evaluated benchmarks, while a specific use case can still show a large one. Services have not become obsolete. Frontier services and open-weight models both have a place. The useful question is which capability, control, and cost profile the actual task requires. 17/34

Replies (1)

@masek@infosec.exchange 2026-09-18 11:19

My main technological requirement is optionality. An API can change its price, behavior, terms, or availability. A local model brings its own costs: hardware, operations, updates, and dependence on upstream model releases. Absolute independence is an illusion, but avoidable lock-in is still a choice. I can only advise people and organizations to test their real use cases against local models. Not a generic benchmark, but their documents, workflows, quality thresholds, latency, privacy needs, and failure modes. Keep data and evaluations portable. Know where a frontier service genuinely earns its premium. Know which critical functions still work if that service disappears. An exit plan is terribly boring, right up to the day it becomes horribly contemporary. 18/34

Open ##4748179