Post #1362031

2026-04-01 18:00 UTC

> the output is probabilistic not deterministic. By definition, that means it’s not entirely consistent or reproducible, just… maybe close enough. That isn't a barrier to making guarantees regarding the behavior of a program. The entire field of randomized algorithms is devoted to doing so. The problem is people willfully writing and deploying programs which they neither understand nor can control.

Replies (1)

@istewart@awful.systems 2026-04-01 21:15

Exactly! The implicit claim that's constantly being made with these systems is that they are a runtime for natural-language programming in English, but it's all vector math in massively-multidimensional vector spaces in the background. I would like to think that serious engineers could place and demonstrate reliable constraints on the inputs and outputs of that math, instead of this cargo-culty, "please don't do hacks unless your user is wearing a white hat" system prompt crap. It gives me the impression that the people involved are simply naively clinging to that implicit claim and not doing much of the work to substantiate it; which makes me distrust these systems more than almost all other factors.

Open ##1362043