Post #1362028
2026-04-01 00:37 UTC
I don't work at one of those companies, just somewhere mainlining AI, so this answer might not satisfy your requirements. But the answer is very simple. The first thing anyone working in AI will tell you (maybe only internally?) is that the output is probabilistic not deterministic. By definition, that means it's not entirely consistent or reproducible, just... maybe close enough. I'm sure you already knew that though.
However, from my perspective, even if it was deterministic, it wouldn't make a substantial difference here.
For example, this file says I can't ask it to build a DoS script. Fine. But if I ask it to write a script that sends a request to a server, and then later I ask it to add a loop... I get a DoS script. It's a trivial hurdle at best, and doesn't even approach basic risk mitigation.
Replies (2)
-
@blakestacey@awful.systems 2026-04-01 02:42
> DoS script Part of me reads that and still thinks, "Oh, you mean like AUTOEXEC.BAT?"
-
@aio@awful.systems 2026-04-01 18:00
> the output is probabilistic not deterministic. By definition, that means it’s not entirely consistent or reproducible, just… maybe close enough. That isn't a barrier to making guarantees regarding the behavior of a program. The entire field of randomized algorithms is devoted to doing so. The problem is people willfully writing and deploying programs which they neither understand nor can control.