Post #2173579
2026-05-04 09:59 UTC
An example. There was a recent article proving that AI, if let to process its own outputs, cannot reach superintelligence. Quite the opposite. And that was proven mathematically (or so I understood, I'm not an actual smart person).
Has anyone put thoughts into how it applies on smaller scale, i.e. on agents being left more or less alone on software, with only design guidance from humans? Can it be seen as a proven fact that the code will turn into chaos? Or is it likely but who knows?
Replies (2)
-
@vadim@m.vrutkovs.eu 2026-05-04 10:50
@creepy_owlet have a look at user experience on Reddit fir the largest vibecoded and vibemaintained project as of today - OpenClaw. I think they just had a more or less stable version last week (and then obviously broke it)
-
@dogfox@kpop.social 2026-05-04 12:02
I read that paper. It was a good one! It has to do with *training* a model (of finite capacity) on it's own output. The simplest part of the argument is that it will loose the rarest data: It's less likely to come out, so it won't survive repeated rounds of training. Repeat ad infinitum until the data is just the dumbest shit. So an agent left on it's own on code won't necessarily do that since it is not re-training itself as it goes. @creepy_owlet