Post #1688713

2026-04-01 11:52 UTC

@sfoskett @maxmortillaro Ive tried them over and over again and try all the whacky rigamarole skills and mcps and everything that gets thrown against the pipe. They can be useful as expensive rubber ducks, limited restructures that fall just outside the tedium of regex and keyboard shortcuts, boilerplate, and other tasks that are in-domain for statistical models of token sequences. I have done several "me first, then LLM" comparisons where I write the prompt text I would use to describe the feature, write my best version first, and then without showing it to the model, give it my honest best attempt to use the pre-written prompt starter and keep coaching it without giving away my implementation. Never once has the LLM ever managed to write a coherent feature, no matter how small and well defined it is. Even when I cheat by reminding it of some caveat that I realized during my own writing, it cannot take into account the existing code and find the smart path from here to there because that's not what it does: it generates text in waves of context, and so all it can do is lay down a new wave. I need to write those up and do a few more systematically and observed.

Replies (2)

@jonny@neuromatch.social 2026-04-01 12:00

@sfoskett @maxmortillaro Opus-4.6 fully kitted out with graph index and LSP tools and a set of tests I pre-wrote for it as a target could not get out of a local minimum where it wanted to solve a race condition where stochastic events needed to be consumed in the order they were produced rather than received by... Sleeping... And hoping that when it woke up it had received all the events. There was already an event store and an indexing system in place. I told it pretty much verbatim which existing methods to call and how, It just hit an input pattern that its training data saw as "sleep" and couldn't do anything else. It concocted an elaborate final report that presented that as the only possible way because of some invented property of the OS sockets. If I hadn't written mine before and already known how to do it I might have just shrugged and merged. That shrug and merge trash is just most programming now.

Open ##1688714
@sfoskett@techfieldday.net 2026-04-01 12:04

@jonny @maxmortillaro The insane thing is the output of LLM tools I’ve tried actually get worse with further revision instead of better. Ask it to produce a thing, examine and give feedback, and the revision comes back worse. Every iteration is impressive, but none is usable. I’m not a coder, but it’s just as bad with prose. It looks like it makes sense, but then doesn’t as you read it. And as you keep working through it, it makes less and less sense, until a credible editor just shitcans it.

Open ##1688716