Elektrine lite

← Feed

@jonny@neuromatch.social

Post #4213254

2026-07-29 20:43 UTC

I had never top to bottom vibe coded something before - like full on "do not look at the code, just keep rolling the dice" - and I was sort of confused about how people in forums would say they were "working for months" on vibe coding something. like what does that look like, what is the work like, what takes time, etc. So I have been trying it in the background when I take breaks from normal work, and while I have had lots of exposure to and used LLMs a lot before in existing projects, but this is the most profoundly frustrating and mind blowingly bad LLM experiment I have ever done and I am coming away with even less understanding of how people can think this rocks. I had an old idea for a game I never followed through on, and the game is extremely simple (and actually pretty fun, unlike my other games): mechanical core is basically the game board is a graph, and moves consist of hops between the nodes. The LLM was able to get a playable prototype running in like an hour, and I was like "whoa maybe this is good!" But none of the actual game part worked - there are invalid moves, but those were allowed. There were valid moves that were blocked, the game board was wrong, etc. OK, no problem, I can just refine from here. Just work through it and keep prompting until its done. The moving target is unbelievable - every single change would bring with it new, unpredictable bugs. Since I hadn't been looking at the code, the LLM had made dozens of assumptions and hardcoded rules I never asked for and made no sense, and was re-reading those implicitly on every command so that internally it was fighting against me, trapped in a prison of its own wrongness. Initially it handrolled its own graph layout algo which was terrible. I intervened and said "just use d3-force" since what I need for the game is at the level of something you could literally copy/paste from the docs examples. It did that, but the layout was still inexplicably bad - successive chains of nodes would hop back and forth across the whole board despite being connected to nothing else. Monday morning I started with prompting for a fix for the "chains of nodes always crisscross the board" problem. I checked in every few hours between work to find some new elaborate explanation of the problem, a few thousand more lines written, and claims it was fixed, and the problem being even worse somehow. Finally at the end of the day I got frustrated and broke the rule of the experiment and went and looked at the code - how could this be so hard, this should be a couple dozen lines. In about 30 seconds of skimming, I found an elaborately commented function whose entire purpose was to guarantee that connected nodes were placed on opposite sides of the board. What??? There is absolutely nothing about what I wrote or could be implied from the structure of the game that would have suggested this as a thing to do. The nodes in the graph correspond to words, and the function specifically made sure that words that started with an even numbered ascii character were on one side and odds were on another. this was allegedly to "help with spacing between the nodes in layout." It had been churning for a full day and generated thousands of lines of code to counteract a function that should not exist and makes no sense but was nonetheless clearly labeled as doing the exact thing I was telling it was a bug and should be fixed. if you told even the most inexperienced programmer to "fix the do x bug, the program should not do x," and in 30 seconds of skimming they found def always_do_x(), their first instinct would probably be to remove it. I looked at its tool use log and it had ingested this whole file, and this specific function dozens of times. The graph layout code was now like 3000 lines of spaghetti trying to counteract the function. I removed it and the problem immediately disappeared. Every single thing the LLM had said, elaborate, jargon heavy, math sounding excuses was 100% wrong, not even close. The LLM dutifully made a memory about "always read the code to see if there is a function for doing the thing it is told not to do." The gaslighting is mind blowing. Later, I told it to adjust the forces on the layout to make clusters clump up more and to spread out low degree nodes. This is an absolutely base level thing that force directed graph layouts do. This was fiddling with two or three constants. The LLM invented entire graph theory concepts and metrics to tell me that it was provably not possible to have less space around hubs and more space between them. It showed me series of fake fucking equations calculating some made up phrase like "unit density" in graph neighborhoods. IT TURNS OUT that it had just been redefining the force direction simulation in a LOOP such that for each node added, the ENTIRE GRAPH was excluded from the force direction calculation - only the newly connected node had any kind of layout forces applied and the rest were static. I removed that and it worked perfectly - as I knew it does, because I have done this a thousand times before. I have never been more convinced that the driving force here is gambling addiction. I did indeed feel the addictive pull of just pulling the bar over and over and something happening. The code jittered, window refreshed... Dang not this time. Roll again. And when it finally sort of worked, here is the reward, this is amazing. I think that this game is good enough that ive been going in and trying to clean it up and make it actually playable, and credit where credit is due, it did program it so badly that I am now working on it out of anger in my free time rather than letting it languish. Anyway, I wasn't planning on writing a blog post about this, but after its done and ive cleaned it up from the slop I will break out the raw message logs and write it up because this has been a completely perplexing look into the present and future of what people are telling me programming will work like forever now. edit since it's been asked twice: claude code with opus 5, and yes i've tried the plugins and the skills and all the whoopsiedoozles that are supposed to make it good and not bad.

Replies (6)

  • @ra6bit@infosec.exchange 2026-07-29 21:19

    @jonny@neuromatch.social Every conversation I have had with engineers about how they iterate this process has moved toward "I engineer prompts that eliminate as much ambiguity as possible, and it's frustrating because the limiting factor is often English itself." And I keep thinking, if only we had some sort of less ambiguous symbolic instruction code with which to model what we want the computer to do..

    Open ##4222410

  • @gjm@mathstodon.xyz 2026-07-29 20:59

    @jonny@neuromatch.social Ouch! Which LLM, and were you using it in something like Codex / Claude Code / Antigravity or doing something else? (For the avoidance of doubt, I'm not asking in order to say "aha, well, you should have used X instead, dumbass" or anything like that. For all I know every model may be equally terrible in this way. I've no vibecoding experience myself. But knowing a specific thing that _definitely_ misbehaves seems like it might be useful.)

    Open ##4227659

  • @foxylad@mastodon.nz 2026-07-30 01:48

    @jonny@neuromatch.social Interesting take on the gambling compulsion, which harmonises with my instinct that the randomness at the heart of LLMs is at odds with the determinism that is at the heart of good coding.

    Open ##4227815

  • @bonkers@nerdculture.de 2026-07-29 21:00

    @jonny@neuromatch.social which LLM engine was that? I vibecoded a few hobby projects in Claude, and the experience was much less frustrating than what you described. But yes, it kept forgetting things until I asked it to update agents.md, and there were a few moments, like, here's a set of methods that will implement what you want, but currently they're empty. So I had to ask the clanker explicitly to actually implement them. It's also inconsistent in structuring the code: sometimes it splits the code in files and components nicely, sometimes it just dumps everything in one file. Often it forgets to update the documentation until I ask it explicitly. So, it's really a junior on crack. The clanker is able to spew tons of code, but you need to actually check if it's worth anything.

    Open ##4278284

  • @happyborg@fosstodon.org 2026-07-29 21:13

    @jonny@neuromatch.social :rofl: @jonny@neuromatch.social you are truly taking one for the rest of us. Reading made me think of some of my coding stupids (with d3) - but also of my suggestion to you to look at a project I know well which claims to be doing this well. I can't quite let go and keep dropping by to watch the Discord and wonder wtf people still hang out there, in the vein hope this is it. And me too, if mainly lurking after being a mainstay. Ick.

    Open ##4278286

  • @jonny@neuromatch.social what worked in my own experiment was giving the harness a stack of explorations with defined success criteria what i got was a microkernel that met defined success criteria but had poor design. in fact, at one point, its refusal to code proper data structures forced me to interrupt it and give it an immediate task of writing a real red-black tree for my own sanity. i didn't like the experience. it was amusing, but not any way i would ever want to write real software.

    Open ##4278288