Post #2248261

@giacomo @jneen@unstable.systems the size of the archive makes this unusable as a predictive model of the LLM's behavior though, at least for the kinds of predictions necessary to support use for building programs. To be useful as a tool in the way that a programming language is, humans would need to be able to build a mental model of its operation that could fit in our heads and allow us to predict what it would do. I can understand the math behind an LLM fairly well and can even make *some* kinds of predictions like "it will tend to fail in these ways." I might even be able to leverage this to exert some level of control over the system if all I want to make it do is fail in a particular way or email me the user's github tokens or something. But to control it precisely in the ways necessary to effectively program with it is harder-to-impossible. It might *seem* to be giving me what I want because (especially for simple programming tasks) it sometimes or even often gives me what I ask for. But that's an illusion of control as it's not dependable and when it fails I've got little recourse other than to abandon it for a different tool.