Post #1286702

2026-04-16 23:13 UTC

There is a 20 GB file on my laptop that can just... do this? (see alt text on the first image) I tried a traditional OCR to compare: the result useful but worse than clank-o-vision. I attached the original image in case anyone wants to try their own hand (source: https://dailyportalz.jp/kiji/ji-ga-kitanai-otona) There are mistakes in the clanker's output. If I prompt it with my guess for a correction, it responds with a plausible summary of why the correction makes sense both visually and in context (third image). It doesn't "think" but it regresses its response from the full context of the image, its original output and the correction, which can yield useful synthesis of those three. A couple more asides: * The way these multimodal models work is super weird: they embed the image in the same vector space as text input so it sees it as a string of highly-dimensional "words". Honestly a neat hack that has no right to work as well as it does. * I showed it one of the new scans of the N-1 moon rocket photos (pic 4) and it identified it as an R-36 ICBM, which is *quite different*, so the model is still dumb as dirt and the "blurry JPEG of the internet" analogy still holds.

Replies (1)

@wren6991@types.pl 2026-04-16 23:15

If there is any place for LLMs in my work it's one that: * Is 100% locally hosted. * Does not place trust in the model (somewhat equivalently, solves a problem to which existing solutions are also fallible). * Does not allow model output to contaminate my codebase. I think this is possible for some niche uses: basically, fuzzy context-aware lint. As an example, I pushed in the float library source and the relevant docs. It found a bad register allocation in the f32 to i64 conversion (not yet tested). It also went down a dumb rabbit hole with the f32 to u64 conversion, and when prompted with "I don't think that's right" it generated a couple of plausible-looking execution traces with reference to the documentation for the new shift instructions, and concluded it was fine. Overall I'm not sure if that was a good use of my time or not. I would have hit the bad allocation as soon as I tested that routine, but it might have taken me a couple minutes to debug. I'm still not sure where I stand on being complicit in models' creation by using them locally. tl;dr normal clanker skepticism service will be resumed shortly

Open ##1286703