Post #3634534
2026-07-06 18:37 UTC
@phil@fed.bajsicki.com @munin@infosec.exchange
An LLM doesn’t actually do anything, except copy-blend-paste from the training set based on the “weights” of words in your prompt.
Exhibit A: promting it to compute an md5. Your freaking washing machine can calculate an MD5 hash (it is probably running some form of Linux). Side note: the response says “by hand”. Anthropomorphise much? 🤦♂️
Exhibit B: prompting it to give the fraction notation of a calculation I found somewhere (I was trying to help the stochastic plastic parrot, guessing it would be in the training data).
The result (read: regurgitated plausible bullshit), presented without any uncertainty, is clearly from some LaTeX source. Not that it is correct LaTeX, nooooo, it only looks that way. And it remained completely wrong crap, never being able to copy-blend-paste with an actual fraction notation, until I ran out of prompts trying (I ain’t paying for this bullshit).
TLDR: LLMs output ONLY plausible bullshit, ALL the time, and you get to decide if you think it is good enough for what your paying. h/t Soulwax https://youtu.be/EF3bu9fmheA
Replies (1)
-
@munin@infosec.exchange 2026-07-06 18:39
@avuko@infosec.exchange @phil@fed.bajsicki.com hey so I full well know exactly how this works and really don't need an explanation for the thing I'm writing policy about for work.