Post #2804224

2026-05-23 23:58 UTC

I think the very design of LLMs makes them not very useful even when used “correctly”. They are basically machines that are very good at sounding plausible. But they have absolutely 0 way of checking if info is correct or not. It’s just whatever is reinforced the most in their model is treated as correct. I think the underlying technology likely has uses. But the way it is currently being produced into products is something that, even if you tried to use it correctly, would simply end up tricking you with some plausible bullshit. Maybe you tell it to edit a paper and it decides to “fix” one of your opinions to be the most common one. Maybe you tell it to tell you the nuance behind a historical fact and it makes up a very likely sounding story that is entirely bullshit which you then repeat to someone else without realizing. The ability it has to sound plausible is its biggest flaw. Because an LLM will VERY rarely if ever say the words, “I don’t know.” You’d basically have to have gone in and coded it to respond to that specific question to respond with “I don’t know”. Otherwise it’ll just make something up.

Replies (2)

@Melobol@lemmy.ml 2026-05-24 00:40

The problem with the current technology, that the future technology will never be worse than what we have freely available. If you check the last 3 years of development where it was and where it is now - it is worrisome and awesome at the same time. And human slop isn’t better than AI slop. If you put the same effort in a human product (literally 2 sentence to prompt the AI for results) - the outcome is predictable. I believe the expression is: shit in - shit out. So if you are able to spend the time to write an article, ask the ai to proofread it and telling it to show you what it needs to change (you can even adk for ourside links and can verify one by one) - then you will be able manually edit your own document - without any AI slop dripping in. Is it extra work? Nope - if you get a human proofreading done - you still have to do these changes yourself. And you will be responsible for what you do with the information. The thing is that we expect AI to do those last steps and making editoral decisions - is our mistake and laziness. That’s the human slop.

Open ##2804223
@yogthos@lemmy.ml 2026-05-24 15:16

You could’ve said the exact same thing about the internet in the early 2000s when the bubble was at its peak and you had a bunch of companies trying to make products that made no sense. But when the hype died down, and the bubble popped, we got a lot of very useful tech out of it. The situation with LLMs is exactly the same. While LLMs are stochastic in nature, that doesn’t in any way make them useless. There are plenty of scenarios where they work extremely well. For example, just last week, I wanted to figure out how to decode RAW files from my camera. I have a Nikon, and it uses NEF format which is proprietary and has no open source decoder right now. I threw an LLM at decompiling a binary and tracing it in memory as it was doing the decoding. After a few days, the LLM managed to write the code that decodes the images. This is absolutely not something I would’ve been able to do on my own. And the fact that there aren’t any open source drivers yet, shows that it’s a very difficult task to accomplish. That’s just one real world example.

Open ##2811530