hoppolito

I may be misunderstanding your argument but just to make sure I want to point out that

desperate people will do desperate things to survive

does not run counter to

if you can’t afford to live, then you certainly can’t afford to move to another country

As far as I know that’s generally what is often done, but it’s a surprisingly hard problem to solve ‘completely’ for two reasons: The more obvious one - how do you define quality? When you’re working with the amount of data LLMs require as input and need to be checked for on output you’re going to have to automate these quality checks, and in one way or another it comes back around to some system having to define and judge against this score. There’s many different benchmarks out there nowadays, but it’s still virtually impossible to just have ‘a’ quality score for such a complex task. Perhaps the less obvious one - you generally don’t want to ‘overfit’ your model to whatever quality scoring system you set up. If you get too close to it, your model typically won’t be generally useful anymore, rather just always outputting things which exactly satisfy the scoring principle, nothing else. If it reaches a theoretical perfect score, it would just end up being a replication of the quality score itself.

I think you really nailed the crux of the matter. With the ‘autocomplete-like’ nature of current LLMs the issue is precisely that you can never be sure of any answer’s validity. Some approaches try by giving ‘sources’ next to it, but that doesn’t mean those sources’ findings actually match the text output and it’s not a given that the sources themselves are reputable - thus you’re back to perusing those to make sure anyway. If there was a meter of certainty next to the answers this would be much more meaningful for serious use-cases, but of course by design such a thing seems impossible to implement with the current approaches. I will say that in my personal (hobby) projects I have found a few good use cases of letting the models spit out some guesses, e.g. for the causes of a programming bug or proposing directions to research in, but I am just not sold that the heaviness of all the costs (cognitive, social, and of course environmental) is worth it for that alone.

Holyy, thanks for this. I can finally put a name to it. Was wondering with my partner for ages what sometimes suddenly befalls us, especially if we’re lying in a weird position.