766
0
126
In reply to
6 earlier posts
So, like with Godwin’s law, the probability of a LLM being poisoned as it harvests enough data to become useful approaches 1.
0
0
0
I mean, if they didn’t piss in the pool, they’d have a lower chance of encountering piss. Godwin’s law is more benign and incidental. This is someone maliciously handing out extra Hitlers in a game of secret Hitler and then feeling shocked at the breakdown in the game
0
0
0
Yeah but they don’t have the money to introduce quality governance into this. So the brain trust of Reddit it is. Which explains why LLMs have gotten all weirdly socially combative too; like two neckbeards having at it with Google skill vs Google skill is a rich source of A+++ knowledge and social behaviour.
0
0
0
If I’m creating a corpus for an LLM to consume, I feel like I would probably create some data source quality score and drop anything that makes my model worse.
0
0
0
Then you have to create a framework for evaluating the effect of the addition of each source into “positive” or “negative”. Good luck with that. They can’t even map input objects in the training data to their actual source correctly or consistently.
It’s absolutely possible, but pretty much anything that adds more overhead per each individual input in the training data is going to be too costly for any of them to try and pursue.
O(n) isn’t bad, but when your n is as absurdly big as the training corpuses these things use, that has big effects. And there’s no telling if it would actually only be an O(n) cost.
It’s absolutely possible, but pretty much anything that adds more overhead per each individual input in the training data is going to be too costly for any of them to try and pursue.
O(n) isn’t bad, but when your n is as absurdly big as the training corpuses these things use, that has big effects. And there’s no telling if it would actually only be an O(n) cost.
0
0
0
Yeah, after reading a bit into it. It seems like most of the work is up front, pre filtering and classifying before it hits the model, to your point the model training part is expensive…
I think broadly though, the idea that they are just including the kitchen sink into the models without any consideration of source quality isn’t true
I think broadly though, the idea that they are just including the kitchen sink into the models without any consideration of source quality isn’t true
0
0
0
Loading comments...
About Community
Technology
!technology@lemmy.world
This is a most excellent place for technology news and articles.
Our Rules
Follow the lemmy.world rules.
Only tech related news or articles.
Be excellent to each other!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
Check for duplicates before posting, duplicates may be removed
Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
@L4s@lemmy.world
@autotldr@lemmings.world
@PipedLinkBot@feddit.rocks
@wikibot@lemmy.world
83979
Members
18845
Posts
Created:
June 11, 2023
View All Posts