@UnrepentantAlgebra@lemmy.world
Post #1302704
2026-03-27 12:52 UTC
Replies (6)
-
@KairuByte@lemmy.dbzer0.com 2026-03-27 13:58
I assume it’s an hourly wage or something. Just because humans can work for free if they choose, doesn’t mean they have no cost associated with them. Just like a company could choose to give away unlimited tokens, those tokens still have a standard cost.
-
@FrankFrankson@lemmy.world 2026-03-27 13:48
That is how much individual testing humans cost when you buy them in bulk.
-
@aesopjah@sh.itjust.works 2026-03-27 15:15
it's also an odd metric since only 20-60% of the humans completed it. Very 60% of the time they complete it everytime energy. Ideally they'd run the bots multiple times through (with no context or training of previous run), but I guess that is cost prohibitive?
-
@mapleseedfall@lemmy.world 2026-03-27 13:27
Youd have to eat $250 worth of burgers to pass it.
-
@ExLisper@lemmy.curiana.net 2026-03-27 13:56
Because I ain't doing this shit for free.
-
@brianpeiris@lemmy.ca 2026-03-27 16:28
This is my rough upper-bound estimate based on the Technical Report. Human participants were paid to complete and evaluate the tasks at an average fixed fee of $128 plus $5 for solved tasks. So if a panel of humans were tasked with solving the 25 tasks in the public test set, it would be an average of $250 per person. Although, looking at it again, the costs listed for the LLMs is per task, so it would actually be more like $10 per human per task. In any case it's one or two orders of magnitude less than the LLMs. > Participants received a fixed participation fee of $115–$140 for completing the session, along with a $5 performance-based incentive for each environment successfully solved https://arcprize.org/media/ARC_AGI_3_Technical_Report.pdf