Post #4143419

2026-07-27 17:58 UTC

Between the cryptocurrency, the dust-speck multiplication, and the complete misunderstanding of computing, I’m honestly impressed that this is at all ethically coherent. But she does seem to care, even if her facts are all wrong. I suppose that it’s hard to avoid sneering Robin Hanson if one has any ethics at all. Top comment is my choice sneer: I think this post is directionally correct, extremely important, and also kind of waffling and unhinged (though I do get that some topics are inherently hard to be hinged about, and I appreciate the effort). I don’t think she’s using GPT in the comments. Quoting from her comment on one of her posts: The things I’m saying are roughly (1) slavery is bad, (2) if AI are sapient and being made to engage in labor without pay then it is probably slavery, and (3) since slavery is bad and this might be slavery, this is probably bad, and (4) no one seems to be acting like it is bad and (5) I’m confused about how this isn’t some sort of killshot on the general moral adequacy of our entire civilization right now. (2) has a big “if” in there, but otherwise yeah, makes sense. I think it rhymes with my post on this from last year: Nobody wants to admit that we only care whether robots aren’t human because we mistreat the non-humans in our society and want permission to mistreat robots as well. Bring this topic up amongst most beneficiaries of the current AI summer, or those addicted to chatting with a BERT, and you’ll get a faceful of apologies about capitalism and productivity; bring it up amongst skeptics or sneerers and you’ll be mocked for taking the field of AI with any sincerity or seriousness. I was confused too, but then I conceptualized capitalism and the sheer hatred lurking within human hearts. Humans are gleefully horrible towards each other. Our civilization isn’t morally adequate. I guess it is cool to learn that somebody addicted to ChatGPT can still perceive the issue; I was too cynical. I also put Bryson 2009, “Robots Should Be Slaves” on my reading list, which I surely will not regret.

Replies (4)

@scruiser@awful.systems 2026-07-27 23:16

(2) has a big “if” in there Its a super big if, but it is one lesswrong (and Anthropic, with their “model welfare” pseudoscience) is allegedly seriously considering as a possibility (since, you know, they think LLMs are already AGI and Eliezer was accusing AI-dungeon, you know, GPT-2, of deliberately scheming). If anything, it speaks to some possible hypocrisy/motivated reasoning that they consider LLMs AGI but not possibly morally relevant. conceptualized capitalism This seems to be a lesson most lesswrongers have had rubbed in their faces repeatedly over the past 5 years with everything about LLM company’s behavior, but they just don’t want to learn.

Open ##4149685
@swlabr@awful.systems 2026-07-28 05:03

Wait does LW even think slavery is bad

Open ##4155144
@YourNetworkIsHaunted@awful.systems 2026-07-28 04:08

I don’t think anyone here is likely to disagree that current LLM systems can’t coherently be said to have some kind of internal experience and the design and functioning of those systems seems to fundamentally preclude the kinds of conscious experience that we want to assign moral agency to. But the key to understanding this as a moral issue imo is that the people designing and building these things seem like they really want to create something that ought to merit that kind of moral consideration. For all that the humanizing language of agency, intelligence, welfare, etc. is in reality part of the grift, I think there’s a strong catch-22 to be argued that if it’s not part of the grift then slavery is pretty implicit in the value proposition of these companies.

Open ##4156468
@TheLazyHase@awful.systems 2026-07-28 07:54

These bozos A - actively want slaves and B - work on enslaving humans first. That thing being said, the notion of slavery and AI is a complicated one to me. The first reference point is a (non mistreated) sled dog. Is slavery an appropriate term for them ? On one hand, they are forced to work, are not paid, and their freedom is infringed. On the other, it’s not like it’s against their will, the life of a wild dog is meaningfully different but not better than a sleigh dog, and (most) sleigh dogs don’t actively try to escape and be free. And while dogs are not person, they are sentient being ; it’s very possible IF we ever succeed at making an AI it end on that intelligence level. The second reference point is a though experiment. Let’s say you have created a sentient AI, and you want to ethically employ that person. What is the appropriate compensation ? We give money and free time to humans. But that work because of human specific factors : they appreciate different ways of getting sustainance, and have goals inherent to them. But an artificial intelligence may have neither of that. Like while I like duck and some people prefer beef, but an AI by default don’t necessarily have preference in term of electricity, and may not be able to move from server to server. They also may not have anything to do outside of work ? Like, it would not be surprising that they would just get bored if given free time, and they may not have urges or hobbies outside of work. So if we ever get to that point - which isn’t a certainty to me, it’s certainly something harder than fusion to achieve - there will be a lot of anthropology (AI-logy ?) to determine they need, which may plausibly be “nothing outside of sustainance and having a job” We often conflate human-level intelligence with litterally being a human. But, any hypothetical AI is likely to not be a human at all, even if they have the same level of intelligence.

Open ##4158015