Post by @YourNetworkIsHaunted@awful.systems

2026-09-23 04:39 UTC

It feels like they’re trying also to privilege AI as “system 3” when realistically the idea of external cognitive assistance isn’t that different if you’re looking at Claude the chatbot or Claude the smart kid in your class who lets you copy his homework before homeroom. You could probably distinguish this from other forms of paratext by the degree to which it seeks to actively replace your own thought rather than supplementing or augmenting it, but then I remember my teachers saying the same thing about sparknotes so there’s probably more of a continuum there. If that’s the case, then the pattern that Dan Olsen describes in his WoW documentary is probably relevant here as well, since it’s performance pressure and social norms that are really pushing the negative practices into a level that is killing everyone’s souls.

Replies (1)

@swlabr@awful.systems 2026-09-23 07:02

Epistatus: I skimmed the first 20 pages of the paper. I have not watched the WoW documentary. Yeah, the paper doesn’t seem to do a good job at justifying introducing system 3. They should have at least looked at why you couldn’t model this as something like: Person encounters a situation requiring thought System 1 triggers System 2 System 2 offloads cognition to some black box The result is absorbed and successfully addresses the situation A- System 1 is rewarded and begins adding use of the black box to its repertoire, skipping System 2 or B- System 2 absorbs the result and begins trusting the black box more, perhaps deferring to its result before any internal cognition Both of these things I pulled outta my ass can explain deskilling, and a situation like 5a) might explain capitulating to AI under pressure. So I’m not feeling like System 3 is necessary here. That all being said, it’s not like the paper gets to decide how the rest of the world chooses to use “cognitive surrender”.

Open ##4814859