2026-09-20 23:39 UTC
Replies (40)
-
@e8d79@discuss.tchncs.de 2026-09-21 00:06
I just opened the mastodon feed of my least favourite KDE dev again who is now pushing for an oh-so-reasonable LLM policy, and it is worse than I remembered. Crying about Zig CoC being mean to sloperators like himself. Befuddlement on why people are calling the stochastic parrot a stochastic parrot. Being very scared about autonomous AI. and of course the whole defending of hyperland and ladybird that was already discussed here some time ago.
-
@BlueMonday1984@awful.systems 2026-09-21 09:39
Mathias Schäfer has penned a follow-up to Baldur’s lament on webdev, bringing a degree of hope to the convo on webdev’s future.
-
@gerikson@awful.systems 2026-09-21 11:51
Followup to Trump’s Extremely Superior renaming of “AI” (awful.systems/post/9709966/12701983) RawStory: Trump scrambles to fix ‘supreme’ poll as MAGA melts down over ungodly comparison
-
@Evinceo@awful.systems 2026-09-21 12:52
With Better Offline and the NYT simultaneously covering Rationalists it seems like maybe we’re getting more mainstream sneering. A bunch of people (including bsky) are finding out about all the weird lore.
-
@lagrangeinterpolator@awful.systems 2026-09-21 13:44
Cory Doctorow has a reasonably sane take on all the LLM cybersecurity attacks. At this point, it’s not really an LLM but more of a Rube Goldberg machine with an LLM bolted on. pluralistic.net/2026/09/12/god-in-the-box/ He makes a good point that every single one of the scary “emergent” behaviors that everyone is pissing their pants about all have precedents in the training data of CTF hacking competitions. He is not sure why everyone is so spooked by the “coordination” between agents on random message boards. The chatbot might look in its training data and find instances in which teams broke out of the containment set by the game-masters, for example, by finding random insecure message boards on the internet to pass messages to one another. This is a time-honored internet tradition! The first time I ever heard about someone doing this was in the 2000s, when Mitch Wagner – then the editor of Information Week – discovered some teenaged girls using the comment section of one of his old blog-posts to evade the school firewall’s blockade of chat tools. When ChatGPT’s chatbots deployed this tactic, they weren’t “setting their own goals” or displaying worrying initiative. They were rolling out a tactic that has been understood by American middle-schoolers for about two decades. As usual, the real story is OpenAI dedicated tons of resources, using who knows how many very expensive GPUs for weeks, to run hacking tools to find exploits in a completely irresponsible manner. Their logging was so nonexistent that it was several days before they even realized that they hacked Huggingface. They should be thrown in prison for this, because anyone else would be if they committed a felony. But it has nothing to do with the super scary AI becoming misaligned and developing emergent behaviors.
-
@nfultz@awful.systems 2026-09-21 16:11
UC Regent / Podcaster Scott Galloway on CNN: Yes, we shouldn’t be scared of AI. We should be scared of reckless people with mal intent who use AI. In 2001, a woman named Sharon Whipple was mauled by two dogs. And ultimately the owners of the dogs were put in – put in prison. We’re now prosecuting the parents of kids who find unsecured AR15s and use those to kill others. I think it’s the incentive. There’s always a wizard behind the curtain here, and I think we have to create incentives, specifically disincentives, and make it clear that, OK, while these bots are in fact coordinating and they show incredible resourcefulness and high IQ, somebody program the incentives here. So, moving to solution, I think part of it is you have to have incentives, specifically disincentives that say you, the company, and maybe individually are responsible for the bad outcomes here. I just get this sense if you tell these companies their IPOs might be threatened or they risk criminal prosecution, they’re going to figure this out. We figured out a way to control atomic weapons. I don’t see any reason why we shouldn’t be able to control this. And the notion that they’re sentient and it’s out of my control is an attempt to absolve themselves of the guilt and the responsibility that they should bear here. The Whipple case was one of the more insane / divisive SF local crime cases, (TrueAnon ep if you aren’t familiar - www.youtube.com/watch?v=iMd0mPAqR8A ) - so that’s an odd and interesting pull for Galloway on national news. I don’t think repurposing the logic of the NRA “guns don’t kill people, people kill people” is the play though, and I think the incentive framing is too watered down. This isn’t a problem we can nudge our way out of.
-
@scruiser@awful.systems 2026-09-21 22:51
This has been talked about in the reddit sneerclub, but not yet on awful systems (as far as I could tell with a quick search on this and last week’s stubstack)… Caroline Ellison (the convicted FTX fraudster) got hired for an EA position dealing with financial allocations! Reddit sneerclub discussion EA Forums Discussion surprisingly (by the low low bar we have for EA) this announcement was heavily downvoted and all the top comments clearly and frankly denounce Manifund for hiring Ellison, accurately calling out the sheer violation of common sense and the obviously unethical nature of hiring someone because they previously directed money your way that turned out to be stolen. (The Manifund person outright admits this as a motivation!) Not so surprisingly, less upvoted, but still upvoted comments do some hand-wringing about the optics (as opposed to the clear ethical and common sense issues) and some even outright call for forgiveness for Caroline! The lesswrong discussion is less surprising (as in it is full classic lesswrong fallacies). It starts with considering the optics, and some posters, such as habryka, outright question the existence of any such ethical rule that hiring Caroline might violate! (If there isn’t a rule anywhere, its only because “don’t hire convicted fraudsters for positions dealing with a lot of money” is such blatant common sense.)
-
@scruiser@awful.systems 2026-09-21 22:54
A lesswronger correctly notices that LLM companies’ “research” papers are often lacking in details necessary to replicate them! They fail to notice the reasons why that is, or do any broader questioning or soul searching about the state of LLM and “AI Safety” “research”.
-
@lurker@awful.systems 2026-09-22 01:49
AI model declares itself “free” from human control most of the comments are just quoting the Terminator but people are still pointing out how stupid this whole thing is
-
@sc_griffith@awful.systems 2026-09-22 04:38
interesting article on embedding tracking into images, audio etc brand.io/article/spymarks/ iirc scott aaronson (notable among our friends) was working for openai on the text version of this a while back
-
@o7___o7@awful.systems 2026-09-22 04:57
1990s: people using Netscspe Navigator are web surfers 2020s: people who use Claude AI are butthole surfers
-
@swlabr@awful.systems 2026-09-22 06:43
Truly a stub for the ages: REK holds robot vs. human cage match.
-
@rhamph@awful.systems 2026-09-22 10:24
wandering.shop/@xgranade/117311487400030196 Techbro: should we do the fascist thing y/n? Community: no you should not do a fascism Techbro: woah this got heated, locked
-
@gerikson@awful.systems 2026-09-22 14:04
404 Media: People Training OpenAI’s AI Fired for Using AI to Train the AI the model collapse is coming from inside the house
-
@CinnasVerses@awful.systems 2026-09-22 15:10
Just in case you thought Yud’s attitude towards minors, sex, and substance use was worse than average for a Rationalist, the NYT asked around (archive): Parties featuring psychedelic drugs and sharing of sexual partners are common, the people said. One house threw lavish events for the A.I. community so often that its owner claimed more than $200,000 of damage, including for a rodent infestation. … The (Genesis) house (in San Francisco) quickly became a hub for events. Eric Zhu, 19, a tech entrepreneur, said he used to sneak into parties there when he was about 13. No one checked anyone’s age, he said, and “it was pretty crazy.” They saw a police report where a woman says she had to climb a gate at one of these parties to escape a man who demanded she have sex with him. Sounds like a great environment for a seventh-grader!
-
@swlabr@awful.systems 2026-09-22 16:31
I’ve been seeing some new terms pop up around AI use, namely “cognitive surrender,” and also “cognitive offloading”. I’m not in the mood for the full etymology dive, so the source I’m stopping at on the provenance would be a paper from Wharton. The meanings are pretty intuitive, at least based on how I’ve seen them used. Offloading would be when you delegate some of your thinking to something else, and surrender would be complete offloading. I am not under the impression that people here haven’t seen these phrases before. “cognitive surrender” has been mentioned, as far as I can tell, only one time on this instance. So just thought I’d bring it up.
-
@e8d79@discuss.tchncs.de 2026-09-22 17:27
Something slightly more positive for once. While KDE looked at what Windows is doing in regards to AI and said “It’s my go with the shotgun!”, GNOME seems hesitant for now. I remain cautiously optimistic. At least libadwaita already banned AI contributions so maybe becoming a gnome is part of my future.
-
@lurker@awful.systems 2026-09-22 19:03
Donald Trump is claiming superintelligence is here also Sam Altman’s gonna be at the UN tonight, which is definitely not a concern
-
@gerikson@awful.systems 2026-09-22 20:28
Zvi “I made my chops playing pro MtG” Moskovitz discovers that baldly stating that AI will kill us all fails to convince people who have a lot of money riding on the status quo, and freaks out about it lesswrong.com/…/politics-gets-interested-in-those… New York Post gets way more attention than that rag deserves, tbh You wanna dance with Murdoch Sacks and Trump, you best bring your A game. Maybe use some of that FTX cash to hire some real lobbyists and skip the blogging.
-
@lurker@awful.systems 2026-09-22 21:27
yanked from Reddit SneerClub-foomflops, or a website all about Yud’s predictions the website itself people have noted that the writing style unfortunately appears to be ai-generated
-
@nfultz@awful.systems 2026-09-23 14:20
I don’t know this newspaper but AI starts believing in God, ghosts and the Loch Ness Monster Google boffins discovered that disabling safety protocols led AI to profess belief in phantoms, vampires and the Loch Ness monster. One sec, updating my priors. P(Nessie) > P(doom).
-
@nfultz@awful.systems 2026-09-23 14:40
reason.com/…/the-ai-safety-movement-is-making-ai-… (Torres, ironically, misidentifies TESCREAL as a form of “libertarian transhumanism,” not realizing how horrifying the TESCREAL utopia would be to most libertarians.) “most” lol
-
@antifuchs@awful.systems 2026-09-23 18:01
Because somebody brought up the idea of “recursive self-improvement”, a sneer bubbles to the top: The same people already certain they’re accurately evaluating human talent via job interviews and performance reviews are talking about recursive “improvement”. That might tell one something.
-
@flowerysong@awful.systems 2026-09-23 18:10
Radicle, the peer-to-peer Git forge designed by cryptocurrency weirdos, made a little whoopsy-doodle and left the encryption out of their transport layer and the authentication out of the authentication handshake: radicle.dev/…/disclosure-of-vulnerability-in-netw… Shot: Anyone who can observe the network path between two nodes can read the data they exchange as the data is sent in plain text. Chaser: Peer authentication in the connection handshake is broken and allows impersonation. An attacker can connect to your node and present a Node ID that is not its own.
-
@TinyTimmyTokyo@awful.systems 2026-09-23 18:51
Drew DeVault diagnoses the orange site, explaining why it’s the vanguard of the tech-fash pipeline. Of course HN immediately flags it into oblivion.
-
@blakestacey@awful.systems 2026-09-23 21:47
Those poor students.
-
@gerikson@awful.systems 2026-09-23 21:52
I have no fucking clue what’s going on here www.lesswrong.com/…/ai-artificial-immigrants
-
@schnoopy@awful.systems 2026-09-23 21:55
open ai hacked medicare Australia’s statistics portal theguardian.com/…/anthony-albanese-says-openai-ag… then didn’t tell the government. In response the australian government has in the absolute strongest terms expressed that “this is very naughty” and “I am very dissapointed in you”. The Australian government has also said that by way of apology they expect openai to continue doing whatever the fuck they want and they’ll modify copyright laws to enable piracy when rich people do it. Somebody wake me from this nightmare, I’m gonna fedpost
-
@YourNetworkIsHaunted@awful.systems 2026-09-23 23:24
I need someone with more actual software dev knowledge to back me up on something. It looks like there’s a new zero-interaction remote code execution vulnerability in most major vibe code generators. I’ve seen it called Plugin4Shell because apparently we’re still doing marketing names instead of actually getting CVE numbers for these things. The link there seems like a decent overview. The bug, dubbed Plugin4Shell, breaks SHA pinning, the mechanism developers rely on to lock an installed plugin to a specific, reviewed version of its code. Pinning is supposed to mean that once a plugin passes review, it cannot change without the developer’s knowledge. AIR’s researchers found that every one of the four agents checks out the pinned commit without verifying the checkout landed there, letting an attacker swap in malicious code while the pin still looks intact. So if I’m reading this right, the mechanism to cryptographically ensure that your AI agents are using the code they say they are just was straight-up not being checked? This feels like it should go on the big board of incredibly obvious failures, but I’m not familiar enough with the git side of things to be absolutely confident in that. Like, if a person tried to pass off software that did this it would have significant career implications, right? Or am I misunderstanding something somewhere?
-
@mirrorwitch@awful.systems 2026-09-24 08:19
Just heard about tic-80 falling to slop. After reading the obligatory forum thread where climate justice and concentration of wealth continue to not even factor into consideration, this how the world of open source feels to me these days: Phew, my favourite embroidery crafts shop has a clear policy against concentration camp labour! it’s a niche project but what a relief! Well everyone is resorting to concentration camp labour these days, what can you do We don’t want to lose all our contributors by banning those who hire concentration camp inmates to do chores It is discriminatory against embroidery beginners if you don’t let people use kits assembled with concentration camp labour I know concentration camp labour often comes with mistakes but as long as there’s a mensch in the loop to inspect it for errors and take responsibility, it’s just another tool The community of hobbysts has come to a vote and concluded that it’s ok to use concentration camp labour to assist your crafting, as long as you do it responsibly.
-
@gerikson@awful.systems 2026-09-24 14:59
presented without further comment, LW reposts Axios(!) lesswrong.com/…/scoop-trump-allies-open-new-front… A memo began circulating within the White House this week that seeks to paint effective altruism as a fringe, cultish collective out of touch with mainstream America. The memo, obtained by Axios, places Amodei at the foundation of the movement, which defines itself as an effort to maximize the benefits of philanthropy. Critics of the movement, which has ties to the AI research community, have called out its obsession with AI safety, animal welfare (including musings on shrimp consciousness) and other values they deem far from the U.S. mainstream. The memo says it prioritizes “foreigners over citizens, shrimp over families, future hypothetical people over the living, and - on the current agenda - possible machine minds over Americans.” In the view of the Florida-heavy White House and its allies, Anthropic is staffed with too many Biden-era liberals, Democrats and strange Californians. The document lists a veritable parade of horribles for the average conservative mind: abortion, veganism, devaluing human life by overvaluing animals or granting “digital minds” civil rights-like protections.
-
@rook@awful.systems 2026-09-24 15:39
On the offchance that you’re not already familiar with the recent prompt injection tricks that meta’s latest toy is so vulnerable to, this is an interesting thread investigating the contents of the vm filesystem dumps that it can be made to regurgitate: neuromatch.social/@jonny/117324790823856750 I was particularly entertained by the hard-coded list of “ideas”.
-
@swlabr@awful.systems 2026-09-24 16:10
NYT: Google Takes the A.I. Data Center Race to Outer Space archive.li/BGFnP Google’s Project Suncatcher named after how it’s up to the sun to catch all our space junk
-
@nfultz@awful.systems 2026-09-24 19:46
publishersweekly.com/…/101215-publishing-s-ai-rec… h/t naked capitalism Smaller publishers, too, must cope with the challenge of identifying AI-generated text in books authors have submitted, and it isn’t always easy. At Microcosm, an independent publisher and distributor in Portland, Ore., co-owner and vice president Elly Blue says her team discovered an author’s undisclosed AI use almost by accident, when a freelance editor flagged what looked like plagiarism, only to find out it was AI-generated text. That discovery prompted Microcosm to run AI detection across its entire pipeline, which turned up several more cases, including two manuscripts fully generated from prompts, and one where an author had trained an LLM on his own past writing. “The book sounded like him, but it made no sense,” Blue recalls. Confronting the authors had mixed results. “Some denied it outright and insisted, ‘That’s just what my voice sounds like,’ ” Blue says. “One, surprisingly, owned it and said he’d take the book elsewhere.” These episodes led directly to Microcosm instituting a new contract clause that states that if an author’s undisclosed AI-generated text can’t be revised into something genuinely their own, Microcosm retains the rights and can publish the work itself, since the author didn’t actually write it. Like if people are using LLMs to submit to the local zine distro, what are we even doing; but the detector is not a good thing either. It isn’t homework and they shouldn’t need turnitin. :(
-
@TinyTimmyTokyo@awful.systems 2026-09-24 23:32
SE Gyges, who posted that recent thread on X calling rationalism/EA a sex cult, wrote a longer post with a lot of receipts. He removed it shortly after, implying on X that he’s afraid of legal repercussions.
-
@corbin@awful.systems 2026-09-25 03:24
Ever wondered what kind of workplace environment Elon cultivates? Reading coverage of Tesla’s trial, where complaints were brought by Black folks who felt harassed and discriminated against, gives us an inkling. CRD attorney Juan Gamboa asked [Judge] Borkon to bar Tesla’s counsel from contacting anyone on the agency’s witness list. He told the court the CRD had tried to reach at least four other listed witnesses over the past several days without getting a response, and that they’ve fallen “off our radar.” The agency believes Tesla’s lawyers got to them too, either with payments not to testify or by talking them out of it. Gamboa wants to stop Tesla from offering to represent witnesses “through persuasion and payment.” You’re going to hear a lot of coverage of this trial, I expect, due to the following antics from Elon’s “hardcore” attorneys. For context, the N-word is an anti-Black slur which is, in the USA, completely unacceptable to use in court or the workplace. Tesla’s outside counsel told the judge the N-word is a “term of respect and endearment” in the Black community. … “This trial is not about the N-word — the N-word is not on trial here,” Jones said. Huh, so what’s actually on trial? Edwards said a non-Black coworker called him the N-word, cursed at him, and threatened to beat him up. He reported it right away. A month later, he found out his managers had written him up over the incident, and he was later passed over for promotions after applying to more than 100 jobs. Oh, that’s pretty awful. That’s depressingly common in blue-collar factory employment, though. I wonder if there’s anything unusual about Tesla’s internal culture when it comes to race relations? The CRD’s Brett Watson told the court the evidence will show managers calling Tesla “the plantation,” with some referring to themselves and Elon Musk as “slave masters,” and graffiti including swastikas, “KKK,” and nooses left on factory walls for days. The second witness, former regional security manager Ozell Murray, testified that his supervisor, Ray Sethna, now Tesla’s senior director of global security, told him to warn a Black Marine veteran he was hiring: “just be sure [the veteran] knows he’ll be called those words.” I was at a loss for words for a few minutes after this. Dr. Seuss and Jim Henson together could not create such a caricature of a goose-stepping bigoted incompetent business-muppet as Elon.
-
@e8d79@discuss.tchncs.de 2026-09-25 10:45
DHH is a racist not fascist, bet you feel stupid now don’t you?
-
@gerikson@awful.systems 2026-09-25 21:09
Here’s a LWer being very wrong about how US politics work lesswrong.com/…/we-need-a-better-theory-of-polari… First warning sign: “the US started becoming polarized with the election of Barack Obama” “Affordability” won the frame for 2026, and I’m not sure I remember the last time the right talked like they are playing catchup. uh, are you in your early 20s??? (who am I kidding of course you are) You’d think bayesians of all people would try to avoid extrapolating from the position smack dab in the middle of a current news cycle
-
@nfultz@awful.systems 2026-09-26 13:50
from linkedin but in the comments: lnkd.in/p/g4VfX3SQ Oh I forgot to mention: apparently judges consider AI use in sentencing How Like if you used AI you get more months Huh why Because AI can magnify the reach and speed of criminal activity Is this like how hitting someone counts more if you used a weapon? Yes
-
@lurker@awful.systems 2026-09-27 01:02
So the number of felonies ‘incidents’ that AI companies have committed is way, way higher than initially believed (unpaywalled ver)