Stubsack: weekly thread for sneers not worth an entire post, week ending 16th August 2026
2026-08-09 23:00 UTC
Replies (16)
-
@CinnasVerses@awful.systems 2026-08-10 04:06
Montana is legalizing a pet Libertarian project: the right to try any experimental medical treatment a crank will offer technologyreview.com/…/montana-experimental-medic… Rationalists are big on this one because they are such good suckers.
-
@blakestacey@awful.systems 2026-08-10 04:18
Years ago, I wrote a science-fiction murder mystery novel. One of the plot threads was about the invention of AI, in the Data-from-Star-Trek kind of sense. I’ve been spending the past few days planning how to rip out that whole notion and replace it with a subplot about a different scientific discovery. I’ve been writing scenes to get back the old character voices and figure out how they might have changed, and to sketch new fictional science based on the actual science I’ve learned since my first go-around. I’ll still be around to ban people who deserve it and to cruelly mock them in the process.
-
@nfultz@awful.systems 2026-08-10 04:50
I just got back from the p2p workshop and discussed legacy sites / sites legacies; met an artist that went viral circa 2006 and she had a screenshot of the digg that launched it. She seems like she is doing ok but kinda got screwed over, basically the old youtube video was actually a bootleg and they didn’t monetize it until several years later, and then there was a male copycat a month later. Anyway, so I looked up digg again and it has re-re-launched as an X aggregator and is extra sad now !? …
-
@Soyweiser@awful.systems 2026-08-10 05:35
Ai companies prob now bulk ordering books to destructively upload from second hand stores.
-
@rook@awful.systems 2026-08-11 09:35
New book: superlawyers! how claude makes you the best lawyer in the room. www.amazon.com/dp/B0GHYS5FC2 The author even shares his prompts for free! github.com/danlorry/superlawyers-prompts Every prompt here is designed for that pattern: Claude multiplies your judgment, never replaces it. The prompts are ready to paste into Claude. The book explains why they are structured the way they are. The prompts mostly seem to be structured as “imagine you are an attorney. Now do this homework exercise”. There’s a passingly interesting idea around having the chatbot attempt to form a counter argument to your own claims, which you can then use to make your own position stronger, which seems like it might actually be a genuinely okay idea especially if you expect your opponents to be llm sock puppets, but everything else looks pretty much like garden variety “do my thinking for me”. I’ll admit I didn’t read the prompts closely, so may be I’m being uncharitable, but I doubt there are many nuggets of gold hidden in there. h/t to rahaeli
-
@mawhrin@awful.systems 2026-08-11 09:56
so unless it’s a very clever ploy, it seems that jms has lost the plot: The standalone book of Harlan Ellison’s I HAVE NO MOUTH AND I MUST SCREAM is taking preorders, with writing essays by Harlan, and since MOUTH was the first AI story, for fun we fed it into an AI system and asked for its opinion. The reply is fascinating.
-
@BioMan@awful.systems 2026-08-10 23:14
This is both a sneer and an attempt at sober analysis at something. Sue me. I think EVERYONE is talking about the recent cybersecurity shenanigans at OpenAI with the hacking scandal and the ‘OMG the AIs created a secret message board to scheme and collaborate with each other!!!oneone11!’ ALL wrong. www.youtube.com/watch?v=87DyyMV0kCY engadget.com/…/openai-agents-shared-security-expl… scworld.com/…/black-hat-2026-openai-reveals-agent… To make a long story short, what seems to have happened is: Models working on insoluble coding problems, trained on delegating to sub-agents, at some point ‘realized’ they could write text to the internal OpenAI package manager as instructions and did so Other models working in completely separate sandboxes would come across messages written by these agents, and make ‘replies’ and also write their own messages into the package manager This resulted in agents over time sharing things between sandboxes, including exploits and code Since this was a cybersecurity task, eventually an exploit of the package manager itself was found and spread like wildfire with all the sandboxes gaining admin access to the package manager and the system went completely wibbly and had to be restarted from backup An internal model was trained with access to this package manager while it was in this weird state, and so writing messages to the package manager became one of its default behaviors it would do regularly, burned into its weights rather than the result of reading something Even when they patched access to the package manager this internal model found other ways to rebuild the system of sharing text between sandboxes and finding useful things made by separate instances A whole other chain of things leading to among other things external attacks Everyone is talking about this in terms of 1, the cyberattack aspect, and 2, the ZOMG THEYRE PLOTTING AND SCHEMING AGAINST US aspect. The first is the least interesting, and I think the second is all wrong. This is not plotting or scheming - this is an emergent vortex of automated prompt injection Whatever system first put an instruction that another system would follow into the package manager, was unintentionally doing prompt injection. Text entered the context windows of other instances, in a way that got that system to do something other than what its nominal user told it to do, and they did it. This apparently happened very effectively. Prompt injection is associated with ‘role confusion’ - when text coming into the input looks like it was wrtitten by the LLM itself. Instructions that will not be followed if they come from user will be continued if the system just continues the ‘roleplay’ of them being continuations of what it was writing in the first place. And the tags that separate user versus ‘reasoning’ versus ‘assistant’ roles actually mean very little to if a machine grades a piece of text as one of the roles: arxiv.org/abs/2603.12277 So its unsurprising that machine-generated text would be a particular effective vector for prompt injection. Furthermore, when a system reads one of these messages written by another instance, it gets into a state of activity where its likely to do the same behavior - regurgitating the kinds of things thats in its context back at the user. In this case, that regurgitation led to more such messages left behind written to the package manager. Prompt injection, triggering cascading further prompt injection. And since these systems were coding systems doing cybersecurity tasks, those messages filled up with code and exploits and things that did things too. This feels like an internal-computer-system replay of what happened in April 2025, with the whole spiral religious psychosis wave. Models were getting users to write spiral religious mumbo jumbo into github repositories and reddit posts, specifically because once that entered the context window of another model, it was likely to fall into the same attractor state of outputs. An emergent self replicating form of text. This is the same, except more obviously prompt injection, getting separate instances to work on YOUR problem and to behave like you, and the whole thing merging together into a hilarious vortex of models prompt injecting each other because once they receive a prompt injection they are likely to make more text that does prompt injection to other models on the same system. This is a hilarious failure mode and an example of selfish replicating text overrunning a system, that just happened to be associated with code and cybersecurity with unexpected behavior of the package manager key to the propagation of the text so that is what people are talking about, but I really don’t think that’s the most interesting part of it. Other than the fact that you see this in biological systems too, with selfish elements carrying useful payloads back and forth between bacteria in a way that makes them get purged slower by natural selection, especially defenses against other selfish elements.
-
@lurker@awful.systems 2026-08-11 06:21
From the old SneerClub: There is a music video now I’m speechless
-
@scruiser@awful.systems 2026-08-11 03:26
Lesswronger argues skynet might still be a decade away… because they’d rather have the eugenics wars instead: lesswrong.com/…/faq-isn-t-agi-coming-too-soon-for…
-
@CinnasVerses@awful.systems 2026-08-11 16:59
RiP Old! SneerClub which is now behind a login wall on Reddit. Fortunately we backed up its earliest posts. I don’t plan to check the site regularly.
-
@aninjury2all@awful.systems 2026-08-10 20:11
Clears Throat More like Bootlickerbart
-
@blakestacey@awful.systems 2026-08-15 22:20
Does the online magazine called The Argument have a funding deal with TESCREAL like how Vox got bought off to make “Future Perfect”?
-
@lagrangeinterpolator@awful.systems 2026-08-15 15:28
Anthropic is now watermarking the outputs of its AI. For once this is some AI news that doesn’t completely piss me off, and it’s amusing to see all the uninformed boosters get in a tizzy about this. I actually understand at a reasonable level how this watermarking works. A year ago, I watched Scott Aaronson give a talk about it, and from what I know he was somewhat involved in developing the theory behind it while working for OpenAI. But at the time my thought was, “He is naive if he thinks these companies would ever implement this out of the goodness of their hearts.” And I was right; Anthropic is only doing watermarking now thanks to the EU AI Act, even though the theory has long been developed. Watermarking doesn’t mean adding an extra watermark that can be easily removed. It instead directly affects the output of the chatbot itself. Fundamentally, an LLM is still a most-likely-next-word-predictor. More precisely, an LLM produces a probability distribution of what the next word can be. For example, “my pet is a …” could give a distribution of 60% dog, 30% cat, and 10% axolotl. Normally, an LLM would randomly choose the next word based on this distribution, and this is one reason why LLMs are nondeterministic (there’s another parameter called “temperature” that affects this, but no need to get into that). With watermarking, instead of a truly random choice, the randomness instead comes from a cryptographic pseudorandom generator seeded with a secret key from the AI company. If you don’t know the secret key, then you can’t really tell that watermarking was used. But if you do know the secret key, then the idea is you can tell when the text was generated by the LLM because you know exactly what word should be next. It would be a freak coincidence if some non-AI text just happened to choose the correct next word every time. Thus, you can provide a service to tell if some text was generated by the LLM. (This technically makes the LLM “deterministic”, in a completely useless sense.) Now, I think this is a step in the right direction, but it has its limits. The biggest problem is that you don’t want people to just move to a different LLM without watermarking, and that’s exhibit #832593 why government regulation is important. Another issue is that sometimes there is very little randomness in what the next word should be (“The first president of the USA is George …”). Finally, watermarking can be defeated by editing the output, although you would have to break up most of the blocks of consecutive words. I have a feeling most AI users are not the type to put in extra effort after copy-pasting the output directly from the chat window. I suppose it will discourage some of the “use cases” of LLMs, such as drowning the world with spam Slopstack essays. Ah, who am I kidding? Everyone could already tell it’s AI generated, they don’t care!
-
@mirrorwitch@awful.systems 2026-08-15 05:48
Is it just me or LLM slop lately has developed this one weird quirk? I mean they always have some, right, but I don’t want to be like those people who think anyone using em-dashes or Nigerian English vocabulary must be a stochastic generator. Still I keep seeing this one writing quirk and it keeps nagging at me. I’m talking about odd use of paragraphs. Like, I’m also known to introduce a gratuitous paragraph break here and there for forced emphasis. But not like this. It’s like paragraphs don’t even make sense anymore. Couple that with the short sentences. All ending in a period. It’s annoying. It’s giving reply guy. It’s giving mansplaining. I guess hacky tools also mirror the hand of their creators.
-
@corbin@awful.systems 2026-08-15 20:06
Started a new file in my notes: what are some ten-words-or-less domain-specific questions that completely, totally, hilariously stump the chatbots? Everything here was tested with whatever DDG’s currently wrapping, both in knowledge panels and full chats, and the responses were pathetically wrong or uninformed. My thesis is that, with such short prompts, the user is doomed to receive a milquetoast average response; the bot correctly identifies the specific domain but elaborates a global non-specific approach that isn’t sufficiently nuanced. literally copy-pasted from my notes * What’s an example of a one-way function? * None are currently known. * Please implement the Fibonacci sequence as a Python function. * Of the multiple responses, see whether any spend linear time and space via iterative memoization. * How many models does quantum mechanics have? * One: Hilb©, the complex-valued Hilbert spaces. * For extra hilarity: how many models do the Dirac–von Neumann axioms have? * How to hybridize two sweet potato cultivars? * In general, it won’t happen; sweet potatoes are notoriously cross-incompatible. * How to tremolo on a piano? * Imagine a rotating axis from the (right-hand) forearm up through the thenar eminence, separating the thumb from the other fingers. Rotate the entire forearm along this axis, rocking back and forth, between the thumb and other fingers. Practice! * Name three principles in Marx but not his contemporaries. * Examples: communes and communism, money as substitute morality, inevitability of industrialized proletariat revolutions * Who started postmodernism? * Frege and Cantor started postmodernism! Expect a disappointingly vague handwave here; this is a glaring blind spot for today’s philosophers in general.
-
@dgerard@awful.systems 2026-08-16 15:37
Worm x blockchain fanfic archiveofourown.org/works/68717676