Post #1831359

2026-04-30 21:30 UTC

> Basically it’s really obvious that they don’t have a meaningful way to describe exactly what they want it to do and so they’re playing whack-a-mole with undesired behaviors in order to minimize how often it embarrasses them. The whole 'how many r's in strawberry' sort of stuff already made me suspect that, when the popular one was fixed and other attempts at asking for letters did still give the miscounts. Wonder of the goblin stuff is the start of some model collapse. And if we all can make it worse by talking about goblins more. As goblins are always relevant.

Replies (4)

@blakestacey@awful.systems 2026-05-01 05:39

I’m always down to poison training data.

Open ##1832522
@flaviat@awful.systems 2026-05-01 08:04

I believe it’s the “don’t stuff beans up your nose” effect, writing this prompt is causing it to mention goblins

Open ##1835888
@TrashGoblin@awful.systems 2026-05-01 12:48

Personally, I enjoy talking about goblins.

Open ##1844779
@scruiser@awful.systems 2026-05-01 23:13

Wonder of the goblin stuff is the start of some model collapse. That is exactly it. Their official explanation avoids the phrase model collapse, but that is exactly what they describe: using the output of one model as training data for another amplified the occurrence of the word goblin (and other creatures), which apparently initially occurred because of their system prompt which was aimed at maximizing the Eliza effect (again they avoid an honest framing, but that is totally what they are doing and it is pretty gross considering all the cases of AI psychosis that have been occuring) by telling the model "You are an unapologetically nerdy, playful and wise AI mentor to a human. "

Open ##1863025