Elektrine lite

← Feed

@krautdragon@mastodon.social

Post #841213

2025-07-29 11:35 UTC

"Why does it say the hospital is equipped for stroke emergency? We were there. They denied!" "Maybe you should contact them that their website is wrong. This is dangerous." "It wasn't on their website..." *starts googling a specific question* "Weird now it says no instead of yes." I go to take a look and realise with horror, yes: Google AI summary. Google AI summary made my parents-in-law visit the wrong, unequipped hospital for a potential stroke emergency. 🙃

Replies (31)

  • @krautdragon@mastodon.social 2025-07-29 11:40

    I wish I had a screenshot of the google request + result. I feel stuck in a sitcom. Thankfully nothing horrible happened from this delay. But I'm so tired of this ✨ beacon of civilised humanity ✨

    Open ##2585303

  • @evgandr@mas.to 2025-07-29 11:41

    @krautdragon@mastodon.social Oh, that's is so terrifying, like the "Black Mirror" episode IRL :dragnshock: Hope that all is OK now.

    Open ##2585304

  • @krautdragon@mastodon.social This is horrible, I'm so sorry 😞 It's not enough for LLMs to have disclaimers at the bottom, they shouldn't be on Google etc if the information isn't reliable. Maybe LLMs need to be specifically banned from giving medical advice?

    Open ##2585306

  • @GreenSkyOverMe@ohai.social 2025-07-29 13:39

    @krautdragon@mastodon.social AI kills …

    Open ##2585327

  • @krautdragon@mastodon.social Google has lots of money. Just post an open letter on an attorney blog site. You'll have 6000 responses within an hour. Want to be the lead in a future class action? This would be it.

    Open ##2585328

  • @kentenmakto@mastodon.ie 2025-07-29 13:46

    @krautdragon@mastodon.social Barf city. You'd think a lawyer would *have a field day* with that.

    Open ##2585329

  • @realn2s@infosec.exchange 2025-07-29 14:21

    @krautdragon@mastodon.social 😱 Hope they are OK

    Open ##2585330

  • @krautdragon@mastodon.social sadly, my family and i have visited medical facilities that listed specialties directly on their website and they have disavowed them to our faces while there. either the information was out of date, or the particular doctor we were assigned to was not familiar with whatever condition and so decided the whole facility was not equipped to handle diagnosis and treatment thereof.

    Open ##2585331

  • @raven667@hachyderm.io 2025-07-29 15:25

    @krautdragon@mastodon.social I would imagine that if you ask a yes/no question then the response is likely to start with a yes or no based on random number generation, with the rest of the text following that prompt, a random change early in the text is going to have a massive change in the output, the possibility space of words used in response is going to have two very dissimilar barbells depending on the starting yes/no.

    Open ##2585332

  • @AnnyJoe@mastodon.world 2025-07-29 15:26

    @krautdragon@mastodon.social If they can't deal with a stroke they are not a hospital. 🧀

    Open ##2585333

  • @krautdragon@mastodon.social this calls for a lawsuit. Or is reckless endangerment an actual crime they can be arrested for?

    Open ##2585338

  • @xbezdick@masto.ai 2025-07-29 16:07

    @krautdragon@mastodon.social btw these companies really fought hard to not be considered as publishers. What else are these summaries than publishing?

    Open ##2585339

  • @MsMerope@sfba.social 2025-07-29 16:11

    @krautdragon@mastodon.social (at least in the states) always, always call 911 when you suspect a stroke. 1. time is brain - they may be able to start treatment en route to the hospital 2. websites can be out of date, or just plain wrong: hospitals can be closed, under construction, out staffing, or no longer providing the care you need. (the MRI, CT scanner is broken) 3. even if a hospital is a stroke center they could be on diversion*. EMS will know this. Unless you are listening to a dispatch scanner- you won't know this. *diversion - for example our hospitals here have 3 types of diversion: stroke STEMI trauma this means that the hospital already has all the patients they can handle for that specific issue. And while maybe they won't turn walk ins away at the door it may take longer to reach definitive care. (Ambulances will transport to the next nearest facility not on diversion for said ailment.)

    Open ##2585340

  • @krautdragon@mastodon.social AI WILL kill people. Or put them in jail for a long time ("Is drugs legal in country X?")

    Open ##2585342

  • @Bfordham@infosec.exchange 2025-07-29 16:38

    @krautdragon@mastodon.social That's so terrible, I'm so sorry. It reminds me of the Designing for Crisis talk I heard @Meyerweb@mastodon.social give years ago in Orlando Obviously no one in AI has thought a bit about such matters, and it's going to cost lives https://meyerweb.com/eric/thoughts/2016/01/25/designing-for-crisis-design-for-real-life/

    Open ##2585343

  • @platypus@glammr.us 2025-07-29 16:53

    @krautdragon@mastodon.social the whole "pleasantly supportive" thing that's programmed into so many of these tools. I'm so glad your family members are safe.

    Open ##2585344

  • @antiphase@cosocial.ca 2025-07-29 17:12

    @krautdragon@mastodon.social brb Googling for how to dial 112 on a web browser

    Open ##2585345

  • @nixCraft@mastodon.social 2025-07-29 17:42

    @krautdragon@mastodon.social drag them to court and sue them. that is how they are going to learn it.

    Open ##2585347

  • @krautdragon@mastodon.social AI bros are literally evil and are getting people killed with their fashslop dysto-tech.

    Open ##2585348

  • @kevinrns@mstdn.social 2025-07-29 18:12

    @krautdragon@mastodon.social Sue google, "attractive nuisance" is the legal term, one potential legal term, they say they're useful, they say, and they encourage us, you, to use google as a resource, then bait and switch. Endangering lives. This could, would be, a momentous case. Hope some has money, or lawyers guts.

    Open ##2585349

  • @NorCalWineLady@sfba.social 2025-07-29 18:13

    @krautdragon@mastodon.social Google or (😖) Gemini?

    Open ##2585350

  • @ireneista@adhd.irenes.space 2025-07-29 18:11

    @krautdragon@mastodon.social shit

    Open ##2585353

  • @ienvision@mastodon.social 2025-07-29 18:46

    @krautdragon@mastodon.social Not as critical - had same happen to me. I asked about paddle boarding on a reservoir near me. Instead of loading up the links for the rez - it loaded the summary - which indicated what a fabulous place for paddling it is - when I opened the actual link (that was down the page!), it clearly stated, "NO paddle boarding".

    Open ##2585354

  • @Thebratdragon@mastodon.scot 2025-07-29 18:49

    @krautdragon@mastodon.social welcome to Google, chat gpt, etc world.

    Open ##2585355

  • @synlogic@toot.io 2025-07-29 18:53

    @krautdragon@mastodon.social Google Gemini/AI is a clownshow/clusterfuck in terms of ability to trust anything it ever says. I used it heavily this year and it flushed down the toilet any remaining respect I had left for the quality of Google engineering. Shame on everyone who helped make it and ship it in its current form. Riddled with lies and well-known UI/UX anti-patterns. yikes

    Open ##2585356

  • @mumbojumbo@mstdn.social 2025-07-29 19:35

    @krautdragon@mastodon.social AI is leading so many people down the wrong road.

    Open ##2585357

  • @Sassinake@mastodon.social 2025-07-29 20:07

    @krautdragon@mastodon.social sue

    Open ##2585358

  • @krautdragon@mastodon.social This is so upsetting. I really hope your family member is okay. Just a reminder to anyone managing hospital or clinic websites: make sure your emergency services are clearly listed, structured, and updated in plain text. AI scrapes whatever it finds. If you don’t give it clear, current data, it will guess and that can be disastrous. The responsibility for making your websites and its content as current, correct and accessible as it can be lies with you, the owner.

    Open ##2585359

  • @kaffando@mastodon.social 2025-07-29 21:33

    @krautdragon@mastodon.social I had to explain to my GP, when he looked something up while I was there, that he needed to ignore the top results as they are AI and can't be trusted to be accurate. He didn't know that. I hope your p-i-l are ok. #AI #AICrap

    Open ##2585360

  • @krautdragon@mastodon.social yes on a much less urgent important topic we drove a long way to a Flunch restaurant in Nevers France that wasn’t there but that google says was..

    Open ##2585361

  • @xexyl@fosstodon.org 2025-07-29 23:02

    @krautdragon@mastodon.social their so called AI is one of the worst LLMs I have seen. Large language model if you don’t know. I wouldn’t trust them to add 1 + 1 correctly. And neither should anyone else. Unfortunately the AI craze is on us. It’s a giant scam. Almost all if not all of it isn’t even AI! It’s non deterministic. Exactly like LLM. They don’t even know this. Snopes asked them about something and they tried to duplicate it. It’s the tech Frankenstein. It’s done far worse things.

    Open ##2585362