Post #662375
2026-03-15 17:44 UTC
I saw something on here that's been bothering me - and it's that an underserved group is getting medical advice from LLMs because doctors just give up when they can't figure it out.
So first, I'm sorry medicine is failing you.
Second, LLMs are purpose-built to tell you what you want to hear and make up shit. Doctors at least swore an oath to not do harm (and are subject to malpractice laws). OpenAI is not. Please do not take ChatGPT's medical advice. There's a nonzero chance it'll be harmful.
Replies (2)
-
@cthos@mastodon.cthos.dev 2026-03-15 17:45
Like, this seems to be desparate people clinging on to whatever source of information they can, and that will not end well for folks that just trust the output. Of a machine that is designed to lull you into trusting the output.
-
@SnoopJ@hachyderm.io 2026-03-16 03:13
@cthos I'd say it even more strongly: consulting a language model for medical advice is VERY LIKELY to lead to harm Even if the generated text happens to contain incidentally-correct information about what the user is querying, this feedback loop will drive them *away* from proper action upon that information, and improper action in this context is already well-established to be deadly, even when the machine that turns your reasoning skills to pudding is not involved