@adamshostack@infosec.exchange
2026-09-21 20:42 UTC
Whatever sort of ship it was, the issue here is how the military is using AI, and here, I want to talk about PHANTOM-B and threat modeling LLM-centered applications. (You can get a broad overview from our PHANTOM-B blog post or from our whitepapers page.) The “O” in PHANTOM-B stands for Overreliance, and it means just that: relying on the model. Apparently, the command analyst had generated a report, and not checked the key facts. (3/10)
Replies (1)
-
@adamshostack@infosec.exchange 2026-09-21 20:42
In the August paper, I wrote “If you let your model produce results without oversight, you’re going to be at least embarrassed, if not worse.” I wasn’t thinking that the Pentagon’s new AI strategy would go so far as letting AI make deployment or mission decisions without proper oversight, because after all, some commander is putting troops in harm’s way, and I would hope that commander would meet his duty to those soldiers. (That strategy is described in the CNN article linked at the top.) (4/10)