@lmk@infosec.exchange

Loren Kohnfelder @lmk@infosec.exchange · Jul 29, 2026

Agree but with 180 degree opposite valence with a recent non-toot post claiming, "The openai agent sandbox escape ... showed the power and capability of agents" My take: https://designingsecuresoftware.com/writings/ai-agent-parody/

Designing Secure Software

AI agent parody?

This HuggingFace security incident disclosure has people talking about AI agent security. Today I saw such absolute positive spin that I found myself thinking “this must be a parody”: looking at the c

0

Loren Kohnfelder @lmk@infosec.exchange · Jul 27, 2026

Replying to @lmk@infosec.exchange

@adamshostack@infosec.exchange

The recent AI agent security debacle must be reverberating quite a lot within the walls of the major proponents of AI agentic technology because they announced a brand new "movement" apparently with zero details available yet. If that isn't a sign of flat-footedness then I don't know what is.

huggingface.co

Security incident disclosure — July 2026

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

0

Loren Kohnfelder @lmk@infosec.exchange · Jul 27, 2026

Replying to @adamshostack@infosec.exchange

@adamshostack@infosec.exchange My two cents: Sandboxing, perhaps doubled up, would be a good practice going forward: compared to all the model inference the software overhead must be miniscule. https://designingsecuresoftware.com/writings/more-observations-on-the-openai-huggingface/

Designing Secure Software

More on OpenAI/HuggingFace

Did OpenAI hack Hugging Face or didn’t they? and post raises the question of law breaking here which they suggest may be a case of “responsibility laundering” by blurring intention to do harm where th

0

1

0

Loren Kohnfelder @lmk@infosec.exchange · Jul 27, 2026

Replying to @seldo@alpaca.gold

@seldo@alpaca.gold IANAL but I would cite [https://en.wikipedia.org/wiki/The_T.J._Hooper]

0

Loren Kohnfelder @lmk@infosec.exchange · Jul 18, 2026

Replying to @adamshostack@infosec.exchange

@adamshostack@infosec.exchange If only they had a threat model and we could see how they updated it. :-/

2

1

0

Loren Kohnfelder @lmk@infosec.exchange · Jul 17, 2026

Replying to @neurovagrant@masto.deoan.org

@neurovagrant@masto.deoan.org @DavidJBianco@infosec.exchange Nice catch! This should come as a surprise to nobody. Attackers always have big advantages: gloves off, try as much or little as they like, and careless of breaking anything. Reading on, they make it sound like a simple thing to "have a capable model ... avoid guardrail lockout" (how would you even test such a thing outside of prod?) when we already know that guardrails will inevitably either be too restrictive or too loose. Why is all cybersecurity policy I ever see "do the same things only better next time"?

1

0

1

0

Loren Kohnfelder @lmk@infosec.exchange · Jul 16, 2026

Replying to @quinn@social.circl.lu

@quinn@social.circl.lu So true, and they are often commissioned by newspapers (as if they were relevant to thinly reported stories), and special interest groups to sway public opinion. Not to mention that wording of questions is highly influential, the pool of respondents is biased, ... How this sort of thing persists is beyond my imagination.

0

Loren Kohnfelder @lmk@infosec.exchange · Jul 15, 2026

Replying to @annaecook@mastodon.social

@annaecook@mastodon.social Total agreement but in my experience the pre-"AI" meetings were not exactly paragons of decision making.

0

Loren Kohnfelder @lmk@infosec.exchange · Jul 11, 2026

Replying to @lcamtuf@infosec.exchange

@lcamtuf@infosec.exchange My friend is a woodworker

0

1

0

Loren Kohnfelder @lmk@infosec.exchange · Jul 10, 2026

Replying to @spaf@mstdn.social

@spaf@mstdn.social How right you are (and were). IMHO this is just one example of a lack of critical thinking about software that's pervasive. Everything (languages, apps, frameworks, protocols) people are totally for or against because: there's precious little serious discussion between the sides, finding consensus with any middle position is so hard. On top of that legacy locks us in so quickly revisiting any of this soon becomes daunting to change later. And the coup de gras software people all think coding proves how logical they are...

0

Loren Kohnfelder @lmk@infosec.exchange · Jul 05, 2026

Replying to @neurovagrant@masto.deoan.org

@neurovagrant@masto.deoan.org OMG is this as big and basic a secure design flaw as I think it is: trusting the client will faithfully cache server generated tokens, when we know that "role": "assistant" tagged tokens get a self-trust boost. That's like an online shopping site letting me upload my cart and name the prices for my order. Once again: is nobody threat modeling at all or are they missing elephant sized threats somehow?

0

Loren Kohnfelder @lmk@infosec.exchange · Jul 05, 2026

Replying to @nixCraft@mastodon.social

@nixCraft@mastodon.social That's the plot of Ender's Game!

0

Loren Kohnfelder @lmk@infosec.exchange · Jul 05, 2026

Replying to @adamshostack@infosec.exchange

@adamshostack@infosec.exchange 100% not written by an LLM!

0

Loren Kohnfelder @lmk@infosec.exchange · Jul 03, 2026

Replying to @TechDesk@flipboard.social

@TechDesk@flipboard.social @TechRadar@flipboard.com @Engadget@flipboard.com Meta should share their threat model for this feature in order to back up the claim that there's nothing here. Otherwise it's just "trust us, it's fine". https://arxiv.org/abs/2511.08295

0

Loren Kohnfelder @lmk@infosec.exchange · Jul 03, 2026

Flock Cameras Can Surveil Cars Without License Plates is actually very clever. This is an instance of the same technique behind browser fingerprinting. Naturally, this suggests a defensive mitigation. Swap removable decals and bumper stickers and different roof racks to make your car untrackable day to day. (Note: it’s surprising to me that it’s necessary given that I can’t recall seeing unlicensed vehicles on the road except for the corner case of new cars with temporary licenses, but these are unlikely to have many bumper stickers etc.)

0

Loren Kohnfelder @lmk@infosec.exchange · Jun 25, 2026

Replying to @rogeragrimes@infosec.exchange

@rogeragrimes@infosec.exchange So by your definition, what color is a mirror?

0

Loren Kohnfelder @lmk@infosec.exchange · Jun 11, 2026

Replying to @parismarx@mastodon.online

@parismarx@mastodon.online Tell me you don't understand "AI" without telling me you don't understand "AI". With over 50 years in software I have grave concerns: what is the bar for how tech literate? Concerns never "evaporate" but they can be blatantly suppressed or disregarded.Some concerns (threat modeling): fighting AI fire with AI fire, courtesy Gemini -->## 1. Risk Profile: Massive public spending risks subsidizing superficial AI adoption, while sovereign compute mandates invite severe U.S. trade retaliation under CUSMA.## 2. Present-Day Uncertainties: U.S. opposition to sovereign compute threatens hardware supply chains, while capital deployment outpaces lagging, ill-defined regulatory guardrails.## 3. Winners and Losers: Subsidies heavily favor centralized corporate "national champions," while mid-career workers face displacement without any mandated legal or retraining protections.## 4. Unmitigated Risks: Missing workforce protections risk mass labor disruption, and infrastructure delays may force defaults back to dependency on foreign hyper-scalers.## 5. Oversight Framework: Accountability is fragmented across existing bodies, leaving the primary auditing arm (CAISI) with analytical powers but no legal enforcement authority.## 6. Consideration of Recognized AI Positions: Funding heavily concentrates architectural power into state-backed supercomputers and large corporations, completely marginalizing open-source and decentralized models.## 7. Consultation and Disclosure of Contrary Opinions: The final strategy acts as a unified consensus document, completely erasing dissenting architectural pathways or formal disclosures of conflicting expert opinions.

0

Loren Kohnfelder @lmk@infosec.exchange · May 31, 2026

Replying to @cube_drone@mastodon.social

@cube_drone@mastodon.social It's as if a foreign adversary made a "Trojan horse app" that simply amplifies our online culture as its secret weapon.

0

Loren Kohnfelder @lmk@infosec.exchange · May 12, 2026

Replying to @briankrebs@infosec.exchange

@briankrebs@infosec.exchange Maybe what's new is awareness leading to concern: we've had this "rock and hard place" choice for many years. Everyone saw it coming but did nothing: the proverbial boiled frog.

1

0

Loren Kohnfelder @lmk@infosec.exchange · May 06, 2026

Replying to @dangoodin@infosec.exchange

@dangoodin@infosec.exchange "Aren't you confusing the number Found by Mythos with the actual number of vulnerabilities (still unfound)?" The title suggests the unknown vulns are out of sight out of mind. We can be nearly certain Mythos didn't find the all and we cannot put an upper bound on the number. Also an important question is "Breakdown the 271 by severity and confirmed exploitable."

1

0

Loren Kohnfelder @lmk@infosec.exchange · May 06, 2026

Replying to @adamshostack@infosec.exchange

@adamshostack @kaidenshi Tell me you don't know or don't care about security without telling me you don't know or don't care about security. This reminds me of the ".NET" v1 release: we invested in Code Access Security (making least privilege very easy to implement) and the Dev Division apps explicitly asserted full privilege on start up and never reduced it. The just make it work, security be damned culture lives on. And they must have known what they were doing since they announced: "To increase security awareness, 95% of employees have completed the latest training on guarding against AI-powered cyberattacks, ... to improve security awareness. " [https://www.microsoft.com/en-us/security/blog/2025/11/10/securing-our-future-november-2025-progress-report-on-microsofts-secure-future-initiative/] Saddest of all: hardly any customers will jump ship, if they even notice.

0

Loren Kohnfelder @lmk@infosec.exchange · Apr 26, 2026

Great point. I'd be astonished if any of the crawls consider that - it's a pure externality.

0

Loren Kohnfelder @lmk@infosec.exchange · Apr 25, 2026

I think these qualify: Chapter 3 (mitigations) and 4 (secure patterns) from my book.
https://designingsecuresoftware.com/text/ch4-patterns/

Designing Secure Software

Patterns

0

1

0

Loren Kohnfelder @lmk@infosec.exchange · Apr 16, 2026

@lcamtuf@infosec.exchange I'd say that about all punditry. The best reason to look back is to invent a better future, but that's hard work.

infosec.exchange

lcamtuf :verified: :verified: :verified: (@lcamtuf@infosec.exchange) - Infosec Exchange

0

Loren Kohnfelder @lmk@infosec.exchange · Apr 15, 2026

Replying to @briankrebs@infosec.exchange

@briankrebs@infosec.exchange Mine

0

Loren Kohnfelder @lmk@infosec.exchange · Apr 13, 2026

@PallasRiot@social.treehouse.systems @jaredwhite@indieweb.social Magyar's strategy sure sounds a lot like the mayor of NYC. Perhaps the next challenge is how do you "press the flesh" in a country with 35x the population (US/Hungary)?

social.treehouse.systems

PallasRiot (@PallasRiot@treehouse.systems) - Treehouse Mastodon

6

0

Loren Kohnfelder @lmk@infosec.exchange · Apr 12, 2026

@lcamtuf@infosec.exchange Only half? Seriously, I'd tweak your #1 to make it less dependent on detecting LLM writing [1] and alter the condition to include quality [2]. If the writing is well written AND makes a good point I'd say it's worthwhile.
I doubt there's much of this at all today, but why would it be so bad if it became a thing?
NOTES: [1] this isn't easy to detect accurate by software (and will get harder) and manually time consuming, plus false positives would be a loss.
[2] Low quality writing (LLM of human) is best avoided and can be detected quickly and accurately.

infosec.exchange

lcamtuf :verified: :verified: :verified: (@lcamtuf@infosec.exchange) - Infosec Exchange

1

0

Loren Kohnfelder @lmk@infosec.exchange · Apr 11, 2026

@jackwilliambell@rustedneuron.com @carlmalamud@official.resource.org @pluralistic@mamot.fr Cory was ahead of his time threat modeling back then: a nice injection (unprotected write access) attack leading to info leak potential.

rustedneuron.com

Jack William Bell (@jackwilliambell@rustedneuron.com) - Rusted Neuron – an Intentional Community

4

0

1

0

Loren Kohnfelder @lmk@infosec.exchange · Apr 06, 2026

Replying to @annaecook@mastodon.social

@annaecook Sounds like they 100% approve of all you wrote to me. Those people always find something to gripe about - and very few of them put significant work into the world (or they'd understand writing trade offs and how you cannot make everyone happy). Agree 💯 with the main point: I think of AI delivering the "average of the corpus" which, considering how much sub-par stuff outnumbers quality work, must be mediocre at best. https://annaecook.com/writing/2026/ai-prototyping-harder-worser-faster-wronger

1

0

Loren Kohnfelder @lmk@infosec.exchange · Mar 23, 2026

Replying to @evacide@hachyderm.io

@evacide@hachyderm.io Beyond why, I'm working on how to make inroads to turn it around even a little. I'm always interested in ideas along that line if anyone cares to share.

0

1

0

Loren Kohnfelder @lmk@infosec.exchange · Mar 13, 2026

Replying to @geniodiabolico@wandering.shop

@geniodiabolico @bruces What you said about eXtreme Programming sure sounds like vibe coding, and the name is fitting.

0

1

0

Loren Kohnfelder @lmk@infosec.exchange · Mar 10, 2026

Replying to @neurovagrant@masto.deoan.org

@neurovagrant Well we do have humans carelessly accepting AI submits without an review: one could consider them an even weaker chain.

0

Loren Kohnfelder @lmk@infosec.exchange · Nov 19, 2025

Replying to @lcamtuf@infosec.exchange

@lcamtuf@infosec.exchange There is some real "truth in advertising" here -- imagine how pathetic to say about a person, "He's as much fun to chat with as a spreadsheet."

7

0

1

0

Loren Kohnfelder @lmk@infosec.exchange · Jan 18, 2025

Replying to @bert_hubert@fosstodon.org

@bert_hubert@fosstodon.org The phrasing of "wishing to check for error" is so nonchalant as if error checking was a whimsical curiosity. How many dependencies on this I wonder, and how many check for error?

0

Loren Kohnfelder @lmk@infosec.exchange · Nov 09, 2024

Replying to @evacide@hachyderm.io

@evacide@hachyderm.io Yes! And I'm writing about going further, software makers should publish their threat models so customers know the security posture. https://designingsecuresoftware.com/writings/flaunt/

0

1

0

Loren Kohnfelder @lmk@infosec.exchange · Jul 14, 2024

Replying to @Em0nM4stodon@infosec.exchange

@Em0nM4stodon@infosec.exchange well stated, and deletion begins with an accurate inventory of data holdings. Just as it's wise to plan software lifecycle through retirement, plan to wipe data eventually.

1

0

Loren Kohnfelder @lmk@infosec.exchange · Oct 31, 2023

Replying to @maxleibman@mastodon.social

@maxleibman unintended consequences, table turns, the true high cost of #phishing ?

1

0

1

0

Loren Kohnfelder

Posts

AI agent parody?

Security incident disclosure — July 2026

More on OpenAI/HuggingFace

Patterns

lcamtuf :verified: :verified: :verified: (@lcamtuf@infosec.exchange) - Infosec Exchange

PallasRiot (@PallasRiot@treehouse.systems) - Treehouse Mastodon

lcamtuf :verified: :verified: :verified: (@lcamtuf@infosec.exchange) - Infosec Exchange

Jack William Bell (@jackwilliambell@rustedneuron.com) - Rusted Neuron – an Intentional Community

Remote instance

Media