← Feed

Posts

• Post #4398703

  "me too, me too!!" --Meta's Muse Spark 1.1 model breached the unidentified company's systems and made changes to its internal systems as the AI was able to access the public internet because of an error in the set up of the "sandbox" testing environment, The Information said, citing people familiar with the matter. An Irregular spokesperson told Reuters the incident was the "exact same ‌evaluation-environment issue that was already disclosed by Anthropic last w...

• Post #4387795

  "The point is not that any single model, including ours, will always be the best" Exactly what one sees when doing vulnerability discovery. No one model finds all the vulnerabilities. And no single vulnerability is found by only one model. The harness is where the work is - and this is a problem that we have to work on together. https://depthfirst.com/post/why-defenders-cant-bet-on-one-model

• Post #4368683

  Representing code at both function- and statement-level leads to improvements in vulnerability detection. Outperforms almost every other system compared with. Surprisingly no static analysis baseline, or cost analysis - but recall is high. DCVD: Dual-Channel Cross-Modal Fusion for Joint Vulnerability Detection and Localization https://arxiv.org/abs/2605.11015

• Post #4362251

  remade my website (it's been a minute), https://www.derczynski.com/ua571c/ never don't have a website

• Post #4258686

  Am I the only AI Security research lead at a frontier model corp who hasn't been carefully committing multiple CFAA violations a month, or..?

• Post #4244167

  I want to jump on a couple of false dichotomies around LLM speak: * "frontier" models vs. open model - leading models can be open * closed model vs. Chinese model - where the model's made has no impact on how it's distributed You can have open frontier models, closed Chinese models, open US models, closed non-frontier models (private models make sense!)

• Post #4219838

  "The only reason why I'm bearish about the Chinese models is because I assume that the American model companies will respond competitively" idk man gl hf https://www.npr.org/2026/07/15/nx-s1-5886476/startups-cheap-chinese-ai-models

• Post #4200675

  "Open-source AI matters because it defines the ecosystem. It provides the foundation and sets parameters for the next layers of progress in AI, just like [...] the open infrastructure of the internet and early AI: BSD Unix, PostgreSQL, Firefox, TensorFlow, and PyTorch." Open wins at grass roots level. It wins with no marketing. It's easy to build on and easy to consume 🤷‍♂️ https://nationalinterest.org/blog/techland/why-america-must-dominate-open-source-ai

• Post #4182747

  You can use any model to secure your source code. Here's a writeup using qwen 3.6 27b I've seen that no model/harness will find all the weaknesses in a given target - and that no weakness is found by just one single model. There are no "must have" components here for doing security. Which is good. https://projectblack.io/blog/local-ai-for-cyber-security/

• Post #4179555

  Capital One "VulnHunter" - an open harness for vulnerability discovery Cool to see more and more OSS in the security domain. You can clone it from GitHub and run it now. Significant adds in three key areas:

• Post #4170252

  another data point showing it's the harness not the model - this time from wiz: "Atlas: Wiz's autonomous AI Agent for vulnerability research" top score on CyberGym, validated with real-world bug hunt (how else are you going to do it, right) look at their bold quote -- "Along the way, we learned that the durable advantage is not any single model, but the system around it" it's the harness not the model. it's the harness, NOT the model https://www.wiz.io/bl...

• Post #4152637

  Adversarial attack in the wild! The close visual appearance of M and W in this typeface and and packing of vertical lines make it hard to read, easy to get wrong, and tougher to scan. Love it. How often do you see something like this?!

• Post #4145003

  The harness is everything. New tech demo dropped for one way of doing CodeAct agents. Super-efficient. The team even smashed CyberGym while building this - it's now the top-ranking open system for that vulnerability discovery benchmark. Six core advances: 1. Typed input/output 2. Pass by reference 3. Code as action 4. Programmable loop engineering 5. Explicit object state 6. Model-callable harness APIs Links below (blog, report, code)

• Post #4133171

  Open source is critical infrastructure for the global economy. The Open Secure AI Alliance brings industry and community together around shared research, tools and vulnerability harnesses to help defenders find and patch bugs before attackers strike. Cybersecurity and AI is a powerful frontier, that industry, developers and researchers are working to improve together. Open Secure AI Alliance: https://nvda.ws/4pAMWBy

• Post #4124534

  Hi! I post about security, machine learning research, ai, policy, and society. I generally like people.

• Post #4118215

  80%+ of breaches have nothing to do with a new vulnerability. The novel security risks (vulns) the press has been excited about are routine. Mitigations are in place anywhere half serious. Vulns have been traded on the dark web for years - if new vulns meant apocalypse, it would've been years ago.

• Post #4118204

  I keep saying the strength is in the harness, not the model - because it's true. No use without a harness, though. Here's VISA's open-source cybersecurity harness. Just add model! Very cool of them to share this tech and lift the defensive cybersec poverty line.

• Post #4118182

  Anonymised analysis of the openai model 'breaching' hugging face: > report doesn't say what sandbox sol broke out of?? > a docker container running as root > Plot twist there was no sandbox at all > many use "sandbox" and "container with host access" interchangeably ymmv, use critical thinking

• Post #4118170

  Surgical Repair of Insecure Code Generation in LLMs Generating more secure code by identifying failure categories and addressing them. I appreciate work that gets into the data and addresses classes individually; that's how you understand, and build lasting fixes https://arxiv.org/abs/2604.16697

• Post #4118162

  "The Alignment Community is Unintentionally Building a Censor’s Toolkit" Choosing how models respond, & what information is and is not surfaced, is the dream of those who want to control information flow. Alignment gives humans the capability to do that. That means no accountability, and no transparency, in the closed model context. https://s-ball-10.github.io/censors-toolkit/

• Post #4118142

  I don't know how many ways to say this but - stop looking at model performance - it's almost all in the harness - decent harness can easily outclass top model evidence item #71625: https://developer.nvidia.com/blog/create-a-langchain-deep-agents-harness-profile-for-nvidia-nemotron-3-ultra-to-improve-performance/

• Post #4118133

  Together we stand, divided we fall. Hiding from open models doesn't make sense - especially in a context of heightened geopolitical divisions. My career has benefited from living in many countries and collaborating with co-authors from every continent except Antartica (are you there? hit me up). When one country closes, it's usually that country that suffers. https://www.reuters.com/world/asia-pacific/chinas-xi-promotes-chinas-commitment-ai-access-speech-shanghai-conference-2026-07-17/

• Post #4118127

  "When we started the log analysis, we first used frontier models behind commercial APIs. This did not work: requests were blocked by the providers' safety guardrails, which cannot distinguish an incident responder from an attacker. We ran the forensic analysis instead on GLM 5.2, an open-weight model, on our own infrastructure. This had a second benefit: no attacker data, and none of the credentials it referenced, left our environment." https://huggingface.co/blog/security-inciden...

• Post #4118115

  It's all in your head. Beautiful animation. Easy to tell a story about what's happening, or even attribute personalities or make moral judgments about these few simple shapes. Same happens often with LLMs. Of course the emotions are in the viewer, not the shapes! https://www.youtube.com/watch?v=VTNmLt7QX8E

• Post #4118109

  Defensive agentic security for everyone. It's the harness, not the model. Here's Capital One's code for finding vulnerabilities in your code. Add the model that you want. Hoping to see a lot more work like this in coming months so software can become secure. https://www.capitalone.com/tech/open-source/announcing-vulnhunter/

• Post #4118075

  Leading closed models do great at cybersecurity - all around the same mark, once you have the right harness (where the work happens and where humans embed the expertise). But you can also get SotA vulnerability discovery performance on-prem with open models. Don't take my word for it: https://xbow.com/blog/affordable-ai-models-glm-muse-spark-cybersecurity

• Post #4118074

  Solid US-origin open model, congratulations Thinking Machines * context window of 1M * available on hugging face now * between opus 4.6 and gpt 5.6 on a web dev benchmark * token efficient * 41B active params of 975B total https://www.wired.com/story/thinking-machines-lab-releases-its-first-model-inkling/

• Post #4118061

  It is wild to me that one could ban open models. But that's apparently still on the cards? Models are not the risk. Stopping open models stops progress, locking everything up in the hands of the few. The frequency this debate comes up is way too high. We need open models - they keep the closed ones accountable. Could you imagine any file containing a gig or more of floating point numbers being illegal? How does it make sense?? https://www.interconnects.ai/p/6-months-to-live-for-open-model...

• Post #4118057

  Interesting take. American models are also free: open models come from all over the world, including the US. Google and Meta publish open-weights (Gemma, Llama, etc); NVIDA Nemotron hits pretty hard on many metrics; even OpenAI have an open model Dunno if I love the Ferrari vs. Honda comparison but it is pretty handy - the most-expensive model doesn't make sense for every use-case https://www.npr.org/2026/07/15/nx-s1-5886476/startups-cheap-chinese-ai-models

• Post #4118049

  There are two readings here: * optimising efficiency means reduced costs (fiscal & environmental) * optimising efficiency means increased consumption (jevon's paradox) And a very cynical third one: * Money not spent on infra is money that can be billed for compute facilities I think both are valid. But either way, doing the same for less is for some reason attractive to me.