Post #4384138

2026-08-04 21:54 UTC

@tehfishman@ioc.exchange Yeah the deskilling one is really scary! But at the same time, especially with malware reversing, that skill is so specific to getting the defensive result for me, that I don't mind being somewhat mercenary about it. Also, FWIW, I recently used a model on a Golang malware sample using a very detailed set of instructions to make me do a lot of the analysis during the process, and the result was I ended up knowing more about Golang malware than when I started. I didn't lose time on address arithmetic, writing decrypt algos, etc. So yeah. It's fuzzy as hell.

Replies (1)

@tehfishman@ioc.exchange 2026-08-04 23:25

@mttaggart@infosec.exchange cory doctorow recently wrote about the hazards of using LLMs as a teaching aid that I think has some overlap here. The short version being that LLMs when used for learning are hazardous because an expert knows when an LLM is producing bullshit, but a novice doesn't and is more susceptible to accepting bullshit as fact. You have probably done enough malware analysis to know when an LLM based tool is way off the mark. But a novice security analyst might throw a malware sample at an LLM, get a terribly misleading result, and be completely lacking in the skills to refute it. To some degree, that's true of many tools in the security space. That same novice analyst let loose on the full contents of an enterprise Splunk environment will easily get lost in the endless sea of scary looking log events. But many tools don't lie so... convincingly. I guess those limits, and where they translate into added danger, are up to every individual defender to gauge for themselves though. So many thorns.

Open ##4385864