Alexander Mackie

Cloud, Security, research, AI and other such things.
@datadogHQ

[Alexander Mackie @ZanderMackie]

@IceSolst Any sufficiently advance LLM tool is indistinguishable from malware.

[Alexander Mackie @ZanderMackie]

@GrahamHelton3 I was (am?) CVP verified. Doing research juggling codex and claude back and forth at some point claude just nopes out on me when building a PoC. Codex seems much more willing to help

[Jim Manico from Manicode Security @manicode]

Opus 4.7: rewards engineers who write specs and punishes everyone who doesn't

Opus 4.7: a senior engineer's model in a vibe coder's market

Opus 4.7: is great if you prompt like it's a senior engineer you're paying by the hour. If you prompt like it's a chatbot, it'll burn tokens and frustrate you

[Thomas H. Ptacek @tqbf]

"Replacing long-lived keys with ephemeral keys is, for my money, one of the best uses of security engineering effort." is the best sentence I've read pertaining to my field in awhile. More at:

https://t.co/HY8WhrYJjp

[Alexander Mackie @ZanderMackie]

@Jhaddix With SFT and RLVR they’re matching 4.6 for specific domains. At 1/100 the price too.

[Alexander Mackie @ZanderMackie]

@manicode The shape of an upgrade with LLMs can be very jarring. It’s almost never just a straightforward “bump the version” like in other software.

(Caveat that this is just something I’ve accepted from 1.5 years tinkering with these things)

[Alexander Mackie @ZanderMackie]

Both?

Where SAST enables determinism and an LLM enables the search loop (ai orchestration to bugs, write semgrep codeQL to cheaply find again, write code to prove exploitability).

[Jim Manico from Manicode Security @manicode]

The most important takeaway is to drop your SAST vendors now and shift to LLM-based code review.

This is not something I say lightly. But it’s my experience that this is the way to roll and now. The difference in depth is just too much.

I see SAST capturing less than 10% of what LLM’s accomplish.

SAST was made to go broad and wide. LLM-based code review does this AND goes deep.

PS: I use LLM’s to do code review and then use that same LLM to review the results (LLM’s do better with context).

[Alexander Mackie @ZanderMackie]

@Grady_Booch How much of that same material was covered in your interview with @Pragmatic_Eng ? Curious what's available to reference? TY!

[@mikko @mikko]

Three years ago, I said in my talks that generative AI would eventually start discovering zero-day vulnerabilities. At the time, many people dismissed the idea as unrealistic. It is no longer unrealistic.

[Alexander Mackie @ZanderMackie]

@anton_chuvakin Unstoppable force meets immovable obect.

[pdawg @prathamgrv]

I made a Claude Code skill that turns any arxiv paper into working code.

Every line traces back to the paper section it came from & any implementation detail the paper skips will be flagged, and not assumed.

open sourcing it -
https://t.co/sSio4JfpIo

[Samir @SBousseaden]

very cool research! that's why we already pushed some GenAI related EDR detection like this one

https://t.co/pb9eQKjoCk

[Danielle Fong 🔆 @DanielleFong]

we live in a reality of agent swarms, so if software has this class of bug deep down we should set up to just migrate every class of heap overflow bug until we solve it, among others

[Liam Fedus @LiamFedus]

RL against verifiable rewards in LLMs has clearly opened a very powerful regime. It works, and because it works, there is a strong tendency to view more and more problems through that lens.

You optimize for tasks where the reward is clean, where success is easy to check, where the feedback loop closes quickly. This is productive and will keep paying off. But it also creates a bias: you start emphasizing what is legible to the training setup, not necessarily what is most valuable.

Scientific reasoning is a good example. Not every step in science is something that can be cleanly graded at the moment it is produced. A hypothesis can later fail experimentally and still have been exactly the right kind of thinking at the time: creative, mechanistically grounded, and responsive to the available evidence. “Turns out to be wrong” does not imply “was low-quality thinking”.

A big part of the next frontier will be AI systems that can operate well under this kind of uncertainty, just like a big part of the last one was RL against verifiable rewards.

[Ryan Naraine @ryanaraine]

"I remind you that this present you're so concerned about losing, you hated it in the first place." @juanandres_gs on why security practitioners should stop clinging to the broken thing and start imagining what the fixed thing looks like.

New episode is live 👇
https://t.co/35cXmcOGxf

[Brendan Dolan-Gavitt @moyix]

One of the really interesting things about this is that they think their dataset is *already* saturated if you can give the models a realistic amount of compute/tokens. 2M tokens is a tiny amount to spend on an exploit!

[Alexander Mackie @ZanderMackie]

This is great!

Here’s my recipe:
- take a domain or attack surface where there is a deterministic test of whether a security property has been violated
- find existing research
- distill vulnerability patterns
- use an LLM to build a massive bucket of candidates where these patterns may exist
- us an LLm to build a filtration system to orchestrate deterministic tests against said candidates against said attack surface (ie exploits)
- use LLms and human judgement to build increasingly improved validations of success / failure of
Exploitation
- repeat and improve system

[Simon Willison @simonw]

Started a new tag on my blog to track stories about AI-powered security research, which is very much having a moment right now - 11 posts so far already https://t.co/rlEjS0Ho1h

[Sebastian Raschka @rasbt]

Components of a coding agent: a little write-up on the building blocks behind coding agents, from repo context and tool use to memory and delegation.

Link: https://t.co/iF4DsMcnhj

[Ryan Naraine @ryanaraine]

If AI finds the zero-day, writes the exploit, and patches the code, who trains the next generation of security researchers? Chris St. Myers' "Cognitive Rust Belt" essay kicked off a debate we couldn't stop having.

Apple Podcasts
https://t.co/caiDEI1fpt

[s1r1us (mohan) @S1r1u5_]

this year's pwn2own isn't just interesting because there will be lots of entries with AI+human.

it is also interesting because

a) anthropic burned a ton of tokens on firefox, basically running claude in a loop until it found something for a month, probably exhausting whatever claude can one shot.

b) if someone submits full chain without much use of ai, it tells you one shotting plateaus and these models are bit like fuzzers than seasoned security reseachers.

c) even if they used an llm to find the bug, this tells us scaffolding/harnesss design, prompting, and the operator matters a lot.