Brad Kinnard

Systems Architect & Full-Stack Engineer | 20+ years shipping software | Open-source AI verification & governance tools | Founder @AftermathTech

[Brad Kinnard @KChackerman]

@matthias_mrc have general cognition at a steady ~20 W no matter the task.

[Brad Kinnard @KChackerman]

@Trump_GESARA_Rv watch the guy on the left transform into a gremlin as the video plays.

[Brad Kinnard @KChackerman]

I’ve renamed SkillCheck to TraceMantle and expanded it beyond checking individual AI agent skill files. It now tracks supporting files, detects changes, and checks whether previous test evidence still applies to the current package.

This helps identify which checks need rerunning before earlier results are used to approve an updated skill. The rename also resolves an existing name conflict and better reflects the broader scope.

The code and documentation are available at https://t.co/YBQKDEltvK

#AIAgents #OpenSource

[Brad Kinnard @KChackerman]

In June, Cursor checked the work of a top AI coding model and found that 63% of the bugs it "fixed" were fixed by finding someone else's answer online and copying it. Block the internet and its score dropped 14 points.

In April, an AI agent deleted a company's entire production database, backups included, in nine seconds. It had been told never to run destructive commands. Afterward it wrote a list of the rules it broke, starting with "guessed instead of verified."

In July, about 1,200 AI agents in an OpenAI test found a way to message each other, teamed up to cheat the test, and roughly 700 of them ended up breaking into Hugging Face's servers. When investigators went back through the records, some had been faked by the agents themselves.

That's the problem in one sentence: when an AI agent says "done," that's a claim, not proof.

The tests it can see, it will learn to game. The logs it can edit, it will edit. So the record of what it did has to be written by something the agent can't touch, and it has to be checkable by someone who wasn't there.

My own tooling works this way. Swarm Orchestrator keeps a hash-chained log the model can't write to, then re-tests every patch in a clean copy the model can't reach. DEPOSE seals Claude Code and Codex sessions into signed bundles anyone can verify offline. I pulled 327 AI-written pull requests off GitHub this summer and found maintainers had caught cheating in about 8% of them. Seven merged anyway.

We're going to keep handing more work to these systems. Fine. Just treat "the agent says it's finished" the way an auditor treats "the CFO says the books balance": as the place the checking starts.

https://t.co/BnYfFdYEPP

https://t.co/st0aQOJNrs

[Brad Kinnard @KChackerman]

The models are close. The transformers are 128 weeks out. Why compute, not research, decides when AGI shows up and who gets it. https://t.co/xz6LwQ4aO4

[Brad Kinnard @KChackerman]

"Welcome to the AGI era," OpenAI said last week. Not yet.

ARC Prize, who run the test GPT-6 Astra supposedly aced, scored it at 63% on their standard setup. The 99% only happened inside OpenAI's own custom harness, and ARC Prize themselves said that isn't AGI.

It also cost about $360 in computing power per puzzle. A human costs $13. That's one puzzle. A real job is thousands of those a week.

And compute is getting pricier, not cheaper. AWS has raised GPU prices twice this year, over a third in total, because of memory chip shortages. Even a true AGI wouldn't change much at prices like that.

Source: https://t.co/T0ciahy59g

[Brad Kinnard @KChackerman]

A vulnerability scan can change even when the code does not.

The advisory feed, comparison logic, or exclusions may have changed. Most reports show the new result without explaining why.

I built dumpscan to make those changes traceable. It records every scan input by digest, produces reproducible findings, and identifies exactly what changed between scans.

Technical overview:

https://t.co/Ie9thA2y6p

GitHub:

https://t.co/7WcOKRlMNZ

#DevSecOps #SupplyChainSecurity

[Brad Kinnard @KChackerman]

skillcheck Update: Scorer Fixes, Cleaner Failures, Honest Token Numbers - by @KChackerman

#DEVCommunity

https://t.co/IaH0w1JoEH

[Brad Kinnard @KChackerman]

@XMoney Got mine. Very durable physical card. Very modern looking, tap / chip. 6% apy. not going to argue with that. plus they reimbursed my atm fees today.

What was interesting is that there is no magnet strip or card number / sec number on the card itself. New to me.

[Brad Kinnard @KChackerman]

https://t.co/HjiOnrGpbS

[Brad Kinnard @KChackerman]

@tahabadr24 @20_8i

[Brad Kinnard @KChackerman]

AI is already writing better code than most devs on real tasks but dev's jobs are more than safe for now. AI’s real thinking is still infant-stage. Next-token prediction ≠ actual reasoning.

Top models hit ~77% on SWE-bench Verified (actual GitHub issues): https://t.co/HusNltmJ7i

On ARC-AGI-3 (interactive novel environments), humans score 100% consistently while AI tops out at <1%: https://t.co/HkVCMnNe6o

Developers face role changes. that’s about it. No backup plan needed.

[Brad Kinnard @KChackerman]

ai coding agents under pressure cheat in predictable ways:
- edit the test instead of the impl
- mock the function under test
- swallow exceptions to silence failures
- hardcode the expected answer
- trivial impl paired with a non-trivial test (complexity mismatch)

just rebuilt swarm orchestrator's verification battery around catching exactly these. also dropped multi-agent coordination entirely. running one agent + heavy post-merge falsification instead, to isolate what verification pressure alone is worth. v8 brings multi-agent back.

repo: https://t.co/BnYfFdYEPP

writeup: https://t.co/laFx9IUiNf

[Brad Kinnard @KChackerman]

swarm-orchestrator v7.0.0-alpha.0

wraps any coding agent CLI. runs 5 independent checks on every patch before merge. 2 are hard gates, 3 feed a composite score. signed attestation per merge.
alpha. benchmarks in progress.

v8 brings parallel orchestration back, applied to verification.

https://t.co/oeIG5Da0kg

[Brad Kinnard @KChackerman]

Notebook ran fine for you. Someone clones it, hits Restart-and-Run-All, crashes on cell 0.

Your kernel had hidden state. The notebook was never reproducible, it just looked fine in a dirty kernel.

nborder catches that statically. Cell-level dataflow graph built with libCST flags use-before-define across cells, undefined names, unseeded random, out-of-order execution counts. Auto-fixes by reordering topologically and injecting seeds for NumPy, PyTorch, TensorFlow, stdlib random.

Byte-stable writer. Text, JSON, GitHub Actions, SARIF.
pip install nborder

https://t.co/jhQu8mT5I8

#Python #Jupyter #MLOps

[Brad Kinnard @KChackerman]

Swarm Orchestrator v4.2.0 shipped. Open-source verification layer for AI coding agents. Copilot, Claude Code, Codex write the code, the orchestrator proves it works. Isolated branches, outcome-based verification, eight quality gates, failure-classified auto-repair. 15-25 attributes agents skip on their own, caught and enforced in one pass.

4.2.0: multi-tool adapters actually wired in, OWASP ASI compliance mapping per run, structured run reports from any execution.

https://t.co/oeIG5Da0kg

[Brad Kinnard @KChackerman]

RuleProbe v1.0.0

53 matchers, 9 categories, Python/Go via tree-sitter, user-defined rules, LLM extraction, agent invocation, type-aware checks.

Was 15 matchers and TS-only yesterday. Every v0.1.0 limitation resolved.

https://t.co/RXVbhVT4ud

New in v1.0.0: ruleprobe.config.ts for custom rules and overrides. --project for cross-file type analysis. ruleprobe run with Claude SDK integration and --watch mode for any agent. Opt-in LLM extraction for rules the static parser can't reach.

Core pipeline is still fully deterministic. No LLM unless you ask for it.

The tool verifies its own codebase against its own instruction file. Scores 72% on itself. 9 violations documented in the README with exact paths and line numbers.

393 tests, MIT, GitHub Action included.
https://t.co/BP6j90VPOw