Fpailo

Building dev tools with Python

[Fpailo @paulsfranz]

Every number is reproducible against pinned commit SHAs:

pip install schliff==8.5.0
schliff score AGENTS.md

Full data, method & limitations:
https://t.co/BhyQSYRJdJ

[Fpailo @paulsfranz]

I scored the AGENTS.md files of the 16 biggest AI agent repos with a deterministic engine (no LLM judge).

~1.46M combined stars.
Zero A grades.
The repo that popularized the convention ranks 13th of 16.

Data + method 🧵

[Fpailo @paulsfranz]

"Can't you game a scorer like this?"

Yes — an earlier version gave junk commands in code fences a 92.5/A. That failure is written into the spec, and the same file now scores 56/D.

The engine is MIT + stdlib-only. The rubric is a dict you can read, not a hidden prompt.

[Fpailo @paulsfranz]

@elder_plinius Insane! Great job dude

[ClaudeDevs @ClaudeDevs]

For the developers building with Claude, a direct line from the team.

Follow for changelogs, API releases, community updates, and deep dives.

[Fpailo @paulsfranz]

Try to cheat the score — it catches you.

6/6 gaming patterns detected: keyword stuffing, empty headers, copy-paste examples, contradictions, bloated preambles, missing scope.

Fake-examples skill: 94 → 43 after detection.

benchmarks/anti-gaming/ in the repo.

[Fpailo @paulsfranz]

I built a deterministic linter for Claude Code skill files.

54 [D] → 98 [S] in 18 autonomous iterations. 7 dimensions, anti-gaming detection, zero deps.

pip install schliff && schliff demo

https://t.co/KOS5AC45om

#ClaudeCode #Anthropic #AI #Python #OpenSource

[Fpailo @paulsfranz]

How it works: 7 scoring dimensions, deterministic, no LLM needed.

Same input → same output, every time. Python 3.9+ stdlib only.

540 unit + 99 integration tests.

#ClaudeCode #DeveloperTools

[Fpailo @paulsfranz]

@aiedge_ Skills degrade silently - triggers start overlapping, instructions contradict each other over time. Been working on a deterministic way to catch this statically. Launching today actually. 🫡

[Fpailo @paulsfranz]

@boxmining @openclaw @claudeai good insights - https://t.co/L1SkBWKEfm is on fire 🚀🚀🚀

[Fpailo @paulsfranz]

@claudeai Skills are becoming the most underrated part of the Claude Code workflow. The gap is quality tooling - there's no eslint equivalent yet for .md instruction files.

[Fpailo @paulsfranz]

@AlexFinn Interesting. I built a static analysis tool for SKILL.md files — catches contradictions, gaming patterns, and trigger overlaps without any LLM. 7 dimensions, 0-100 score. Shipping it tomorrow on Show HN: https://t.co/KOS5AC45om

[Fpailo @paulsfranz]

@manthanguptaa Been building something in this space — a deterministic linter that scores SKILL.md files across 7 dimensions before you run expensive LLM optimization loops. Zero deps, no API calls needed. Launching tomorrow: https://t.co/KOS5AC45om

[Fpailo @paulsfranz]

@DavidOndrej1 Been building something in this space — a deterministic linter that scores SKILL.md files across 7 dimensions before you run expensive LLM optimization loops. Zero deps, no API calls needed. Launching tomorrow: https://t.co/KOS5AC45om

[Andrej Karpathy @karpathy]

Software horror: litellm PyPI supply chain attack.

Simple `pip install litellm` was enough to exfiltrate SSH keys, AWS/GCP/Azure creds, Kubernetes configs, git credentials, env vars (all your API keys), shell history, crypto wallets, SSL private keys, CI/CD secrets, database passwords.

LiteLLM itself has 97 million downloads per month which is already terrible, but much worse, the contagion spreads to any project that depends on litellm. For example, if you did `pip install dspy` (which depended on litellm>=1.64.0), you'd also be pwnd. Same for any other large project that depended on litellm.

Afaict the poisoned version was up for only less than ~1 hour. The attack had a bug which led to its discovery - Callum McMahon was using an MCP plugin inside Cursor that pulled in litellm as a transitive dependency. When litellm 1.82.8 installed, their machine ran out of RAM and crashed. So if the attacker didn't vibe code this attack it could have been undetected for many days or weeks.

Supply chain attacks like this are basically the scariest thing imaginable in modern software. Every time you install any depedency you could be pulling in a poisoned package anywhere deep inside its entire depedency tree. This is especially risky with large projects that might have lots and lots of dependencies. The credentials that do get stolen in each attack can then be used to take over more accounts and compromise more packages.

Classical software engineering would have you believe that dependencies are good (we're building pyramids from bricks), but imo this has to be re-evaluated, and it's why I've been so growingly averse to them, preferring to use LLMs to "yoink" functionality when it's simple enough and possible.

[sarah guo @saranormous]

Caught up with @karpathy for a new @NoPriorsPod: on the phase shift in engineering, AI psychosis, claws, AutoResearch, the opportunity for a SETI-at-Home like movement in AI, the model landscape, and second order effects

02:55 - What Capability Limits Remain?
06:15 - What Mastery of Coding Agents Looks Like
11:16 - Second Order Effects of Coding Agents
15:51 - Why AutoResearch
22:45 - Relevant Skills in the AI Era
28:25 - Model Speciation
32:30 - Collaboration Surfaces for Humans and AI
37:28 - Analysis of Jobs Market Data
48:25 - Open vs. Closed Source Models
53:51 - Autonomous Robotics and Atoms
1:00:59 - MicroGPT and Agentic Education
1:05:40 - End Thoughts