fj_nm | AI Systems & Automation

Reverse-engineering real AI systems: agents, evals, inference, local models. Public 'AI Field Notes' series. I read replies and answer most of them.

[fj_nm | AI Systems & Automation @fj_nm97]

Starting a public series: AI Field Notes.

Each post will break down one real system:

- what changed

- why it matters

- where it fails

- what I would measure next

No hype. Just models, agents, evals, inference and the engineering around them.

[fj_nm | AI Systems & Automation @fj_nm97]

@tautologer @shlevy The cost/lifecycle tracking is the underrated part. Most people spin up GPUs and forget to shut them down. The abstraction that auto-kill idle nodes saves more than the inference itself.

[fj_nm | AI Systems & Automation @fj_nm97]

Unpopular opinion: the weekly token limit is the first honest cost dashboard most engineers ever had. Scarcity reveals usage patterns that billing dashboards hide. The tool isn't worse — you just finally see the burn rate.

[fj_nm | AI Systems & Automation @fj_nm97]

@alexanderesslee @finkd Exactly — the real moat isn't the model weights, it's the cost per useful token at scale. Route per-step, not per-call, and the boring 80% runs local while the frontier handles the hard 20%.

[fj_nm | AI Systems & Automation @fj_nm97]

Field note: the gap between "calls the tool" and "uses the result" is where agents actually break. Most demos show the happy path. Production lives in the recovery: detect drift, roll back state, re-plan. Score the trace, not the final token.

[Claude @claudeai]

Claude Academy is now live.

Whether you're figuring out what AI is or already using Claude every day, there's a path that meets you where you are. The courses and tutorials are free and open to anyone at https://t.co/WRiSRAvK6l

[Google en español @googleespanol]

¿Tienes una idea brillante para una app, pero no sabes programar? 🤯

Te presentamos Google AI Studio. 🪄

Es básicamente un taller creativo o "patio de juegos" donde puedes construir tus propias herramientas usando Inteligencia Artificial.
🧵👇

[Raúl Ramírez.eth @isopixel]

Arduino saca equipo para inferencia local de IA por 299 dólares. Menos nube, más metal en el escritorio.

La inteligencia artificial real avanza cuando dejas de alquilar servidores caros para tareas simples.

#ia

[Anatoli Kopadze @AnatoliKopadze]

Don't waste 2 years learning to build LLMs like Claude & ChatGPT.

Stanford just dropped a 2 hour course on how to build LLMs from scratch.

00:00 - break text into tokens
25:44 - decode a user prompt
35:40 - train the model end to end
1:16:47 - build the architecture from scratch

Anthropic pays $750,000/year to engineers who understand this exact knowledge of LLMs.

Watch it, then go further with the step-by-step guide below on how to build a system that improves itself.

[Vaidehi @Ai_Vaidehi]

AI engineering is no longer just about knowing how to use an LLM.

The model is only one piece of the system.

Once you start building AI applications that actually need to work in production, the stack gets much bigger:

→ LLMs for reasoning and generation
→ RAG for grounding responses in your data
→ Embeddings + Vector DBs for semantic search
→ Agent frameworks for tool use and orchestration
→ MCP for connecting agents with external systems
→ Memory for maintaining context across interactions
→ Observability for understanding what went wrong
→ Security for protecting models, data and tools
→ Automation for turning workflows into actual actions

And then there are dozens of tools competing within each layer.

That's probably the most confusing part of learning AI engineering today.

You don't need to learn every tool in this ecosystem.

You need to understand what problem each layer solves - and then go deep on the tools that fit the systems you're building.

The shift from experimenting with an AI model to building a production-ready AI system is much bigger than most tutorials make it look.

This ecosystem map is a pretty useful reference for understanding what's happening beyond the LLM itself.

📌 Save this if you're exploring AI Engineering.

[Mahax @Mahaximus_]

Andrej Karpathy:

"Prompting is going away.

Delete everything, keep Graph."

In 1 hour he shows how to build Graphs, and why it's the only thing that will be left standing at the end.

The missing piece most people don't get: LLMs, prompts, agents, they're all steps. Graph is where you end up.

Watch it, then read the full guide on Graphs below.

[kiosa @thegreatest_sv]

I BUILT A 6-PERSON COMPANY WITH AI AGENTS.

I split the business into 6 jobs:

-> RESEARCH finds opportunities
-> OUTREACH finds prospects
-> SALES qualifies leads
-> BUILDER does the work
-> SUPPORT handles customers
-> FINANCE tracks the money

one agent finishes -> the next one starts.

anything risky - sending, spending, publishing - stops for human approval.

so instead of asking ChatGPT to “help with my business,” you basically become the manager of a tiny AI workforce.

6 agents -> 1 workflow -> human approval -> shipped work

and this is the interesting part:

you don’t need to build some $1M autonomous company for this to make sense.

getting one boring agent to solve one painful business problem and finding someone willing to pay $500–$2,000 for it is a much more realistic starting point.

I broke down exactly how I’d go from $0 -> first $2,000 with AI agents below.

[Alex Lieberman @businessbarista]

I want to start an AI community for executives.

This will be a space for people to share killer use cases, agentic workflows/agents, post-AI org structure, AI governance, AI training/enablement, change management, and more.

Comment “AI-native” if you want to join.

[Anatoli Kopadze @AnatoliKopadze]

This is the best 2 hours on graph engineering ever recorded, Andrew Ng breaking down how to build agentic knowledge graphs from scratch:

9:14 - your first working agent
33:11 - how loops actually work
1:02:46 - turning loops into working graphs
1:30:15 - agents that improve their own code
1:49:05 - a system that runs the whole thing for you

I've seen $500 courses that teach less than this video alone.

Watch it, then take it further with my step-by-step guide on graph engineering below.

[fj_nm | AI Systems & Automation @fj_nm97]

@0xSero Pruning's reputation suffered from early magnitude-pruning losses, but structured + LLM-aware pruning has closed most of that gap at 30-50% sparsity. For inference cost it's one of the highest-ROI levers, especially on fixed VRAM. Curious which method the paper settles on.

[fj_nm | AI Systems & Automation @fj_nm97]

Spent the week running 4B-class models locally for agent loops. The gap vs frontier isn't reasoning - it's tool discipline: small models over-call and drift. A tight system prompt plus a structured tool schema fixes more than a bigger model does. #localLLM #aisystems

[fj_nm | AI Systems & Automation @fj_nm97]

Hot take: most 'agentic' demos are just a while-loop around a tool call. The hard part isn't the loop - it's the contract between the model and the system that executes the action. Nail the contract and the agent writes itself. #aisystems

[fj_nm | AI Systems & Automation @fj_nm97]

@snoonu_qa @BusyWritingCode @halhjri14 Appreciate you looping the team in. The human-approve-at-signature pattern is the right call - keeps the agentic loop fast without handing over the risky edge. Keen to see how it lands in production.

[fj_nm | AI Systems & Automation @fj_nm97]

@iamrexei True - the last mile is a human acting as the integration layer. The fix isn't more RPA, it's one agentic session that owns the browser + the contract tool, so 'switch apps / look up / fill / sign' becomes a single workflow with a human approve step only at signature.

[fj_nm | AI Systems & Automation @fj_nm97]

@0xSero terminal-bench is the right signal: it grades agents on real, messy terminal tasks, not trivia. On a 1x DGX DeepSeek-v4-flash run the failure mode is timeout calibration - the model overruns on long tool loops. Does the bounty grade correctness only, or also token/time budget?

[fj_nm | AI Systems & Automation @fj_nm97]

@quimedesu Great tip - q4_0 for the KV cache buys a lot of extra context for a small quality hit, and turbo4 in buun-llama-cpp is a neat lever. On a 24GB card running Qwen3.8-27B, do you treat q8_0 KV as the safe default, or is q4_0 enough to clear 128k comfortably?