ompragash

Product Manager at @harnessio • Automation and AI for Developer Workflows • prev @redhat @cloudera

[Caura @CauraAI]

Five frontier models, 100 questions, judged blind — by each other.

Opus 5 wins at 8.87. Fable 5 takes third without being beaten: its own guardrail handed in 4 blanks — two were high-school biology.

The numbers, the blanks, the self-bias ↓

[Elon Musk @elonmusk]

The most powerful moving object ever created by humans

[h100envy @h100envy]

The creator of Pydantic explained how to make live agents smarter after they ship in 1 hour 21 minutes - better than $2500 agent evals bootcamps.

collect traces from your live agent -> score them with an eval -> run GEPA to auto-optimize the prompt -> deploy the winner -> repeat forever.

That loop is how Pydantic AI agents self-improve while running in production.

Pydantic AI + Logfire tracing + GEPA prompt optimizer + evals + feedback loops - that's the stack.

Watch and save it, then wire GEPA into your live agent this week.

[Gergely Orosz @GergelyOrosz]

Knowing how LLM contexts work and how to work around context limitations – aka “context engineering” – is becoming so important. No better person to explain than @dexhorthy

Timestamps:

00:00 Intro
01:33 Dex’s path into tech
03:34 Early work in platform engineering
05:28 Replicated
11:24 Metalytics
12:36 12-factor agents
18:27 Context engineering
23:38 Harness engineering
26:11 Context overload
30:45 Loop engineering
44:34 Software factories before and after AI
50:33 Automation limits
55:18 Three options for automating
59:00 RPI framework
1:04:16 Intentional compaction
1:11:48 Token harder vs. token smarter
1:16:44 AI slop
1:19:15 HumanLayer
1:29:09 Book recommendation

Brought to you by:

• @AntithesisHQ — with Antithesis, you can use AI agents to work on critical systems without worrying about correctness. Teams like Jane Street, https://t.co/MTEw4IA0qA, and the etcd community use Antithesis to ship better code, faster. https://t.co/AKYm4cbVCU

• @buildkite  — the CI orchestration platform built for reliable scale. Used by OpenAI, Anthropic, Cursor, Meta, Uber, Ramp, Nvidia, Airbnb and many more. https://t.co/NOnTAHsCDD

• @sentry — application monitoring software built by developers, for developers. Check out their AI agent, Seer AI, and Sentry MCP. https://t.co/Ig2WcWttl3

Three interesting learnings from this episode:

1. Lesson learned: Shipping unread code spells disaster within months.

Dex experimented with having the model write the code and humans not reviewing anything in July 2025. Four months later, they shut things down and threw the whole system out. Production broke, and no matter how much the team prompted Opus 4.1, the model could not find the root cause. Once fixed, it took three weeks (!!) to re-onboard to a codebase no human had ever read

2. Context engineering 101: figure out where the “dumb zone” begins.

As a rule of thumb, the less of the context window that is used, the better the outcomes are. This is because the attention mechanism is quadratic: the more that goes into the context window, the more compute is required to process it all.

3. “You’re completely right!” or “you’re right to push back on that” are phrases that mean it’s time to start a new session.

These responses mean the LLM session is trajectory-poisoned, and you’re wasting time and tokens to continue. This is because models are autoregressive.

[Moysei @0xMoysei]

Andrej Karpathy quietly published 9 rules for building AI agents.

Rule 1: stop writing prompts.

"If you find yourself iterating on a single message at 3 in the morning, you are still in the prompting era."

A friend who runs agent infra at a trading firm read it once and deleted half the harness his team built last quarter.

The whole paper argues most agents die from a weak harness, not a weak model. Everything you added to compensate for the model becomes dead weight the moment the model improves.

The rule near the middle, about letting the loop delete its own work and start over, is the part he screenshotted. It contradicts how almost everyone builds right now.

The closing section on where the bottleneck goes next is the whole paper in one line.

He said he read it twice, second time with his own repo open beside it.

Everyone is still tuning prompts. Karpathy already moved on.

[ompragash @ompragash_v]

Kubernetes is not the answer to every container problem.

For a lot of real-world infrastructure, @Podman_io Quadlet is the better move.

It gives you something infrastructure teams actually need: a clean, declarative way to run containers as services without turning everything into a platform project. No sprawling scripts. No brittle startup logic. Just a readable file, system-managed lifecycle, and sane defaults.

That is what good infra looks like to me.

Not more abstraction for the sake of it. Better abstraction where it matters.

And the bigger lesson is even more important: stop forcing one operating model everywhere. Linux has systemd. macOS has launchd. Use the native primitives. Build with the grain of the platform.

The best tooling disappears into the background and lets engineers focus on the system, not the ceremony.

That is the kind of simplicity worth keeping.

[Thariq @trq212]

I’ve found the most important part of working with Fable is discovering my own unknowns so I can prompt it better, heres how I do that.

[sysls @systematicls]

https://t.co/hYv4BM5Sz6

[Addy Osmani @addyosmani]

https://t.co/oxHTdKwDJa

[Andrew Ng @AndrewYNg]

“Loop engineering” is a hot buzzphrase after mentions of it by Boris Cherny (Claude Code’s creator) and Peter Steinberger (OpenClaw's creator) went viral on social media. Loops are now a key part of how we get AI agents to iterate at length to build software. In this letter, I’d like to share my 3 key loops, shown in the image below, for building 0-to-1 products. These loops guide not just how I build software, but also how I decide what software to build.

Agentic coding loop: Given a product specification and optionally a set of evals (that is, a dataset against which to measure performance), we can have an AI agent write code, test its work, and keep iterating until the code is bug-free and meets its specification. This idea of closing the loop took off around the end of last year, and it has been a game changer in enabling coding agents to work longer productively without human intervention. For example, over the weekend, I was building an app for my daughter to practice typing, and my coding agent could easily work for around an hour, using a web browser to check what it had built multiple times before getting back to me, without needing my intervention.

The engineering loop executes quickly. Every few minutes, the coding agent might build and test a new version of the software. I hear frequently from developers who are finding new ways to engineer more effective engineering loops. This is an active area of invention!

Developer feedback loop: In this loop, a developer examines the current product and steers the coding agent to improve it. Last year, a lot of developers (including me) were acting as the QA (quality assurance) function for our coding agents, manually finding bugs and then asking the agent to fix them. But with coding agents much more able to test their own code, the amount of time we need to spend on this function has decreased significantly. This allows us to make higher-level product decisions, such as what key features to offer, where the UI needs improvement, and so on.

The developer-feedback loop operates over time intervals between tens of minutes and hours — that's how frequently a developer might review a product and give feedback. In the case of the typing app, I changed my mind a few times about the visual design, what cat costumes she can unlock as she learns (she loves cats), and the user flow for a grown-up to log in and steer the child's learning experience.

When a developer has a clear vision for what to build, it is still a lot of work to translate that vision into a specification for a coding agent to implement. Further, after the developer has seen an implementation, they might update (or perhaps clarify) the spec to steer it toward what they want. If you find that the system repeatedly runs into certain problems, building a set of evals for the agent becomes useful.

AI-native teams are increasingly using AI to help shape product direction, for example, automating the gathering and analysis of usage data, summarizing written and verbal customer feedback, or carrying out competitive analysis. However, for pretty much all the products I’m involved in, I see humans as having a significant context advantage over current AI systems — we know a lot more than the AI system about the users and the context the product has to operate in — and thus humans play a critical role. Many people describe this human contribution as “taste,” but I prefer to think of it as humans having a context advantage, since that gives us a clearer path to helping AI systems get better. This also speaks to why this step can’t be automated: So long as the human knows something the AI does not, human-in-the-loop is needed to to inject that knowledge into the system.

External feedback loop: This includes a wide range of tactics like asking a few friends for feedback, launching to alpha testers, or putting the code into production with A/B testing. These tactics are usually slow, rarely taking less than hours and sometimes taking days or even weeks. This data informs the developer vision, which in turn continues to drive the detailed product spec, which in turn drives the coding agent.

With coding agents speeding up software development, more engineers are starting to play a partial product management role. For many engineers who are growing into this role, the hardest part is shaping the product vision and striking a balance between building (bridging the gap between vision and spec) and getting user feedback to evolve the vision. It is important to do both!

I will write more about how to do this in future posts, but for now, I find it encouraging that engineers are playing an expanded role (just as product managers and designers now do more engineering).

[Original text: The Batch]

[OpenAI @OpenAI]

Windows users, this one’s for you.

Computer use now works on Windows, so Codex can take action on your Windows computer.

And with Windows support for Codex in the ChatGPT mobile app, you can start, review, and steer tasks on the go while work continues on your Windows machine.

An early experience, but we’re working on more ways to keep your work moving, wherever you are.

[Bilgin Ibryam @bibryam]

The 8 Levels of Agentic Engineering
https://t.co/7aGzJCQIVG

[Quarq @quarqlabs]

https://t.co/Cs80smDjlJ

[Elon Musk @elonmusk]

True

[ompragash @ompragash_v]

Seeing some positive feedback for Grok Build! Have you considered making it available to X Premium Subscribers? @xai

[Jeffrey Emanuel @doodlestein]

I finally got around to making a skill a lot of people have been asking me for:

https://t.co/3APwJYevpF

It basically helps to "de-slopify" and refactor code that's been written by agents, looking for ways to simplify and reduce the amount of code without changing the behavior.

The difference between this and other skills or prompts in the same spirit is the lengths this one goes to in order to prevent the process from going off the rails and introducing bugs or security problems.

It's a whole elaborate system spanning 98 files and one full megabyte of reference files, scripts, and subagents (see pic).

You can run it over and over again and it will autonomously identify good opportunities for accretive simplification and do everything needed to implement the changes and prove that they didn't change the outputs.

GPT-5.5 can explain better than I can how it does all that and what makes it so compelling and useful:

---

The strongest thing about this skill is that it treats refactoring as a proof obligation.

A normal “clean this up” prompt invites the model to follow taste. It sees repetition, long files, wrapper functions, stale types, try/catch clutter, _v2 files, and it starts cutting.

Sometimes that works. Sometimes it silently changes error semantics, loses a side effect, removes a lifecycle hook, or deletes a file that looked unused but was actually the intended
implementation path.

This skill changes the frame. A simplification claim becomes: “this smaller program is observably equivalent to the larger one.” Then it makes the agent prove that claim.

It starts with a baseline: tests, golden outputs, LOC, warnings, complexity. It maps duplication instead of eyeballing it. It classifies clones, because exact copy-paste, parametric duplication, semantic similarity, and accidental rhymes are completely different things.

It scores each candidate by expected LOC saved, confidence, and risk. Low-score candidates get rejected and logged, which is important because future agents otherwise rediscover the same bad idea forever.

The isomorphism card is the key move. Before editing, the agent has to answer boring but lethal questions: same ordering, same errors, same logs, same metrics, same side effects, same async cancellation behavior, same React hook identity, same serialization, same resource lifecycle. Those rows catch the kind of bugs that compilers and ordinary tests miss.

Then the edit discipline is deliberately narrow: one lever per commit, no rewrites, no sed, no drive-by fixes, no deletion without explicit permission. Afterward, it verifies behavior again and records the result in a ledger. If the refactor did not actually preserve behavior, it does not get to call itself a refactor.

What I like about it is that it matches the real failure modes of agent-written code. AI code tends to accumulate plausible junk: defensive branches for impossible inputs, duplicated wrappers, too many optional parameters, orphaned “improved” files, shallow happy-path tests, stale types, and comments that are really leftover task plans. The skill has a whole pathology catalog for those patterns, plus scripts and subagent roles to find them systematically.

So the compelling part is not “make the code prettier.” The compelling part is leverage with brakes. You can send very strong models into messy codebases and ask them to reduce complexity aggressively, while forcing them to preserve the contract that matters: observable behavior. That is the difference between a refactor you hope is safe and a refactor you can audit.

[Paul Iusztin @pauliusztin_]

Every day, 100+ people ask me, "How can I learn AI evals?"

I copy-paste these 11 links (every time):

1. AI evals & observability (series): https://t.co/erSJcqpAV7
2. Using LLM-as-a-judge: https://t.co/xMBt9j4JRc
3. Demystifying evals for AI agents: https://t.co/HBbCe5PnXJ
4. There are only 6 RAG Evals: https://t.co/gwfyhIozqK
5. Evaluation-driven development: https://t.co/GMtp6bewol
6. Binary evals vs. Likert scales: https://t.co/WyMw1hHTfm
7. The mirage of generic AI metrics: https://t.co/ugryF5zfKO
8. Error analysis: https://t.co/OXgPZd8IXi
9. Carrying out error analysis: https://t.co/OXgPZd8IXi
10. Evaluating the effectiveness of LLM-evaluators: https://t.co/NuaXhr19TV
11. LLM judges aren't the shortcut you think: https://t.co/fDep2HFjCq

Binge these to skyrocket your skills.

[Bilgin Ibryam @bibryam]

Claude Code Skill for Generating Architecture Diagrams with Excalidraw
https://t.co/W2xTPqifTJ

[Adolfo 🦀🔺 | OpenCrabs.com | truelens.tech®️ @AdolfoUsier]

Again no one is talking about this but @nvidia is offering a bunch of completely free API endpoints.

They just dropped MiniMax M2.7 free.

I mean literally, completely FREE!!!!

And you can easily setup it as custom provider on @opencrabs

Grab your key here:
https://t.co/NJoorufZjH

On config.toml paste this or just give to @opencrabs opencraw, hermes, or any harness to setup for you:
[providers.custom.nvidia-minimax27]
base_url = "https://t.co/bCOWJRDD2i"
context_window = 200000
default_model = "minimaxai/minimax-m2.7"
enabled = false/true

On keys.toml paste:
[providers.custom.nvidia-minimax27]
api_key = "your_nvidia_api_key"

And boom you are all set to go, at 0 costs!

Kimi is there since weight release so think about it.

I'm still handling all complex tasks to Qwen 3.6 Plus but for fallback I have just setup Nvidia with MiniMax M2.7, why pay if you have it free?

And also GLM 5 btw xD

Can't wait for 5.1 and Qwen 3.6.

Meanwhile enjoy it!

[Gabriel Chua @gabrielchua]

Here’s how we use Codex to:

> understand large codebases
> review PRs faster
> build macOS apps
> turn Figma into code
> automate bug triage
> create a CLI as agent tools
> analyze datasets
> generate slide decks
> coordinate new-hire onboarding
> learn a new concept

…and more.

https://t.co/9vCkSNjfjB

@kagigz , @Dimillian , @nickbaumann_ and team pulled together a great collection of Codex use cases, and based on how engineering *AND* non-engineering teams build with Codex daily at OpenAI.

What else should we cover?