Theodoros "Theo" Kokosioulis

Senior software engineer. 10 years shipping. Now figuring out what coding agents are actually good for.

[lauren @poteto]

i feel like swe interviews could probably collapse into just 2 technical rounds:

• system design (how well can you articulate your ideas, do you actually know how to engineer something)
• onsite project to build a real thing with agents (can you translate your intent into high quality outcomes)

everything else we used to ask is no longer necessary

[Hamel Husain @HamelHusain]

Q: Why do you recommend binary (pass/fail) evaluations instead of 1-5 ratings (Likert scales)?

A: Binary labels force clearer thinking and more consistent labeling, and involves significantly lower complexity to operationalize.

https://t.co/dKYUiOnfNN

[Sam Altman @sama]

dot is my favorite openai product so far!

it is amazing to me that each day it feels noticably better as it learns more of my workflow and style.

having it do the stuff i don't like doing--and usually just builds up as a gravity well of dread--has me very happy.

[Mark Ajzenstadt @mardehaym]

I think internal evals will become one of the most valuable AI assets a company owns.

A good evaluation captures something specific about the business: what doing the job correctly actually means, including the exceptions its most experienced people have spent years learning.

Coding agents have an advantage here. Software teams already have tests, build systems and code review. Those checks are imperfect, but they give agents feedback they can use to improve.

Taking agents into operations, finance or customer service means building an equivalent for those workflows.

Did the agent apply the right refund policy? Match the payment to the correct account? Recognize an exception that needed a person’s approval?

To answer those questions consistently, you need real cases, outcomes your operators agree are correct, and an environment where the agent can attempt the work.

That gives you a repeatable way to improve performance. Run the agent, inspect its failures, change the instructions or tools, then check whether the changes also work on cases it hasn’t been optimized against.

Anthropic describes doing this with its internal Slack and Asana tools, improving performance beyond its initial expert implementations and checking the gains on separate test cases.

The improvement comes from acting on what the evals reveal. Running the tests is only one step.

Over time, that process can turn operational experience into an asset: a growing collection of real situations, expert judgments and checks that help you improve each new version of the agent.

You can buy evaluation software. Building that collection takes access to the business and the people who understand it.

As models become more widely available, a company’s ability to define and verify good work could become a lasting advantage.

[Cursor @cursor_ai]

GLM 5.3 and GLM 5.3 Flash are now available in Cursor!

GLM 5.3 Max is the best-scoring open-weight model on CursorBench 4.0.

[Gipp 🦅 @gippp69]

AN ATLAN ENGINEER JUST SHOWED WHY A PRODUCTION AGENT SHOULD KNOW WHEN TO GIVE UP

only 14% of incoming alerts ever reach the agent. instead of making intelligence the default path, Atlan makes it the exception and forces routine work through cheaper deterministic logic first.

context → loop → Jev → harness → evals

3 attempts is the ceiling. stay below 50% confidence after that and Python takes the wheel, which turns uncertainty into a designed branch instead of an endless reasoning bill.

zoom out and the same logic applies across all 5 layers: shrink what the model sees, gate what deserves reasoning, restrict what it can touch, then replay failures until the weak layer becomes obvious.

that’s what makes the architecture interesting to me. the upgrade isn’t giving an agent more freedom, it’s engineering exactly where that freedom begins and ends.

[Andrej Karpathy @karpathy]

Cool eval. Simply ask an LLM “Land or Water?” and give it a latitude and longitude coordinate as text. Ask 16,200 times, plot as image. The models know. From compressing the internet.

[Zach Lloyd @zachlloydtweets]

Agentic engineering is really two disciplines now:

- Product engineering: ensuring the quality and usefulness of product
- Factory engineering: deploying agents to build that product, optimizing system efficiency and output quality

Here's how to adapt: https://t.co/XylpSn7MkG

[Andrej Karpathy @karpathy]

We'll be spending a lot more time trying to understand the outputs of language models. A few thoughts, tips & tricks:

Writing. Something I've had success with: Ask your LLM to explain something in ASD-STE100, it's a controlled language specification originally developed for aerospace maintenance documentation. LLMs well-versed in this language and it comes with heavy constraints on clean writing style that I often find a lot more readable. Sometimes I've tried to soften it a bit e.g. ask for "80% of the way to ASD-STE100" because the spec is quite stringent. But even better:

Diagrams / images. Instead of writing, ask your LLM to create a diagram. These can be a lot easier to process, parse, and understand. But even better:

Web pages. Ask for output "in HTML" to get a beautiful, interactive webpage. LLMs are getting really good at frontend and can create beautiful experiences, animations, etc. But even better:

Explainer videos. The output format I am most bullish on is fully custom / bespoke explainer videos generated on any arbitrary topic. Experiment with things like "Create a 3b1b style video explainer on X. Use my ElevenLabs API key for audio narration". (you'd need an API key for the latter or you can ask your LLM to find you decent free alternatives that use your local compute). This is actually starting to work!

In summary:
- As LLMs get better, they will do more and more of the legwork autonomously, and a lot more of our work will rise up the abstractions into oversight and understanding.
- Luckily, LLMs can help here too because as intelligence and code are increasingly abundant, you can ask for large, custom, discardable software artifacts (e.g. web apps, video explainers) that would have never made sense to create before. Push the boundaries here and you'll be surprised.

[arXiv Bangers @arXivBangers]

CliffCompaction: Cost-Efficient Compaction for Long-Horizon Coding Agents

CliffCompaction cuts agent costs by up to 50% by dropping or truncating original context without rewriting, preventing context drift across million-token sessions.

https://t.co/3t9fulXPaU

[Mauro Erta @mauro_erta]

“Your agent is stupid.”

We got that feedback while our evals were passing ~100%.

That failure taught us more than the tests did.

I wrote down a few lessons from years of building AI agents in production: evals, feedback loops, and why a good model isn’t enough.

[Sam Altman @sama]

I think there is much more potential energy in Sign In With ChatGPT/Plugin Extensions than we realize

[Hamel Husain @HamelHusain]

Q: Should I build automated evaluators for every failure mode I find?

A: No. Different types of evaluators have different costs (code vs. LLM judge), which you need to weigh before building an eval.

https://t.co/knHmTMH4DP

[Puneet Patwari @system_monarch]

If you're a backend engineer scared of AI, you already know half of it:

- Caching → semantic caching
- Rate limiting → token budgets
- Load balancing → model routing
- Idempotency → safe agent retries
- Timeouts → streaming + partial answers
- Monitoring → evals and traces
- Input validation → prompt injection defense

The half nobody teaches you:

- deciding what "correct" means
- managing context, instead of just prompts
- cost per successful task instead of cost per call

Backend engineers aren't behind in AI. They're halfway there.

[Cursor @cursor_ai]

Innate is a team of 7 building capable robots for everyday life.

"The seven of us here do the work of 50," says Vignesh Anand, co-founder of @innate_bot.

We spent time with the team to capture what building means to them.

[Prince Kushwaha @AIWithPrince]

Half the price per token can mean twice the price per task.

A failed step isn't just one retry: every later turn re-reads the error too.

In one 3,500-run study, failed coding-agent runs used 1.8x the tool calls but 2.7x the tokens.

Budget on cost per successful task.

[Poorvith M P @poorvith_mp]

There is a question that keeps bugging me: how did frontier labs give us 1,000,000-token windows only for coding agents to get slower, dumber, and trash git trees after turn 30?

Here is why. 👇 https://t.co/gyxAbyd42v

[Rahul Kumar @RahulTechX]

Most people are still prompting AI like it’s 2023.
I was too.

More instructions ≠ better results.

The real shift is from “writing better prompts” to building better AI systems.

I broke down what’s changing in prompting for 2026 👇 https://t.co/JvfJXmSbEH

[Tech with Mak @techNmak]

If you are trying to understand where AI agents are going, learn harness engineering.

A capable model is only one part of an agent system. Once the model begins reading files, calling tools, modifying state and working across many steps, the quality of the system depends increasingly on the software around it.

Consider a coding agent working through a large repository. The model can decide that it needs to inspect a file, search for a symbol, make an edit or run a test, but those decisions do not execute themselves.

The surrounding runtime has to decide which resources are available, whether the requested action is permitted, how the operation should be performed, what result should be retained, and what information should be presented to the model on the next step.

This becomes harder as the run gets longer. As history accumulates, replaying everything can become costly and less effective. The harness has to decide what should remain in context, what should be summarized or retrieved later, and what belongs in persistent state outside the context window.

Execution has similar problems. A long-running agent may need to survive an interruption, avoid repeating completed work, enforce permissions around consequential actions, and preserve enough history to reconstruct what happened when the final result is wrong.

These are harness problems.

The harness is the layer that manages context, tools, execution, state, checkpoints, limits and traces around the model.

Harness engineering is the work of designing and improving that layer. Engineers inspect execution traces, evaluate agents on representative tasks, look for recurring failure modes, and then change things such as context selection, tool interfaces, state handling or execution controls.

That last part matters because agent failures are often not fixed by changing the model. Sometimes the useful change is in what the model sees, how a tool is exposed, what state is preserved, or what the runtime does after a failed step.

As agents take on longer tasks, the demands on this surrounding software grow. Model capability remains essential, but harness engineering is what turns that capability into an execution process that can be controlled, inspected, tested and improved.

[Victor Motricala @VictorMotricala]

This is the N+1 query problem, just moved into agents.

Left alone, a coding agent calls a decision API once per question, the way a lazy ORM fires one SELECT per row.

Batch the questions into one request and the bill drops.

TypeSafe measured 12.2x with 13 questions.