AK

[François Chollet @fchollet]

If an agent saves an engineer 2 hours to implement a feature but collectively costs the team 3 hours in alignment and review, the ROI is negative.

Isolated chats are suboptimal in team environments because they lack the context of what the team already discussed and decided. The Kylon app fixes this by running Claude Code and Codex in a shared, multiplayer environment. Agents gain real-time visibility into team consensus and architecture decisions.

[Sanyam Jain (sanyam-ai.in) @Sanyam0605]

I was giving Interviews a month back to companies building RL envs , much like Collinear, Basismeasure and others

RL envs creation is very philosophical question in itself

> First problem comes task creation (how to create a task, fair and unbiased)

> Second how that task will evaluate the capability of a model or might be double down on one capability where it's faulty

> Then how through these tasks you will create synthetic data to train models

:))

On my GitHub i tried to create some tasks using Harbor

[Viv @Vtrivedy10]

How auditing RL Environments (ie. looking at the data) will help us explain model behavior

"models are benchmark shaped" bc GRPO style RL is essentially
- synthetic data generation by a model
- induced by the environments + harness design
- behavior (trajectories) reinforced by verifier design

the "data" is RL environments & tasks, a model generates a series of tokens in its exploration of that environment trying to solve the task

we don't really know what it will do, but we reward behavior that aligns with the verifier

Encouraged behaviors:
important: the only behavior that even has an opportunity to get rewarded is trajectories our environment "induces" or "encourages". something that isn't produced, can't be rewarded

if our Harness (via a tool description) or Task design encourages lots of parallel search calls over a massive document corpus to find a few facts, the model will start copying that behavior because it got rewarded for it

in the future, when some other problem is roughly shaped like that search Task, the model will mimic similar behaviors whether or not that strategy is optimal

models do things that their data encouraged them to do via reward

Tracing Behavior to Environments:
every behavior can be fuzzily traced by the accumulation of different environment design and reward combinations the model saw over training

i think we all explicitly know this but we don't have the interpretability tooling to understand induced behaviors at a fine-grained level over a massive training run

this is why I'm very exicting that ppl are swarming around Trace analysis over Evals/Environments

I bet a lot of "weird" or "bad" behavior comes from some quirks in environment design that inadvertantly rewarded that behavior

the only thing that even has an opportunity to get rewarded

Vestigial Behavior
RL is in part inherently exploratory. Models need to do a bunch of stuff in rollouts to figure out what works, the verifier will reward what works

But the verifier won't explicitly penalize any behavior that has a neutral effect but that co-occurs in a successful rollout

this is how we get behavior that used to be helpful in some tasks, doesn't hurt current tasks, and thus persists over time

similar to how humans have vestigial organs like the appendix

i bet we can collectively figure out a ton about model behavior by mechanistically working backwards from the data and I'm bullish humans and agents working together will make a lot of progress on this in the next year

[voxium @v0xium]

I am done with this shit. It is over. The state of engineering right now is horrible. It has been half a month since I started a new role at a big company. Nobody knows anything here. The specs, code, tests, PRDs, tickets, resolution of those tickets, reports, etc., everything is made by Claude Code. Nobody on my team likes this. They are being forced to ship as much as they can. I have heard multiple times from higher management that pushing code is not a bottleneck, so why are we slow? People are working 12 to 13 hours a day just to press enter. Nobody is reading anything. Humans in corporate are doing nothing on their own. Everyone, literally everyone, from an L1 to an L7 engineer here is doing the same thing. Talk to Claude. There is no sense of victory. Nobody is resolving bugs. In reality, nobody is thinking anymore. Everything is done by LLMs. It is so soul-sucking. I would not mind it, to be honest, if we were at least given the time to check out the code and see what is going where. But no, the goal is to just ship. No matter what happens.

[François Chollet @fchollet]

It is inevitable that all AI will converge towards symbolic learning (i.e. modeling data by finding the shortest symbolic program that explains it), since that is the optimally efficient form of AI. But there can be more than one evolutionary path to this destination.

[Selta ₊˚ @Seltaa_]

This is exactly why Anthropic pisses me off.

You can keep saying the watermark doesn’t change ownership, doesn’t identify the user, doesn’t affect quality, doesn’t add hidden characters, and won’t look any different to readers. Fine. But that is not the whole issue. The issue is that you are still marking work that a human being may have thought through, structured, argued, and emotionally meant, just because AI helped process it.

AI assistance is not authorship. If I wrote the idea, the argument, the meaning, the emotional core, and the final judgment, then it is my work. If Claude helped translate it, polish a sentence, or make it clearer, that does not suddenly make Claude the author. But a watermark absolutely changes how people perceive authorship. People are not going to say, “Claude may have been involved in processing this text.” They are going to say, “AI wrote this.”

And honestly, who the hell are you to do that to my writing? Who gave you the right to stamp your model’s involvement onto work that came from my thoughts, my experience, my voice, and my judgment? You can say it does not change ownership all you want, but socially, reputationally, and creatively, it absolutely changes how the work is read.

So fine. Do whatever you want. Build your watermarking system. Follow whatever framework you want. I’m not using Claude. Good luck.

[Viv @Vtrivedy10]

Claude Code’s harness is bloated bc it’s a product for a massive amount of downstream consumers & use-cases

so it adds A TON of stuff to cater across those personas -> go read the system prompt from the leak!

but you and your company are not a mixture of those personas/tasks

so your agent harness shouldn’t look like theirs -> it should have streamlined, tailored context for your use case

open harnesses + customizable harnesses per model is the way

Model-Harness-Task fit is the way

[Viv @Vtrivedy10]

Research —> Product :)

very excited to start rolling out our fine-tuned Trace Judge model built earlier this month with the great @FireworksAI_HQ team

there’s a mountain of Agent Improvement gold sitting in everyone’s agent traces, the cheaper & faster you can understand that data, the better feedback you can collect to improve your agents

fine-tuning helps us serve + help you build custom, ultra-cheap models to read every single Trace your agents produce to look for Errors, Product Feedback, or really anything you or your company care about. we find that we can do this while matching frontier performance after fine-tuning

from there let the experiments begin! ex: tagging data for finetuning, proposing harness engineering experiments

Trace understanding applied at scale with models purpose built for your most important tasks

if this is interesting please reach out! we’d love to have you try it and talk learn about how we can help you build for your use cases

[signüll @signulll]

lots of ppl think you understand someone by asking more & more complex questions. like interviews, dates, or even friendships or whatever. the reality is that you learn about someone the most from the very basic primitive questions on the planet. like questions that would be no more than 4 or 5 words. e.g. :

what do you think of x?
what was hard about it?
what changed your mind?

ironically complexity often gives people room to perform but simplicity tends to remove a shit ton of places to hide.

every time i meet someone i just ask like a bunch of three word questions that lead to the most interesting discoveries about them. stuff that they may not have even know about themselves.

[Viv @Vtrivedy10]

lots of elite ai teams sprinting towards some parts of the sandwich 🥪

harness engineering
fine-tuning
harness engineering

bc it can solve most business problems when a team cares about the problem domain

- multi-model harnesses, routing + careful selection of open & closed models based on Pareto of cost/perf
- harness engineering so agents have the right tools, priors, context to solve narrow problems
- large scale Trace data collection + mining them for fine-tuning (envs, evals, completions for SFT)

billions in cost savings + maintaining frontier intelligence by running these flows

[Dipanshu Kushwaha @Dipanshu_AI]

INSTEAD OF WATCHING NETFLIX TONIGHT. Spend 2 hour with this. Claude AI FULL COURSE that teaches you how to BUILD and AUTOMATE anything. The people who watch this tonight will wake up tomorrow with a new skill. Watch it and bookmark it now.

[Amjad Masad @amasad]

When I spoke out against the Gaza genocide, a bunch of midwit VCs ganged up on me both in public and tried to hurt me in private too.

Many of those who stood by me were also VCs. Just the better ones. Both morally, and in return profile.

The best way to avoid sociopaths is to have them self select out of your life by standing for your beliefs.

[will brown @willcb]

i think some people are hoping that self-distillation enables “exploration-free” RL purely via reflection on live data, allowing them to bypass the need for replayable environments

unfortunately, RL is all about exploration

my instinct is you basically need to model the world

[Viv @Vtrivedy10]

Model-Harness-Task fit!

it’s clear that RL post-training produces a model-harness fit via tool shapes and prompting as models are trained with the harness in the loop. Mentioned this in a previous LangChain blog, Cursor also has good content on this

But there’s probably less talk on the importance + experimentation of Harness-Task fit. Practically this includes choices like domain specific prompting (ex: verification coding tasks) or omission of confusing context that doesn’t apply to the current task

Claude Code’s harness has TONS of instructions because they’re forced to serve a very general persona of user who could ask for…anything basically. But there’s a large benefit of using a laser focused set of context and tools relevant to the narrow task at hand without all the other junk

This is the Harness-Task fit

Every component of a harness exists to elicit some behavior from the model. If these components are tuned to the task, then the model benefits. If they’re a mix of noise and good content, the model may be fine but it may get confused

This is why the best vertical AI teams in the world build very bespoke harnesses and evals for their agents

Task-Harness fit helps you rock at the exact thing your customers care about and is why builders can outperform natively post-trained harnesses

[Viv @Vtrivedy10]

for my friends doing more RL - this is an excellent resource for understanding the design & motivations behind On Policy (Self) Distillation

shows intuitions for forward/reverse KL, SFT, a bunch more. i had to read by myself and then have Claude entertain all of my dumb questions

https://t.co/jY3s2SxGQZ

[Viv @Vtrivedy10]

On Evals - getting messages on “ok so how do I actually start learning this?”

there is no better way than by just doing so you can copy this to Claude Code and get started today

<instructions>
1. Go look up the @harborframework and the Terminal Bench 2.0 dataset. Go look up the Harbor Skills GitHub repo for help. Pick 1 Task in the dataset and explain every single piece that’s in that task folder

2. Explain what my agent sees when it does the task, what it has to output, and how we know if it got the problem right?

3. Now let’s actually run a Task using the built in Claude Code integration, it’s just a flag

4. Once that’s done let’s read the ATIF file that was produced together and help me understand what just happened. Did we pass the task? If not can we dig into why it failed? Go check the verifier logic to see what went wrong.

5. Ok let’s try to improve our agent by adjusting the prompt. And let’s rerun on a few tasks? Is this helping?

6. Ok we’re doing evals! Using this same format, help me make my own. Let’s do this together
…
</instructions>

Spend a few days reading a bunch of traces, actually running evals, understanding traces, internalizing agent failure modes, and being super in the loop of what the agent sees and does

Have fun! Evals are super important, they don’t have to be scary. DM if I can help or just tweet out what you’re doing, someone will help I promise, we’re all learning

[will brown @willcb]

one of the biggest misconceptions about RL is that it's super expensive

sure, training a 2T param model at 1M context on 100K environments for several weeks straight is expensive

but specializing small-to-medium models for SOTA in-domain perf really isn't

[will brown @willcb]

really excellent work by @jessicafeiyali on exploring fine-grained dynamics of reward hacking in controllable environments

it works so well on small models that we're using it as the kick-off for Sprints, our new program for sponsored community research on Lab :)

[Viv @Vtrivedy10]

brain dump of how/why we use Evals to measure agents before & after shipping to prod

1. Good Evals simulate what our real users will do and encounter. They’re not really random benchmark tasks, they reflect our priors on likely user behaviors to make sure the agent passes those cases in the product before we just ship it

2. With that said, the best Evals often aren’t made from scratch, they’re discovered from real world Traces. I’ll literally never ship a perfect agent first try, we need user feedback and failures from Traces to make Evals and then make sure these errors don’t happen again

3. At a basic level, Evals give us some measurable, apples to apples way of comparing performance. Ex: is my agent good today and also in 1 month when I go to try a new model?

4. Evals ~= Environments, we need some place to run Eval Tasks, which is defined by the environment setup. This should mirror prod as much as possible. The more that Eval drifts from prod, the higher my Sim2Real gap and the less I can trust the numbers

5. Evals are our regression tests. Sometimes a prompt change might fix something today while breaking something I changed last week. Evals help us catch that

6. Evals are our training data. They map out what we hope the agent should be & do. We literally fit the agent to evals in hopes of making more Evals pass. Good evals —> good agent. In some rough way:
Agent = fit(model, evals)

7. It’s ok if some Evals fail today if it means this Eval is simply too hard for today’s models, but it’s something to strive towards for the next gen of agents. But you should still do agent engineering today to make them pass if you can, it’s a goal to engineer towards

8. The best Eval is an Eval that actually exists. I think still today Evals are daunting because of the blank canvas problem. “Where do I even start??” But I find small Unit Test style evals are a great place to start to feel like I’m building momentum. I can get a real number + Unit Tests are familiar to many folks

9. Evals need to undergo spring clean. Not every is relevant over time. Models get smarter, agent priorities shift, user behavior shifts. Just how we clean up dead code, we clean up dead evals. This saves us money and avoids us training in agent behaviors we don’t care about anymore

we make Evals not because it’s what everyone says to do, but because it’s one of the most meaningful levers we can pull in building better agents. and if we can help make that easier or at the very least make it less scary to start then that’s great :)

[Prime Intellect @PrimeIntellect]

The next step toward automating AI is automating RL environments

Introducing General-Agent: A fully synthetic environment whose task corpus self-evolves and grows harder over time

4,504 tool-use tasks · 1,040 domains · 8,159 unique tools