Aaron Ellis-Bloor

Post-Training @baseten

[Jake Broekhuizen @jakebroekhuizen]

Stoked to launch 𝚜𝚖𝚒𝚝𝚑𝚝𝚞𝚗𝚎 in public beta!

Post-training your agent's policy has never been easier, with training APIs from @baseten and @FireworksAI_HQ becoming increasingly accessible. But sourcing high-quality data to use them effectively remains a challenge.

LangSmith processes hundreds of millions of traces a day, making it a rich system of record for your agent’s behavior.

𝚜𝚖𝚒𝚝𝚑𝚝𝚞𝚗𝚎 turns those trajectories into the data you need for supervised fine-tuning, with an end-to-end pipeline for curation, preparation, training and deployment for models you want to SFT for specific tasks.

We care deeply about giving teams ways to own their own intelligence, and as the open model frontier continues to advance, it’s becoming increasingly clear that post-training on your own agent data will become a key part of that.

Excited to get this into people's hands!

[Aaron Ellis-Bloor @AEllisBloor]

Always extremely fun working with @Vtrivedy10 and @jakebroekhuizen.

Super proud of our partnership - you can now turn your traces into a fine-tuned model via @LangChain’s smithtune cli and @baseten Loops.

[Viv @Vtrivedy10]

Traces -> Curated Data/Environments -> Fine-tuning Infra -> Continuously Better Agents

always a pleasure cooking with the Baseten team (shoutout @AEllisBloor), our Labs team uses Baseten Loops to go from idea to model training experiment in legit minutes

the cycle of continuously improving your agents using your data gets more accessible every month

goal is to help every team own this loop with their custom data & models optimized for intelligence, cost & latency 🤝

[Viv @Vtrivedy10]

Traces -> Curated Data/Environments -> Fine-tuning Infra -> Continuously Better Agents

always a pleasure cooking with the Baseten team (shoutout @AEllisBloor), our Labs team uses Baseten Loops to go from idea to model training experiment in legit minutes

the cycle of continuously improving your agents using your data gets more accessible every month

goal is to help every team own this loop with their custom data & models optimized for intelligence, cost & latency 🤝

[Viv @Vtrivedy10]

The future is about continuously turning your valuable data & ideas into better/faster/cheaper models -> Owning your Intelligence

the Baseten team rocks at helping us run tons of experiments to make this happen @AEllisBloor @mudithj 🤝

A lot of work in our Labs team looks like:
- sitting down with a buddy @jakebroekhuizen
- picking a problem/agent to make better, cheaper, faster
- curating data + evals/environments
- doing tons of training experiments, with continuous measurement of every tweak
- Loop (pun intended)

the infrastructure and tooling is here to allow every team to run hundreds of experiments and get the best quality/cost/latency tradeoff for their customers and use-cases

also ngl training is fun and your agents will love the experience

[Niko @nikogrupen]

https://t.co/Fg8FniKQv3

[Charlie O'Neill @oneill_c]

Today we're launching Base Labs, a research lab by Baseten. Our mandate is to make open-source as useful as possible, and our one rule is that we publish without exception, including what fails.

Up until pretty recently I thought the way to get the world onto open models was to train them for one company at a time. @mudithj, @maxkirkby and I cofounded @parsedlabs on that bet, @baseten acquired us, and we spent the last year running their training team doing it for customers one by one. Every one of those engagements taught us something new about how models learn, forget, specialise and get cheaper, and almost none of it got spoken about. Sadly, in general that's the field's default in that the people who know the most about training have the least freedom to say it.

To be clear I don't think the closed labs are the villains here. They get to new capabilities first, which buys the rest of us time to harden the world before that stuff is everywhere, and they're the ones paying to find out what's actually possible. But I'm fairly convinced the only real advantage they have is data and scale, and their incentives point squarely at the frontier. You can't do slow, public science on how these things learn when your job is the best model by end of quarter. Someone without that pressure has to, and there are very few of those someones around. Hence Base Labs.

We have the broad remit of making open-source as useful as possible and our one rule is that we publish without exception, including what fails. The first problem is continual learning, which I have come to think is several problems wearing one name. We are also working on the open RL environments and data that open models need and currently can't get, because we have to aggregate data with the same ferocity everyone's been talking about aggregating compute. Plus a bunch of other stuff I'm genuinely excited about, eg a safety stack people can run on top of open deployments, and performance research so these things are cheap for everyone to serve.

I still think open and closed coexist, and that's the good world. It's just that coexistence isn't free, someone has to actually do the work, and this is basically what keeps me up at night. We're hiring researchers, engineers and fellows. Come help distribute the mandate of heaven!

[Shruti Goli @ShrutiGoli]

last night delivered.

@elise_ai x @baseten on betting on open source. three takeaways:

1/ fine-tuned open-source models can move the needle fast: 50%+ latency reduction, ~20x cost savings, and real quality gains. in production, not on a benchmark.

2/ evals are non-negotiable, but most teams are only doing half the job. you have to measure individual outputs and system-wide behavior. one without the other and you'll miss what's actually breaking.

3/ before you fine-tune anything, exhaust prompt and parameter tuning first. then build a retraining pipeline that can evolve as fast as your prompts, evals, and models do.

4/ the real win wasn’t just a better model. the process of fine-tuning forced a much deeper understanding of product behavior at scale.

🎥 full recording coming soon. 👀

great chatting with you all @TheRealAk914 @AEllisBloor Chris Wirick, Mario Martone 🚀🚀

[Charlie O'Neill @oneill_c]

If you want to RL (or sft/opd) glm-5.3-flash, DM me to get access to it on @baseten loops. We're very excited about this model as a great base for further RL for your task. Being smaller than kimi k3, dsv4 pro, glm 5.2 etc means you can really get a lot of RL out of it because inference is much cheaper. And the model has been redesigned around inference efficiency at long context lengths

[Aaron Ellis-Bloor @AEllisBloor]

@Vtrivedy10 "Look at the data", still the unpatched method

[LangChain @LangChain]

As model capabilities converge, more of the differentiation may come from the system around the model: the harness, tools, environments, evals, and feedback loops used to improve it.

Join @hwchase17, co-founder and CEO of LangChain, @Vtrivedy10 Applied Researcher at LangChain, @willcb , Applied Researcher at Prime Intellect, and Aaron Ellis-Bloor Applied Researcher at @baseten for a live panel session on Automating Eval & Environment Engineering.

We’ll explore the improvement loop and what open models and model-harness co-design mean for companies that want to own their intelligence.

Register now: https://t.co/Gbi5QpM1l5

[Akshay Ramaswamy @TheRealAk914]

Extremely bullish on the future of open source -- on the heels of GLM 5.3 dropping today, I'm excited to host a panel with @AEllisBloor, @ShrutiGoli and the @baseten crew on where we see the industry heading!

We'll be talking about topics like harness engineering vs fine-tuning, hill-climbing on the right evals, and the benefits / tradeoffs of open source vs using models from the leading labs

Not to mention, we'll be hosting at the brand new @elise_ai HQ 🚀

[Harry Partridge @part_harry_]

https://t.co/iJsDrlGy45

[Aaron Ellis-Bloor @AEllisBloor]

@ShrutiGoli @baseten @elise_ai @oneill_c Have loved working with everyone at Elise! What a great partnership

[Tuhin Srivastava @tuhinone]

https://t.co/aVf5uLG7j5

[Akshay Ramaswamy @TheRealAk914]

Huge congrats to the @baseten team, it's been super exciting seeing the work our research team has been doing with @oneill_c and @AEllisBloor to push the frontier

For me, it's completely changed how I think about fine-tuning and running our own open-source models -- we've seen massive gains at @elise_ai in reducing cost and latency at the inference layer of our voice orchestration

[Gabe Pereyra @gabepereyra]

Model strategy for @harvey:

We are working on the first model in our legal foundation model series, inspired by @cursor_ai's Composer. Two goals:

1. Allow us to serve frontier intelligence across our product surface areas at an affordable price and a strong security posture.
2. Create the foundations for law firms to build their own specialized models and own their own intelligence.

The model series will focus on complex client matters that span months and take dozens of associates. The agentic system will learn to control legal tech tools, sub agents and ask for help from frontier models or human partners, much like a senior associate.

We’ve open sourced benchmarks for evaluating our initial post training work that represents work done by associates and in-house lawyers. We are scaling these significantly using synthetic and human pipelines as well as building private evals for firms.

Open sourcing this data has allowed us to quickly validate the feasibility of post training open weight models for legal work. With our research partners we’ve already shown promising results post training open source models to approach frontier performance:

1. @baseten - novel compaction strategies for analyzing large data rooms.
2. @FireworksAI_HQ - matching frontier performance by using frontier as an advisor.
3. @appliedcompute - improving performance and reducing cost of large scale review tables.
4. @trajectorylabs & @nvidia - sovereign continual learning over client matters.

We plan to continue to invest heavily in working with research partners and open sourcing our data, models and research as much as possible. We believe open research in legal will be important to building trust in the frontier ecosystem.

We are also scaling our research team. Harvey Labs is our internal research group, responsible for pushing the frontier of legal intelligence and working closely with labs, research partners, and academia to bring the frontier of agent research into Harvey.

Labs is run by @nikogrupen and @ItsJulioPereyra - Niko worked on multi-agent RL at Google Brain and Julio clerked and worked in BigLaw. We believe this pairing is crucial for building frontier legal AI systems. Together they have already made significant progress in scaling our data and training efforts.

The long term goal of Harvey Labs is to contribute to the research and infrastructure required for the legal industry to create a frontier ecosystem. We believe that the best version of legal super intelligence is one where each law firm, enterprise and government owns their own specialized version.

We are hiring for Harvey Labs across the post training, agent and data stack and open to acquiring talented teams / neolabs in this space. If interested please DM me.

[Charlie O'Neill @oneill_c]

1/ You can shrink a language model's KV cache by 200×, in a single forward pass, and it still answers correctly.

At 256k context that's 36 GiB of cache down to ~360 MiB, with no change to the base model.

Here's how we did it 👇

[Niko @nikogrupen]

Pushing the frontier of open weight legal intelligence!

Our experimental results show that combining post-training and harness optimization brings open weight agents to performance parity with closed frontier models on our Legal Agent Benchmark.

Incredible collaboration with @oneill_c, @mudithj, @mblau_, and the @baseten team — lots more to come here, so stay tuned!

[Gabe Pereyra @gabepereyra]

https://t.co/VniSjWQbOI

[Charlie O'Neill @oneill_c]

https://t.co/ERAhd412li