Josh

mts @JudgmentLabs working on agentic search, hill climbing infra. on the side: building a realtime + reasoning minecraft agent, Jeeve.

[Josh @joshjwilkin]

Day 1 of building Jeeve (pronounced like Steve), a realtime + reasoning general purpose Minecraft agent, that plays with you.

You /chat Jeeve <do this> like you would message your coding agent, and Jeeve interacts with the world and learns how to perform any tasks. Being able to rewrite entire parts of its harness to hill climb on perception + learning.

Why?
1. Earlier Minecraft bots were either fast but task specific, or smart but slow (paused entire game to think).
2. Test the theory that decision models can bridge that gap to make the realtime interactions smarter, while reasoning can drive overarching actions without blocking responsiveness.
3. cool as hell (imagine: "Jeeve, build this base I saw on instagram <link>")

High level Design
This initial prototype of Jeeve is built on a System 1 + 2 approach.

System 1 / Decision model (Jev):
- handles real time reactivity + fight or flight decision making. Follows a current task (i.e chop down a tree). Also responds realtime to threats (mobs + hunger + health).
- can use skills / deterministic code controllers that system 2 + async tool builders create

System 2 (Opus 5.5)
- handle longer term planning tasks + handles interrupts on real time decisions needed by System 1 reacting to it's current environment.
- decides which tools Jeeve is missingand has Async Tool Builder build them as code controllers.

Async Tool Builder (Opus 5.5)
- builds low-latency code controllers that behave like control systems for small scoped tasks (e.g. efficient towering, bridge building)
- System 1 then calls this code at low latency, for the appropriate task, and it’s kept for next time.

ofc tracing, evals, hill-climbing using @JudgmentLabs

still lots of ideas to try, stay tuned!

[Mercor @mercor]

Congratulations @judgmentlabs on launching general access and thanks again for the @mercor ice cream. 🍨

[Akshay Ramaswamy @TheRealAk914]

Always blown away by how sharp and thoughtful @alexshander03 and the @JudgmentLabs are - they've taught me more about evals than anyone else in the industry!!

This is a company to watch

[Maude @carolinemaudee]

free ice cream and merch… what more could one ask for!!!! @JudgmentLabs

[Dakota McKenzie @dakatdgp]

@Dynamicfund is long on long-horizon agents and @JudgmentLabs ! As companies finally get more increasingly complex agents in production ranging from AI-natives to enterprises, this becomes an even more critical piece of infrastructure.

Congrats to @alexshander03 @JSCamyre @andrew_li03 on GA. Just getting started!!

[Zaza Pachulia @zaza27]

Judgment Labs is now GA. If you're building AI agents, you need to see where they fail before your users do, that's exactly what Judgment does. Self serve, sign up and start monitoring today. Big congrats to @alexshander03, @Andrewli, @JSCamyre and the team.

[Mercor @mercor]

Join us tomorrow in San Francisco for a free scoop of @Mercor mocha ice cream.🍦

Congratulations @JudgmentLabs on releasing general access and thanks for the sweet treat.

500 Howard St.
San Francisco
Wednesday, October 28, 11AM-1:15PM

[Andy Fang @andyfang]

@alexshander03 @JudgmentLabs congrats Alex on the launch!

[ahh @abhishekg999]

quick osint round 2

[Joseph Camyre @JSCamyre]

Every era of software gets the infra it needs.

Cloud apps got Datadog. You couldn't run prod without knowing what it was doing.

Agents are the next cloud apps. They run in prod and break in prod too. The difference is they generate data faster than any human could ever read it.

That data is how your agent gets better. Most of it ends up in logs nobody opens.

So we're building the infra to actually use it. That's Judgment.

[Josh @joshjwilkin]

@NathenFernandes reading upcoming blogs 👀

[Josh @joshjwilkin]

so free

[Alex Shan @alexshander03]

The @JudgmentLabs free ice cream truck is back in SF.

Tue, Wed, Thu this week. Go get some 🍦

Thank you to our partners! Today, we're serving @typesafeai jev jam, @modal À La Modal, @ExaAILabs EXA-presso.

[Enyu @0xhappier]

People are underestimating how much data a world full of agents will produce.

Long-horizon agents doing work everywhere, all the time. Their traces will dwarf the data we're used to handling today.

We've built Judgment for that future.

Today, it's available to everyone.

[Alex Shan @alexshander03]

Today, @JudgmentLabs opens for general access.

We believe long running agent systems will generate more exhaust data than the world can currently process.

Hidden within agent traces are rich records of how to make these systems better: what users ask for, when agents stumble, and how agents interact with the external world.

Judgment is the most performant monitoring and improvement layer for long-horizon agents.

We’re excited to see what you build!

[Alex Shan @alexshander03]

@omarsar0 @JudgmentLabs come try it: https://t.co/4utecUoR7j

[Josh @joshjwilkin]

@alexch3n @JahowYu @Guidepoint @TegusHQ @AlphaSights someone should give this man a raise !

[Josh @joshjwilkin]

@seancfong how many cumulative offline test hours

[Josh @joshjwilkin]

@alexshander03 @JudgmentLabs this is load-bearing !!!

[Josh @joshjwilkin]

@bernirov @alexshander03 @JudgmentLabs some may even say load-bearing

[Josh @joshjwilkin]

@ivanleomk Thank you Ivan! Excited to see what folks build!

[Ali Tekin @alitekintr]

Being able to see what happened between giving an agent a job and it saying 'done' is reassuring. This looks worth a try.