sathvik

Semper Prorsum

DevEx Engineer @warpdotdev, Georgia Tech Alum, ex-Perplexity

[sathvik @vikvang1]

3d game demos aren't the best way to judge a model's coding ability according to @theo. neither is SWE-bench, CursorBench, or TermBench.

instead, you should benchmark models against your own traces to find the best one for your tasks.

here's how the Warp team benchmarks various harness/model configs, which helped reduce cost / PR from $80 to $20 in our software factory (alongside uncovering a number of unintuitive findings!)

Demo:

[sathvik @vikvang1]

@AravSrinivas wow

[sathvik @vikvang1]

Catch us at AIE code summit in SF @warpdotdev

[sathvik @vikvang1]

I’m sorry… Fermat’s Last Theorem????

[Lance Martin @RLanceMartin]

i'm happy to share some recent work on safely scaling a team of 1,000 Fable 5.2 agent to tackle ambitious, long-horizon work. the team was able to formalize a proof of Fermat's Last Theorem through coordination within our safety boundaries.

read it here:
https://t.co/IIsNzYJBeQ

[sathvik @vikvang1]

@threepointone You might enjoy our approach at @warpdotdev 😃

[Ben Holmes @BHolmesDev]

I was talking to an engineer the other day about this question:

How do you make a codebase easy-to-navigate by both agents and humans, when everything from the hello world is agent-built?

The answer is system architecture. Agents (like humans) copy bad patterns like a virus. It's a ball of clay that needs constant reshaping.

You can write down best practices into your factory's skills, yes, but keeping the codebase itself on the right path is the real goal.

It's too expensive to have an agent review the universe constantly, spitting out architecture change PRs no one wants to look at. But, say, a weekly automation with a frontier reasoning model (Fable 5.2 is a good one to try) that pulls out one improvement to your system architecture is a start. I like @steveruizok's prompt to "look for implicit state machines."

Will experiment with a factory automation like this and see how it goes.

[sathvik @vikvang1]

@BHolmesDev I already added this to our mobile factory :)

Smart to run it on a cron!

[sathvik @vikvang1]

@adcock_brett OOC, does iMessage hold higher retention than the standalone app? I do enjoy Hark's UX

[sathvik @vikvang1]

@ekzhang1 @can @modal @thinkymachines congrats!

[sathvik @vikvang1]

@choeshow Yeah this will definitely get the Republicans reelected in 28!

[sathvik @vikvang1]

lol

[Garry Tan @garrytan]

@bb_fresh Going to change the description of gstack to an abstract software factory provider

[Varoon Kodithala @varoonkodithala]

not true of at least @warpdotdev

over the past few months, we built a factories product from scratch, shifted every engineer’s mode of work to leverage our factory, benchmarked and improved that factory over time, and it has absolutely accelerated our pace of work

last week, we shipped over 400 factory PRs in one day

🏭🏭🏭

[sathvik @vikvang1]

@theo Shilled by t3code 🥲🥲🥲

[Chuck Haines @termleech]

Exactly why we are evaluating @warpdotdev right now and likely will adopt it wholesale (terminal and factory). Gives us the whole package.

[sathvik @vikvang1]

@Shashikant86 🫡

[sathvik @vikvang1]

In SF next week for AI Engineer! Who’s around? 😊

[sathvik @vikvang1]

@lotar_vk can you say more

[sathvik @vikvang1]

@termleech @warpdotdev Dm me with any Qs!

[sathvik @vikvang1]

@JoshARosen Software is art!

[sathvik @vikvang1]

@sethkarten @zachlloydtweets yep

[sathvik @vikvang1]

@petergyang @conductor_build @t3dotcodes @orca_build the @warpdotdev ADE is completely open sourced w 62k+ GitHub stars

https://t.co/UPnbIJwMOF