Bren

10+ years in tech: Engineering + AI + platform. hackathon winner. open source contributor. kickball mvp.

Benchmarking models

[Bren @BrenBuilds]

GPT-6.1 Terra fixed all 20 bugs in my app for about $.06 a run

GPT-6.1 SOL did the same for $.22 and opus 5.5 for $.78

GPT-6 Luna and Haiku 5.5 cost less but missed bugs in every run

[OpenAI @OpenAI]

GPT-6.1 Terra: delivering fast and capable computer use at a lower price.

GPT-6.1 Terra understands what’s on screen, takes action across apps, writes naturally, and is a more capable model for your whole workday with the efficiency to take on more of it.

[Bren @BrenBuilds]

@elshayib_ Cursor just added it, I'll have to give it a few turns in benchmarks

[Bren @BrenBuilds]

No one working on anything serious is using Gemini models.

The adoption numbers are because they shoved Gemini in every google search

[Bren @BrenBuilds]

@AnikaCodes Let's get a few more Colossus going

[Bren @BrenBuilds]

@yacineMTB attention is all you need

[Bren @BrenBuilds]

@PalmerLuckey a little too intentional for something larger to not be going on

[Bren @BrenBuilds]

@mjtechguy is it just me or did grok bot get dumber?

[Bren @BrenBuilds]

@mert_gulsun theres alot more time left on the clock

[Bren @BrenBuilds]

https://t.co/4jU1KW1Uw2

[Bren @BrenBuilds]

GPT-6 Luna at medium often stops before writing any code

from my testing, 26 of its 68 failed runs changed no file. 24 wrote up the fix and ended.

codex tells it to "persist until the user's intended goal is complete".

at xhigh it never stopped early.

[Bren @BrenBuilds]

GPT-6 Luna at medium solved 0 of 30 tries on coding changes over 450 lines.

from my testing, on fixes up to 300 lines it solved 17 of 40.

at xhigh those went to 10 of 30 and 32 of 40.

[Bren @BrenBuilds]

when it rains, it pours.

told Nikita let’s wait for more.

it’s never just one.

[Bren @BrenBuilds]

@nikitabier ...that we know of, so far

[Bren @BrenBuilds]

@levelsio haha when it rains, it pours. its never just one.

[Bren @BrenBuilds]

@ajambrosino @JensenHuang Tibo has to approve

[Bren @BrenBuilds]

@poteto congrats on the 1.0 release!

[Bren @BrenBuilds]

test results: https://t.co/pSmB4hSPT8

[Bren @BrenBuilds]

GPT-6 Luna at medium often stops before writing any code

from my testing, 26 of its 68 failed runs changed no file. 24 wrote up the fix and ended.

codex tells it to "persist until the user's intended goal is complete".

at xhigh it never stopped early.

[Bren @BrenBuilds]

@burkeholland @github nice feature. looking forward to trying it.

[Bren @BrenBuilds]

@shiri_shh dead internet theory

[Bren @BrenBuilds]

@poteto lol tibo . dog in the presentation.

[Bren @BrenBuilds]

@pvergadia @lennysan will check it out, appreciate the resource

[Bren @BrenBuilds]

@AnikaCodes bad business

[Bren @BrenBuilds]

i paid for X Premium+ mostly so i don't see ads.

now my threads are full of them.

it was sold as ad free.

that should hold until the subscription ends.