Vincent

Cooking at Stealth.

Prev. @berkeley_ai

[Vincent @Cveinnt]

today opus 4.8 hallucinated an entire typosquat prompt injection attack all on its own and caused me quite the panic.

how's your saturday going?

[Vincent @Cveinnt]

@andrewchen tasteful people are more amplified than ever

https://t.co/sfialMHXWY

[Vincent @Cveinnt]

But there are those who are amplified:

>AI gets better, which raises the bar for human judgment.
>Humans develop deeper taste, which produces better work for AI to learn from.

Each pushes the other higher. Two forces that don't compete so much as co-evolve.

[Vincent @Cveinnt]

@martin_casado taste is always the quintessential human job
https://t.co/5PYB4hzUp5

[Vincent @Cveinnt]

Paul Graham (@paulg), Sam Altman (@sama), Greg Brockman (@gdb) all landed on the same word this year: Taste.

They're right that it matters. They're wrong that it's new.

Taste was always the job. Execution was the fog that hid it.

https://t.co/soufBbeUCq

[Vincent @Cveinnt]

I've never been more excited to work.

The parts of building I always cared about - the decisions about what should exist and why - used to be buried under months of execution.

Now they're the whole job.

The fog of execution lifted, and what's underneath is the work I wanted to be doing all along.

https://t.co/soufBbeUCq

[Vincent @Cveinnt]

Paul Graham (@paulg), Sam Altman (@sama), Greg Brockman (@gdb) all landed on the same word this year: Taste.

They're right that it matters. They're wrong that it's new.

Taste was always the job. Execution was the fog that hid it.

https://t.co/soufBbeUCq

[Vincent @Cveinnt]

But there are those who are amplified:

>AI gets better, which raises the bar for human judgment.
>Humans develop deeper taste, which produces better work for AI to learn from.

Each pushes the other higher. Two forces that don't compete so much as co-evolve.

[Arena.ai @arena]

Not all questions need GPT-4!

We introduce RouteLLM – a routing framework based on human preference data that directs simple queries to a cheaper model.

With data augmentation techniques, RouteLLM achieves cost reductions of over 85% on MT Bench and 45% on MMLU while maintaining 95% GPT-4 performance.

Comparing against commercial offerings (Martian and Unify AI) on MT Bench and achieve the same performance while being over 40% cheaper.

Our model, datasets, and code for serving and evaluating LLM routers are all open-sourced. We are excited to see what the community will build on top!

1/6 (blog post & links👇)

[Vincent @Cveinnt]

@github Thanks GitHub!