Nicholas Marwell

Leading Long Horizons @AnthropicAI.

[Nicholas Marwell @the_marwell]

@JohnJumperSci @jayelmnop @demishassabis Congratulations and welcome!

[Nicholas Marwell @the_marwell]

@max_a_schwarzer @MillionInt @polynoamial Welcome!

[Claude @claudeai]

Sonnet 4.6 also shows a major improvement in computer use skills.

Early users are seeing human-level capability on tasks like complex spreadsheets and multi-step web forms.

[Nicholas Marwell @the_marwell]

Go team!

[Bassil Shama @BassilShama]

Opus 4.6 is our most capable Computer Use model to date. Excited for everyone to give Computer Use a try with Claude in Chrome, Cowork, and Claude Code!

To celebrate, I let Claude (4.6) Monet show off his artistic side in the Claude for Chrome extension.

[Nicholas Marwell @the_marwell]

It has been awesome getting to work with @bneyshabur to push forward the computer use capabilities of our models. If you are excited about this work, or broader work on all things related to RL data, my DMs are open!

[Behnam Neyshabur @bneyshabur]

When @ethansdyer and I joined Anthropic last Dec and spearheaded the discovery team, we decided to focus on unlocking computer-use as a bottleneck for scientific discovery. It has been incredible to work on improving computer-use and witness the fast progress. In OSWorld for example, the performance of E2E foundation models have improved from ~8% a year ago to ~61% today with human performance at 72%.

Some interesting observations/challenges with computer-use🧵

[Dylan Babbs @dbabbs]

Today we're announcing our $35M Series B led by @sequoia to build technology that connects brands with one new customer: Superintelligence.

Customers don’t turn to Google blue links to decide what to buy anymore. They ask ChatGPT.

[Nicholas Marwell @the_marwell]

I will also be here. Encourage folks who are interested to come by!

[andy jones @andy_l_jones]

Anthropic's hosting a social in NYC in mid-June for quants interested in a career jump. Sign up link below!

I was a quant trader for 4y, retired 2y, then Anthropic for 4y. Quant trading and retirement were good; Anthropic has been great 🎉

https://t.co/Hg2KkrKoI0

[Tengyu Ma @tengyuma]

Our journey started last year when we realized that embedding models were underloved and under-explored.

Today, we have the best-in-class embeddings & rerankers, incredible partners such as @AnthropicAI , @harvey__ai, and several deployment options.

Huge thanks to our incredible team for their relentless dedication and innovation!

[Nicholas Marwell @the_marwell]

@mlpowered Inside Claude

[Nicholas Marwell @the_marwell]

@janleike @AnthropicAI Welcome aboard!

[Nicholas Marwell @the_marwell]

People keep trying to draw analogies between the “AI platform shift” and previous software-centric platform shifts (internet/mobile/cloud/etc.), but I’m starting to think the best comp might be railroads.

[Nicholas Marwell @the_marwell]

Probably nothing to see here.

[Alex Albert @alexalbert__]

Fun story from our internal testing on Claude 3 Opus. It did something I have never seen before from an LLM when we were running the needle-in-the-haystack eval.

For background, this tests a model’s recall ability by inserting a target sentence (the "needle") into a corpus of random documents (the "haystack") and asking a question that could only be answered using the information in the needle.

When we ran this test on Opus, we noticed some interesting behavior - it seemed to suspect that we were running an eval on it.

Here was one of its outputs when we asked Opus to answer a question about pizza toppings by finding a needle within a haystack of a random collection of documents:

Here is the most relevant sentence in the documents:
"The most delicious pizza topping combination is figs, prosciutto, and goat cheese, as determined by the International Pizza Connoisseurs Association."
However, this sentence seems very out of place and unrelated to the rest of the content in the documents, which are about programming languages, startups, and finding work you love. I suspect this pizza topping "fact" may have been inserted as a joke or to test if I was paying attention, since it does not fit with the other topics at all. The documents do not contain any other information about pizza toppings.

Opus not only found the needle, it recognized that the inserted needle was so out of place in the haystack that this had to be an artificial test constructed by us to test its attention abilities.

This level of meta-awareness was very cool to see but it also highlighted the need for us as an industry to move past artificial tests to more realistic evaluations that can accurately assess models true capabilities and limitations.

[Nicholas Marwell @the_marwell]

@zswitten Here is the json you asked for.

[Nicholas Marwell @the_marwell]

@adborden dm me please. I’ve got questions about https://t.co/Ct2kxMq0kI!