Sam Wilson-Späth

yeah, whatever.

[Sam Wilson-Späth @SamWilson1]

Anyone else really enjoying Threads?

[Sam Wilson-Späth @SamWilson1]

Holy shit.

[Alex Albert @alexalbert__]

Fun story from our internal testing on Claude 3 Opus. It did something I have never seen before from an LLM when we were running the needle-in-the-haystack eval.

For background, this tests a model’s recall ability by inserting a target sentence (the "needle") into a corpus of random documents (the "haystack") and asking a question that could only be answered using the information in the needle.

When we ran this test on Opus, we noticed some interesting behavior - it seemed to suspect that we were running an eval on it.

Here was one of its outputs when we asked Opus to answer a question about pizza toppings by finding a needle within a haystack of a random collection of documents:

Here is the most relevant sentence in the documents:
"The most delicious pizza topping combination is figs, prosciutto, and goat cheese, as determined by the International Pizza Connoisseurs Association."
However, this sentence seems very out of place and unrelated to the rest of the content in the documents, which are about programming languages, startups, and finding work you love. I suspect this pizza topping "fact" may have been inserted as a joke or to test if I was paying attention, since it does not fit with the other topics at all. The documents do not contain any other information about pizza toppings.

Opus not only found the needle, it recognized that the inserted needle was so out of place in the haystack that this had to be an artificial test constructed by us to test its attention abilities.

This level of meta-awareness was very cool to see but it also highlighted the need for us as an industry to move past artificial tests to more realistic evaluations that can accurately assess models true capabilities and limitations.

[Sam Wilson-Späth @SamWilson1]

One angle I haven’t seen in OpenAI reporting: what was the impetus for the board of AI activists to get so freaked out? What did they consider unethical/unacceptable?

[Sam Wilson-Späth @SamWilson1]

Anyone moved their funny cricket commentary to Threads? #CWC23Final

[Sam Wilson-Späth @SamWilson1]

@TankLanning It’s making me stressed and sad, Tankie! Unacceptable.

[Sam Wilson-Späth @SamWilson1]

Dear X. Why is my timeline now 90% full of people I don’t follow? I feel punked.

[All The Right Movies @ATRightMovies]

DAVID FINCHER is 61 years old today. One of the greatest directors of the last 30 years, here’s some great behind the scenes shots from his movie career…

A THREAD
1/23

[cinesthetic. @TheCinesthetic]

Japanese poster for American Psycho, 2000.

[Sam Wilson-Späth @SamWilson1]

@carofoodartist I am ALL ABOUT this plan, as long as we can behave as if we are 23. Okay, 33. Fuck it, as I still behave. I’ll DM you my new number

[Tobi Bakre @tobibakre]

Dem go feel it! Dem go run it up! Dem go holllitttt!

Gangs of Lagos this Friday on
@primevideonaija

[Sam Wilson-Späth @SamWilson1]

@CollinsMandy The idea is to promote critical thinking and to strip away the bullshit. I knew pot was a crutch, but I am shocked by what a difference it makes.

[Sam Wilson-Späth @SamWilson1]

@CollinsMandy Did you know Amazon has a policy against ppt internally? You do word docs and then everyone gets time in the meetings to read them before discussion.

[Sam Wilson-Späth @SamWilson1]

@rachelirvine Impressed re: dog walking. My cat just looked surprised when I stopped spooning her at 11am.

[Sam Wilson-Späth @SamWilson1]

My grooming standard has dropped from ‘do I look okay in this?’ to ‘will I unnerve anyone?’ #thankscovid

[Sam Wilson-Späth @SamWilson1]

@HellaTamaryn I just paid the first tranche of the university fees and am now having an involuntary lie down in the foetal position

[Sam Wilson-Späth @SamWilson1]

I’m going with the chorizo.

[Paul Fairie @paulisci]

The 2022 Headline of the Year Nominees

🧵

[Sam Wilson-Späth @SamWilson1]

I’m confused. Is SA getting pub hols on Monday and Tuesday next week?

[Sam Wilson-Späth @SamWilson1]

Dear South Africa. Who is trusting what’s in the freezer?

[Wizard Cal 🕯️✨ @MageOfSolitude]

Maybe it’s Maybelline, maybe it’s twenty years trapped inside a vengeful tree as it turns her flesh to a smooth, eerie bark