Freddie Bickford Smith

ML postdoc at Oxford with @tom_rainforth

[Freddie Bickford Smith @fbickfordsmith]

There’s a lot of confusion around uncertainty in machine learning.

We argue the "aleatoric vs epistemic" view has contributed to this and present a rigorous alternative.

#ICML2025 with @janundnik @eleanortrollope @markvanderwilk @adamefoster @tom_rainforth

1/5

[The Wall Street Journal @WSJ]

Anthropic, the AI company known for its devotion to safety, is scaling back that commitment https://t.co/qOdXDbP9zb

[Jon Barron @jon_barron]

This idea that intelligence is solely a function of what you've observed since birth and not also a function of the 500 million years of evolution that preceded your birth is surprisingly sticky despite being demonstrably untrue.

[Freddie Bickford Smith @fbickfordsmith]

@SumedhaChugh @janundnik 11am on Friday :)

[Freddie Bickford Smith @fbickfordsmith]

Active testing enables label-efficient model evals but can be computationally expensive.

We show how to reduce costs and scale up to LLMs.

https://t.co/rXkpQrJ7DY

Work led by Gabrielle Berrada. Find her at EurIPS, or @janundnik and me at NeurIPS in San Diego.

[Jannik Kossen @janundnik]

Come and check out our work on how to evaluate LLMs with less compute and fewer labels!

Find first author Gabrielle at EurIPS or Freddie and I at poster 110 at the 11 AM session on Friday.

[Dimitris Papailiopoulos @DimitrisPapail]

Grandest endorsement of doing a PhD that I've seen recently :)

[Timothy Gowers @wtgowers @wtgowers]

I crossed an interesting threshold yesterday, which I think many other mathematicians have been crossing recently as well. In the middle of trying to prove a result, I identified a statement that looked true and that would, if true, be useful to me. 1/3

[Toby Ord @tobyordoxford]

New post on RL scaling:
Careful analysis of OpenAI’s public benchmarks reveals RL scales far worse than inference: to match each 10x scale-up of inference compute, you need 100x the RL-training compute. The only reason it has been cost-effective is starting from a tiny base.
🧵

[Helen Toner @hlntnr]

Every so often, OpenAI employees ask me how I see the co now.

It's always tough to give a simple answer. Some things they're doing, eg on CoT monitoring or building out system cards, are great.

But the dishonesty & intimidation tactics in their policy work are really not. E.g:

[Oxford Statistics @OxfordStats]

With @JesusOxford we are looking for a Professor of Statistics.

Become part of a historic institution and a community focused on academic excellence, innovative thinking, and significant practical application.

About the role: https://t.co/mzBacAoCiv
Deadline: 15 September

[Cas (Stephen Casper) @StephenLCasper]

Research on AI "sandbagging" is getting more popular recently. In this 🧵, I'll give some reasons that I think it's not a useful research paradigm.

TL;DR, I think it's a confusing reframing of fairly well studied and previously solved problems.

[David Krueger 🦥 ⏸️ ⏹️ ⏪ @DavidSKrueger]

It's great the governments (and others) continue to demonstrate that the models companies release are incredibly insecure.

It's terrible that governments aren't penalizing companies for releasing such insecure models, and instead just help them patch them.

[Freddie Bickford Smith @fbickfordsmith]

@_rockt @andrewgwils More like (1) the future resembles the past and (2) you can capture the resemblances in your model, right? The existence of the future doesn’t imply you can predict it.

[Lorenz Kuhn @_lorenzkuhn]

Just two years ago, our smartest models could barely solve the easiest competitive programming problems. Last week, our latest reasoning models achieved a gold medal score at the International Olympiads of Informatics. Competitive programming is one of the cleanest examples of scaling up RL training for LLMs. Soon, with experimental approaches like the one used for IMO, we might see similar scaling on real-world coding problems.

[William MacAskill @willmacaskill]

Today I’m releasing an essay series called Better Futures.

It’s been something like eight years in the making, so I’m pretty happy it’s finally out!

It asks: when looking to the future, should we focus on surviving, or on flourishing?

[Freddie Bickford Smith @fbickfordsmith]

Join RainML! Lots of exciting work going on :)

[Tom Rainforth @tom_rainforth]

I have an opening for a 2-year postdoc in probabilistic machine learning and/or experimental design. The application deadline is the 3rd of September. See here for details and how to apply: https://t.co/ht9n9cEviw

[Freddie Bickford Smith @fbickfordsmith]

Strongly recommend working with Rob!

[Rob Cornish @rob_cornish]

I'm looking for talented and ambitious PhD students to join me at Nanyang Technological University Singapore to work on safe and robust AI systems!

Full scholarships covering tuition and a stipend are available, and are open to local and international students alike.

[Freddie Bickford Smith @fbickfordsmith]

@ClementineDomi6 @GatsbyUCL @SaxeLab Congrats!

[Freddie Bickford Smith @fbickfordsmith]

Come and chat at the poster session today at 4:30-7pm :)

Poster E-1403, East Exhibition Hall A-B

[Freddie Bickford Smith @fbickfordsmith]

There’s a lot of confusion around uncertainty in machine learning.

We argue the "aleatoric vs epistemic" view has contributed to this and present a rigorous alternative.

#ICML2025 with @janundnik @eleanortrollope @markvanderwilk @adamefoster @tom_rainforth

1/5

[summerfieldlab @summerfieldlab.bsky.social @summerfieldlab]

In a new paper, we examine recent claims that AI systems have been observed ‘scheming’, or making strategic attempts to mislead humans. We argue that to test these claims properly, more rigorous methods are needed.