Andy R Forbes

London lawyer interested in AI / ML technology, data protection, health tech and green tech. Views are my own.

[Thang Luong @lmthang]

A bit of a history of IMO-Bench and our IMO efforts:
a. We started building IMO-Bench around early 2024, which was the precursor of ProofBench (basic).

b. IMO-Bench was first mentioned in the Gemini 1.5 paper around May 2024. At that time, Gemini Math-specialized 1.5 Pro scored only 25% (whereas the performance on Hendrycks’s MATH was 81%, a big breakthrough!).

c. Fast forward to April 2025, Gemini 2.5 Pro scored 55% on ProofBench (basic) and 35% on ProofBench (advanced). Nobody talked about the MATH benchmark anymore (it has served its purpose well!).

d. Also in April 2025, our paper was accepted to ACL 2025. We were asking leadership to share it publicly, first to arXiv, but we were asked to wait until IMO 2025, so we postponed ACL. At the time, it felt like a major setback to me because it was uncertain at the time if we were going to surpass ourselves at IMO 2025 (we got Silver at IMO 2024). We kept marching on.

e. At IMO 2025, our generalist model (advanced Gemini Deep Think), scored 89.0% on ProofBench (basic) and 66% on ProofBench (advanced). And the rest was history 🙂

In case people missed it, this project page has all the infos about IMO-Bench https://t.co/jB67vqApu3.

[Tim Brooks @_tim_brooks]

DeepMind has ambitious plans to make massive generative models that simulate the world. I'm hiring for a new team with this mission. Come build with us!

https://t.co/pqvALtAvLs https://t.co/vtwgeXl9Dl

[Andy R Forbes @AndyRForbes]

@DesMoffett @laureneldon @O2academybrix @O2 @EatYourOwnEars @FourTet Yea it was terrible. Hopefully they will take on the feedback and fix it for tonight. Let us know how it goes
🔊🧏‍♂️🙉 #fourtetbrixtonacedemy #brixtonacademy #eatyourownears

[Andy R Forbes @AndyRForbes]

@_ianwalker @O2academybrix @O2 @EatYourOwnEars @FourTet It was just poor sound and far too loud. London venues are way beyond that. It’s not Fabric Room 1 but Brixton Academy usually has a great system. Shows the value of decent sound engineers!

[Andy R Forbes @AndyRForbes]

@EatYourOwnEars @FourTet @caribou The sound was terrible. I have tinnitus now because of it. Loads left before FourTet came on because it was so bad. Why did you not have anyone at the sound desk? Feel bad for the artists who rely on the venue & promoters to sort this stuff out #tinnitus #basics #seewhatyoucando

[Andy R Forbes @AndyRForbes]

@Acnap94 @O2academybrix @O2 @EatYourOwnEars @FourTet Yea it was awful. We left. 0/10. Feel really sorry for anyone who stayed…

[Andy R Forbes @AndyRForbes]

@anthillsocial hello!

[Reuters Tech News @ReutersTech]

The Supreme Court has sided with Samsung in its smartphone patent fight with Apple. More here: https://t.co/cmkh5BdEnp

[Andy R Forbes @AndyRForbes]

Wise words https://t.co/mU4RT7nnHz

[Resident Advisor @residentadvisor]

Richie Hawtin and Pioneer DJ team up for Get Played Get Paid https://t.co/DZu90FwL5y

[Andy R Forbes @AndyRForbes]

Japan's @SoftBank to acquire @ARMHoldings. An example of Asian interest in bargains following #brexit? https://t.co/Dw1RS9cwvd

[The Telegraph @Telegraph]

Pokémon GO players gather at 1am in seemingly crazy search for Snorlax https://t.co/4FQjGPgE3Q

[TechCrunch @TechCrunch]

Facebook sneaked a seriously addictive basketball game into Messenger https://t.co/zLoP7St88h by @jonrussell

[Andy R Forbes @AndyRForbes]

Amazing scenes! Google 2-0 up in clash with board game pro https://t.co/yUBWZkmnDl History being made by @demishassabis @DeepMindAI #AlphaGo

[Bloomberg @business]

BREAKING: Google to make driverless cars an Alphabet company in 2016 https://t.co/DzqtL1UiPP

[Andy R Forbes @AndyRForbes]

Tencent, owners of #wechat, invest in @Kik. Now competing with @Facebook / @WhatsApp in USA? https://t.co/IOWS9Tt4de via @WIRED #China #tech

[WIRED @WIRED]

Nokia’s Ozo VR camera marks a rebirth for the phone giant https://t.co/YzrUHeqA2Y

[WSJ China Real Time @ChinaRealTime]

Video: Alibaba's Jack Ma is saying, 'Open Sesame.' Will SCMP let him in? https://t.co/qiijpZDEcU

[Andy R Forbes @AndyRForbes]

UK fan raises €24,000 on @gofundme for musicians behind the #AmenBreak https://t.co/Y5KBbhQTpE via @ukthump #copyright #sampling #goodnews