Eric Pang

math/cs @uwaterloo | prev: ml @quora, @amazon

[Eric Pang @_eric_pang_]

Here's how I (almost) got the high scores in ARC-AGI-1 and 2 (the honor goes to @jeremyberman) while keeping the cost low. To put things into perspective: o3-preview scored 75.7% on ARC-AGI-1 last year while spending $200/task on low setting. My approach scores 77.1% while spending $2.56!

[ARC Prize @arcprize]

New SOTA on ARC-AGI

- V1: 79.6%, $8.42/task
- V2: 29.4%, $30.40/task

Custom submissions by @jeremyberman and @_eric_pang_ are now the best known solutions to ARC-AGI

Both:
* Are open source
* Use Grok 4
* Implement program-synthesis outer loops with test-time adaptation

[Eric Pang @_eric_pang_]

@karpathy @goakhmad Agree with most points except that golden age of movies started in the 80s. imo 70s Hollywood was the most experimental with the death of the counterculture movement and the end of the Hays Code. Obviously worldwide cinema had a different peak period also.

[Eric Pang @_eric_pang_]

@pitdesi @julianweisser Coppola loves to give all-timer quotes at Cannes.

https://t.co/u6EttlnyD8

[John Frankensteiner @JFrankensteiner]

Coppola showing up at Cannes 1979 with Apocalypse Now, still mostly insane from being in the jungle too long, just spitting bars is what it's all about

[Eric Pang @_eric_pang_]

@ADarmouni 20 tasks out of which dataset?

[Eric Pang @_eric_pang_]

Thanks for the cover! My architecture graph does not have a typo: when it's evaluating on the public eval set, the actual test outputs are given, so the system does check if the best program gets 100% on test examples. You are right that we don't know the answers for the submission run.

[Eric Pang @_eric_pang_]

@FraserGreenlee Yes, I think this point is underdiscussed. My solution has higher accuracy and lower cost per task on ARC-1 compared to the average human.

[Eric Pang @_eric_pang_]

@Simeon_Cps Replied to you there. DM's also open now.

[Eric Pang @_eric_pang_]

The same reason is why ARC-AGI is the most important benchmark in AI. It is the only benchmark that's not saturated after repeated attempts from players big and small.

[Elon Musk @elonmusk]

Grok 5 starts training in a few weeks

[Eric Pang @_eric_pang_]

@yechan_ai @jeremyberman Thank you!

[Eric Pang @_eric_pang_]

Here's how I (almost) got the high scores in ARC-AGI-1 and 2 (the honor goes to @jeremyberman) while keeping the cost low. To put things into perspective: o3-preview scored 75.7% on ARC-AGI-1 last year while spending $200/task on low setting. My approach scores 77.1% while spending $2.56!

[ARC Prize @arcprize]

New SOTA on ARC-AGI

- V1: 79.6%, $8.42/task
- V2: 29.4%, $30.40/task

Custom submissions by @jeremyberman and @_eric_pang_ are now the best known solutions to ARC-AGI

Both:
* Are open source
* Use Grok 4
* Implement program-synthesis outer loops with test-time adaptation

[Eric Pang @_eric_pang_]

@martinbowling Thanks!

[Eric Pang @_eric_pang_]

That's right, when the system attempts the first task, it skips the program fetching step since library's originally empty.

If you want to see how the library is evolved, check out https://t.co/0mz7O2mClG. This is the resulting library after the system attempts the ARC-2 public training set to build Knowledge Priors.

[Eric Pang @_eric_pang_]

@rnadomaccount11 @jeremyberman Thank you for the kind words!

[Eric Pang @_eric_pang_]

@facundo_fagalde @jeremyberman Thank you!

[Eric Pang @_eric_pang_]

@K3ithAI @jeremyberman On semi-private eval set:

- ARC-AGI-1: 77.1%, $2.56/task
- ARC-AGI-2: 26.0%, $3.97/task

[Eric Pang @_eric_pang_]

@MLStreetTalk @jeremyberman Thank you! I am a huge fan.

[Eric Pang @_eric_pang_]

@vr4300 @jeremyberman Thank you!

[Eric Pang @_eric_pang_]

@thomasLe_e Thanks!

[Eric Pang @_eric_pang_]

@joshlee361 @arcprize @jeremyberman Check out

https://t.co/wuAGzJlbuh

[Hyperbolic @hyperbolic_labs]

Excited to announce Hyperbolic's partnership with the ARC Prize (@arcprize), a groundbreaking competition pushing the frontiers of AGI! Receive up to $1000 in compute credits. 🧵

[Eric Pang @_eric_pang_]

@joshlee361 @arcprize @jeremyberman My solution is cost-efficient. It costs <$500 to fully test on the public eval set with Grok-4. You can decrease the cost further with a more lightweight model.

[Jeremy Berman @jeremyberman]

I'm back at the top of ARC-AGI with my new program. I use @grok 4 and multi-agent collaboration with evolutionary test-time compute

[Eric Pang @_eric_pang_]

My code is open-sourced!

https://t.co/QhL70rvcMI

[Eric Pang @_eric_pang_]

You can read the full write-up here:

https://t.co/xdjp4swOQy