Albert Zuo

research @ percepta

[Athul Paul Jacob @apjacob03]

1/ What would it take for LLMs to be able to grow their own capabilities?

We introduce spotlight, a memory architecture with growing capacity but with linear computational complexity. We show that this suffices to host evolving ecosystems like software by building Python directly into an LLM.

We also share some very early language modeling results where it outperforms attention and linear attention on long context capabilities!

[Albert Zuo @AlbertYZuo]

Individualized care has long been a pipe dream. We’re doing the work to make it technologically and financially feasible

[Thomas Mathew @tmathew0309]

At Percepta we build decision making systems for the hardest operational problems at a business. Running them at scale gets expensive fast, so @shohamshitrit, @AlbertYZuo and team built infra to make private evals for our customers and then hillclimb the pareto frontier of cost vs accuracy.

In our latest example, we built a system that reads the full clinical timelines of 100k patients every day and picks the next best actions for each one.

The task is prohibitively expensive even with Gemini Flash models, so we trained a distilled 0.8B open model that beat every frontier model we tested at >20x lower cost.

[Albert Zuo @AlbertYZuo]

@AlexDGoldie @AIatMeta Congrats!!

[Hirsh Jain @hirshjain]

Reupping this seeking architects in new york. Come build the future w us. Healthy referral bonus to anyone who we hire as a staff platform eng :)

[Albert Zuo @AlbertYZuo]

@hirshjain @generalcatalyst Got em

[Hirsh Jain @hirshjain]

https://t.co/AZra0rINxA

[Albert Zuo @AlbertYZuo]

Make data structures great again

[Christos Tzamos @ChristosTzamos]

1/4 Want to build a computer inside a transformer?

Given the wide interest in our project, we are releasing the code and the weights so that others can build on our construction.

[Albert Zuo @AlbertYZuo]

For the record, this was a weekend project. If you’re interested in working with people of this caliber, reach out!

[Christos Tzamos @ChristosTzamos]

1/4 LLMs solve research grade math problems but struggle with basic calculations. We bridge this gap by turning them to computers.

We built a computer INSIDE a transformer that can run programs for millions of steps in seconds solving even the hardest Sudokus with 100% accuracy

[Athul Paul Jacob @apjacob03]

1/N Over the past year, one thing that has become increasingly clear to me is that timelines are compressing very very quickly.

That changes what is worth attempting. The problems that would have sounded too early, or too ambitious even recently are starting to look tractable.

[Hirsh Jain @hirshjain]

1/6 Palantir sued 3 of Percepta's employees (including me) last week. They're trying to bully us. We should all be concerned when a $400B+ giant is attacking a 35-person start-up over baseless allegations.

[Albert Zuo @AlbertYZuo]

If you’re excited about fixing the *real* bottleneck in enterprise AI, come chat!

[Athul Paul Jacob @apjacob03]

Percepta is hiring for research (RL, modeling, OR), engineering, and product roles in Europe and on the US East Coast. A few of us are at NeurIPS this week. Reach out to me, @EugeneVinitsky, @ChristosTzamos, @justintchiu, or @zzzzgq to learn more!