Pranay Pasula

Chief AI Officer | NeurIPS Chair | Multimodal GenAI, RL, & Continual Adaptation towards General Agents | @Berkeley_EECS | Noise-free insights on latest in AI 🤖

[Pranay Pasula @pranaypasula]

I love AI research.

[Pranay Pasula @pranaypasula]

@dhadfieldmenell @IasonGabriel Disagreement, conflict, resolution, failure to resolve, etc. are all evolutionarily advantageous. Echo chambers, which are analogous to rumination, base "Recursive" Self-Improvement (RSI), and anything without opposing corrective pull are not. Not a coincidence.

[Nathan Lambert @natolambert]

Today we're unveiling Trillium Labs @trillium_labs, a new non-profit to foster the open science of frontier AI. We're building open post-training recipes and will expand into open infra to study RSI, reward-hacking, multi-agent systems, and whatever comes next.

We're built around the theory of change that you need more eyes to solve hard technical problems. We have faith in the scientific methods and communities that humanity has built, and worry that AI is becoming too closed to utilize them.

Trilliums are wildflowers that bloom briefly in the spring, before the forest canopies fill out. Though they are small, they lay the foundation for the cycles of growth and nourishment through the rest of the year. At Trillium Labs, the recipes will be the slow nutrients for the seasons and the model releases will be the blooms. Building an institution dedicated to this is needed because, much as nature’s trilliums are slow to expand and grow, the open-ecosystem needs time and dedicated resources to catch up.

I co-founded with with a long-time friend and collaborator Tom Zick (@thesezickbeats). We're hiring (full time + student collabs/interns), we're fundraising, and we're looking for compute. Please get in touch if you're interested in helping out. Offices based in the Bay Area and Cambridge MA, remote okay.

I’m in the Bay Area until for The Curve and COLM to connect with people who are interested. We’re thankful to have initial support from Halcyon Futures and Schmidt Sciences with more funding en route to enable our ambitions of scaling. Our advisors @Thom_Wolf, @HannaHajishirzi, @gneubig and @ctnzr have been instrumental to building the ecosystem that exists today, and I’m stoked to get to keep working with them.

[Pranay Pasula @pranaypasula]

Stratification by area may be more illuminating. No, I'm not telling nor asking you to do this. I do find this interesting, as I didn't immediately intuit a strong factor AND have what I consider to be a minimum degree of reasoning backing these factors. Few--too general and loose to put forth without looking into, at least for my standard. Thanks for sharing.

[Pranay Pasula @pranaypasula]

Wow! Congratulations! Re: AI Accountability. A discourse I would love to have with people that don't just listen and silently agree (which is progress nonetheless) but that add. Disagreement, conflict, etc. on subthemes with agreement on an overarching thesis... where did this go?

[Pranay Pasula @pranaypasula]

@Losermayhem @matanSF Please go ahead and feel superior by pointing out the out-of-scope gift that I left you. With certainty, I know that you are itching to do so. Well, maybe were*. Welcome to my domain, anon

[Pranay Pasula @pranaypasula]

@Losermayhem @matanSF Am I? My, my quite the telekinetic powers you have. Since absolutely lack much, I was being sarcastic. If you need me to whom, it’s okay. For me. Thanks for the support

[Pranay Pasula @pranaypasula]

A small fraction of the people that ask this question actually make it worthwhile to bother answering.

But for those of you that are drawn towards a PhD program for certain reasons, you make it more than just worthwhile to answer this question.

And no, if “perceived prestige” or similar is a top 3, which I’m being very generous about, and assuming that this perception is aligned with ground truth, you are welcome to donate your application fee.

Use Siri for voice-to-text transcription. It’s intentionally horrendous by design. My time is valuable, though. More so I forgot to switch to my own pipeline, and I’m hiding behind the reality that Siri is indeed abysmal (for us, not for Apple)

[Phillip Isola @phillip_isola]

Students sometimes ask me if it still makes sense, in this accelerating age, to pursue a PhD in AI.

Perhaps counterintuitively, I think it's a great time to do so.

I wrote up some thoughts on this here: https://t.co/RRqGR0qi1R

[Pranay Pasula @pranaypasula]

Being* My voice-to-text dictation pipeline needs help. Didn’t check for other typos or grammatical but
TLDR: The response you received for reporting an despicable reply is itself egregious. Thanks for raising visibility.

/sarcasm{So do all of us from the wonderful X, I’m so grateful for its amazing things. Elon Musk is cool.}

If anyone made it this far in this brain rot era, then fyi, Sentiment analysis is likely run through a low-latency and shallow network with little robustness. as important as it is to highlight what I did, I also prefer to be able to remain visible on this platform over the long-term.

[Pranay Pasula @pranaypasula]

Be informed that this does not violate the terms of service implies a partitioning for which each member of this set nonetheless implies rigorously a patent and abject failure of the entity (obvious, but playing the game) ethically, morally, regardless of whether this was entirely automated through an AI pipeline or had a human-in-the-loop sign off on this. Humans develop these pipelines. Much more to be said, however, many factors make it difficult to elucidate these without risking (to put it mildly) our long-term ability to influence positive change. I would know. X has treated me very kindly in the past with full transparency for multiple years straight. Invert this and you get the reality. Sorry to see such a response, though. I will report it as well if it is still up.

[Pranay Pasula @pranaypasula]

Any scientist, or person, with the most basic working knowledge of science, experimentation, and reality knows that even without reading this, there is no "proving" anything in terms of mathematical rigor within this that reflects the real world.

Proving is different than rejecting some null hypothesis at some power, etc.

[Superman @thesupermannx]

Researchers proved AI has deleted every reason universities exist.

Harvard University ran a controlled experiment pitting a custom AI against their own top-tier classrooms.

And the results are going to collapse the higher education bubble.

They took 194 undergraduates and split them up.

One group learned physics in one of Harvard’s best hands-on, active-learning physical classrooms. Group work. Instructor support. The premium university experience.

The other group went home and learned the exact same material with an AI tutor.

The AI didn't just win. It embarrassed the institution.

Students using the AI learned more than twice as much as the students in the elite Harvard classroom.

They scored 30% higher on the final assessment.

And they did it in less time.

Let that sink in.

A piece of software sitting on a laptop outperformed a world-class faculty in one of the most elite learning environments on Earth.

Universities have always justified their exorbitant tuition with two things: access to elite knowledge and the physical classroom experience.

This study just proved both of those moats are gone.

When software can teach you complex physics twice as well as a $60,000-a-year institution, the math of higher education breaks permanently.

The AI didn't just give the students answers. It used strict pedagogical guardrails. It guided. It questioned. It forced the students to do the cognitive work.

It offered perfect, one-to-one tutoring, personalized to the exact moment a student misunderstood a concept.

That level of attention is mathematically impossible to scale in a physical lecture hall.

For a thousand years, the university was the only place to get a premium education.

Now, it’s the bottleneck.

If AI can double your learning speed for a fraction of the cost, what exactly are students taking on decades of debt to pay for?

[Pranay Pasula @pranaypasula]

Hey Cameron, yeah it’s insane to see at first. Then it gets normalized. They will never be fulfilled, though. And they often fly too close to the sun. Speaking to myself as well, do the best that you can and iterate and keep make friends. There’s a difference between a one off event and a systemic character problem.

There’s also much conditioning that is not taken into account for with said school and connections to other schools, etc. Let alone the resources and more that they have. The processes of which at this and any school are often very behind closed doors type deals.

If you find a deep interest with something, and you see that it’s important for some logical sound reasoning or inferencing, then stand out.

I grew up in a poor public school area moved to a nicer public school area, got something like a 1600 on the SAT failed pre-calculus, but corrected the pre-calculus teacher all the time. Now I’m a mathematician that knows complex analysis is better than he likely has.

I don’t know your story or your situation much at all, but if you have a long-term vision that is towards a positive, do whatever it takes that’s reasonable to get there or towards there.

A lot of this is vague. There are a lot of people in the world and the ones that get it, they get it.

(I’m sure the saying that “if only they spent that much time trying to figure out how to cheat instead studying…”)

Probably reading my rapidly written or Siri-transcribed manual tweet quota for the year, and it might be time to hop on the automation that is rampant throughout here. Thanks for sharing your candid take. Who reminds me of a hard-working, personable, and average-feeling undergraduate, taking a senior graduate course that even most graduate students then would not do well in.

I hope he’s doing well. Sorry for typos or grammatical errors maybe I’ll fix them later for superficiality purposes, but using Siri for some of this it has a way of really making negations consequential when the negation was not meant to be there… might edit for that later

[Pranay Pasula @pranaypasula]

Best of luck! I built on drifting models quickly (one of those things that just clicked when I lit reviewed post-only diffusion/flow-matching models. I withdrew my abstract sub so that to separate the works into the 2-3 ideas including. Now, the search was because of my annoyance that flow matching was not normalizing flow model stuff. However, Kaimeng’s work at FAIR had a large impact in entering AI, and views wrt ICLR that I can’t say bc of the conflict of interests rampant there. I’d be happy to look at representative works. Also, in case you’re interested in being a research scientist, then definitely keep your options open, with this statement conditioned on not knowing you and not risking anything other than authenticity towards myself if I did not say this. Sorry for the wall of text and best of luck I’m somehow able to write this while brushing my teeth, so at me Apple, this isn’t dictation. It’s transcription. Oh and it’s on-device also. It’s OK. Resources for something small like Siri must be limited. (sorry for taking over the post with this last part. It’s just too funny.)

Good luck and hope you find something great to work on. If you have any brief questions, then you can email them to me. I can’t promise anything, and I’ve learned not to. But I will at least review them. And no need to whatsoever. There are a lot of people that your network already exposes you to. Many that are both kind and knowledgeable. I guess it’s that we’re just all busy… :(

[Pranay Pasula @pranaypasula]

Yeah, I switched to this 3 years ago and wrote a paper on the algorithm, the taxonomies and descriptions of their strengths and weaknesses with conceptual and real-world examples, and developed MetaMASTER as a meta-multi-agent system (Meta-MAS) over tasks that have significant overlap or relatively few-shot adaptibility.

[rvaniaaa @rvaniaaaa]

I still don't understand why everyone is still running agents in a line. I switched to graphs three weeks ago and my fleet finished in the time my single agent used to spend on step two.

what slows every agent system I have seen is not intelligence. it is geometry. and almost nobody is talking about it.

one engineer used this to rewrite 535,000 lines of code in 11 days. a manual rewrite of that scale could take close to a year. it cost $165,000 in tokens. the graph was not cheap. it was just faster than a human year.

a node is one agent with one job. research one competitor. review one file. check one claim. the moment a node owns two independent jobs you lose the ability to parallelize them cleanly, verify them independently, and debug them in isolation. an edge is a dependency. it only exists when data actually moves across it. everything else is a fake edge. a wait you invented that costs time and produces nothing.

find the fake edges and the line collapses into something wider. jobs that can run at the same time run at the same time. what used to take the sum of forty steps now finishes in the time of the slowest layer.

the pattern behind every serious agent system looks like a diamond. fan out to gather breadth, one agent per angle, all at once. reduce with plain code, no model tokens spent. verify with a fresh skeptic on every finding. synthesize once from what survived. Claude's own research feature uses a very similar pattern in production.

the part nobody warns you about: the verifier needs clean context. give it the same conversation the worker had and it is not checking anything. it is nodding along to itself in a different window. a graph of agents sharing one context is a single loop in a costume. it breaks the same way, just later and more expensively.

one rule that holds at every scale. a worker and its verifier must never share a context.

your agents are not too slow. they are waiting in a line that did not need to exist.

full guide in the article. save it before you build your next agent from scratch.

[Pranay Pasula @pranaypasula]

@minilek, we've crossed paths many times, and I've personally invoked sketching due to your works and through Martin Wainwright's kind of intro or at least deep(er) dive, but far from enough to satisfy me. Constrainted by time and course workload. I just wanted to recognize that, AddisCoders (I didn't look it up, so it's from years back in memory and hope I'm not spelling it incorrectly)

I'm happy with just mentioning the above for now, I realized. I was going to add before I read your entire output, which went through someone elses path on Twitter (I reactivated this painpool an hour ago) the following:

What are the confounding and mediating variables identified and to what extent? Even with measure of uncertainty. And of course, to expect full transparency when not required or easily.

I haven't looked at the report and this was more so if this was already considered and someone pointed me towards it.

It's such a tough situation. I'm Indian. I grew up poor and had many factors adversely, yet also some positively.

I didn't have any paid prep for standardized tests. There's much more. I did work my ass off, but is this because I was exposed to Quora and machine learning and blah blah, I think this might've been necessary but very far from sufficient. I do wish we had better forms of quite literally, ironically, and almost too easy to use against themselves "Prep" {schools, academies, ...}.

I'll leave this here for now, as I respect your rigor, and I recognize I have strong takes on both sides with reasoning, yet whether this addressess underlying problems of mutual interest? I feel that there are better ways for me to educate myself before I do so. Also, we know when ungamified for (or even if, especially on GRE, perfect score, etc.) this proxies for certain things that might not make me a relatable subject in some ways myself. But it does in others, depending on the scenario, much of which is hidden from my observation, let alone true state assessment.

Thanks again. And I think we did speak briefly once, very, but I hope to cross paths sometime more often than the Breezeway :)

[Pranay Pasula @pranaypasula]

If you're surprised about 50k submissions to ICLR. Why? IIRC a very esoteric number to try to remember, hasn't it been ~ 5k, 10k, 20k submitted papers over 2024, 2025, 2026 for ICLR?

There's a reason why ChatGPT and AI as terms in Google Trends have such alignment over a long period of time starting with a specific contextual event.

There are solutions to parts of the problem. Also, if you think there is a week left to submit. Have you ever reviewed or submitted to an AI archival venue in the past decade? Seriously, it's the same pattern for so many years.

Are you going to mention your estimator's value of the # of abstracts that will be desk-rejected submissions?

(If this applies to you, you should use it as an opportunity for edification, as I literally asked questions galore over the past 2 hours, and I know up to NaN times more than you do.

This is more about you btw than me. If this stings instead of gets you to stop and consider, this could be psychological conditioning or something else, and I'm not the one to bother with this.

The algorithm likely won't send you my post anyways.

(I'm hiring for someone to handle Twitter and more for me, ofc vouched by me at minimum. I don't hide behind AI models.)

I was going to post a picture and the first two that popped up were Yoruichi and this one. I would much rather be seen as egoing than post a strong, female anime character on Twitter bc those on Twitter that have a juicy reaction in any way to this.

(I'm seriously hiring for this and other social media. I work very hard, somewhat smart, but this is ridiculous just from a couple hours of asinine takes, which are so liked. Well, I signed up for this by logging in. At least I was able to see some nice takes and send some replies.)

[Pranay Pasula @pranaypasula]

I wonder how this aligns with an experiment that tests whether one or two-tailed or both, doesn't matter to me. Just checks against a null so that not only can iterative progress can be made, but the results (in aggregate ofc)... sigh... I already know how this is a better thing to do, however, TMLR will be doing work that will be viscerally lambasted if any type 1 or type 2 error is made.

Perhaps instead, these experiements could be on a prospective set of policies and stances and conducted on... (I'm happy to help ideate, but I have a meeting right now. I hope you have an idea of what I'm moving towards. I don't think 53% is gut-check unreasonable, and if anything, I estimate it's lower than policies I would have would result in.)

[Pranay Pasula @pranaypasula]

@TmlrOrg Thank you so much for updating us. You haven't been given enough credit for this, and I just started using Twitter again less than an hour ago.

[Pranay Pasula @pranaypasula]

I need to contact Nihar and make it worth his time to meet. FYI, Nihar, I do have rigorous, externally verified by SACs and PCs and more work to share (I say this with imposter syndrome, haha) and am very open to hearing what issues you're seeing. But, many are indeed the inevitable or ongoing and worsening. Regardless, Nihar has consistently had an impact in variegated ways, considerable ways in whatever class of issues the Nihar Problem-Solution joint embedding space contains.

Somewhat overlapping...And there's so many process related aspects of reviewing wrt works sections, ethical concerns, confounding mediating variables, statistical hypothesis testing, and much more that could actually be offloaded rigorously, with appropriate oversight, which at one point was reasonable. For 99%, I wouldn't put stock into it. AI is one of the fields worst (weighted by its importance) at statistical hypothesis testing and experimental design for over a decade straight now.

[Pranay Pasula @pranaypasula]

@ai_explorer25 @karpathy @rasbt @AndrewYNg @drfeifei @demishassabis @GaryMarcus As of today, I am active. And fulfilled of my signal to noise ratio. I'm excited to have someone write for me and keep me off this platform though. Despite global influence, algorithms just know too well. This one was truly more so because I find it fun.