Eric Xing

Researcher, educator, entrepreneur, and administrator in computer science, artificial intelligence, and healthcare.

[Institute of Foundation Models @IFM_AI]

The most detailed open LLM data release yet: K2 Horizon's pre-training data and post-training data-generation pipelines are now open!

13 trillion pre-training tokens across 5 datasets and 24 subsets. Over 10 trillion tokens of synthetic data spanning mathematics, reasoning, coding, planning, and problem-solving.

Beyond the scale, four highlights for model builders (details in the threads):

- Scaling synthetic data without losing diversity. Our generation methods produce diverse training examples at scale, nearly matching natural web text in measured diversity.
- More verification isn't always better. Strict verification can throw away useful training signal.
- Small-scale wins can mislead. Our chosen data mix actually lost in smaller experiments.
- Beyond teacher imitation. Our post-training data generation methods produce training trajectories the teacher couldn't generate on its own.

Open pre-training datasets, post-training data-generation pipelines, and processing tools for researchers to inspect, reuse, and build on.

Pre-training data: https://t.co/3Lb28JhyG9

Post-training data-generation code & tools: https://t.co/YWgJpOnzVe

[Institute of Foundation Models @IFM_AI]

Jais-ASR ranks #1 on the Open Universal Arabic ASR Leaderboard.

Our open model is built for Modern Standard Arabic and dialects from 19 countries.

Beyond the leaderboard, it handles:
- Natural code-switching with English, French, Spanish, and Italian
- Robustness to background noise
- Faithful transcription, resistant to hallucinations
- Arabic diacritization
- Built-in language and dialect identification

Weights released under Apache 2.0: https://t.co/kXjQEqVXvU

[Eric Xing @ericxing]

Time also for university leaders to reflect due process, apoliticalness, meritocracy and many many now unfashionable principles that used to make universities great, fair, safe, and inspiring. What happened to Dr Sabatini hurts not only him and his family, it hurts science, universities, academia, and public trust of academic institutions, and many other women and men who are the real victims of professional misconduct but may have to experience mistrust because of such acts as from Dr Sabatini’s accuser. Universities must do better in upholding integrity and due process.

[The Boston Globe @BostonGlobe]

A civil jury ruled in favor of ex-MIT scientist David Sabatini, who was accused by a younger colleague of inappropriate sexual conduct in his Cambridge research lab, and awarded him $6 million in damages. https://t.co/lXDcUxIKmx

[Institute of Foundation Models @IFM_AI]

Jais 3 is built for Arabic from the start.

Speech: the most accurate open Arabic ASR, across MSA and dialects

Writing: fluent, native Arabic from the first line to the last

Agents: leads open models on Arabic agentic tasks

Download weights: https://t.co/Q2ZI1PJxAW

[Institute of Foundation Models @IFM_AI]

Huge thanks to the vLLM team!

With Day-0 support on @vLLM, Jais 3 now runs on both of the serving stacks most teams already have.

Weights: https://t.co/Q2ZI1PIZLo
API: https://t.co/W6IrYtBTGD

[Institute of Foundation Models @IFM_AI]

Introducing Jais 3, Arabic-native agentic AI.

Jais 3 375B matches the best open models, and with Jais Speech offers the most advanced Arabic AI stack available.

→ Hugging Face: https://t.co/Ecvx8FhZvG
→ Technical blog (Jais 3): https://t.co/9bHZeFSU2D
→ Technical blog (Jais Speech): https://t.co/SuiqdPKrF2
→ Learn more: https://t.co/ZIAg1xmS6l

Jais 3 was created by the Institute of Foundation Models (IFM) at MBZUAI, in collaboration with Inception42 and Cerebras Systems.

#Jais3 #IFM #MBZUAI

[Michael Jordan @JordanMich45361]

(1/7) I'd like to provide some additional context for the remarks that can be found in this interview.

(2/7) My discomfort with the attention being paid to existential risk---and with the ongoing dominance of the public discussion of such risks by a few so-called thought leaders---is that there's an elephant in the room that is being missed. That elephant is the effect of all of this discussion on young people, and indeed the effect---beginning with social media a decade ago---of information technology on young lives broadly speaking.

(3/7) I recently read an interview by Jill Lepore in which she reports on discussions that she had recently with a group of college students: "They had just three things to say. One, social media completely destroyed their childhoods. Two, AI is destroying their college education and their likely prospects for employment. And three, what the hell are we all doing?" To add a fourth bullet that I've heard in similar discussions: "I'm not going to have children, because the existential risk is just too high."

(4/7) Please contemplate such a statement, those of you who think that it's OK that the public, widely diffused, discussion coming from seemingly authoritative figures is so adamant about existential risk.

(5/7) Let me add that I have little interest in a debate about what "superintelligence" might mean, or whether we're currently seeing the emergence of superintelligence. For what it's worth, I would certainly agree that the solution of extremely challenging math problems (for example) exhibits a very impressive form of intelligence, although I would want to argue that such intelligence is impossible to dissociate from the collective of humans that provided data to the machine learning system (which itself has become a kind of collective).

(6/7) Such philosophical debates are entertaining and even important, but when they then head in the direction of "when such superintelligence emerges, there's a good chance that it will want to kill all humans", I say "science fiction". Again, I don't think that it's bad to consider such thoughts, but I strongly believe that it's bad for society, and especially young people, for such thoughts, unaccompanied by any serious real-world analysis, to become a dominant form of discourse of the era.
(7/7) We are damaging human lives, and the consequences of that damage will no doubt be a major theme of the next several decades.

[nxthompson @nxthompson]

This summer, the Institute for Progress and the Foundation for American Innovation organized a letter urging Congress to help prevent the misuse of AI for biological weapons. Dario Amodei, Sam Altman, and Demis Hassabis all signed. But @ericxing, a renowned computer scientist who, among many other things, cofounded an AI-driven biology company, didn’t. I asked him why.

Watch our full conversation here: https://t.co/efaAPc1vvo

[steve hsu @hsu_steve]

Michael I. Jordan, machine learning demigod, UC Berkeley

... the young people who work at these companies operate in a world [Silicon Valley, Berkeley, etc.] where these kinds of discussions are commonplace. Many of them aren’t really engineers; they come from the humanities, philosophy, or other disciplines. They’re 25 years old, they’re naive, they don’t do engineering work, they’ve never worked at large companies where real products are developed for real people, with discipline. And they get together, a bit like in Parisian cafés in the 1920s, to talk about these grand science-fiction ideas. When these young people resign, they know they’re going to attract attention. It allows them to feel virtuous and to be seen as very important figures. They claim to be giving up money, but in reality, they already have their stock options. So all of this is a grand moralizing performance that doesn’t serve the real objective it claims to pursue. There’s a great deal of naiveté here, which goes hand in hand with a certain cynicism…

https://t.co/GwP9zcz7vZ

[Eric Xing @ericxing]

It was a true pleasure to sit down with @nxthompson of Atlantic to share my thoughts on Open Source AI, World Model, and AI for Science. If you like to know my views on what to expect from a world model and how it should be built; how to transform medicine by simulating life across scales using a biological world model and virtual cell, and transparency and reproducibility in AI, the full-length conversation is available bellow.

Nick is an extraordinary journalist and deep thinker with a quick mind and sharp questions! Thank you for having me in your studio and look forward to catching up again.

[nxthompson @nxthompson]

You might think that @ericxing, one of the most prominent advocates of open-source, would support China’s approach to AI, since all of its leading models are open. Instead, he says those models give the illusion of openness, but actually hold their users hostage.

We had a fascinating conversation about the world models he’s building, and his strong views on open-source: https://t.co/efaAPc1vvo

[Eric Xing @ericxing]

I am with @pmddomingos , it is about who discover it last when it stays discovered!

[Pedro Domingos @pmddomingos]

@rsalakhu By the Columbus test, the real discoverer of something is not the first one to discover it but the last (when it stays discovered).

[Institute of Foundation Models @IFM_AI]

Rework is inevitable in large-scale LLM pre-training and fine-tuning. What it costs isn't.

Introducing xLLM, an efficient and flexible infrastructure for pre-training and fine-tuning dense and MoE LLMs. It keeps key training decisions changeable without giving up throughput.

Efficient, at 6,295 tokens/sec per GPU on K2-Horizon-MoVA-36B-A4B and 10,050 tokens/sec per GPU on Llama3-8B on H200s.

Flexible, because the tokenizer, data mixture, model architecture, and training stages can change without rebuilding the dataset or the system around them.

xLLM ships with the checkpoints, training logs, and recipes behind K2 Horizon: https://t.co/FJ2mg3OMdb

K2-Horizon-MoVA-36B-A4B: https://t.co/lH2BYBdmaC

[Institute of Foundation Models @IFM_AI]

How big is this model? At @ICML in July we previewed K2-Horizon-3.7B and 7B and asked attendees to guess their size from watching them conduct real-work and coding tasks. Average answers, 89.9B and 62.6B.

Both off by an order of magnitude.

Density of intelligence matters more than parameter count now. The need for capable models at smaller sizes, lower cost and higher speed was overlooked. K2 Horizon is built around it.

Try them:
https://t.co/VuT1kI0nyK; https://t.co/aAdKO9tKLJ

[Eric Xing @ericxing]

Let me copy-paste my LinkedIn post here — not a solution, but a point worth debating:

In light of recent news that 60K+ submissions have flooded #ICLR, I resurface this thread, along with my comments, for discussion and debate. I know it is controversial and I don't have a perfect solution, but the system will break if we don't do anything. It is important to rethink "technical conference" operation:

Same as bellow (aggressive desk rejection, like 80-90%) should be applied to conference submissions, NeurIPS, ICML, ICLR, ACL/EMNLP, CVPR/ICCV ...to spare the reviewers (or their AI-agents) reading these submissions and produce a review, all top tier conferences should do this, or the conferences will be soon (possibly already) be destroyed by the tsunami of random submissions.

If they are the biggest, how they can at the same time be the best? The problem started long ago from the soft position against low quality submissions by the PCs, under a cultural/political (not a operational/technical) excuse (everyone deserves attention, respect, not missing a single good one, not smashing a young dream, etc), which is predictably exploited by gamers to flood the system. To be honest, for experienced senior reviewers, determining a poor paper takes no more than 3 minutes just from title, writing style, author credential, an wrong chart, etc, it is not that they can not determine, but because they are afraid to be retributed with holy excuses. A good venue does not entitle a whole community for a spot or full attention, can miss and turn down good submissions, but should not publish bad submissions, like Nature, Science, Annals of Mathematics ... -- why they are not flooded? Because people know these journals will desk-reject obvious low quality submissions from editors not afraid of holding the bar high.

So if the top conferences still want to exist, be meaningful and do their job of providing scientifically credible assessment/endorsement of the accepted papers, the key is to empower and protect PC/ACs, to fully exercise their judgement without fear, so the low quality submitters can learn that they can game nothing, and can even pay a price. I tried to advocate this tougher process in ML community years ago without success, but with current disastrous situation of all conferences, hopeful we can raise the bar substantially— 80-90% desk rejection, 10-20% acceptance of the remaining, until situation becomes better. Nothing stops people from publishing, people who wish to share their work to public can use platforms like arXiv, OpenReview, Github ..., faculty promotion, student application, can still refer to those publications if they made an impact there.

[Institute of Foundation Models @IFM_AI]

Michigan builders: K2 Horizon is coming to the University of Michigan this Saturday. ⛰️

Build with IFM’s open 375B-A23B flagship in the “Best Use of K2 Horizon” challenge.

🏆 100M tokens for every winning team member
📅 Sept. 19 | 9 AM–9 PM
📍 East Hall 1324

#K2Horizon #OpenSource

[Eric Xing @ericxing]

Glad to be quoted here (thank you @CNN and @TM_Brown!) of my standing on AI dystopia -- an idea, a software (which AI is), a blueprint on a file (which AI generates), is NOT a physical actuation. Closing this gap takes a long way and can/should be guardrailed. I'd quote (approximately) words of wisdom I remembered from X:

"regulate the world of atoms, not algorithms; penalize the effects from misalignment, not misalignment; and monitor supply-chains/factories/infrastructures in real world, not blueprints on files"

[Teddy (T.M.) Brown @TM_Brown]

A lot of people are saying AI is going to kill everyone. But how exactly?

For @CNN I talked to some influential doomers about how the world will end—and some leading experts in existential risk about why they think that prospect is pure science fiction.

https://t.co/rp4s0gH9U9

feat. @HeidyKhlaaf @So8res @HerbLinCyber @ericxing @thlarsen

[GenBio AI @genbioai]

Out today in @cellpressnews: "A world model of the virtual cell," from our co-founders @ericxing and @dasongle.

Their definition of a virtual cell is a stateful, multi-modal, multi-scale simulator, judged on whether it stays coherent across a sequence of interventions.

Open access: https://t.co/BLHUOmQHki