Wenhan Xiong

ex @xAI @AIatMeta

[Simon Willison @simonw]

"Anthropic told investors on Wednesday distillation accounted for 5 per cent to 7 per cent of revenue"

If true, and Q2 revenue was $11.5B, then that's more than half a billion dollars spent by other labs to distill Anthropic's models!
https://t.co/szd2oJxtrv

[Byron Hsu @hsu_byron]

@periodiclabs Neon is an early glimpse of the future: physical labs owning their own AI that beats closed models and keeps getting better from their data.

At Periodic, we do hardcore atoms-and-bits work. We build **the full stack of AI for Science** — from physical labs, simulation tools, and lab software (atoms) to training, inference, RL, and sandbox infra (bits) — and we optimize across the full stack to squeeze the juice out of the data and GPUs. I am grateful for the close collaboration from @radixark @sgl_project @nvidia @raydistributed for the infra piece.

Periodic Neon is a 1T-parameter model that we post-trained to analyze experimental results and deployed to our physical labs. This is a step toward putting autonomous discovery into the hands of scientists.

[Wenhan Xiong @xwhan42]

The limits of my language (with my agents) mean the limits of my world.

Increasingly, the limit goes both ways

[Wenhan Xiong @xwhan42]

@Guodzh there’s actually a great sci-fi story about it. “story of your life” by Ted Chiang, diffusion indeed

[Wenhan Xiong @xwhan42]

Nice read about how IPO works if you are watching $SPCX

and congrats to everyone who has poured their hard work to the incredible mission of @SpaceX!

[Yishan @yishan]

Now is a good time to explain certain things about the mechanics of the IPO process that most people don't know. I will explain various dynamics around IPOs that you've probably wondered about (or just felt were odd but ignored).

To the financially sophisticated: this post elides certain details and attempts to be simple enough for a lay audience. There's no novel reveal at the end so if you already know how IPOs work, you can skip this post.

When a company wants to IPO (sell shares, then have the shares float for public trading - two separate things - hence this explainer), they don't actually just sell them to the public.

Rather, they hire bankers to round up a bunch of institutional buyers or wealth investors. We'll call these "big buyers." The company does presentations to these buyers, and then the buyers indicate their willingness to buy - like how much, and at what price - and the bankers mediate this whole process and arrange the transaction.

The reason this happens is because when most companies come to the market, no one really knows what price it should start trading at, and it's largely an unknown entity (to the public investors), so you need experienced bankers who understand business and equity markets to figure it out and get it approximately correct. This is the actual function of the bankers who "take companies public" and they are called underwriters.

The way it works is this:

The bankers go to the big buyers (often investors who already have a lot of business with the bank) and ask them to participate in this process - the one I described above. Buying the shares is risky, so the big buyers need an incentive. This is where the "first day pop" comes from.

Because the bankers have the most information of any party at this stage, they are the most likely to be able to guess at the likely "real" market trading price of the stock. So they price the stock a little bit under so that when the actual offering to the public happens on opening day, the stock "pops" by about 10% - 20%. This allows the big buyers to make an instant profit flipping the stock, compensating them for taking the risk of buying an "unknown" stock with no trading history.

Let me summarize:

- The company sells its shares in a negotiated sale to big buyers.
- The company receives cash for those shares.
- Now the company is done.
- The next day, the big buyers sell their shares on the open market at whatever price public buyers are willing to pay.

There are, typically, rules around flipping the shares but obviously not for everyone, not for all the shares, and some big buyers just break the rule (and may not be invited to future IPOs). But obviously the shares that the public actually buys comes from somewhere, and it's these big buyers who bought them from the company in the banker-negotiated sale. They do not come directly from the company.

At no point does any member of the "retail" investor public actually give money to the company itself in return for any shares. It all goes through the banker-mediated sale process (called a "road show") where big buyers get a discount for taking the risk, and make their profits on the first-day flip to the buying public.

(The bankers also make a huge fee for doing this)

That seems kind of like "rich people enriching other rich people" but it's only partially that.

Only a small percentage of IPOs do really well. Many of them stink, or fail on the first day with no pop. So if you're a big buyer who participates in a lot of IPOs, you are taking on a real risk. If all goes well, there is a modest 10-20% pop on the first day, and the IPO is deemed a success from the standpoint of the financial industry, and in particular, the bankers who arranged it.

Here are two major ways it can go wrong:

If the bankers misjudge where the public price will end up and price it too low, the pop will be HUGE. The common reaction to this is "Wow, what a great IPO!" because most people just get excited when Money-Number-Go-Up. But if the pop settles to something roughly close to its peak - the first-day close is taken as a proxy for what the market considers a fair valuation of the company - it means the company left money on the table: it sold to the big buyers at far too low of a price. All the big buyers and opening-bell first-day retail buyers captured a huge part of that value. Remember that the purpose of an IPO is to raise money for the company's operations and if they just sold a bunch of stock for less than the market was willing to pay, well, that's bad for the company.

Another way it can go "wrong" is if it's "priced to perfection" where the negotiated sale is arranged at exactly what the public ends up being willing to pay, and there is no pop. Even "worse," if it's priced above that, and the bankers completely misjudge the price in the wrong direction, the stock will fall on the first day and close below the IPO price.

But "wrong" is a matter of perspective: this is bad for the big buyers (who didn't make money on flipping the IPO), bad for the first-day retail buyers who now own a plunging stock, and bad for the bankers, who lose credibility. But it is the best financial outcome for the company itself. The company was able to sell its shares at the maximum price the market would bear, and it walks away with the cash.

When an IPO like this happens, the financial headline that dominates is "it's a failed IPO." But in terms of raising money for the company, it's the best-case outcome! This is an instance where the incentives of all the parties involved are not the same.

When the bankers price the IPO for a modest pop, that's the default compromise between these interests: the company makes a little bit less than it would get from the public, the big buyers take a risk and get a profitable flip sometimes, and the bankers lend their market expertise and get paid their fees. When it skews in either direction, one or more of those parties takes a hit - but the others benefit.

The reason that the "priced to perfection" scenario is often excoriated in the press is because the financial press is largely controlled by the financial industry. It's not a conspiracy, it's just that financial news will mostly ask their network (i.e. finance folks) to give their opinion, and because the finance folks (bankers or big buyers) didn't come out ahead, they think of it as a failure. But it's a Great Success for the company itself!

Outlier IPOs:

In my life, I've had a front-row seat to two outlier IPOs (Google and Facebook), and two "standard default" IPOs (PayPal and Reddit). I'll talk a bit about the interesting effects in the outliers, and how they compare to the defaults, and then a bit about what could happen with SpaceX.

One of the ways the default IPO process can vary is when a company is already very well-known to the public. Most IPO-ing companies are unknown to the public, and that's a big reason why the bankers have to be involved: they form a bridge of trust to the big buyers and bring value in their specialized expertise about market sentiment.

But a company that's already very well-known doesn't get as much value from that. Especially if there's already demand for the company's shares, the company can often find enough buyers for its offering. The contract terms (services, fees) for underwriting an IPO are always negotiable, and so certain companies can negotiate lower fees and do things differently.

Google did this in 2004.

Now, one funny thing that's typically true in a default IPO is that the stock will open between $15 and $25. The reason for this is that most people are not financially sophisticated and if a stock opens at $100, they will think it's too expensive.

The real value of the stock is what percentage of the company it represents + the company's financial performance. So the numbers $15 and $25 are chosen because most people will think that's a reasonable price to buy - they compare it to buying something at the store. No joke.

Now, because companies and their existing stock can have a large range of values, what they do prior to the offering is simply do a stock split (or reverse stock split) so that the effective per-share price falls into that range. It's entirely just optics because most people don't understand math and finance.

In 2004, Google IPO'd at $85/share.

If you are thinking "omg, that's a lot!" then you are one of the people I just described. It doesn't matter that it was $85/share.

Google, because it was well-known and there was a lot of demand for its stock, did not have the underwriting bankers negotiate their sales to the big buyers! Because they had a lot of internal expertise (and preference for) fancy auction mechanics as a price discovery mechanism, they set up a "Dutch auction" for their shares.

Briefly, the Dutch auction is an auction format that is considered better at reaching the real market value of whatever's being sold (compared to a regular auction, which seeks to maximize the buying price).

They ran this Dutch auction and asked everyone who wanted to invest to submit their bids and amounts, and then assigned a price and (modulo some regulatory details) opened at $85/share.

This was the first time in tech for an "unusual" IPO. It was met with positive regard because Google didn't have to pay the bankers as much money, probably got a fairer price for its shares, and the buying public got in at a reasonable price, cutting out a lot of middlemen (e.g. big buyers, though they sold to the big buyers too). And Google was known for innovation and being quirky, so this fit their brand.

Today, by the way, the split-adjusted price for that offering is about $2/share.

Facebook also had an unusual IPO process.

Facebook engaged the underwriters from a position of absurd negotiating superiority. They were already globally known, and was probably the company most well-known at IPO (in terms of name recognition) in history.

Typical banker fees for underwriting can be ~4%. Facebook reportedly negotiated an underwriting fee of 1%.

Why? Because there was massive demand for its shares, and everyone already knew what Facebook was about. So who cares about the bankers?

Not only that, but Facebook priced itself to perfection.

It opened at $38/share, and closed at $38.23/share, implying that Facebook had exactly hit the market price and gotten the maximum amount of money, with nearly no spread between what Facebook sold for and what the public ended up paying for it.

Further, over the next few months, its stock trended downwards. This caused no end of hand-wringing from people who bought on first-offering, but it implies even more strongly that Facebook got top dollar for selling its shares.

(Anyone who held on for longer 16 months after that saw huge gains - today the price is at ~$590)

The financial press absolutely excoriated the Facebook IPO, calling it a huge failure. This drove the mainstream conversation about it, which also depicted it as a failure, highlighting stories of investors like an old lady who'd put her life savings into the IPO (.... which you are never supposed to do). The bad press went on for months.

At the same time, Facebook execs and informed insiders quietly understood that it had been a perfectly-executed IPO, in terms of raising money for the company.

And, if people like the old lady held on to her stock for a couple years, she still made mad bank.

Those were the outliers. Now the regular ones:

One of the features of an IPO is that typically most shareholders are subject to what is called a "lockup."

The default lockup is often for 6 months, but the terms can be negotiated. During the lockup, shareholders cannot sell their shares.

To understand this, first realize that "shareholders in the company" are different from the company itself.

In an IPO, the company (the corporate entity) issues stock and sells it to investors, taking in cash to fund the company's operations.

This is different from shareholders of the company - existing investors, employees, and executives - selling stock. These parties personally own stock (i.e. ownership) in the company and if they sell it the cash goes to them, not the company.

The lockup typically applies to some or all of these parties, and the reason is because when the company floats shares in its offering, if on the next day (or month) many large shareholders were to also sell their shares - some of which could be a block of comparable size to the IPO offering itself - it would tank the market. This would reflect very poorly on the company because it would mean that all the investors who bought in the IPO (big buyers, but also people who believed in the company and bought on the first day/week) would see steep declines while "insiders" made off with profits.

But the exact configuration of lockups varies, because it's all negotiable.

The common default is that most private company shares are locked up for 180 days.

Sometimes, the shares floated (sold to the big buyers) for the IPO aren't newly issued shares by the company. Sometimes the major shareholders negotiate to sell some percentage of their holdings - say 10% - and those shares are the ones sold to the big buyers and then later into the regular market. The rest of the shares held by the shareholders may remain subject to the lockup. The negotiation ends up balancing the desire of shareholders (prior investors, executives, employees) for liquidity vs the signaling effect it has on the market - no one wants an IPO to look like insiders dumping on the market.

When Reddit went public, the underwriting situation was pretty vanilla (road show, sell to big buyers, modest pop on first day).

The shares offered for sale at the Reddit IPO weren't all issued by the company, a significant component were employee shares. Many employees had been at the company for a long time, so a program was set up whereby the employees could elect to sell some percentage of their vested holdings, and these were some of the shares offered to the big buyers. All of the large existing shareholders - the venture capitalists and mutual funds that had already bought into Reddit at far below the IPO price - didn't sell a single share in the IPO.

(Subsequent market performance seems to have borne out the financial wisdom of that decision)

One thing to understand here then is the divergent effects on the company vs existing shareholders. If the company is "priced to perfection" and subsequently the stock price falls, and existing shareholders did not participate in the IPO sale itself, they are in the same boat as retail investors: the stock value is dropping. Further, if they're subject to a lockup, they have no way to exit the stock for a long time.

====

Now that we have all that background, we can talk about the SpaceX IPO:

SpaceX is an outlier, if for no other reason than the fact that size of the offering is the largest in history. When outliers happen, rules often get broken. Not because of corruption (though sometimes it's that), but because an outlier will often create conditions that are outside the anticipated range of what the existing rules were set up to handle.

The first thing is that SpaceX is one of those "already really well-known" companies and one with a lot of pent-up demand for its stock. In the last few years, SpaceX funding rounds have been massively oversubscribed.

This means that SpaceX is in a position to not only negotiate the sorts of terms that Facebook got with its underwriters (very low fees), but it has negotiating power on key terms like pricing, sizing, and lockup periods.

Remember that in terms of "cash raised" in the IPO, the amount the company raises is simply the amount they sell to the big buyers for, NOT how much the stock trades up (or down) once the markets open.

Elon's stated intention is that the IPO is necessary to raise the huge amount of funds needed to complete Starship and fund a mission to Mars.

People can quibble about whether that's his main motivation or if he's just grifter unloading on the retail market, but it's a very telling point that his actual compensation package involves actual Mars-based metrics like establishing a colony with a million people on it. If he's a grifter, and he basically controls his board, he there'd be no need for a comp package like that.

So if the goal of the IPO is not to cash out for insiders, but actually "raise a huge amount of money for the company to carry out its insanely ambitious goals," there would be a strong incentive to "price to perfection," i.e. push the bankers to price the stock at what they think the market really will bear, and reduce the profit the big buyers would make on the first-day pop.

And if any hiccup occurs, the stock could tumble, much like what happened with the Facebook IPO - but SpaceX itself would have the cash it needs.

Based on what I've explained much earlier, you can now also see that if the stock being floated in the IPO is newly-issued by the company and none of the existing shareholders are allowed to sell into the IPO, and the IPO is "priced to perfection," it's less likely that it's a dump on retail investors, because the stock will tumble before any of the major shareholders can sell. The company as an entity makes cash, but its shareholders share the fate of the market (actually slightly worse because of the lockup's effects on their liquidity).

On the other hand, having learned from that, SpaceX might not want a year of bad press, with the entire financial press discussing how bad an investment SpaceX is. Elon and SpaceX already have to fight a culture war and lots of people demonize them. So there's a chance the pricing has been set up to be something like the default - a modest pop on the first day. The question is basically whether the company wants to optimize for cash or public perception - compelling arguments for both could be made.

Having said all that, people are probably underestimating the degree of retail investor interest. The allure and romance of space flight, the exploration of space - all of those are long-held dreams that are older than Google or Facebook or even the internet itself. Mankind has dreamt of walking among the stars for decades.

Although the smart money makes decisions on the basis of P/E ratios and the like, a regular Joe with a Robinhood account who has dreamed of space and remembers the magnificence of seeing twin rocket boosters landing side-by-side will probably want to grab a few shares if he can. A LOT of people probably feel this way, and not many of them will be able to get IPO allocation.

Thus, it's possible that no matter where the offering price is set, there will be an absolutely insane, possibly record-setting pop on the first day. SpaceX is not just a selling Starlink, or compute or whatever you think - SpaceX is selling dreams. And it has been steadily making them real.

Incidentally, if this happens, after the euphoria wears off, the stock will probably tumble, providing lots of fodder for negative news coverage.

SpaceX's lockup policy is also unusual. Instead of either allowing some shareholders to sell immediately, or locking everyone up for 180 days, there is a staged and gradual unlock over the span of the 180 days, with a fraction of one's holdings allowed to be sold. One of the stages even requires that the stock price be over some threshold, presumably to hold the stock price in a certain range of values. It's unclear how this staged unlocking will affect price dynamics; it feels like an engineer's solution to trying to manage market volatility. (My suspicion is that the magnitude of public sentiment - both positive and negative - will drive more of the volatility than any pricing or lockup schedule)

Well, now you know everything I know about IPOs.

If I were to guess at outcomes, my probability distribution is:

70% likely to see a huge first-day pop (sustained for at least a week), and 30% likely that it's priced to perfection and closes below its IPO price.

This situation is such an outlier and all of the conditions necessary for any of those things to happen are in play, and it's not clear which forces will dominate.

Either way, good luck!

🚀

[Guodong Zhang @Guodzh]

SpaceX, even now, feels like something that should only exist in sci-fi. It will surely inspire generations of kids to dream bigger.

Congrats @elonmusk and @SpaceX. Looking forward to a future where we, as a civilization, harness a meaningful fraction of the sun’s energy.

[Andrej Karpathy @karpathy]

In awe of SpaceX and its story - past, present and the future. You can think about it in 10+ different ways and continue re-blowing your mind in circles. Huge congrats to the team! 🚀

[Yunlong Liu @LiuYunlong63318]

@Guodzh @_arohan_ @itsclivetime Thanks Guodong for the clarification.
Essentially both JAX and torch become kernel launchers and JAX runtime is fully in c++. As GD mentioned, implementing pp in a customized jax stack can either by lifing the dim outside jit or arranging pp stages inside jit with shard_map.

[Lisan al Gaib @scaling01]

Claude Mythos is insanely token-efficient

[Wenhan Xiong @xwhan42]

@Guodzh What a legendary run, Guodong! It’s sooo sad to see you go. I’m grateful for the journey we shared over the past years. It’s been a great honor!

[Wenhan Xiong @xwhan42]

@JustinLin610 thx for all the great contribution to open research!

[SpaceX Super Intelligence @SpaceXSI]

Understanding requires imagining. Grok Imagine lets you bring what’s in your brain to life, and now it’s available via the world’s fastest, and most powerful video API: https://t.co/tqQwQVgCEI

Try it out and let your Imagination run wild.

[Hexiang (Frank) Hu @hexiang]

@Arden_2210

[Wenhan Xiong @xwhan42]

@ying11231 @radixark @lmsysorg @slime_framework @jxwuyi @xai Congrats!

[Ying Sheng @ying11231]

We've been running @radixark for a few months, started by many core developers in SGLang @lmsysorg and its extended ecosystem (slime @slime_framework , AReaL @jxwuyi). I left @xai in August — a place where I built deep emotions and countless beautiful memories. It was the best place I’ve ever worked, the place I watched grow from a few dozen people to hundreds, and it truly felt like home. What pushed me to make such a hard decision is the momentum of building SGLang open source and the mission of creating an ambitious future, within an open spirit that I learnt from my first job at @databricks after my PhD.

We started SGLang in the summer of 2023 and made it public in January 2024. Over the past 2 years, hundreds of people have made great efforts to get to where they are today. We experienced several waves of growth after its first release. I still remember the many dark nights in the summer of 2024, I spent with @lm_zheng , @lsyincs , and @zhyncs42 debugging, while @ispobaoke single-handedly took on DeepSeek inference optimizations, seeing @GenAI_is_real and the community strike team tag-teaming on-call shifts non-stop. There are so many more who have joined that I'm out of space to call out, but they're recorded on the GitHub contributor list forever. The demands grow exponentially, and we have been pushed to make it a dedicated effort supported by RadixArk. It’s the step-by-step journey of a thousand miles that has carried us here today, and the same relentless Long March that will lead us into the tens of thousands of miles yet to come.

The story never stops growing. Over the past year, we’ve seen something very clear:
The world is full of people eager to build AI, but the infrastructure that makes it possible is not shared. The most advanced inference and training stacks live inside a few companies. Everyone else is forced to rebuild the same schedulers, compilers, serving engines, and training pipelines again and again — often under enormous pressure, with lots of duplicated effort and wasted insight.

RadixArk was born to change that. Today, we’re building an infrastructure-first, deep-tech company with a simple and ambitious mission:
"Make frontier-level AI infrastructure open and accessible to everyone."

If the two values below resonate with you, come talk to us:

(1) Engineering as an art.
Infrastructure is a first-class citizen in RadixArk. We care about elegant design and code that lasts. Beneath every line of code lies the soul of the engineer who wrote it.

(2) A belief in openness.
We share what we build. We bet on long-term compounding through community, contribution, and giving more than we take.

A product is defined by its users, yet it truly comes alive the moment functionality transcends mere utility and begins to embody aesthetics.

Thanks to all the miles (the name of our first released RL framework; see below).

https://t.co/2vio4Eiiac

[Wenhan Xiong @xwhan42]

@WenhuChen congrats!

[Wenhan Xiong @xwhan42]

@zhs05232838 Nice work, congrats!

[Wenhan Xiong @xwhan42]

@WenhuChen 👀

[Yuandong Tian @tydsh]

Several of my team members + myself are impacted by this layoff today. Welcome to connect :)

[Wenhan Xiong @xwhan42]

@aurko79 This explanation feels kind of superficial, how this connected to the lmsys event