Longxuan Yu

Generative AI researcher & builder | Diffusion language models, efficient inference & reliable AI

[Longxuan Yu @Loy004Yu]

@ciruai Let's see!

[Longxuan Yu @Loy004Yu]

I also try so many test for what is the difference between dllm and ar. I think beyond fast may be representation is more robust, and dllm is good at global constrain task. But for most of text problems, the constrain could be expressed as long text. Like what has been shown in https://t.co/VWx6eL8y5o
If a task could be expressed as a text problem, if we could give ar more time, it should be possible for ar to solve it. Even the problem in this paper, I have tried with nanbeige 4.2, the model could solve it as the same level with the fine-tuned llada.

[Longxuan Yu @Loy004Yu]

@StamblerLev This block size is only 8. I don't do any tuning for this, I think curriculum learning for block like dream reasoner's strategy maybe helpful here, so could be explored here.

[Longxuan Yu @Loy004Yu]

Hot take: the community keeps asking whether diffusion LLMs can beat AR. That's the wrong question. Diffusion is a sampling method, and it can sit on top of an AR model to make it faster instead of replacing it.

Speculative decoding already speeds up AR, but it has two long-standing problems. Training a drafter takes a long time, and you have to build a complete pretraining corpus for it. Even then, token acceptance drops sharply on out-of-domain data.

Uno @ssahoo_ offers an elegant way out: it decouples sampling training from capability training. There's no separate drafter. The model itself, plus one LoRA, drafts several tokens in a single forward pass, and the LoRA learns to match the model's own next-token distribution, so there's no drafter corpus to build and no labels to collect.

We applied this to the Looped Transformer Nanbeige4.2-3B: 1 hour of LoRA training gives a lossless 2× speedup on agentic task. We haven't optimized anything yet, and we believe the acceleration can go much further.

The takeaway: diffusion doesn't need to beat AR. Any model that can score its next token can get a fast diffusion mode in an hour.

Page: https://t.co/SCkbkrtf2z
Lora Adapters: https://t.co/Awks2jdkx0

[Longxuan Yu @Loy004Yu]

@Laz4rz I think the conversion is not new (DiffusionLLaMA → Dream → SDAR → LLaDA 2.2), but the cost was always high before. And diffusion training is expensive and would hurt capacity. I think DFlash is a good way, but training the drafter is still too costly.

[Longxuan Yu @Loy004Yu]

Nice question! It could be hardware optimzied and software optimized. I think at the end of day 2-3x time speed up would be possible for general agentic task, and 5-10x time speed up for low-entropy task generation.

First, Uno drafting is a large model drafting that is expensive. There should some way to make this draft cheaper.

Second, token accpetence could be larger with larger block, that is underexplored now.

Third, there is a lora load and unload process. I think there is some kernels scheduling which could be optimized deeper.

[Longxuan Yu @Loy004Yu]

@ssahoo_ Hi Sahoo, I just reproduce the Uno lora training, the result is pretty nice! Easily I get 2x speed up for agentic task for a loop transformer model.
https://t.co/RUvG39ZqF9

[Nanbeige @nanbeige]

🚀Excited to announce Nanbeige4.3-33B!
To push the performance ceiling of small agents, our strategy in Nanbeige4.3 starts with building a powerful teacher first.
Nanbeige4.3-33B delivers SOTA agentic performance among open models at ~30B parameters:
* Strong results on agentic coding and cowork benchmarks.
* When integrated into mainstream agent harnesses, it tackles real-world tasks with cost-efficient inference.
* It will serve as a teacher for upcoming smaller Nanbeige models, starting with Nanbeige4.3-3B.

https://t.co/tSToQgZf2O
https://t.co/D228euvybz

#LLM #Agent

[SparkLLM @SparkLLM]

🎉 Spark-X2.6-4B reached #4 on Hugging Face Trending!
Snapshot: All · last 7 days, Oct 25, 2026.

Thanks for your support, hands-on reports & constructive feedback. If you find it useful, give it a ❤�� Like:
https://t.co/5GdI4SGeTo

[Sander Dieleman @sedielem]

The magic of flow map language models is that they capture dependencies between tokens in just 1-2 model passes.

Using this as a drafter for speculative decoding makes a ton of sense, and it's exciting to see this working as well as it does! (Love the shuffle test as well btw!)

[Sander Dieleman @sedielem]

A key difference between continuous and discrete diffusion for language is that the former is (in principle) able to represent uncertainty in intermediate noisy states (i.e. superpositions of possible outcomes).

Binary discrete diffusion is another way to achieve this. Neat!

[Longxuan Yu @Loy004Yu]

@Chonghan_Liu wow 365 following vs 2000 follower

[Longxuan Yu @Loy004Yu]

@missckroach follwowed

[Institute of Foundation Models @IFM_AI]

The most detailed open LLM data release yet: K2 Horizon's pre-training data and post-training data-generation pipelines are now open!

13 trillion pre-training tokens across 5 datasets and 24 subsets. Over 10 trillion tokens of synthetic data spanning mathematics, reasoning, coding, planning, and problem-solving.

Beyond the scale, four highlights for model builders (details in the threads):

- Scaling synthetic data without losing diversity. Our generation methods produce diverse training examples at scale, nearly matching natural web text in measured diversity.
- More verification isn't always better. Strict verification can throw away useful training signal.
- Small-scale wins can mislead. Our chosen data mix actually lost in smaller experiments.
- Beyond teacher imitation. Our post-training data generation methods produce training trajectories the teacher couldn't generate on its own.

Open pre-training datasets, post-training data-generation pipelines, and processing tools for researchers to inspect, reuse, and build on.

Pre-training data: https://t.co/3Lb28JhyG9

Post-training data-generation code & tools: https://t.co/YWgJpOnzVe

[Longxuan Yu @Loy004Yu]

@sedielem Hhh, want to share this paper https://t.co/mHw3al2c1i

[Longxuan Yu @Loy004Yu]

https://t.co/bAnpksS85M We also have tried this, one bottle neck is dependency, I think for poem, vision multi objects understanding, this would be nice. line ar provides sentence modeling, and and multi line dependency could depend on image. Entropy could be used to defer some token.

[Wei Guo @WeiGuo01]

Another great work on large-scale continuous diffusion LLMs 👀
Also huge thanks for sharing our work Sigma https://t.co/SzrFQTD1yc in your post! I learned a lot from the insightful discussion with @chandavidlee and @wognsfjq96 during ICLR 2026 on flow language models

[Hongcan Guo @HongcanGuo]

1/5 A little self-promo 🥹 Excited to share Cola2, our follow-up to Cola!

A hierarchical framework bringing continuous semantic modeling and discrete text generation together.

🌍 Project: https://t.co/TNZXTSXAH8