Cong Zhou

@CongZhou1

Researcher @Anuttacon_

Bay Area

Joined January 2020

416 Following

134 Followers

96 Posts

CongZhou1 retweeted

Xie Zhifei

@XieZhifei14110

about 1 month ago

Hi everyone, 如果你的项目中有语音识别，你一定要看这个帖子。在去年的一些项目中我们发现，面对远场、混响、电子录制杂音等场景时，即使是目前最好的开源/闭源模型表现也不理想，常常出现大范围幻觉和句子丢失（最典型的就是远场场景，识别率往往不到一半）。为此我们做了 Mega-ASR。我们首先构建了 Voices-in-the-Wild-2M 数据集，以 7 种经典音频扰动为基础扩展出 54 种叠加场景，并通过 A2S-SFT 和 DG-WGPO 方法对模型进行了大规模的后训练。实验结果显示，Mega-ASR 在各项指标上相比开源/闭源的顶尖模型有 10%–30% 的明显提升。更重要的是，在很多人耳都难以听清的场景下，经过专项训练后，它在整体语义恢复、降低内容缺失和幻觉方面都有显著改善。如果大家的应用中遇到类似场景，欢迎尝试。 Mega-ASR 已完全开源！技术报告：https://t.co/gzopfj9oUp 项目主页：https://t.co/eynzC9fB71

520

523

88K

CongZhou1 retweeted

Ailing Zeng

@AilingZeng81332

3 months ago

Introducing LPM 1.0 — a video-based character performance model that speaks, sings, listens, reacts, and emotes in real time. - Generating full-duplex conversation, identity-consistent infinite-length generation, and nuanced human-like performance. - Building across a co-designed data pipeline, Base model, Online model, and streaming inference optimization. - Key advantages over other video generation models: performance quality, emotional conversation, precise lip-sync, identity preservation, and lifelike naturalness. Turning an image into a performance video, LPM 1.0 serves as a visual engine for conversational agents, live streaming characters, and game NPCs. Page: https://t.co/Ve2c2YNuqj Arxiv: https://t.co/cM54T3KSPs

288

323

25K

CongZhou1 retweeted

Nathan Lambert

@natolambert

7 months ago

We present Olmo 3, our next family of fully open, leading language models. This family of 7B and 32B models represents: 1. The best 32B base model. 2. The best 7B Western thinking & instruct models. 3. The first 32B (or larger) fully open reasoning model. This is a big milestone for Ai2 and the Olmo project. These aren’t huge models (more on that later), but it’s crucial for the viability of fully open-source models that they are competitive on performance – not just replications of models that came out 6 to 12 months ago. As always, all of our models come with full training data, code, intermediate checkpoints, training logs, and a detailed technical report. All are available today, with some more additions coming before the end of the year. As with OLMo 2 32B at its release, OLMo 3 32B is the best open-source language model ever released. It’s an awesome privilege to get to provide these models to the broader community researching and understanding what is happening in AI today. Base models – a strong foundation Pretraining’s demise is now regularly overstated. 2025 has marked a year where the entire industry rebuilt their training stack to focus on reasoning and agentic tasks, but some established base model sizes haven’t seen a new leading model since @alibaba_qwen's Qwen 2.5 in 2024. The Olmo 3 32B base model could be our most impactful artifact here, as Qwen3 did not release their 32B base model (likely for competitive reasons). We show that our 7B recipe competes with Qwen 3, and the 32B size enables a starting point for strong reasoning models or specialized agents. Our base model’s performance is in the same ballpark as Qwen 2.5, surpassing the likes of Stanford’s Marin (@stanfordAILab) and Gemma 3 (@GoogleDeepMind), but with pretraining data and code available, it should be more accessible to the community to learn how to finetune it (and be confident in our results). We’re excited to see the community take Olmo 3 32B base in many directions. 32B is a loved size for easy deployment on single 80GB+ memory GPUs and even on many laptops, like the MacBook I’m using to write this on. A model flow – the lifecycle of creating a model With these strong base models, we’ve created a variety of post-training checkpoints to showcase the many ways post-training can be done to suit different needs. We’re calling this a “Model Flow.” For post-training, we’re releasing Instruct versions – short, snappy, intelligent, and useful especially for synthetic data en masse (e.g. recent work by Datology @datologyai on OLMo 2 Instruct), Think versions – thoughtful reasoners with the performance you expect from a leading thinking model on math, code, etc. and RL Zero versions – controlled experiments for researchers understanding how to build post-training recipes that start with large-scale RL on the base model. The first two post-training recipes are distilled from a variety of leading, open and closed, language models. At the 32B and smaller scale, direct distillation with further preference finetuning and reinforcement learning with verifiable rewards (RLVR) is becoming an accessible and highly capable pipeline. Our post-training recipe follows our recent models: 1) create an excellent SFT set, 2) use direct preference optimization (DPO) as a highly iterable, cheap, and stable preference learning method despite its critics, and 3) finish up with scaled up RLVR. All of these stages confer meaningful improvements on the models’ final performance. Instruct models – low latency workhorse Instruct models today are often somewhat forgotten, but the likes of @aiatmeta Llama 3.1 Instruct and smaller, concise models are some of the most adopted open models of all time. The instruct models we’re building are a major polishing and evolution of the Tülu 3 pipeline – you’ll see many similar datasets and methods, but with pretty much every datapoint or training code being refreshed. Olmo 3 Instruct should be a clear upgrade on Llama 3.1 8B, representing the best 7B scale model from a Western or American company. As scientists we don’t like to condition the quality of our work based on its geographic origins, but this is a very real consideration to many enterprises looking to open models as a solution for trusted AI deployments with sensitive data. Building a thinking model What people have most likely been waiting for are our thinking or reasoning models, both because every company needs to have a reasoning model in 2025, but also to clearly open the black box for the most recent evolution of language models. Olmo 3 Think, particularly the 32B, are flagship models of this release, where we considered what would be best for a reasoning model at every stage of training. Extensive effort (ask me IRL about more war stories) went into every stage of the post-training of the Think models. We’re impressed by the magnitude of gains that can be achieved in each stage – neither SFT nor RL is all you need at these intermediate model scales. First we built an extensive reasoning dataset for supervised finetuning (SFT), called Dolci-Think-SFT, building on very impactful open projects like OpenThoughts3, Nvidia’s Nemotron Post-training, Prime Intellect’s SYNETHIC-2, and many more open prompt sources we pulled forward from Tülu 3 / OLMo 2. Datasets like this are often some of our most impactful contributions (see the Tülu 3 dataset as an example in Thinking Machine’s Tinker :D @thinkymachines @tinker_api – please add Dolci-Think-SFT too, and Olmo 3 while you’re at it, the architecture is very similar to Qwen which you have). For DPO with reasoning, we converged on a very similar method as HuggingFace’s (@huggingface) SmolLM 3 with Qwen3 32B as the chosen model and Qwen3 0.6B as the rejected. Our intuition is that the delta between the chosen and rejected samples is what the model learns from, rather than the overall quality of the chosen answer alone. These two models provide a very consistent delta, which provides way stronger gains than expected. Same goes for the Instruct model. It is likely that DPO is helping the model converge on more stable reasoning strategies and softening the post-SFT model, as seen by large gains even on frontier evaluations such as AIME. Our DPO approach was an expansion of Geng, Scott, et al. "The delta learning hypothesis: Preference tuning on weak data can yield strong gains." arXiv preprint arXiv:2507.06187 (2025). Many early open thinking models that were also distilled from larger, open-weight thinking models likely left a meaningful amount of performance on the table by not including this stage. Finally, we turn to the RL stage. Most of the effort here went into building effective infrastructure to be able to run stable experiments with the long-generations of larger language models. This was an incredible team effort to be a small part of, and reflects work ongoing at many labs right now. Most of the details are in the paper, but our details are a mixture of ideas that have been shown already like ServiceNow’s PipelineRL or algorithmic innovations like DAPO and Dr. GRPO. We have some new tricks too! Some of the exciting contributions of our RL experiments are 1) what we call “active refilling” which is a way of keeping the generations from the learner nodes constantly flowing until there’s a full batch of completions with nonzero gradients (from equal advantages) – a major advantage of our asynchronous approach; and 2) cleaning, documenting, decontaminating, mixing, and proving out the large swaths of work done by the community over the last months. The result is an excellent model that we’re very proud of. It has very strong reasoning benchmarks (AIME, GPQA, etc.) while also being stable, quirky, and fun in chat with excellent instruction following. The 32B range is largely devoid of non-Qwen competition. The scores for both of our Thinkers get within 1-2 points overall with their respective Qwen3 8/32B models – we’re proud of this! A very strong 7B scale, Western thinking model is Nvidia’s (@NVIDIAAI) NVIDIA-Nemotron-Nano-9B-v2 hybrid model. It came out months ago and is extremely strong. I personally suspect it may be due to the hybrid architecture making subtle implementation bugs in popular libraries, but who knows. All in, the Olmo 3 Think recipe gives us a lot of excitement for new things to try in 2026. RL Zero DeepSeek R1 showed us a way to new post-training recipes for frontier models, starting with RL on the base model rather than a big SFT stage (yes, I know about cold-start SFT and so on, but that’s an implementation detail). We used RL on base model as a core feedback cycle when developing the model, such as during intermediate midtraining mixing. This is viewed now as a fundamental, largely innate, capability of the base-model. To facilitate further research on RL Zero, we released 4 datasets and series of checkpoints, showing per-domain RL Zero performance on our 7B model for data mixes focus on math, code, instruction following, and all mixed together. In particular, we’re excited about the future of RL Zero research on Olmo 3 precisely because everything is open. Researchers can study the interaction between the reasoning traces we include at midtraining and the downstream model behavior (qualitative and quantitative). This helps answer questions that have plagued RLVR results on Qwen models, hinting at forms of data contamination particularly on math and reasoning benchmarks (see Shao, Rulin, et al. "Spurious rewards: Rethinking training signals in rlvr." arXiv preprint arXiv:2506.10947 (2025). or Wu, Mingqi, et al. "Reasoning or memorization? unreliable results of reinforcement learning due to data contamination." arXiv preprint arXiv:2507.10532 (2025).) What’s next This is the biggest project we’ve ever taken on at Ai2 (@allen_ai), with 60+ authors and numerous other support staff. In building and observing “thinking” and “instruct” models coming today, it is clear to us that there’s a very wide variety of models that fall into both of these buckets. The way we view it is that thinking and instruct characteristics are on a spectrum, as measured by the number of tokens used per evaluation task. In the future we’re excited to view this thinking budget as a trade-off, and build models that serve different use-cases based on latency/throughput needs. As for a list of next models or things we’ll build, we can give you a list of things you’d expect from a (becoming) frontier lab: MoEs, better character training, pareto efficient instruct vs think, scale, specialized models we actually use at Ai2 internally, and all the normal things. This is one small step towards what I see as a success for my ATOM project. We thank you for all your support of our work at Ai2. We have a lot of work to do. We’re going to be hunting for top talent at NeurIPS to help us scale up our Olmo team in 2026. This post in full also appears on Interconnects – the full links to the artifacts and paper are below. Moo, moo, rawr!

natolambert's tweet photo. We present Olmo 3, our next family of fully open, leading language models.
This family of 7B and 32B models represents:

1. The best 32B base model.
2. The best 7B Western thinking & instruct models.
3. The first 32B (or larger) fully open reasoning model.

This is a big milestone for Ai2 and the Olmo project. These aren’t huge models (more on that later), but it’s crucial for the viability of fully open-source models that they are competitive on performance – not just replications of models that came out 6 to 12 months ago. As always, all of our models come with full training data, code, intermediate checkpoints, training logs, and a detailed technical report. All are available today, with some more additions coming before the end of the year.

As with OLMo 2 32B at its release, OLMo 3 32B is the best open-source language model ever released. It’s an awesome privilege to get to provide these models to the broader community researching and understanding what is happening in AI today.

Base models – a strong foundation

Pretraining’s demise is now regularly overstated. 2025 has marked a year where the entire industry rebuilt their training stack to focus on reasoning and agentic tasks, but some established base model sizes haven’t seen a new leading model since @alibaba_qwen's Qwen 2.5 in 2024. The Olmo 3 32B base model could be our most impactful artifact here, as Qwen3 did not release their 32B base model (likely for competitive reasons). We show that our 7B recipe competes with Qwen 3, and the 32B size enables a starting point for strong reasoning models or specialized agents. Our base model’s performance is in the same ballpark as Qwen 2.5, surpassing the likes of Stanford’s Marin (@stanfordAILab) and Gemma 3 (@GoogleDeepMind), but with pretraining data and code available, it should be more accessible to the community to learn how to finetune it (and be confident in our results).

We’re excited to see the community take Olmo 3 32B base in many directions. 32B is a loved size for easy deployment on single 80GB+ memory GPUs and even on many laptops, like the MacBook I’m using to write this on.

A model flow – the lifecycle of creating a model

With these strong base models, we’ve created a variety of post-training checkpoints to showcase the many ways post-training can be done to suit different needs. We’re calling this a “Model Flow.” For post-training, we’re releasing Instruct versions – short, snappy, intelligent, and useful especially for synthetic data en masse (e.g. recent work by Datology @datologyai on OLMo 2 Instruct), Think versions – thoughtful reasoners with the performance you expect from a leading thinking model on math, code, etc. and RL Zero versions – controlled experiments for researchers understanding how to build post-training recipes that start with large-scale RL on the base model.

The first two post-training recipes are distilled from a variety of leading, open and closed, language models. At the 32B and smaller scale, direct distillation with further preference finetuning and reinforcement learning with verifiable rewards (RLVR) is becoming an accessible and highly capable pipeline. Our post-training recipe follows our recent models: 1) create an excellent SFT set, 2) use direct preference optimization (DPO) as a highly iterable, cheap, and stable preference learning method despite its critics, and 3) finish up with scaled up RLVR. All of these stages confer meaningful improvements on the models’ final performance.

Instruct models – low latency workhorse

Instruct models today are often somewhat forgotten, but the likes of @aiatmeta Llama 3.1 Instruct and smaller, concise models are some of the most adopted open models of all time. The instruct models we’re building are a major polishing and evolution of the Tülu 3 pipeline – you’ll see many similar datasets and methods, but with pretty much every datapoint or training code being refreshed. Olmo 3 Instruct should be a clear upgrade on Llama 3.1 8B, representing the best 7B scale model from a Western or American company. As scientists we don’t like to condition the quality of our work based on its geographic origins, but this is a very real consideration to many enterprises looking to open models as a solution for trusted AI deployments with sensitive data.

Building a thinking model

What people have most likely been waiting for are our thinking or reasoning models, both because every company needs to have a reasoning model in 2025, but also to clearly open the black box for the most recent evolution of language models. Olmo 3 Think, particularly the 32B, are flagship models of this release, where we considered what would be best for a reasoning model at every stage of training.

Extensive effort (ask me IRL about more war stories) went into every stage of the post-training of the Think models. We’re impressed by the magnitude of gains that can be achieved in each stage – neither SFT nor RL is all you need at these intermediate model scales.

First we built an extensive reasoning dataset for supervised finetuning (SFT), called Dolci-Think-SFT, building on very impactful open projects like OpenThoughts3, Nvidia’s Nemotron Post-training, Prime Intellect’s SYNETHIC-2, and many more open prompt sources we pulled forward from Tülu 3 / OLMo 2. Datasets like this are often some of our most impactful contributions (see the Tülu 3 dataset as an example in Thinking Machine’s Tinker :D @thinkymachines @tinker_api – please add Dolci-Think-SFT too, and Olmo 3 while you’re at it, the architecture is very similar to Qwen which you have).

For DPO with reasoning, we converged on a very similar method as HuggingFace’s (@huggingface) SmolLM 3 with Qwen3 32B as the chosen model and Qwen3 0.6B as the rejected. Our intuition is that the delta between the chosen and rejected samples is what the model learns from, rather than the overall quality of the chosen answer alone. These two models provide a very consistent delta, which provides way stronger gains than expected. Same goes for the Instruct model. It is likely that DPO is helping the model converge on more stable reasoning strategies and softening the post-SFT model, as seen by large gains even on frontier evaluations such as AIME.

Our DPO approach was an expansion of Geng, Scott, et al. "The delta learning hypothesis: Preference tuning on weak data can yield strong gains." arXiv preprint arXiv:2507.06187 (2025). Many early open thinking models that were also distilled from larger, open-weight thinking models likely left a meaningful amount of performance on the table by not including this stage.

Finally, we turn to the RL stage. Most of the effort here went into building effective infrastructure to be able to run stable experiments with the long-generations of larger language models. This was an incredible team effort to be a small part of, and reflects work ongoing at many labs right now. Most of the details are in the paper, but our details are a mixture of ideas that have been shown already like ServiceNow’s PipelineRL or algorithmic innovations like DAPO and Dr. GRPO. We have some new tricks too!

Some of the exciting contributions of our RL experiments are 1) what we call “active refilling” which is a way of keeping the generations from the learner nodes constantly flowing until there’s a full batch of completions with nonzero gradients (from equal advantages) – a major advantage of our asynchronous approach; and 2) cleaning, documenting, decontaminating, mixing, and proving out the large swaths of work done by the community over the last months.

The result is an excellent model that we’re very proud of. It has very strong reasoning benchmarks (AIME, GPQA, etc.) while also being stable, quirky, and fun in chat with excellent instruction following. The 32B range is largely devoid of non-Qwen competition. The scores for both of our Thinkers get within 1-2 points overall with their respective Qwen3 8/32B models – we’re proud of this!

A very strong 7B scale, Western thinking model is Nvidia’s (@NVIDIAAI) NVIDIA-Nemotron-Nano-9B-v2 hybrid model. It came out months ago and is extremely strong. I personally suspect it may be due to the hybrid architecture making subtle implementation bugs in popular libraries, but who knows.

All in, the Olmo 3 Think recipe gives us a lot of excitement for new things to try in 2026.

RL Zero

DeepSeek R1 showed us a way to new post-training recipes for frontier models, starting with RL on the base model rather than a big SFT stage (yes, I know about cold-start SFT and so on, but that’s an implementation detail). We used RL on base model as a core feedback cycle when developing the model, such as during intermediate midtraining mixing. This is viewed now as a fundamental, largely innate, capability of the base-model.

To facilitate further research on RL Zero, we released 4 datasets and series of checkpoints, showing per-domain RL Zero performance on our 7B model for data mixes focus on math, code, instruction following, and all mixed together.

In particular, we’re excited about the future of RL Zero research on Olmo 3 precisely because everything is open. Researchers can study the interaction between the reasoning traces we include at midtraining and the downstream model behavior (qualitative and quantitative).

This helps answer questions that have plagued RLVR results on Qwen models, hinting at forms of data contamination particularly on math and reasoning benchmarks (see Shao, Rulin, et al. "Spurious rewards: Rethinking training signals in rlvr." arXiv preprint arXiv:2506.10947 (2025). or Wu, Mingqi, et al. "Reasoning or memorization? unreliable results of reinforcement learning due to data contamination." arXiv preprint arXiv:2507.10532 (2025).)

What’s next

This is the biggest project we’ve ever taken on at Ai2 (@allen_ai), with 60+ authors and numerous other support staff.

In building and observing “thinking” and “instruct” models coming today, it is clear to us that there’s a very wide variety of models that fall into both of these buckets. The way we view it is that thinking and instruct characteristics are on a spectrum, as measured by the number of tokens used per evaluation task. In the future we’re excited to view this thinking budget as a trade-off, and build models that serve different use-cases based on latency/throughput needs.

As for a list of next models or things we’ll build, we can give you a list of things you’d expect from a (becoming) frontier lab: MoEs, better character training, pareto efficient instruct vs think, scale, specialized models we actually use at Ai2 internally, and all the normal things.

This is one small step towards what I see as a success for my ATOM project.

We thank you for all your support of our work at Ai2. We have a lot of work to do. We’re going to be hunting for top talent at NeurIPS to help us scale up our Olmo team in 2026.

This post in full also appears on Interconnects – the full links to the artifacts and paper are below.

Moo, moo, rawr!

356

502K

CongZhou1 retweeted

Whispers from the Star

@WFTS_Game

7 months ago

Whispers from the Star is officially live on iOS You can finally play Stella’s story on mobile, and we’re kicking things off with a 50% launch discount. Start on your phone, continue on PC, switch back and forth whenever you want. → Download here: https://t.co/ExoyxUbvFL #WhispersfromtheStar #WFTS #Anuttacon

Who to follow

Chang-Bin Jeon

@jeonchangbin49

Staff Engineer at Samsung Electronics, PhD from MARG Seoul National University. Previous intern @merl_news @Gaudiolab.

Anurag Kumar

@AcouIntel

Research Scientist, @GoogleDeepMind | Prev: @AIatMeta | CMU @SCSatCMU | @IITKanpur | Audio/Speech, Multimodal AI

Ge Zhu

@__gzhu__

Research Scientist/Engineer

CongZhou1 retweeted

jiatongshi @jiatongshi

8 months ago

Speech isn’t just sound -> it’s how we turn thought into expression. Our new work, Speech-DRAME, measures how well speech AI can act, aligning evaluation with human perception. Paper: https://t.co/QzDuj2eJ6Z Code: https://t.co/dOMqq8rgFJ

CongZhou1 retweeted

Shawn Shen

@shawnshenjx

11 months ago

I’m Shawn, founder of https://t.co/DLaNyXWuUn, former researcher at Meta and CS PhD at University of Cambridge. Today we’re launching https://t.co/DLaNyXWuUn: we built the world’s first Large Visual Memory Model - to give AI human-like visual memories. Why visual memory? AI to date is chat-based, which has great applications, but humans are not only chat based. Humans have visual-memories, and are visually driven. https://t.co/DLaNyXWuUn enables AI agents, softwares, and even robots to see and remember the world the way humans do. We just raised an $8m seed round led by @susaventures to build the visual-memory layer for AI. Why is https://t.co/DLaNyXWuUn so groundbreaking? 👇

179

349

Cong Zhou @CongZhou1

about 1 year ago

Congrats on the release!

cory

@Cixelyn

about 1 year ago

new anime video model, with a matching banger OP. team really cooked with this one

257

Cong Zhou @CongZhou1

about 1 year ago

@jordiponsdotme Congrats Jordi!

CongZhou1 retweeted

Wan @Alibaba_Wan

about 1 year ago

1/3 🚀Thrilled to introduce Wan2.1-FLF2V-14B - our first 14B-parameter large model for First-Last-Frame to video generation! Open-source, open-source, open-source! Empowering digital artists with unprecedented efficiency and creative flexibility. #wan #AIGC #alart

286

826

197K

Cong Zhou @CongZhou1

about 1 year ago

@honualx Awesome!

CongZhou1 retweeted

Justin Uberti

@juberti

about 1 year ago

Put another way: we have LLMs with billions of parameters controlled by VAD models with thousands of parameters. There are reasons for this but we need more sophisticated solutions (and evals for them!)

Cong Zhou @CongZhou1

over 1 year ago

@Cixelyn New discoveries?!

165

Cong Zhou @CongZhou1

over 1 year ago

The first trailer for Whispers from the Star is here! 🌟 Thrilled to have contributed to the voice modeling efforts and excited for you to experience it! Join us in shaping immersive AI-driven experiences at @AnuttaconGames! 🎮🚀 https://t.co/QmEUOAamX7

Whispers from the Star

@WFTS_Game

over 1 year ago

Whispers from the Star⭐️ Announce Trailer Your words seal her fate. When a girl named Stella crash-lands on an alien planet called Gaia, you are the only person she can contact through her communicator. Through texts, voice messages, and video calls that unfold throughout your day, guide her through a gripping story where every response could mean life or death. As you stay connected in real time, your open-ended conversations with Stella become her compass through danger. In this innovative story of hope, resilience, and wonder, you are Stella's sole chance at survival. Will your voice guide her home? ––––––– ➤ Register now for the Closed Beta: https://t.co/mt8Pf99Gco ➤ Join the Discord: https://t.co/gqsd7983Mg #WhispersfromtheStar #WFTS #Anuttacon

197

38K

Cong Zhou @CongZhou1

over 1 year ago

Cool!

Boyuan Chen

@BoyuanChen0

over 1 year ago

Announcing Diffusion Forcing Transformer (DFoT), our new video diffusion algorithm that generates ultra-long videos of 800+ frames. DFoT enables History Guidance, a simple add-on to any existing video diffusion models for a quality boost. Website: https://t.co/wdZ19yCgjJ (1/7)

473

214

176K

292

Cong Zhou @CongZhou1

over 1 year ago

It’s cool!

Justin Alvey

@justLV

over 1 year ago

Excited to share a peek of what I’ve been working on We @sesame believe voice is key to unlocking a future where computers are lifelike Here’s an early preview you can try! 👇 We’ll be open sourcing a model, and yes… we’re building hardware! 🧵

187

250

964

453K

207

CongZhou1 retweeted

小互

@xiaohu

over 1 year ago

字节跳动这个新项目效果非常不错 OmniHuman：通过一张图片配合音频或视频，生成非常自然的会说话、唱歌的人类动作视频支持各种不同类型输入（如单一的人物图片和音频、视频等信号）生成非常逼真真人视频动画，涵盖从面部表情到全身动作，无论是说话、唱歌、跳舞等。 OmniHuman 在生成动画时能够保持极高的真实感，能够精确捕捉到人类的面部表情、身体动作、手势变化、物体交��等细节，超越了现有的方法。

675

151

627

236K

CongZhou1 retweeted

최형석 (Hyeong-Seok Choi)

@92HsChoi

over 1 year ago

@sedielem imo just because it more “compressed” doesn’t mean it’s good for “modeling.” In audio/speech space people use semantic token, which is not necessarily optimized for compression. What matters more is the characteristics of representation the encoder has learnt.

529

Cong Zhou @CongZhou1

over 1 year ago

Congratulations, Jordi! I’ll definitely play with it, any plans to go to 32k?

Jordi Pons @jordiponsdotme

over 1 year ago

Weights are out! 🤗 Tokenizing 16kHz speech at very low bitrates. Inference code: https://t.co/eZKbrBzHzw Model code: https://t.co/vLJhpyGa7M Model weights: https://t.co/fFHpte7fey arXiv: https://t.co/ZbslCfppvF Audio demos: https://t.co/J9D46A6prO

jordiponsdotme's tweet photo. Weights are out! 🤗
Tokenizing 16kHz speech at very low bitrates.

Inference code: https://t.co/eZKbrBzHzw
Model code: https://t.co/vLJhpyGa7M
Model weights: https://t.co/fFHpte7fey
arXiv: https://t.co/ZbslCfppvF
Audio demos: https://t.co/J9D46A6prO https://t.co/YCxHw07LE4

116

183

Cong Zhou @CongZhou1

over 1 year ago

You can not miss this one!

cory

@Cixelyn

over 1 year ago

come hang w/ us at neurips! i'm hosting an anime & ai social on dec 11th! will be there along with a bunch of folks who work on @nijijourney then later, we're hosting a diffusion bar event dec 12th with @midjourney! rsvp on the partiful links below!

Cixelyn's tweet photo. come hang w/ us at neurips!

i'm hosting an anime & ai social on dec 11th! will be there along with a bunch of folks who work on @nijijourney

then later, we're hosting a diffusion bar event dec 12th with @midjourney!

rsvp on the partiful links below! https://t.co/pHLFklDgPa

163

Cong Zhou @CongZhou1

over 1 year ago

Congrats on the poc!

Justin Uberti

@juberti

over 1 year ago

In 5 months, Ultravox has gone from a v0.1 proof-of-concept to the leading open-source speech LLM!

146

Cong Zhou

@CongZhou1

Who to follow

Last Seen Users on Sotwe

Trends for you

Most Popular Users