Yavor Ivanov

Research Engineer @Meta, prev @PlayAIOfficial

[Yavor Ivanov @YavorGI]

It is all over. We are the SOTA

[Jonathan Whitaker @johnowhitaker]

@giffmana @tunguz Woah, SOTA BTW (https://t.co/0TJ1G25NX9)
Identifying critters in photos; it gets 72%, beating me (71%) and runner-up Gemini 3.5 Flash (67%).

[Alexandr Wang @alexandr_wang]

@teortaxesTex this isn’t true we’re going to increase access to muse spark incl openrouter

[Yavor Ivanov @YavorGI]

@sakurayukiai @GenAI_is_real Still if large enough X num of requests with large enough Y prompt len come at the exact same time - you will definitely get 100% SM util on a single server for both prefill and decode. That is if you do NOT waste time on the CPU for some reason, which I suspect is the issue.

[Yavor Ivanov @YavorGI]

@GenAI_is_real I would suggest profiling the Scheduler and related components on the CPU under e.g. batch size 256 and seeing what happens compared to e.g. batch size 32. I would not be surprised that you are hitting some Python GIL overhead or there is some bug.

[Yavor Ivanov @YavorGI]

@GenAI_is_real Maybe there is a large bs SGLang-Omni scheduler issue taking more CPU than needed. Otherwise fundamentally if we assume you use overlap scheduler as submit as fast as possible jobs to the GPU, I do NOT understand how this can happen and why just scaling bs does not help.

[Yavor Ivanov @YavorGI]

@YichiZ03 I have done very similar things across inference engines in my work - nice that you put in the work to have them in SGLang Omni.

[Yavor Ivanov @YavorGI]

@JerryCap Definitely higher margin if serving your own AI products

[Yavor Ivanov @YavorGI]

@giffmana @ar0cket1 That seems about right, but 3T Dense will take a ton of compute to train. Might turn out better at reasoning https://t.co/bkChFV4G8U
+ @dylan522p thinks Anthropic's MoEs are denser compared to OpenAI ones - https://t.co/LZzsTC7nHs

[Yavor Ivanov @YavorGI]

@giffmana @ar0cket1 Always thought that MoE is needed to circumvent a 10T Dense being impossible to train and serve economically on modern hardware. In a perfect world where we have hardware serving the 10T Dense cheaply, I would go with it.

[Yavor Ivanov @YavorGI]

Let's see if we catchup soon

[Alexandr Wang @alexandr_wang]

First, Mark was clearly talking about the industry’s progress on agentic capabilities on the whole.

But, while we’re on the topic: Our next Muse Spark update is coming soon. Big improvements in coding and agentic capabilities to be more competitive with other leading models.

Excited to get these into your hands—will be rolling out to Meta AI and our new API!

[Yavor Ivanov @YavorGI]

@ggerganov https://t.co/eefysZxYF4

[Joseph D. Janizek @joejanizek]

Look, I want to not be a hater/crash out, but these images look like absolute dogwater

[Yavor Ivanov @YavorGI]

Video is cool, but not sure how viable it is as I lack the physics expertise to evaluate it although I worked for MRI-Guided Radiation Therapy company. Maybe @ggerganov can help.

[Midjourney @midjourney]

A technical dive inside our new "Midjourney Scanner"

[Yavor Ivanov @YavorGI]

@ggerganov Indeed compared to ViewRay's MRI I worked for the images do not look great.

[Alexandr Wang @alexandr_wang]

try out the new voice AI in the meta ray-bans!

[Fuli Luo @_LuoFuli]

Inference Optimizations Behind the MiMo-V2.5 Series API Price Reductions

Read the full technical blog: https://t.co/B5tp4tdnim

The V2.5 model family, including MiMo-V2.5 and MiMo-V2.5-Pro, is built on a Hybrid Sliding Window Attention (Hybrid SWA) architecture, which compresses KVCache storage to roughly 1/7 that of Full Attention. However, architectural advantages rarely translate directly into measurable gains in production serving. To realize these gains, we redesigned KVCache management, tiered caching, and the prefix-cache tree; addressed key challenges in SWA KVCache handling; and optimized scheduling as well as the Prefill/Decode pipeline.

Validated on real production traffic, these optimizations have increased effective KVCache capacity by nearly 5x, with server-side cache hit rates averaging 93%–95% across mainstream harness frameworks. Together with MoE configuration tuning and multimodal inference optimizations, they enable more efficient long-context inference and form part of what makes the recent API price cuts possible.

[Anthropic @AnthropicAI]

AI can make work faster, but a fear is that relying on it may make it harder to learn new skills on the job.

We ran an experiment with software engineers to learn more. Coding with AI led to a decrease in mastery—but this depended on how people used it.
https://t.co/lbxgP11I4I

[spor @sporadica]

just remembered that time Noam Shazeer dropped the hardest line ever written in an ML paper

[Georgi Gerganov @ggerganov]

Llama.cpp supports the new gpt-oss model in native MXFP4 format

The ggml inference engine (powering llama.cpp) can run the new gpt-oss model with all major backends, including CUDA, Vulkan, Metal and CPU at exceptional performance. This virtually brings the unprecedented quality of gpt-oss in the hands of everyone - from local AI enthusiasts to enterprises doing inference at the edge or in the cloud. The unique inference capabilities of ggml unlock a vast amount of use cases for the entire spectrum of consumer-grade hardware available on the market today - use cases that are impossible to support with any other inference framework in existence. Today, gpt-oss trained with the MXFP4 format, effectively “leaps” over the existing resource barriers and allows us to experience SOTA AI quality on our own personal devices.

The era of natively trained 4-bit local models has officially began and ggml will continue to lead the way forward!

[Yavor Ivanov @YavorGI]

@PlayAIOfficial New link: https://t.co/o41Do65kuJ

[Yavor Ivanov @YavorGI]

Wrote this blog post on part of the inference work I was doing at @PlayAIOfficial before Meta acquired us https://t.co/BOKzjWYnti