Nikolay

AI Engineer

[Nikolay @nekech52]

@gamer3dx можно что бы полицейские ходили по электричкам)

[Nikolay @nekech52]

@Ildar_De так возьми пэрентал лив)

[Nikolay @nekech52]

@tagir_valeev если вдруг хочется поиграть, я сделал еще в 2022 для telegram) https://t.co/qrHiqiGOSN

[Shumpei Takezaki @shumpeiMaxwell]

Gaussian-JEPA: Joint-Embedding Predictive Learning for 3D Gaussian Splats

TL;DR: Brings JEPA to 3DGS: predict hidden Gaussian-token features from visible context rather than reconstruct raw attributes, improving robustness to resampling, partial views, and Gaussian completion.
https://t.co/UZGpC0YzaU

[Nikolay @nekech52]

@Ildar_De мы когда, дочке было 6 месяцев улетели в Турцию, на 3 месяца) но вообще путешествовать с детьми удобно до 1.5 лет и после 3х)

[Nikolay @nekech52]

@Ildar_De я бы обсудил, возможность сдвига, если нет то реализовал бы, но еще зависит от того, а какая вероятность, что дадут переделать потом, и какой импакт не оптимальное решение окажет в будущем на продукт

[Nikolay @nekech52]

@dee_hw looks really good, but why price is so high?

[Nikolay @nekech52]

@Ildar_De зато нижний вон в верху, но интересно почему москва ниже тбилиси) и где мюнхен?

[Sergey Levine @svlevine]

Latest Deep RL class lectures are now online!
https://t.co/GvqI1v3hgD
Thanks to @seohong_park, we now have CS185/285 for spring 2026 available to everyone to watch.

Course website here: https://t.co/U16rasTOAo

Apologies for a few recording glitches (it's not a perfect system).

[Nikolay @nekech52]

@Ildar_De а развесть есть такие зоны?

[Edward Z. Yang @ezyang]

Fable and I made a set of exercises for learning how PP schedules work: https://t.co/LVRHs6yLAp -- unlike the other simulators, it asks you to build out the schedule block-by-block (double click to greedily finish the microbatch), subject to constraints

[Nikolay @nekech52]

@kelstar_ да, но при этом фронтир модели становятся все жирнее, а производительность dgx spark не очень высокая.

[Nikolay @nekech52]

@Ildar_De тяжело, но мне кажется столько сил дает, даже когда нес дочку на одной руке, в другой коляску 4км, супер

[Nikolay @nekech52]

@Crazy_breze вчера было норм, что бы похайкать, сегодня уже жарковато конечно(

[Avi Chawla @_avichawla]

NVIDIA researchers did it again!

They found a way to make KV cache transferable between models.

The target model skips prefill entirely, and the conversion runs 2.7 to 25x faster than processing the context again.

Let's understand why this is so important today.

LLM APIs are stateless, so every turn sends the entire conversation back to the model. The model reads all of it again before writing a single new token, and all of it is billed as input.

Prompt caching allows Anthropic and other providers to hold the KV cache for a stable prefix and bill a hit at roughly 10% of the base input rate, because the compute was already done once.

The 90% reduction is one of the largest lever in LLM serving, which is why so much production work goes into keeping prefixes byte-stable.

But the cache only works on the model that produced it. Keys and values are produced from that model's weights, so no other model can read them.

In pratice, the constraint shows up in LLM routing. If the traffic is shifted to a different model for cost/capability reasons, the accumulated KV cache becomes invalid.

As a result, the accumulated context has to be processed from scratch, and it's billed at full rate.

NVIDIA's recent paper treats this as a representation problem.

Prefill's only output is the KV cache, so to move KV between models, we need to convert one model's cache into the format the other expects.

They first checked whether the conversion has any structure worth exploiting.

They found that moving from Qwen3 14B to 32B, a plain linear regression from a single source layer reconstructed 56% of the variance in the target model's keys.

The two models obviously may have different layer counts, so there is no natural one-to-one pairing between them.

For each target layer they rank every source layer by how well it predicts that layer, then feed the top eight in together, which takes the reconstruction to 79%.

The mapper itself has three parts:

> Each target layer and head gets its own independent linear map, solved in one closed-form step rather than by gradient descent.

> The cross-layer selection described above is the second part, and their ablation shows it carries the most weight of the three.

> Keys also carry a position-dependent rotation from RoPE. They strip that rotation, fit the map in position-free space, then re-apply the target model's rotation at inference.

Across six pairs from Qwen3, Llama 3.1 and Ministral 3, four retain 73 to 98% of the receiving model's standalone accuracy, and the conversion runs 3-25x faster than processing the context again.

Prior work on cross-model KV reuse exists, but it either trains a neural adapter per pair or requires both models to be architecturally identical.

This is probably the first version that is closed-form and training-free, so a lot of it is still open research.

Every pair tested belongs to one family, so it works on Qwen to Qwen and Llama to Llama.

Cross-family transfer is listed as future work.

All six pairs mentioned above also happen to share KV head count and per-head dimension across scales. Mismatched head configurations are currently untested.

The researchers scoped this to dense full-attention only, so sliding-window and attention-recurrent hybrids still need work.

Here's the paper: https://t.co/tMUGhijFbc

Plenty of work is yet to be done. Still, the constraint being solved is genuine.

Every model swap currently invalidates the full KV that was already paid for, and this is the first result showing that work might be recoverable without training anything extra.

That said, all of this only matters because of what the KV cache is doing in the first place.

I wrote a first-principles breakdown of it, covering why the model stores keys and values at all, why the cache grows with every token, and what generation speed looks like with and without it.

Read it below.

[Nikolay @nekech52]

@vas3k да, ладно софт, я купил айпад эир и наслаждаюсь разблокировкой, когда планшет стоит горизонтально, я даже не знал что уменя рука так гнется)

[Shumpei Takezaki @shumpeiMaxwell]

Qwen-3D: A Generalist 3D Vision-Language Model for Spatial Understanding

TL;DR: Qwen-3D unifies multi-view inputs in a shared 3D space using depth and camera poses. With 3D RoPE and a dense mask decoder, it supports spatial reasoning, grounding, segmentation, and VQA while outperforming prior 3D LMMs and retaining 2D capability.
https://t.co/zyinrSnV6V

[Qwen @Alibaba_Qwen]

📢Meet Qwen3.8-Max — our most capable model to date.

Next week, the open weights of Qwen3.8-Max will be released, and Qwen3.8-27B is also going open-weights to meet you all!🎉

Qwen3.8-Max, a new bar for coding and cowork at 2.4T parameters:

- Autonomous coding: 10+ days of self-evolving development, from empty folder to production without hand-holding, complete project trace in the GitHub：https://t.co/iVHZWQoeSo
- Real work, real results: Production-quality deliverables across hundreds of professions.
- Long-horizon mastery: System-level autonomous planning with closed-loop adaptive learning, driving 500+ turns of chip design optimization and 365 days of e-commerce strategy.
- Native multimodal intelligence: Vision isn't just input — it's a continuous feedback loop for planning, execution, and self-correction.

💰Pricing:
Input: $2.0 / M tokens
Output: $6.0 / M tokens
Implicit Caching: $0.25 / M tokens

Start building with Qwen3.8-Max! 🚀

📖 Blog: https://t.co/iwjmQxLBof
✅ Qwen Studio: https://t.co/4V2pFvDovG
⚡ API: https://t.co/gAGqaLQGbN

[Antonio Lupetti @antoniolupetti]

"Understanding Transformers and Attention Mechanisms" is a very interesting paper that presents the Transformer architecture from the perspective of applied mathematics.

It starts by representing text as vectors and explains mathematically how the attention mechanism processes these vectors to encode contextual information. It then develops Multi-Head Attention and shows how the main components of the Transformer architecture are constructed.

The paper also discusses more recent methods designed to reduce the computational and memory costs of attention, including KV caching, Grouped Query Attention, and Latent Attention. I think it is a useful reference for anyone interested in understanding Transformers beyond their high-level architecture and in seeing the linear algebra behind modern language models.

https://t.co/Rlun9QT7zx

[Anush Elangovan @AnushElangovan]

Power of OPEN: Mi455/CDNA5 ISA: https://t.co/xFsF8Cytom