Tejesh Bhalla

The sky is falling, the wind is calling
Stand for something or die in the morning

@theagentic

[Tejesh Bhalla @OG_tejeshbhalla]

@v0xium Agreed

[Tejesh Bhalla @OG_tejeshbhalla]

Elliot has done such a great work on his megakernel book , always a refresher to read it . Im half way through and already know so much more on kernel engineering it would take months to read on docs. Defo Recommend

[Tejesh Bhalla @OG_tejeshbhalla]

GPT work is genuinely insane for me , i can set up a scheduled task , every day on my email i get notified for curated papers , clean filtered to conditions i need them by.

[Loic cabannes @loiccabannes]

Introducing Sparse Delta Memory (SDM) - The first work of my PhD 🎓.

SDM combines the GatedDeltaNet update with Product Key sparsity, enabling a recurrent state 3000x larger at the same FLOPs and significantly improving long-context performance.

Let RNNs be sparse!

[Tejesh Bhalla @OG_tejeshbhalla]

@Chinemerem92227 @elliotarledge so much power in that comment bro!

[Tejesh Bhalla @OG_tejeshbhalla]

@alokbishoyi97 🖐

[Tejesh Bhalla @OG_tejeshbhalla]

Seems like its a custom model trained to output tool chains, function_calls etc , into compacted summary with details . Also to return user,assistant pairs back exactly as we need them .

[Tejesh Bhalla @OG_tejeshbhalla]

Got curious about how Codex CLI keeps its shit together on long tasks without forgetting what it's doing. So I went digging.

[Tejesh Bhalla @OG_tejeshbhalla]

What actually survives across compaction?
User prompt and Developer prompts are kept as is ,
They summarize all tool calls and file info into one compacted history
The crazy part is "As expected from Sam", They encrypt the summary and its decoded server side.

[elie @eliebakouch]

new inference optimization method by @deepseek_ai with an extremely detailed paper, draft model and framework to train them. results in production for dsv4 lead to +50% for throughput and latency (can go to ~80% for latency, crazy).

full explanation of DSpark:

it's about speculative decoding and the idea builds upon DFlash (fully parallel) and Eagle (fully sequential) to create a "semi-parallel" method that keeps the advantages of both

the core equation you want to optimize is the "time to generate each token" which is:
(time to draft + time to verify) / how many tokens are accepted

the advantage of the parallel variant (DFlash) is that it's fast, but when you increase the number of tokens you draft, acceptance rate drops pretty fast (makes sense since there is no dependency on the previous token).

fully sequential is nice but opposite issue: it's slower (you need a much smaller draft to get the same speed) but the autoregressive dependency means you can maintain good acceptance rate at a lot of tokens. since you have a much smaller draft head, the first token acceptance rate is often quite low

idea of DSpark is to combine both: a "heavy" parallel head (you only do it once) and then a small sequential step to bias the logit distribution with information about the previous token. this biasing is done with a small markov head (only depends on t-1)

they also get a confidence score out of the sequential head that allows them to adjust how many tokens they want to verify. verification can get expensive if the gpus are already at maximum utilization, so they use this confidence score to do some load balancing and predict the right number of tokens depending on gpu workload

one small detail: i would have liked to see production numbers if they used DFlash or Eagle instead of MTP-1, but as always, huge work by deepseek and i'm expecting to see this method widely adopted

[Tejesh Bhalla @OG_tejeshbhalla]

@clopenloop loool

[Karthik Elamvazhuthi @clopenloop]

The job market is crazy these days.

[Xuandong Zhao @xuandongzhao]

Happy to introduce our latest work — VIMPO: Value-Implicit Policy Optimization for LLMs

Most RL methods for LLM training face a trade-off:
· PPO-style methods use a value model (critic) for token-level credit, but critics are hard to train.
· GRPO-style methods drop the critic, but give every token the same trajectory-level signal.

Can we get the best of both worlds? 🧵

[Tejesh Bhalla @OG_tejeshbhalla]

Feels like if sub q is real , it must be a hybrid a rough throwaway from my brain is few layers probably do full attn , in middle they have some kind of linear indexer maybe built of same mamba / delta net variants ? .

[Tejesh Bhalla @OG_tejeshbhalla]

This one was obvious from the lawsuit though they illegally got a ton of pretraining data . Now how much is that , whats the split thats to think about .

[Tejesh Bhalla @OG_tejeshbhalla]

Anthropic pays crazy amounts for rl environments to these firms working for them as "data curators" , there's a huge pipeline process to make sure those are critically hard . Along with this seems like their pretrain corpus is way beyond 80-100 trillion.

[Tejesh Bhalla @OG_tejeshbhalla]

Note most oss models are pretrained on roughly 20-35 trillion tokens , and labs find a hard time gathering these because common crawl and hf mostly accumulate a few trillion quality tokens beyond that its probably expert knowledge , books .

[ThePrimeagen @ThePrimeagen]

the masculine urge to use polymorphism to solve all problems is real

I am realizing my last 20 years were a lie and the strategy pattern isn't the best thing ever...

Maybe a function with 200 lines and a switch statement is just easier by all means