Andrei Khrapavitski

Data science engineer and software developer, also a researcher and a musician

[Thariq @trq212]

the right way to use model capabilities is not to ship 10x more features to prod

it's to spend more time understanding your users, trying experiments, building prototypes, learning about things you don't understand so that you can ship things that actually work

[Santiago @svpino]

This new model does something really cool:

It turns speech into text as you speak. This is different from every other audio model.

The model is called R2T2. It processes audio in small chunks and knows which words to publish immediately and which need more context.

Hugging Face link: https://t.co/0VysSGvkwb

[PrismML @PrismML]

Today, we’re announcing Ternary Bonsai 2 27B.

Based on Qwen3.8 27B, Bonsai 2 27B is 9x smaller than its full-precision counterpart while retaining 98.2% of its aggregate benchmark performance.

Two months after the first Bonsai 27B release, the biggest change is quality. The footprint remains 5.9 GB, but the gap to full precision has narrowed materially, with particularly strong gains in agentic coding, multimodal reasoning, and long-horizon tool use.

Ternary Bonsai 2 27B is available today under Apache 2.0.

[Andrei Khrapavitski @endrus]

It’s interesting to see some post-LLM solutions surfacing out of the stealth mode. Curious about Jev

[Diogo Almeida @CompleteSkeptic]

After co-inventing ChatGPT, I kept asking myself: why have superhuman chat models not led to AGI?

I’ve spent the last 2 years in stealth building a new way to train models (RLCD), and a new type of frontier AI model that we are releasing today: Jev

• 20-200x faster
• 40-400x cheaper (w/ output tokens free)
• Frontier composable intelligence optimized for decisions

AFAICT the shortest path to AI-based economic revolution

[Andrei Khrapavitski @endrus]

The agents' swarm that solved the Navier Stokes Millennium Prize problem approximates to 100 years of time. 10,000 agents x 88 agent-hours x (365 days x 24 hrs) = 100 years. Agentic swarms can do extraordinary things with right goals and proper constraints.

[Jakub Pachocki @merettm]

I wrote about the state of AI, why I’m concerned about the next few years, and the choices we need to make to keep the future in humanity’s hands.

An Alien Mind: https://t.co/FeIfWNe0UE

[Boris Cherny @bcherny]

Let us know what you think!

[Prince Canuma @Prince_Canuma]

🚀 Gemma 4 12B is here!

We partnered with @GoogleDeepMind to bring and optimize their new dense and unifed multimodal model for Apple Silicon.

◈ 12B dense · 256K context
◈ Thinking mode (built-in reasoning)
◈ Vision: dynamic res, OCR, UI + charts
◈ Native audio: ASR + speech translation
◈ Function calling for agents
◈ Text + image + audio, interleaved

Runs local. Get started now ⚡

> uv pip install -U mlx-vlm

https://t.co/7BvnEuzKvj

[ClaudeDevs @ClaudeDevs]

We've updated /fork in Claude Code

/fork now runs a background agent with your exact context (system prompt, tools, history, model) and prompt cache. The result gets returned to your session.

/branch (the old /fork) still copies the transcript to a new session you drive.

[Vaibhav (VB) Srivastav @reach_vb]

UPDATE: Came up with an even better version of this prompt after the feedback

Ask Codex to look across your sessions, Memories, and Chronicle, identify patterns, reuse what already exists, and only create the smallest useful skill, subagent, or automation.

"Look back over my recent work from the last 30 days, or all available history if shorter, and identify repeated manual workflows worth packaging.

Use available evidence in this order:
- Recent Codex sessions and task summaries.
- Codex Memories and rollout summaries to find patterns repeated across sessions.
- Chronicle, if enabled, to spot repeated work outside Codex. Use Chronicle for discovery only; confirm important details in the relevant source system when possible.
- Existing skills, custom agents, and automations, so you reuse or extend what already exists instead of duplicating it.

Look broadly for work that is repeated, time-consuming, error-prone, context-heavy, or benefits from a consistent process. Include workflows across coding, research, writing, planning, communication, operations, analysis, and personal administration.

Only act on a candidate when it:
- occurred at least twice, or is clearly likely to recur and costly to repeat;
- has stable inputs, a repeatable procedure, and a clear output or stopping condition;
- would materially improve speed, quality, consistency, or reliability;
- is not already adequately covered.

Choose the smallest appropriate form:
- Skill: a reusable workflow or playbook.
- Custom subagent: a bounded specialist role or investigation task suitable for delegation.
- Automation: a scheduled or recurring check, report, reminder, or monitor.
- Skip: work that is too one-off, ambiguous, sensitive, or poorly evidenced to package.

First produce a compact shortlist with:
- repeated workflow
- supporting evidence and dates
- frequency/confidence
- recommended form: skill, subagent, automation, extend existing, or skip
- why it is or is not worth creating

Then create only the high-confidence missing items. Keep them narrow, practical, source-aware, and easy to validate. Do not create speculative, overlapping, or overly broad assets.

Finish with:
- what you created or extended
- what you deliberately skipped
- what needs more evidence before packaging"

[Boris Cherny @bcherny]

People often ask what my biggest tip is for getting the most out of Claude Code.

These days my #1 tip is: use auto mode

Auto mode means no more permission prompts. It is the key building block for multi-clauding: start a session, then while it runs, work on another session in parallel.

[Akshay 🚀 @akshay_pachaar]

The next step after Karpathy's wiki idea:

Karpathy's wiki works on knowledge that sits still.

A page on how attention works is just as useful today as it was a year ago.

The LLM reads sources, pulls out ideas, writes clean articles, and keeps them cross-linked.

You never have to rebuild the context from scratch when you want to ask something.

But this breaks the moment you ask a question that spans multiple things at once.

"Which authors moved from Google to Anthropic between 2022 and 2024, and what did they publish after the move?"

A Markdown page can't answer that. The answer lives in the connections between people, companies, papers, and dates.

A wiki can describe that pattern only if someone already wrote an article about it.

A graph lets you ask it directly, ask ten variations of it, and get an answer every time without rebuilding anything.

FalkorDB is an open-source graph database built for exactly this kind of question.

The idea underneath it is simple, and it's what makes the whole thing fast enough to be practical.

Most graph databases store connections as chains of pointers and follow them one by one through memory.

FalkorDB stores the entire graph as a grid of zeros and ones (a sparse matrix) where a 1 means "these two things are connected."

Once your graph is a grid, walking through it becomes math. Two hops is one multiplication. Five hops is five multiplications.

That sounds like a small change, but it lets the CPU do work in parallel and reuse decades of math research that nobody had applied to graph queries before.

In practice, this is the difference between a seven-hop question returning in 350ms and the same question timing out.

The wiki and the graph aren't competing. They sit at different layers.

The wiki stores what something is. The graph stores how everything connects.

Any work where the connections matter as much as the things being connected belongs in a graph.

FalkorDB also comes with vector search built in, which matters for GenAI work.

You can find a relevant part of the graph, search for similar items inside it, and return the answer, all in one query.

Most GraphRAG setups build this by hand across two separate databases. Here you get it in one.

You run it through Docker, query it with Cypher, and connect from Python, JavaScript, Rust, Java, Go, or any Redis client.

Open source and multi-tenant by default, so one instance can host thousands of separate graphs without spinning up thousands of servers.

Repo: https://t.co/xZ1rig02JI

Karpathy nailed the foundation. The next layer is here.

[Andrej Karpathy @karpathy]

LLM Knowledge Bases

Something I'm finding very useful recently: using LLMs to build personal knowledge bases for various topics of research interest. In this way, a large fraction of my recent token throughput is going less into manipulating code, and more into manipulating knowledge (stored as markdown and images). The latest LLMs are quite good at it. So:

Data ingest:
I index source documents (articles, papers, repos, datasets, images, etc.) into a raw/ directory, then I use an LLM to incrementally "compile" a wiki, which is just a collection of .md files in a directory structure. The wiki includes summaries of all the data in raw/, backlinks, and then it categorizes data into concepts, writes articles for them, and links them all. To convert web articles into .md files I like to use the Obsidian Web Clipper extension, and then I also use a hotkey to download all the related images to local so that my LLM can easily reference them.

IDE:
I use Obsidian as the IDE "frontend" where I can view the raw data, the the compiled wiki, and the derived visualizations. Important to note that the LLM writes and maintains all of the data of the wiki, I rarely touch it directly. I've played with a few Obsidian plugins to render and view data in other ways (e.g. Marp for slides).

Q&A:
Where things get interesting is that once your wiki is big enough (e.g. mine on some recent research is ~100 articles and ~400K words), you can ask your LLM agent all kinds of complex questions against the wiki, and it will go off, research the answers, etc. I thought I had to reach for fancy RAG, but the LLM has been pretty good about auto-maintaining index files and brief summaries of all the documents and it reads all the important related data fairly easily at this ~small scale.

Output:
Instead of getting answers in text/terminal, I like to have it render markdown files for me, or slide shows (Marp format), or matplotlib images, all of which I then view again in Obsidian. You can imagine many other visual output formats depending on the query. Often, I end up "filing" the outputs back into the wiki to enhance it for further queries. So my own explorations and queries always "add up" in the knowledge base.

Linting:
I've run some LLM "health checks" over the wiki to e.g. find inconsistent data, impute missing data (with web searchers), find interesting connections for new article candidates, etc., to incrementally clean up the wiki and enhance its overall data integrity. The LLMs are quite good at suggesting further questions to ask and look into.

Extra tools:
I find myself developing additional tools to process the data, e.g. I vibe coded a small and naive search engine over the wiki, which I both use directly (in a web ui), but more often I want to hand it off to an LLM via CLI as a tool for larger queries.

Further explorations:
As the repo grows, the natural desire is to also think about synthetic data generation + finetuning to have your LLM "know" the data in its weights instead of just context windows.

TLDR: raw data from a given number of sources is collected, then compiled by an LLM into a .md wiki, then operated on by various CLIs by the LLM to do Q&A and to incrementally enhance the wiki, and all of it viewable in Obsidian. You rarely ever write or edit the wiki manually, it's the domain of the LLM. I think there is room here for an incredible new product instead of a hacky collection of scripts.

[Boris Cherny @bcherny]

I wanted to share a bunch of my favorite hidden and under-utilized features in Claude Code. I'll focus on the ones I use the most.

Here goes.

[Boris Cherny @bcherny]

no 👏 more 👏 permission prompts 👏

[Andrei Khrapavitski @endrus]

Useful https://t.co/32j0OXnXBF

[Avi Chawla @_avichawla]

Big release from Kimi!

They just released a new way to handle residual connections in Transformers.

In a standard Transformer, every sub-layer (attention or MLP) computes an output and adds it back to the input via a residual connection.

If you consider this across 40+ layers, the hidden state at any layer is just the equal-weighted sum of all previous layer outputs.

Every layer contributes with weight=1, so every layer gets equal importance.

This creates a problem called PreNorm dilution, where as the hidden state accumulates layer after layer, its magnitude grows linearly with depth.

And any new layer's contribution gets progressively buried in the already-massive residual. This means deeper layers are then forced to produce increasingly large outputs just to have any influence, which destabilizes training.

Here's what the Kimi team observed and did:

RNNs compress all prior token information into a single state across time, leading to problems with handling long-range dependencies. And residual connections compress all prior layer information into a single state across depth.

Transformers solved the first problem by replacing recurrence with attention. This was applied along the sequence dimension.

Now they introduced Attention Residuals, which applies a similar idea to depth.

Instead of adding all previous layer outputs with a fixed weight of 1, each layer now uses softmax attention to selectively decide how much weight each previous layer's output should receive.

So each layer gets a single learned query vector, and it attends over all previous layer outputs to compute a weighted combination.

The weights are input-dependent, so different tokens can retrieve different layer representations based on what's actually useful.

This is Full Attention Residuals (shown in the second diagram below).

But here's the practical problem with this idea.

Full AttnRes requires keeping all layer outputs in memory and communicating them across pipeline stages during distributed training.

To solve this, they introduce Block Attention Residuals (shown in the third diagram below).

The idea is to group consecutive layers into roughly 8 blocks.

Within each block, layer outputs are summed via standard residuals. But across blocks, the attention mechanism selectively combines block-level representations.

This drops memory from O(Ld) to O(Nd), where N is the number of blocks.

Layers within the current block can also attend to the partial sum of what's been computed so far inside that block, so local information flow isn't lost.

And the raw token embedding is always available as a separate source, which means any layer in the network can selectively reach back to the original input.

Results from the paper:

- Block AttnRes matches the loss of a baseline LLM trained with 1.25x more compute.

- Inference latency overhead is less than 2%, making it a practical drop-in replacement

- On a 48B parameter Kimi Linear model (3B activated) trained on 1.4T tokens, it improved every benchmark they tested: GPQA-Diamond +7.5, Math +3.6, HumanEval +3.1, MMLU +1.1

The residual connection has mostly been unchanged since ResNet in 2015.

This might be the first modification that's both theoretically motivated and practically deployable at scale with negligible overhead.

More details in the post below by Kimi👇
____
Find me → @_avichawla
Every day, I share tutorials and insights on DS, ML, LLMs, and RAGs.

[Andrei Khrapavitski @endrus]

Always On Memory Agent

[Hasan Toor @hasantoxr]

Repo:

https://t.co/jNovhyEZmv

[Akshay 🚀 @akshay_pachaar]

I just fixed a major blind spot in HuggingFace's fine-tuning skill!

HuggingFace released a skill you can plug into Claude or any coding agent that lets you fine-tune open-source LLMs with plain English.

The agent handles GPU selection, job submission, monitoring, and pushes the finished model to the Hub.

But there's a catch.

It assumes you already have a clean dataset on the HuggingFace Hub.

But most of the time, the data you actually want to fine-tune on lives on Twitter, LinkedIn, Reddit, Amazon, and other platforms behind anti-bot walls.

So I integrated Bright Data's Web MCP into the skill so that the coding agent can scrape data from the web before kicking off the training job:

- Scrape data from platforms like YouTube, Amazon, or Reddit, with CAPTCHAs and anti-bot systems handled automatically.
- Structure the scraped content into a formatted fine-tuning dataset
- Validate the dataset, select GPU hardware, and submit the training job to HuggingFace
- Monitor training progress and push the finished model to the Hub

With this update, you can now tell Claude something like: "Scrape the top 500 Python discussions from Reddit, convert them into instruction-response pairs, and fine-tune Qwen3-0.6B on that dataset using SFT."

And the agent takes it from there to do data collection, formatting, training, and deployment in one go.

The original HuggingFace skill did the training side really well, and I didn't touch that.

I just added the missing data collection layer using Bright Data's MCP, which supports 60+ web data tools across 40+ platforms.

Why Bright Data?

Agents accessing the web struggle with IP blocks and CAPTCHA. Bright Data handles all of that behind the scenes.

It lets you scale scraping without getting blocked, simulate real user actions on complex sites, and work with both real-time and historical data from 40+ platforms.

I have shared the GitHub repo of Bright Data MCP in the replies!