SJ

I'm an Engineer, so to save time let's just assume I'm always right

[Lunar @LunarResearcher]

Anthropic engineer just dropped a 2-hour+ workshop on Graph Engineering.

"80% of our engineers are already using self-improving loops.

Now everyone is building agentic graphs."

04:43 - RAG and graphs
26:30 - Nodes and edges
1:09:11 - Index agent data inside graphs
1:30:50 - The three graph layers
1:48:15 - Self-verifying Adaptive RAG
2:16:37 - What comes after agent loops

Most people are still building isolated agents.

Anthropic engineers are already connecting them into graphs.

Loops are the old workflow.

Graphs are the new one.

This 2-hour workshop is worth more than most $500 agent memory courses.

Bookmark and watch it before everyone catches up.

Then read the full graph engineering roadmap below

[Vivek Galatage @vivekgalatage]

The GPU glossary from @modal and @charles_irl is so good that I shared it with a bunch of students, and they want to jump into all the finer and lower details with great curiosity - loved the enthusiasm.

https://t.co/sFCfMFUDfe

[Pau Labarta Bajo @paulabartabajo_]

Highly recommended presentation by the great @maximelabonne for those interested in learning how Small LMs are built.

[Bhavin Jawade @BhavinJawade]

𝗞𝗶𝗺𝗶 𝗞𝟯's 𝗽𝗼𝘀𝘁-𝘁𝗿𝗮𝗶𝗻𝗶𝗻𝗴 𝗿𝗲𝗰𝗶𝗽𝗲

Kimi K3's technical report just dropped. K3 is a 2.8T-parameter open-weight MoE model. I went through the report, and here I share its post-training recipe.

As we have seen through the DeepSeek V4 post-training recipe, training multiple narrow expert models through RL and then using multi-teacher on-policy distillation to consolidate them into a single model is becoming a standard recipe across many of these frontier-model post-trainings. Kimi K3 follows the same skeleton. The report does not describe a separate RLHF or DPO preference-tuning stage here either.

𝗦𝘁𝗮𝗴𝗲 𝟬: 𝗦𝗙𝗧 𝗰𝗼𝗹𝗱 𝘀𝘁𝗮𝗿𝘁

1) Standard cold-start SFT to set up the base policy before RL. They use synthetic agentic trajectories generated by domain-specialist models from the earlier Kimi series, followed by multi-stage verification and human-in-the-loop annotation.

2) They use their own chat format, called XTML, to standardize the data, with separate think, response, and tools channels. The reason this matters later is that it gives the SFT, RL, and distillation stages a common structured interface.

𝗦𝘁𝗮𝗴𝗲 𝟭: 𝗧𝗿𝗮𝗶𝗻𝗶𝗻𝗴 𝘁𝗵𝗲 𝗲𝘅𝗽𝗲𝗿𝘁 𝘁𝗲𝗮𝗰𝗵𝗲𝗿𝘀 𝘄𝗶𝘁𝗵 𝗥𝗟

Kimi does not stop at one expert per domain. It crosses domain specialization with reasoning-effort specialization.

1) They pick three broad domains: general tasks, general agents, and coding agents. This is the specialist-per-domain idea also seen in MiMo-V2-Flash and DeepSeek V4.

2) They train an expert for each domain at three reasoning-effort levels: low / high / max. Three domains times three effort levels gives nine expert teachers, not three.

3) Effort is controlled through a token budget during RL. Each problem gets an initial budget estimated from the cold-start model, and any rollout that goes beyond a scaled version of that budget gets its reward overridden to -1, even if the answer was correct. Going over budget is therefore punished directly.

For general tasks, the budget counts thinking tokens. For agentic tasks, it counts cumulative output tokens, including both reasoning traces and tool-call arguments.

4) They train the max-effort expert first with a generous, but still capped, budget and then anneal the budget multiplier down to obtain the high- and low-effort experts. It is a curriculum over the budget multiplier, not three independent training runs.

Here are a few more papers worth reading to understand this budget-controlled reasoning approach:

- L1 is a closely related example of training a model through RL to follow a target token budget, although its objective differs from Kimi's hard over-budget reward override.

- Kimi K1.5 used a relative length reward to discourage unnecessarily long responses, while K2.5 introduced problem-dependent budget-controlled RL. K3 extends this into explicit low / high / max effort.

- Reasoning-effort-guided training is also used by gpt-oss, which is trained with low / medium / high prompt-conditioned effort levels. @rasbt has a nice recent post on reasoning-effort control.

For non-verifiable general tasks inside this stage, the reward comes from an agentic generative reward model. We have seen DeepSeek V4 post-training also use a GRM for hard-to-verify tasks. Kimi provides more detail about its judging protocol:

1) The judge follows a fixed protocol: read the output, generate a rubric, score each candidate against that rubric, and write the scores to a scorepad. The rubric is generated for the task rather than being globally fixed.

2) They retain the tournament-style group reward with binary comparisons from K2.5 for ranking candidates.

3) To stop the reward model from simply rewarding longer answers, a candidate that goes beyond a verbosity budget automatically loses its comparison. It is the same budget philosophy as the reasoning-effort control, reused to curb reward hacking.

𝗦𝘁𝗮𝗴𝗲 𝟮: 𝗙𝘂𝘀𝗶𝗻𝗴 𝘁𝗵𝗲 𝗻𝗶𝗻𝗲 𝗲𝘅𝗽𝗲𝗿𝘁𝘀 𝗶𝗻𝘁𝗼 𝗼𝗻𝗲 𝗺𝗼𝗱𝗲𝗹 (𝗠𝗢𝗣𝗗)

The nine experts are consolidated into a single student using multi-teacher on-policy distillation. We have also observed this in DeepSeek V4.

1) The student samples its own rollouts. For each domain and sampled effort level, the rollout is matched with the corresponding teacher: a coding rollout at max effort is guided by the coding-max teacher, and so on across the nine experts.

2) The teacher evaluates the tokens sampled by the student. Kimi uses the clipped teacher-versus-student log-probability ratio on each sampled token as a dense OPD reward, corresponding to a sampled-token estimator of the reverse-KL objective.

Because the training is on-policy, the student receives supervision on its own generated prefixes, avoiding the exposure bias associated with off-policy distillation. It also provides dense token-level guidance rather than relying only on the sparse trajectory-level outcome rewards commonly used in RLVR.

3) Kimi K3 uses the simpler sampled-token OPD reward rather than top-k or full-vocabulary distribution distillation. This contrasts with DeepSeek V4, which computes the full-vocabulary reverse-KL objective. Kimi also experimented with more fine-grained top-k objectives but observed no clear advantage in either convergence speed or final performance.

Previous MOPD work has discussed several benefits of integrating specialist capabilities in policy space this way: avoiding the exposure bias of off-policy distillation because the student trains on its own rollout distribution, providing dense token-level supervision, reducing cross-domain interference, outperforming parameter merging in the reported experiments, and enabling a modular and parallel pipeline in which each teacher can be developed independently.

Training independent specialist models with RL and then using MOPD to consolidate them into a unified model is becoming a standard recipe across several of the latest frontier-model post-training reports: DeepSeek V4, Xiaomi's MiMo-V2-Flash, NVIDIA's Nemotron 3 Ultra, and now Kimi K3. 🧵

[codila @0xCodila]

Google just released free 1-hour course on building agentic knowledge Graphs from 0% to 100%:

10% → 4:01 - how to build a GraphRAG agent
30% → 15:00 - Graph Engineering explanation
55% → 30:00 - Agentic search Engineering
80% → 35:48 - Graph Engineering practice
100% → 47:06 - self-improving agents in graphs

this free Google course mass replaces a $500 graph engineering bootcamp - learn it in 60 min to 100%

watch it today - then read the full graph playbook in the article below ↓

[AVB @neural_avb]

Some papers about Looped Transformers arranged chronologically:

Universal Transformers
https://t.co/cr270xYYQW

MoEUT
https://t.co/gndnj7ZJ4N

Relaxed Recursive Transformers https://t.co/CEjpQvbI43

Recurrent Depth Transformers
https://t.co/yhlVvNELpx

Mixture-of-Recursions
https://t.co/oQ2uyjr843

DeepLoop
https://t.co/ynm8e4YtNV

Loop the Loopies
https://t.co/n8jpo8G63m

[The Year of the Graph @TheYotg]

Human-in-the-Loop and Open Schema Tooling: Two New Ways Graphs Are Powering Investigations

What do a homegrown OSINT pipeline and a new open-source library have in common? Both start from the same premise: a relational database can store a network, but it can't ask "who connects to whom" the way a graph can.

RecomendeMe Intelligence, a two-person team operating out of Natal, Brazil, ran an open-source intelligence investigation across eight cities, working exclusively from public-domain sources — tax registries, judicial records, OSINT feeds — a design constraint chosen for LGPD compliance rather than convenience. 

Their architecture is a four-layer pipeline: linguistic decoding of coded language, LLM-based entity extraction, rule-based behavioral pattern matching against known trafficking modus operandi, then a Neo4j graph carrying every validated relationship. 

PageRank surfaced the highest-centrality entities. Louvain clustering found groups no linear reading of the documents would reveal. Shortest-path traversal in Cypher exposed indirect ties between suspect nodes.

The part that made the findings usable rather than merely interesting: a mandatory human-in-the-loop checkpoint. 

Every escalated finding carries its full evidence chain and source citations, and domain experts validate or correct the AI-proposed relationships before anything is treated as intelligence instead of hypothesis. 

The work was selected as a featured case in the ITU AI for Good "Innovate for Impact" Interim Report and formally submitted to Brazil's Federal Public Prosecutor's Office.

C4ADS approached the same underlying problem — unifying disparate investigative data into a queryable network — from the tooling side. Their new open-source library, followthemoney-neomodel, bridges the Follow the Money (FtM) data standard with Neo4j. 

FtM gives anti-corruption investigators a shared vocabulary for entities and their relationships — ownership, directorship, payment — but until now, using that vocabulary in a graph meant hand-writing a Python class for every entity type. 

The toolkit generates FtM's full schema as ready-to-use classes automatically, preserving its inheritance hierarchy through multi-label nodes, so a Person node carries Thing, LegalEntity, and Person all at once. 

Analysts can build and traverse investigative networks — like mapping a Politically Exposed Person's corporate and financial ties — in plain Python, no Cypher required.

What connects both efforts: neither treats the graph as the end product. 

It's the substrate that makes disconnected records answerable to the question investigators actually ask — who connects to whom, and how many hops away — while keeping the output traceable back to its source and, where it matters, checked by a human before anyone acts on it.

By Lucas Matheus and Justin Quick

Building a Human-in-the-Loop Knowledge Graph for OSINT Investigations with Neo4j https://t.co/HPATdTPx7Y
Bringing Follow the Money into the Graph https://t.co/x0ZNDPv8iu

#OSINT #GraphAnalytics #AntiCorruption #OpenSource

--

💬 ‘A great newsletter’ - Claudia Remlinger, former Sr. Marketing Director, Neo4j. 

Join readers from Amazon, Capgemini, Michelin, Neo4j & more

Subscribe to the Year of the Graph newsletter for quarterly updates and insights on all things #KnowledgeGraph, #GraphDB, Graph #Analytics / #DataScience / #AI and #SemTech 👇

https://t.co/7pg6gqWYvw

[lazybutai @lazybutai]

@aiDotEngineer @coyle_frankp @emileifrem @steveonjava Fun way to explore them is with:
https://t.co/zFACEBuxVo

https://t.co/uH9M1OmET0

[AI Engineer @aiDotEngineer]

✨ontologies✨

thanks Prof @coyle_frankp, @emileifrem, Jesús Barrasa, @steveonjava et al for reintroducing these concepts to the AI Engineering world

[swyx @swyx]

first "100k views in 4 days" since @trq212's keynote. especially hard to achieve s.t. no biglab boost, no hypey buzzword bingo.

just a useful concept, presented well, that will stand the test of time.

always glad to see these show up in our CFPs. usually this early traction is a good predictor that @coyle_frankp will join our rare ranks of >1M view speakers.

We've got more professors coming - I'm excited for @JohnOusterhout's keynote! increasingly like the idea of leaning on youtubers and professors as keynoters for industry conferences as 1) they obviously speak for a living so are good at it, 2) they dont have obvious bias for a single specific startup to sell you on (usually), 3) they can condense years of thinking/already public output into a concise talk for you to rabbithole on further if the keynote interests you (others are not nearly as public)

[Tiezhen WANG @Xianbao_QIAN]

Wow DSpark!

[Ahmad @TheAhmadOsman]

ATTENTION

The bible for running LLMs locally is now available online to read for FREE

Covers what to use on

- Laptop / edge / odd hardware
- Mac-first workflows
- Single RTX GPUs
- 2-4+ NVIDIA / CUDA GPUs
- General production serving
- Long-context / MoE / routing
- NVIDIA max performance
- Cluster orchestration

Software

- llama.cpp
- MLX / MLX-LM
- ExLlamaV2
- ExLlamaV3
- vLLM
- SGLang
- TensorRT-LLM
- NVIDIA Dynamo

You should read this, and if you cannot now then you most definitely wanna bookmark it for later

Local & Opensource AI FTW

[ali @waterloo_intern]

I spent 48 hours with the Kimi K3 modeling code.

It took:
- 650 mg of caffeine (mandatory)
- 40 cans of LaCroix (optional... world record (?))
- 8 papers
- 6 months off my lifespan

Finally grokked the entire lineage of Kimi K3 and how we got here... every single step, since 2019 GPT-2

[Philip Kiely @philipkiely]

As an inference engineer, I always discounted tokenization time as negligible.

Turns out there is a traffic pattern where that is not true: 100K-1M token ISL + mostly a prefix cache hit means tokenization is material to TTFT.

So Michael fixed it for Kimi K3.

[Ramchand Kumaresan @Mechramc]

At 38, I had 17 years of program management under my belt. PgMP, PMP, enterprise SaaS. Zero machine learning experience. At that time, as everyone does, I started doing some Coursera courses but everything was vague. There were two things that helped - @karpathy and @rasbt. Once the concepts were broken down - I could build, break, and eventually come up with 100s of ideas that failed.

I decided to pivot into AI research. Not a side hobby. A real pivot. I spend to this day at least a few hours every week reading latest papers and trying to break the implementation of the paper. Even now, I am participating in the ICLR huggingface paper reproduction challenge.

Two years later at 40: 6 papers on arXiv, an NVIDIA Inception membership, founded Murai Labs, and an open source Tamil language model that is coming out soon. The book grew directly out of what I needed to build TamilLM from scratch. If it helped me learn how to ship a real model with no ML background, it can help you too. It is such a beautiful coincidence that TamilLM finished pretraining and Under The Hood launched on Amazon on the same day - it was not planned. (Kadavul irukaan Kumaru moment)

100+ copies sold on Leanpub. #1 bestseller there. Now available worldwide on Amazon.

8 free chapters at https://t.co/ZFffhN8aYv (Chapters 4, 5, 8, 13, 18, 23, 27, 32).

India Kindle: https://t.co/pwHlarFb03
US Kindle: https://t.co/YwjBi7keBB
US Paperback: https://t.co/SdubSGpxLh
Leanpub: https://t.co/9hANb7VaLc

I started late. But not done yet - I am glad I started. 🙏

[General Intuition @gen_intuition]

Introducing MIRA.

A playable, multiplayer world model.
A dream of Rocket League.

Trained on 10k hours of data collected with publicly available bots, MIRA learns the dynamics of a four-player game. The model runs in real time at 20 fps, based on the keys you and the other players press.

Built by General Intuition and @kyutai_labs, in collaboration with Epic Games. Not used to develop Rocket League.

▶️ Play the demo, read the technical report, and explore the open-source code at https://t.co/JjlsamGE1D

At ICML? Find us at Booth 111 to try it yourself and dig into the results with the team.

[vixhaℓ @TheVixhal]

https://t.co/qQKhudPYIL

[How To Prompt @HowToPrompt__]

China has open-sourced a sandbox that:

→ boots in 60ms (docker takes 200ms)
→ uses 5MB of RAM per instance
→ runs thousands of AI agents on one machine
→ isolates every agent at the hardware level
→ drops into your E2B code by swapping one URL

It’s called CubeSandbox, a hardware-isolated sandbox for AI agents that cold-starts in under 60ms with just 5MB of memory overhead.

It's a drop-in E2B replacement. Swap one URL. Keep your code. Kill the bill.

no Docker. no shared kernel. no cold start tax.

100% open source.

[Nathan Barry @nathanrs]

Repo:
https://t.co/lqxeagiyDT