Kyle Dunovan

dad, guitarist, machine learning engineer, research scientist.

[Kyle Dunovan @dunovank]

My entry to the #yusmotokichallenge2
Congrats on 20K, Yus!
@heyitsmeyus
@saitoguitars
@VOXampsJP

[Jackson Atkins @JacksonAtkinsX]

My brain broke when I read this paper.

A tiny 7 Million parameter model just beat DeepSeek-R1, Gemini 2.5 pro, and o3-mini at reasoning on both ARG-AGI 1 and ARC-AGI 2.

It's called Tiny Recursive Model (TRM) from Samsung.

How can a model 10,000x smaller be smarter?

Here's how it works:

1. Draft an Initial Answer: Unlike an LLM that writes word-by-word, TRM first generates a quick, complete "draft" of the solution. Think of this as its first rough guess.

2. Create a "Scratchpad": It then creates a separate space for its internal thoughts, a latent reasoning "scratchpad." This is where the real magic happens.

3. Intensely Self-Critique: The model enters an intense inner loop. It compares its draft answer to the original problem and refines its reasoning on the scratchpad over and over (6 times in a row), asking itself, "Does my logic hold up? Where are the errors?"

4. Revise the Answer: After this focused "thinking," it uses the improved logic from its scratchpad to create a brand new, much better draft of the final answer.

5. Repeat until Confident: The entire process, draft, think, revise, is repeated up to 16 times. Each cycle pushes the model closer to a correct, logically sound solution.

Why this matters:

Business Leaders: This is what algorithmic advantage looks like. While competitors are paying massive inference costs for brute-force scale, a smarter, more efficient model can deliver superior performance for a tiny fraction of the cost.

Researchers: This is a major validation for neuro-symbolic ideas. The model's ability to recursively "think" before "acting" demonstrates that architecture, not just scale, can be a primary driver of reasoning ability.

Practitioners: SOTA reasoning is no longer gated behind billion-dollar GPU clusters. This paper provides a highly efficient, parameter-light blueprint for building specialized reasoners that can run anywhere.

This isn't just scaling down; it's a completely different, more deliberate way of solving problems.

[Kyle Dunovan @dunovank]

@gauri__gupta exceptional, thank you

[Kyle Dunovan @dunovank]

@heyitsmeyus @saitoguitars @VOXampsJP Thanks man!

[Berried Alive @CaswellCharles]

I brought my guitar to a job interview
https://t.co/MTFabGxwKW

[william @wgussml]

good job everyone good job
@langchain must die.

🚀 pip install ell-ai

[Sebastian Raschka @rasbt]

If you are looking for a resource to understand the instruction fine-tuning process in LLMs, I've uploaded a notebook to implement the fine-tuning process from scratch: https://t.co/cocpP75fAM
It explains
1. how to format the data into 1100 instruction-response pairs
2. how to apply a prompt-style template
3. and how to use masking.
Of course, this also includes a section on implementing an LLM-based automated process for evaluation.
Happy coding!

[yobibyte @y0b1byte]

Anyone tried this?

[Kayson Fakhar @Kaysonfakhar]

Oh you’re a computational neuroscientist. Ok, what kind of a computational neuroscientist are you?

[Sebastian Raschka @rasbt]

@deliprao @y0b1byte Notebooks are great for interactive exploration, analysis, and teaching.
Python packages are great for software development and debugging.
Python scripts are great for training runs, deploying code, etc.

[elvis @omarsar0]

MatMul-free LLMs

Proposes an implementation that eliminates matrix multiplication operations from LLMs while maintaining performance at billion-parameter scales.

The performance between full precision Transformers and the MatMul-free models narrows as the model size increases.

Quote from the paper: "By utilizing an optimized kernel during inference, our model’s memory consumption can be reduced by more than 10× compared to unoptimized models."

There is a huge potential to use this type of approach to significantly reduce both memory usage and latency in LLMs. It's much needed especially for building agentic workflows and other types of advanced LLM applications that require better memory usage and latency.

Lots of interesting results in this paper and I am now curious what a 100B+ parameter MatMul free model or even a long-context LLM based on this approach can do. Efficiency is a huge constraint for LLMs so it's always exciting to see works that optimize core Transformer operations and overall architecture.

[Kording Lab 🦖 @KordingLab]

Was just going to briefly and very shamelessly plug our textbook on Bayesian modeling of behavior. As opposed to all those DL models, one can actually deeply understand the Bayesian models. If interested in behavior check it out: https://t.co/iwyBUCjupx (free pdf too)

[will brown @willccbb]

been learning a lot about LLMs etc over the past year, organized some of my favorite explainers into a “textbook-shaped” resource guide

wish i’d had this at the start, maybe it can useful to others on a similar journey

https://t.co/54gZimsOnO

[Grant Sanderson @3blue1brown]

I often prefer to read math than to watch videos on it, especially in reviewing material. For anyone who feels the same, there are now written/illustrated versions of the linear algebra series, thanks to the help of Kurt Bruns.

https://t.co/Dnjc6HfosG

[Akshay Agrawal @akshaykagrawal]

My PhD thesis was a 100+ page monograph on vector embeddings.

When I worked on it, I really wanted embeddings to be tangible: to select points in a 2D embedding, and get the original data back in Python.

I couldn't do that in Jupyter ... but you can do it today in marimo.

[Thomas Ahle @thomasahle]

KANs (NNs with learned functions on the edges) have a quite elegant representation using Tensor Diagrams.

This chart of MLP layers also shows some neat relationship between things like ReGLUs and MoEs.

[Thomas Wolf @Thom_Wolf]

This might be one of the most important 45-mn read you could indulge in today if you want to understand the secret behind high performance large language models like Llama3, GPT-4 or Mixtral

Inspired by the @distillpub interactive graphics papers, we settled to write the most extensive, enjoyable and in-depth tech report we could draft on the science of creating high quality web-scale datasets, detailing all the steps and learnings that came in our recent 15 trillion tokens 🍷FineWeb release

And it's not all, in this article we also introduce 📚FineWeb-Edu a filtered subset of Common Crawl with 1.3T tokens containing only web pages with very high educational content. Up to our knowledge, FineWeb-Edu out-performs all openly release web-scale datasets by a significant margin on knowledge- and reasoning-intensive benchmarks like MMLU, ARC, and OpenBookQA

To close, we make a number of surprising observations on the "quality" of the internet it-self which may challenge some of the general assumptions on web data (not saying more, I'll let you draw your conclusions ;)

Take a walk through it, you won't be disappointed 👇

[Sebastian Raschka @rasbt]

Just read Apple's "OpenELM: An Efficient Language Model Family with Open-source Training and Inference Framework". Similar to the OLMo, it's refreshing to see an LLM paper that shares details discussing the architecture, training methods, and training data.

Let's start with the most interesting tidbits:
- OpenELM comes in 4 relatively small and convenient sizes: 270M, 450M, 1.1B, and 3B
- OpenELM performs slightly better than OLMo even though it's trained on 2x fewer tokens
- The main architecture tweak is a layer-wise scaling strategy

Sharing details is not the same as explaining them, which is what research papers were aimed to do when I was a graduate student. For instance, they sampled a relatively small subset of 1.8T tokens from various publicly available datasets (RefinedWeb, RedPajama, The PILE, and Dolma). This subset was 2x smaller than Dolma, which was used for training OLMo. What was the rationale for this subsampling, and what were the criteria?

The layer-wise scaling strategy (adopted from the "DeLighT: Deep and Light-weight Transformer" paper) is very interesting. I wish there was an ablation studio training an LLM with and without this strategy on the same dataset. But those experiments are expensive, and I can understand why they didn't do it.

An interesting bonus that I didn't expect was that the researchers compared LoRA and DoRA (which I discussed a few weeks ago) for parameter-efficient finetuning! It turns out that there wasn't a noticeable difference between the two methods, though.

Anyways, great work, and big kudos to the researchers (and Apple) for sharing!

[Kyle Dunovan @dunovank]

I don’t often join internet pile-ons. But I’ll always make an exception for @Boeing

[Massimo @Rainmaker1973]

Comparing Boeing and Airbus airline incidents.

[Gustavo Cid @_cgustavo]

I used to beg LLMs for structured outputs.

Most of the time, they understood the job and returned valid JSONs. However, around ~5% of the time, they didn't, and I had to write glue code to avoid breaking downstream modules of my system.

Then, I found @jxnlco's instructor. instructor is a zero-cost abstraction that works by leveraging OpenAI's function calling. The steps are:

1. patch the OpenAI client with instructor.
2. define the Pydantic model for the output I expect.
3. pass the Pydantic model as an extra kwarg to my OpenAI call.

Once that's done, instructor guarantees that what I get back is a valid instance of the Pydantic model. I no longer need to write code to handle the unsuccessful cases.

Another benefit I found of using instructor is that it is a much better way to organize my prompts. I can keep information close to the source, with doctrings, field descriptions, and examples.

The example in the screenshot is simple. However, I'm using instructor to manage close to 100 different Pydantic models, each with 3-5 layers of nested objects. And guess what? It works.

For some complex objects, though, the LLM violates some constraints on the first try. instructor also has a retry mechanism. If a constraint is violated, it sends another request to the LLM with part of the stack trace, so that the LLM can correct the output, as in the screenshot below (traces in @openlayerco).

Overall, I'm pretty satisfied with using instructor, and it solved a major issue for us. Also, excited that instructor 1.0.0 just released!