Sam Clastine

Building Allorc

[Sam Clastine @SamClastine]

A 3B model going toe-to-toe with a 1T model.

VibeThinker 3B is seriously impressive.

This is a reminder that efficiency > parameter count.

No need to spend $100/month when optimization wins. 🔥

[Francesco Bertolotti @f14bertolotti]

Stellar performance from a 3B model. These results were achieved primarily through post-training refinements on Qwen2.5-Coder. The paper doesn't provide many details, but it appears they distill from RL ckpts and then do a final RL-based instruct RL.

🔗https://t.co/FmdRwGNMOg

[voratiq @voratiq]

GLM 5.2 high just won head-to-head against Opus 4.8 xhigh and GPT 5.5 xhigh

The task was a tricky performance optimization in an internal code-analysis product

First time we've seen an open-weight agent outperform the top closed agents

Very interesting result...

[atomic.chat @atomic_chat_hq]

New Kimi K2.7 Code performs at GPT-5.5 level 3x cheaper!

We gave both models the same three prompts: build a self-contained HTML5 canvas sim with real physics, no libraries. A spring pendulum on a stretching coil, a 1 kg block trading collisions with a 100,000 kg block, and 22 balls churning in a spinning hexagon

Outputs:
Kimi K2.7 Code: $0.28 on 52.4k tokens
GPT-5.5: $0.93 on 23.4k tokens

Spring pendulums and blocks came out even. The balls Kimi did better: its pile spins with the drum when GPT's bounce around in pure chaos. On price to quality, K2.7 Code is the clear pick

[Kimi.ai @Kimi_Moonshot]

🌘 Kimi-K2.7-Code, our latest coding model, is now released and open-sourced!

🔷 Improved coding & agent performance over K2.6: +21.8% on Kimi Code Bench v2, +11.0% on Program Bench, and +31.5% on MLS Bench Lite.
🔷 Reasoning efficiency: Less overthinking, with 30% lower reasoning-token usage compared to K2.6.
🔷 Long-horizon coding: Improved instruction following, higher end-to-end coding task success rates.

⚡️ 6x High-Speed Mode coming soon!
🔌 Available today via Kimi API and Kimi Code.

🔗 Kimi Code: https://t.co/uvoSJKyGCY
🔗 API: https://t.co/EOZkbOwCN4

[Sundar Pichai @sundarpichai]

Geminiii

[VraserX e/acc @VraserX]

🔥 GPT-6 may not just be smarter, it might be alive (in the computational sense).

A new research paper called SEAL, Self-Adapting Language Models (arXiv:2506.10943) describes how an AI can continuously learn after deployment, evolving its own internal representations without retraining.

Some of the SEAL researchers are now working at OpenAI. 👀

That’s no coincidence.
SEAL’s architecture enables models to:
•learn from new data in real time
•self-repair degraded knowledge
•form persistent “memories” across sessions

If GPT-6 integrates this, it won’t just use information, it will absorb it.
A model that adapts to the world as it changes.
A system that gets better every single day.

This could be the birth of continuous self-learning AI, the end of the frozen-weights era.
Welcome to the next chapter.

[Jackson Atkins @JacksonAtkinsX]

My brain broke when I read this paper.

A tiny 7 Million parameter model just beat DeepSeek-R1, Gemini 2.5 pro, and o3-mini at reasoning on both ARG-AGI 1 and ARC-AGI 2.

It's called Tiny Recursive Model (TRM) from Samsung.

How can a model 10,000x smaller be smarter?

Here's how it works:

1. Draft an Initial Answer: Unlike an LLM that writes word-by-word, TRM first generates a quick, complete "draft" of the solution. Think of this as its first rough guess.

2. Create a "Scratchpad": It then creates a separate space for its internal thoughts, a latent reasoning "scratchpad." This is where the real magic happens.

3. Intensely Self-Critique: The model enters an intense inner loop. It compares its draft answer to the original problem and refines its reasoning on the scratchpad over and over (6 times in a row), asking itself, "Does my logic hold up? Where are the errors?"

4. Revise the Answer: After this focused "thinking," it uses the improved logic from its scratchpad to create a brand new, much better draft of the final answer.

5. Repeat until Confident: The entire process, draft, think, revise, is repeated up to 16 times. Each cycle pushes the model closer to a correct, logically sound solution.

Why this matters:

Business Leaders: This is what algorithmic advantage looks like. While competitors are paying massive inference costs for brute-force scale, a smarter, more efficient model can deliver superior performance for a tiny fraction of the cost.

Researchers: This is a major validation for neuro-symbolic ideas. The model's ability to recursively "think" before "acting" demonstrates that architecture, not just scale, can be a primary driver of reasoning ability.

Practitioners: SOTA reasoning is no longer gated behind billion-dollar GPU clusters. This paper provides a highly efficient, parameter-light blueprint for building specialized reasoners that can run anywhere.

This isn't just scaling down; it's a completely different, more deliberate way of solving problems.

[Rohan Paul @rohanpaul_ai]

A beautiful paper from MIT+Harvard+ @GoogleDeepMind 👏

Explains why Transformers miss multi digit multiplication and shows a simple bias that fixes it.

The researchers trained two small Transformer models on 4-digit-by-4-digit multiplication.

One used a special training method called implicit chain-of-thought (ICoT), where the model first sees every intermediate reasoning step, and then those steps are slowly removed as training continues.

This forces the model to “think” internally rather than rely on the visible steps.

That model learned the task perfectly — it produced the right answer for every example (100% accuracy).

The other model was trained the normal way, called standard fine-tuning, where it only saw the input numbers and the final answer, not the reasoning steps.

That model almost completely failed — it only got about 1% of the answers correct.

i.e. model trained with implicit chain of thought, called ICoT, gets 100% on 4x4 multiplication while normal training could not learn it at all

[Tom Dörr @tom_doerr]

Run 70B LLMs on a 4GB GPU with layer-wise inference and memory optimization, quantization optional

[FearBuck @FearedBuck]

Mark Zuckerberg’s Meta really put a smartphone inside a pair of glasses

[Thinking Machines @thinkymachines]

Today Thinking Machines Lab is launching our research blog, Connectionism. Our first blog post is “Defeating Nondeterminism in LLM Inference”

We believe that science is better when shared. Connectionism will cover topics as varied as our research is: from kernel numerics to prompt engineering. Here we share what we are working on and connect with the research community frequently and openly.

The name Connectionism is a throwback to an earlier era of AI; it was the name of the subfield in the 1980s that studied neural networks and their similarity to biological brains.

https://t.co/lrJioBmpbT

[Dr Singularity @Dr_Singularity]

Big AI/science news

New amazing AI from Google can write expert level science software, and can do it 100x - 1000x faster than humans.

AI science acceleration

Researchers developed an AI system that can write expert level scientific software to speed up discoveries.

It combines a large language model with tree search to explore ideas and improve results.

The AI created 40 new bioinformatics methods that beat human designed ones, 14 epidemiology models that outperformed the CDC’s COVID-19 forecasts, and produced state of the art tools in areas like geospatial analysis, neuroscience, time series forecasting, and math.

And this is just the beginning.

[Sam Clastine @SamClastine]

More brains don’t always mean better results.

In multi-agent debates, weaker models often disrupt stronger ones instead of improving them. Three agents on one task can end up with less clarity, not more.

The real challenge isn’t adding more voices—it’s choosing the right ones.

[elvis @omarsar0]

A few more insights for AI devs:

Manage heterogeneity carefully. Mixing weak and strong models can drag down outcomes. In some cases, “self-ensembling” a strong model may be safer than debating across diverse models.

Don’t assume debate is a free upgrade. Simply adding debate rounds can lower performance, even with stronger models in the majority. This challenges the default belief that “discussion improves answers”.

Beware sycophancy. Current RLHF-aligned models are tuned to be agreeable, which makes them adopt peers’ mistakes instead of challenging them. Developers need to account for this over-compliance when designing multi-agent systems.

Design for critique, not consensus. Protocols should reward independent reasoning, confidence-weighted voting, or explicit error-checking instead of majority agreement. Otherwise, groups converge on wrong answers.

Building robust debate frameworks will likely require structured roles (e.g., critics, verifiers), calibrated weighting of arguments, and training that encourages questioning over politeness. This applies to other multi-agent systems.

Paper: https://t.co/pOdNHV9xxj

[Sam Clastine @SamClastine]

Context is king, not prompts
#AgenticAI

[Chris @ChrissGPT]

🚨 OpenAI Claims to have found the cause of hallucinations

Maybe this is why, all of those employees posted about AGI just before this paper came out

[Nick St. Pierre @nickfloats]

Midjourney

Day 1 Day 637

[Oscar Gomez @OscarGomez369]

Apple Vision Pro The Simpsons did it again 🤪
#Simpsons
#AppleVisionPro

[Sebastian Raschka @rasbt]

There's a new promising method for finetuning LLMs without modifying their weights called
proxy-tuning (by Liu et al. https://t.co/3PjF0NtlOM).
How does it work? It's a simple decoding-time method where you modify the logits of the target LLM. In particular, you compute the logits' difference between a smaller base and finetuning model, then apply the difference to the target model's logits.

More concretely, suppose the goal is to improve a large target model (M1).

The main idea is to take two small models:
- a small base model (M2)
- a finetuned base model (M3)

Then, you simply apply the difference in the smaller models' predictions (logits over the output vocabulary) to the target model M1.

The improved target model's outputs are calculated as M1*(x) = M1(x) + [M3(x) - M2(x)]

Based on the experimental results, this works surprisingly well. The authors tested this on
A. instruction-tuning
B. domain adaptation
C. task-specific finetuning

For brevity, focusing only on point A, here's a concrete example:

1) The goal was to improve a Llama 2 70B Base model to the level of Llama 2 70B Chat but without doing any RLHF to get the model from Base -> Chat.

2) They took a 10x smaller Llama 2 7B model and instruction-finetuned it.

3) After finetuning, they computed the difference in logits over the output vocabulary between 7B Base and 7B Finetuned

4) They applied the difference from 3) to the Llama 2 70B Base model. This pushed the 70B Base model's performance pretty close to 70B Chat.

The only caveat of this method is, of course, that your smaller models have to be trained on the same vocabulary as the larger model. Theoretically, if one knew the GPT-4 vocabulary and had access to its logit outputs, one could create new specialized GPT-4 models with this approach.

[elvis @omarsar0]

I've been exploring RAG use cases more than anything these days.

This has pushed me to explore newer techniques (which I post daily) to improve retriever-augmented systems and tools.

While there are a lot of new approaches emerging, the tooling has room for improvement, especially on the scaling and affordability side of things.

It's becoming clear that combining LLMs (closed or open-source) with external data is a powerful approach to solving all kinds of problems and building useful agents, especially for enterprise where there is a ton of data available.

So it's exciting to see Abacus AI expanding its offering to cheaper inference, LLMOps APIs, and vector store APIs.

For the RAG use cases, it's important to have a scalable vector store for storing and retrieving embeddings so I think that's where Abacus AI can really help developers. The vector store can store billions of embeddings, has millisecond latencies, and can deliver 100% recall.

They already offer a great no-code solution for building LLM-powered chat systems but offering these solutions in the form of APIs will enable interesting knowledge-intensive applications.

Learn more here: https://t.co/q7LpqIhuwy

[elvis @omarsar0]

Introduction to Modern Statistics

If you are studying computer science or machine learning, it's worth every minute learning about Statistics.

This online book looks like a great place to start.

FREE PDF is also available.

This is an absolute gem!

https://t.co/YTTZnidbAs