Fernando Velasco

Mathematician. Data Scientist @EA. Deep Learning, algorithms, visualizations y del Atleti.

[Juan Julián @juanjucm]

Last month NVIDIA released Nemotron 3 Nano Omni, a highly efficient open model that unifies video, audio, image, and text understanding.

Today you will learn how to deploy it in Microsoft Azure Foundry with Hugging Face and leverage its multimodal power 🚀

[AYi @AYi_AInotes]

全网都在吹的LeCun新论文，90%的解读都是错的。

他们说生成式AI是死路，说过去三年花的几百亿全白费了，说15M参数的小模型就能吊打万亿大模型。

这些全是营销号的夸张，
我觉得这篇论文的真正分量比他们吹的还要重。

Yann LeCun团队这次解决了JEPA困扰了好几年的表征坍缩问题。

以前的世界模型，学着学着就会把狗车人都压成一模一样的向量，什么都学不到。

这次他们只加了一个极其优雅的数学正则化器SIGReg，
没有复杂的trick 和六个超参数要调，训练稳得离谱。

单张GPU几个小时就能训完，在机器人控制任务上，规划速度比巨型世界模型快48倍，成功率还更高。

最厉害的是它的隐空间里天然就编码了物理规律。
不用教，它自己就知道物体不能瞬移，知道速度和位置的关系。
能瞬间检测出物理上不可能发生的事。

这不是啥范式革命，也不会让GPT和Claude明天就死掉。
语言和创意生成，依然是自回归大模型的天下。

但它打开了一扇全新的门，
原来懂物理不需要万亿参数，不需要云端超算。
原来世界模型可以小到跑在机器人的本地芯片上。

过去三年，整个行业都在一条路上狂奔，堆参数，堆算力，堆数据。
所有人都以为只要足够大，就能懂世界。

现在我们终于知道，还有另一条路，一条更高效，更优雅，更接近真实世界运行方式的路。

生成式AI不会死，
但未来的智能体不会是只会聊天的大模型，
它会是一个懂物理的小世界模型，加一个大语言接口，

这才是这篇论文真正的意义所在吧 hhh

[Maarten Grootendorst @MaartenGr]

A Visual Guide to Gemma 4

With almost 40 (!) custom visuals, explore the new models from Google DeepMind. We explore various techniques, ranging from Mixture of Experts and the Vision Encoder all the way up to Per-Layer Embeddings and the Audio Encoder.

Link below 👇

[Tolkien World @TolkienWorldG]

Billy and Dom vs Colbert in Tolkien Trivia!

Did you know all the answers to these questions?

[Andrej Karpathy @karpathy]

Excited to release new repo: nanochat!
(it's among the most unhinged I've written).

Unlike my earlier similar repo nanoGPT which only covered pretraining, nanochat is a minimal, from scratch, full-stack training/inference pipeline of a simple ChatGPT clone in a single, dependency-minimal codebase. You boot up a cloud GPU box, run a single script and in as little as 4 hours later you can talk to your own LLM in a ChatGPT-like web UI.

It weighs ~8,000 lines of imo quite clean code to:

- Train the tokenizer using a new Rust implementation
- Pretrain a Transformer LLM on FineWeb, evaluate CORE score across a number of metrics
- Midtrain on user-assistant conversations from SmolTalk, multiple choice questions, tool use.
- SFT, evaluate the chat model on world knowledge multiple choice (ARC-E/C, MMLU), math (GSM8K), code (HumanEval)
- RL the model optionally on GSM8K with "GRPO"
- Efficient inference the model in an Engine with KV cache, simple prefill/decode, tool use (Python interpreter in a lightweight sandbox), talk to it over CLI or ChatGPT-like WebUI.
- Write a single markdown report card, summarizing and gamifying the whole thing.

Even for as low as ~$100 in cost (~4 hours on an 8XH100 node), you can train a little ChatGPT clone that you can kind of talk to, and which can write stories/poems, answer simple questions. About ~12 hours surpasses GPT-2 CORE metric. As you further scale up towards ~$1000 (~41.6 hours of training), it quickly becomes a lot more coherent and can solve simple math/code problems and take multiple choice tests. E.g. a depth 30 model trained for 24 hours (this is about equal to FLOPs of GPT-3 Small 125M and 1/1000th of GPT-3) gets into 40s on MMLU and 70s on ARC-Easy, 20s on GSM8K, etc.

My goal is to get the full "strong baseline" stack into one cohesive, minimal, readable, hackable, maximally forkable repo. nanochat will be the capstone project of LLM101n (which is still being developed). I think it also has potential to grow into a research harness, or a benchmark, similar to nanoGPT before it. It is by no means finished, tuned or optimized (actually I think there's likely quite a bit of low-hanging fruit), but I think it's at a place where the overall skeleton is ok enough that it can go up on GitHub where all the parts of it can be improved.

Link to repo and a detailed walkthrough of the nanochat speedrun is in the reply.

[Andrej Karpathy @karpathy]

I am (slowly) re-reading the Tolkien legendarium (of which Lord of the Rings is a small part). The whole body of work is so incredible and there's nothing else like it... it dilutes other worlds of fiction. Wait - your story doesn't have a comprehensive history/mythology spanning multiple ages all the way back to a creation myth as detailed in separate volumes? You didn't first invent new languages and dialects for your characters? You didn't pack it with powerful themes and stories written it in a beautiful, archaic style and compose poems and songs alongside? It didn't take you multiple decades of iteration? And what of all the uncharted territory still remaining? Is Tom Bombadil one of the Ainur. Where are the Entwives. What happened to the two unaccounted Istari. Can we hear more about what it was like in Cuiviénen when the elves first awoke? Or to see the light of the two trees of Valinor. Or of the splendor of the caves of Aglarond.

What's most on my mind though - the Tolkien legendarium is imo a concrete example of a height of culture. Does AI, today or soon, make it easier to reach this high via empowerment in both writing and ideation? Or harder, when quick wins are tempting and ~free, and an independent ability to create is stifled. If such a body of work is made again but now with heavy AI assistance, does it inspire the same wonder? What if thousands of them come out on demand with just a prompt? Why do you feel cheated when you learn that something your read was AI generated? Is it transient or a function of capability? Is it slop? What is slop? Or is wonder inseparable from its own creation myth of a lifelong obsession of a mind like your own? So many questions.

[Julien Chaumond @julien_c]

The Ultra-Scale Playbook (the large-scale LLM training guide from the @huggingface science team) is out now!🔥

It is a 246-page, very nicely designed PDF that walks you through learning how to train your own DeepSeek-V3 model using:
• 5D parallelism,
• ZeRO,
• fast kernels,
compute/comm overlap and bottlenecks with theory, interactive plots and 4000+ scaling experiments and audio!

It's available with a HF PRO subscription ✨

[Balaji @balajis]

AI PROMPTING → AI VERIFYING

AI prompting scales, because prompting is just typing.

But AI verifying doesn’t scale, because verifying AI output involves much more than just typing.

Sometimes you can verify by eye, which is why AI is great for frontend, images, and video. But for anything subtle, you need to read the code or text deeply — and that means knowing the topic well enough to correct the AI.

Researchers are well aware of this, which is why there’s so much work on evals and hallucination.

However, the concept of verification as the bottleneck for AI users is under-discussed. Yes, you can try formal verification, or critic models where one AI checks another, or other techniques. But to even be aware of the issue as a first class problem is half the battle.

For users: AI verifying is as important as AI prompting.

[Anthony Bonato @Anthony_Bonato]

Checks out

[Fermat's Library @fermatslibrary]

Évariste Galois died exactly 193 years ago at age 20 from wounds suffered in a duel.

At 17, Galois was able to determine a necessary and sufficient condition for a polynomial to be solvable by radicals, thereby solving a problem standing for 350 years.

[Jeremy Howard @jeremyphoward]

Announcing fasttransform: a Python lib that makes data transformations reversible/extensible. No more writing inverse functions to see what your model sees. Debug pipelines by actually looking at your data.

Built on multi-dispatch. Work w/ @R_Dimm
https://t.co/OGDrBFhnfP

[Aleksa Gordić (水平问题) @gordic_aleksa]

How does OpenAI's o1 exactly work? Here is a list of papers & summaries on LLM reasoning that I've recently read.

I'll split them into 2 categories:

1) prompt-based - enforce step by step reasoning & self-correcting flow purely using prompts

2) learning-based - bake in the above into the policy model's weights (or into a verifier - usually a PRM; process reward model)

--- Prompt-based

0) CoT et al: https://t.co/j5pxO5AdeJ

Ground zero (plus I'm a C guy :P iykyk). It started with Chain-of-Thought paper. This class of methods boils down to asking the LLM nicely to reveal its internal thoughts (e.g. "Let's think step by step"; more generally telling the model to disclose the intermediate computation steps in someway).

A simple variation on CoT would be "CoT self-consistency" -> i.e. sample multiple CoT traces in parallel and use majority voting to find the "right" answer.

1) ToT (tree of thought): https://t.co/NtdPu84pkt

Further complexifies the above (in CS terms we go from linear list -> tree): build up an m-ary tree of intermediate thoughts (thoughts = intermediate steps of CoT); at each thought/node:

a) run the “propose next thoughts” prompt (or just sample completions m times)
b) evaluate those thoughts (either independently or jointly)
c) keep the top m

cons: very expensive & slow
pro: works with off-the-shelf LLMs

2) Self-Reflection: https://t.co/E9huynHP8s

If incorrect response pass the self-reflection feedback back to an LLM before attempting to re-answer again;

As an input self-reflection gets a gold answer and is prompted to explain how it would now solve the problem. The results are redacted before passing back the feedback to avoid leaking the solution.

Even re-answering given only a binary feedback (“your previous answer was incorrect”) is significantly stronger than the baseline (no feedback, just sample a response once).

3) Self-Contrast: https://t.co/LJpT2qCoHv

a) create multiple solutions by evaluating diverse prompts derived from the original question (yields diverse perspectives on how to solve the problem
b) pairwise contrast the solutions
c) generate a todo checklist in order to revise the generations from a)

4) Think Before You Speak: https://t.co/NFm8OykRnk

Introduces the CSIM method: 5 prompts that make the model better at dialogue applications.

5 prompts they use to help improve the communication skills are "empathy", "topic transition", "proactively asking questions", "concept guidance", "summarizing often".

Their LLM has 2 roles: thinking and speaking. The thinking role, or the “inner monologue” is occasionally triggered by the 5 prompts and is not displayed to the user but is instead used as the input for the user-facing speaking role.

I think these 5 bullet points nicely captures the main modes of prompt-based methods i've observed -> lmk if I missed an important one!

--- learning-based -> coming up in the follow-up post

[Sepp Hochreiter @HochreiterSepp]

I am so excited that xLSTM is out. LSTM is close to my heart - for more than 30 years now. With xLSTM we close the gap to existing state-of-the-art LLMs. With NXAI we have started to build our own European LLMs. I am very proud of my team. https://t.co/IH7giCe3gd

[Ruben QC @RubenQC_Lab]

Extremely grateful to @WorldwideCancer, the support of #Curestarters and @ContraCancerEs AECC, for teaming up and give us the chance to investigate mitochondrial transfer as a novel approach against #glioblastoma @InstitutoCajal
@CSIC @CSICdivulga https://t.co/np43Gb0Gcu

[Yann LeCun @ylecun]

By assuming that technical talent is fungible and interchangeable, a previously-successful engineering company will be run into the ground...
...along with a few of its own products and everyone on board.

It's astonishing how certain types of managers will prefer less-talented-but-more-controlable scientists and engineers over more brilliant and creative ones who may be more difficult to control, somewhat unpredictable, and sometimes annoying.

Here is the thing: scientific and technical breakthroughs are unpredictable by management, and progress often comes from folks with unconventional personalities and backgrounds.

As I was told by my management at Bell Labs: "You don't have to be crazy to work here, but it helps" 😄

[Fernando Velasco @fer_maat]

@jmgargon @pablodeolavide Cuantísimo me alegro. Enhorabuena, tío!!

[María Grandury @mariagrandury]

🔥 ¡Lanzamos oficialmente las inscripciones y colaboraciones para el Hackathon #Somos600M!

En @SomosNLP contamos con tooodo el mundo para conseguir los ambiciosos objetivos que nos hemos propuesto 🚀

➡️ ¡Únete ya! https://t.co/NAE9JxyeuP

[Ruben QC @RubenQC_Lab]

📢 Ya está publicada la propuesta de resolución definitiva de las JAE Intro 2023 https://t.co/Bxbtj1fTh0. Contáctanos antes del⏰27 de agosto si quieres unirte a nuestro proyecto de transferencia mitocondrial intercelular JAEINT23_EX_0079 @InstitutoCajal @CSIC

[Andrés Trasado @andres_trasado]

Unos le conocen como el Duque, otros como la Ambición Rubia, el genio de Brooklyn o el titoclint... Pero todos le conocemos como Al Paccino, y os voy a contar los 10 datos sobre Al Paccino que siempre habéis temido preguntar.