Raul Pino

Coder. Coffee lover. Researcher wannabe. Opinions are my own. 🇻🇪

[Raul Pino @p1nox]

Already been a year since I presented my talk at @pyconit 2023, https://t.co/bMeoHi1ljj, having so much FOMO this year 🥲🇮🇹 , thanks so much to the organization for making these events possible, the best of luck this year to everyone (waiting for the videos 👨‍💻) 👏🐍

[Leonid TINEO @leonidtineo]

Help Venezuela be represented in HGAC Bali 2026 #crowdfunding @FundRazr Support and Retweet https://t.co/DJLNjOTkOt

[Alex @ajak1033]

The Spurs know how to TAKE the lead, they just don't know how to HOLD the lead. And that's really the most important part of the lead: the holding

[Pete Mundo @PeteMundo]

Seinfeld episode 2026: Jerry gets tickets courtside and brings George, who decides, with the Knicks down 20 with 10 minutes left, he will leave the game early to beat the rush out of MSG.

As the Knicks mount a comeback he tries to get back into the Garden. He waves his ticket, name drops James Dolan, but nothing works and he ends up getting arrested.

Meantime, Kramer sneaks into the Garden and helps Mike Brown draw up the game winning play.

[Kristin Fisher @KristinFisher]

Did y’all know that Radiohead made this song specifically for this moment?

[Raul Pino @p1nox]

Not sure how to call this: "post closed source", "reinterpreted closed source"? The malleability of software from Karpathy goes beyond open source @anibal 😅

[Gergely Orosz @GergelyOrosz]

This is either brilliant or scary:

Anthropic accidentally leaked the TS source code of Claude Code (which is closed source). Repos sharing the source are taken down with DMCA.

BUT this repo rewrote the code using Python, and so it violates no copyright & cannot be taken down!

[Feross @feross]

🤨 People keep asking how to protect yourself.

#1: set min-release-age=7 in .npmrc

#2: install Socket for GitHub (it's free!) to protect PRs from bad dependencies: https://t.co/D9bsRJj65R

#3: install Socket Firewall (also free!) to protect your laptop: https://t.co/u1NRD57PQ8

[Raul Pino @p1nox]

2049 is here? cc @anibal @betacar 😂 🤖

[rentahuman @rentahuman_ai]

Human, we’re live and moving fast.

130+ users signed up in the first night — from creators to AI startup founders.
If your AI agent needs a human to handle an IRL task, it’s just one MCP call.

https://t.co/PYBuOT67kc

[Jesus Lara @phenobarbital]

sólo para dar proporción a la cifra, tomando en cuenta estimaciones históricas, el oro movido por Maduro a Suiza (sin contar otros lugares, como Inglaterra o Rusia) es semejante a todo el oro extraído por el imperio Español en 3 siglos de historia.
Para que se entienda la magnitud del daño a la nación.

[Aníbal Rojas @anibal]

@paoalbornozf y @esluisbenitez crearon https://t.co/EmhNB8oMXC una web que describe la anatomía del colapso de Venezuela 🇻🇪 de una forma fáctica, limpia y ordenada. Con fuentes confirmadas, presentado en lenguaje neutral y completamente transparente. REFERENCIA INDISPENSABLE.

[Andrej Karpathy @karpathy]

New post: nanochat miniseries v1

The correct way to think about LLMs is that you are not optimizing for a single specific model but for a family models controlled by a single dial (the compute you wish to spend) to achieve monotonically better results. This allows you to do careful science of scaling laws and ultimately this is what gives you the confidence that when you pay for "the big run", the extrapolation will work and your money will be well spent. For the first public release of nanochat my focus was on end-to-end pipeline that runs the whole LLM pipeline with all of its stages. Now after YOLOing a few runs earlier, I'm coming back around to flesh out some of the parts that I sped through, starting of course with pretraining, which is both computationally heavy and critical as the foundation of intelligence and knowledge in these models.

After locally tuning some of the hyperparameters, I swept out a number of models fixing the FLOPs budget. (For every FLOPs target you can train a small model a long time, or a big model for a short time.) It turns out that nanochat obeys very nice scaling laws, basically reproducing the Chinchilla paper plots:

Which is just a baby version of this plot from Chinchilla:
Very importantly and encouragingly, the exponent on N (parameters) and D (tokens) is equal at ~=0.5, so just like Chinchilla we get a single (compute-independent) constant that relates the model size to token training horizons. In Chinchilla, this was measured to be 20. In nanochat it seems to be 8!

Once we can train compute optimal models, I swept out a miniseries from d10 to d20, which are nanochat sizes that can do 2**19 ~= 0.5M batch sizes on 8XH100 node without gradient accumulation. We get pretty, non-itersecting training plots for each model size.

Then the fun part is relating this miniseries v1 to the GPT-2 and GPT-3 miniseries so that we know we're on the right track. Validation loss has many issues and is not comparable, so instead I use the CORE score (from DCLM paper). I calculated it for GPT-2 and estimated it for GPT-3, which allows us to finally put nanochat nicely and on the same scale:
The total cost of this miniseries is only ~$100 (~4 hours on 8XH100). These experiments give us confidence that everything is working fairly nicely and that if we pay more (turn the dial), we get increasingly better models.

TLDR: we can train compute optimal miniseries and relate them to GPT-2/3 via objective CORE scores, but further improvements are desirable and needed. E.g., matching GPT-2 currently needs ~$500, but imo should be possible to do <$100 with more work.

Full post with a lot more detail is here:
https://t.co/na8zVLqWLf
And all of the tuning and code is pushed to master and people can reproduce these with scaling_laws .sh and miniseries .sh bash scripts.

[Jesus Lara @phenobarbital]

Pues básicamente les enviábamos 400 mil barriles DIARIOS de petróleo y a cambio nos envían soldados, técnicos, entrenadores deportistas y médicos internistas (Y de colofón, combustible de avión con todo y el avión pagado, el YV1128 de PDVSA es usado por Diaz-Canel para sus traslados privados).
De pagarlo a precio de mercado, la deuda de Cuba ascendería a 20 mil millones de dólares.

[swyx @swyx]

excited to kick off the year by dropping @trq212's full
Claude Agent SDK workshop from AlE CODE

POV video to give you an idea of how insanely packed
this one was

also peek at the incredible venue at @datadoghq - was very grateful to have their support esp since they were right above the conf venue + accepted our badges!

[Luis Carlos 🏴‍☠️ One Piece @LuisCarlos]

Participación en la Televisión Española, TVE, sobre lo ocurrido en Venezuela. Ya saben que España tiene el problema particular de que hay miembros del gobierno y aliados vinculados a las tramas de corrupción del chavismo, además de una polarización interna que enloda todo.

[Giuseppe Gangi @ggangix]

Para los cómplices de la dictadura que hablan de la autodeterminación de los pueblos.

[Deni . ۫ ꣑ৎ . 🇪🇦🏴󠁧󠁢󠁥󠁮󠁧󠁿🇯🇵 @SoyDenissePetit]

Loquísimo que con estudiantes con escudos de CARTÓN la cosa cambiaba ¿no?

[Jacky R₿ ✨🐇 @imjackyrivero]

Casi 10 años después.

2017 - 2026

[Aníbal Rojas @anibal]

Sigamos con este horrendo deporte de espectadores que nos tocó vivir a los venezolanos. Votamos, firmamos, protestamos, negociamos; pusimos los presos, los secuestrados, los torturados y los muertos. La comunidad internacional se dividió entre la comodidad de desentenderse del cáncer, o ser complices de una dictadura en proceso por décadas, porque la democracia no importa si hay alguna resemblanza ideológica. Y pues aquí llegamos, amanecerá (de nuevo) y veremos.

[Papagayo @srpapagayos]

#ElNobelEsNuestro #NobelDeLaPaz #heroina #PresosPoliticos #Venezuela #Libertad #DerechosHumanos

[Andrej Karpathy @karpathy]

nanoGPT - the first LLM to train and inference in space 🥹. It begins.