CronicasVibeCodding

Vende el problema que resuelves, no el producto.

[CronicasVibeCodding @vibeCodder]

@javsolgar @Gonzalo_stba Conjunto de limitaciones programáticas sobre el contexto, capacidades o tools y datos.

[CronicasVibeCodding @vibeCodder]

@matiasblnc Eso es por seguir redes sociales. Tenemos que construir agentes que nos quiten esta mierda de que todo el mundo mola menos tu

[Jaime @DragonLadyU2]

El derecho de un niño ceutí a jugar a la pelota en un parque sin tener miedo a que le hagan una atrocidad o le contagien una enfermedad medieval está por encima de cualquier pseudoderecho de un no ciudadano cuya primera acción en mi país ha sido violar la frontera.

[CronicasVibeCodding @vibeCodder]

@juanmacias @harvey @WeAreLegora Todos estáis mirando donde no debéis... eso es papel de interpretación y no es oki matemático es emocional técnico político...

[CronicasVibeCodding @vibeCodder]

@felipepoloruiz La verdad que yo le di a todos... incluso de uno de ellos me pegaba con coreanos ....

[Kim Philby @LOQUEDIGAELFMI]

- Tertsch: "El gobierno marroquí es patriota"
- Abascal: "Marruecos no es el enemigo"
- Tertsch: "Reconcer el Sáhara marroquí a cambio de Ceuta y Melilla"
- Abascal: "Tropas argelinas"
- Ariza: "Marruecos no es el enemigo"

No sé, parece que hay un patrón (y que paga en shekels)

[CronicasVibeCodding @vibeCodder]

@barckcode

[Akshay 🚀 @akshay_pachaar]

UC Berkeley just open-sourced FreeToken.

(2–4x faster local LLM inference than Ollama)

the results are wild:

- Qwen3.6-35B on an 8GB GPU at 39.3 tokens/s
- DeepSeek-V4-Flash 284B on a 32GB GPU at 22 tokens/s
- GLM-5.2 753B on a 96GB GPU at 14.9 tokens/s

a 35B model at 16-bit precision needs about 70GB just for its weights. even at 4 bits it is close to 18GB, and FreeToken serves it on an 8GB GPU.

let me explain how:

all three models mentioned above are Mixture-of-Experts, and that is what FreeToken takes advantage of. each layer holds hundreds of separate experts plus a small router that picks a few of them per token.

Qwen3.6-35B activates roughly 3B of its 35B parameters per token. DeepSeek-V4-Flash picks 6 of 256 experts per layer, so 13B of its 284B run at a time.

so compute was never the bottleneck. the weights a single step touches fit comfortably on a consumer GPU.

every expert the router might pick still has to exist somewhere. they sit in system RAM, and the GPU keeps a cache of the ones the model has been using recently.

so everything comes down to what happens when the router picks an expert that is not on the GPU.

there are two ways to serve that miss:

1. copy it over PCIe and run it on the GPU
2. run it on the CPU, where it already lives

both read from the same system memory, so they compete for one pool of bandwidth instead of adding to each other. existing engines pick one option and freeze it when the model loads.

but routing changes on every token, so a fixed choice misses most of what the model asks for.

FreeToken measures both bandwidths on your machine and splits each step's misses between the two paths in proportion. the GPU and CPU results then merge exactly, with no approximation.

two machines with the same GPU can end up wanting opposite strategies, which I did not expect. a 5090 in a gaming desktop should push nearly everything over PCIe, while an 8GB laptop is better off computing most misses on the CPU.

none of that is readable off a spec sheet, so the engine profiles it once per machine.

the second half of the design is about agents. coding agents constantly rewrite their own history, and every edit normally forces thousands of tokens back through prefill.

FreeToken saves its checkpoints at the exact boundaries agent frameworks cut on, so it only reprocesses the new part. its slowest first token stays under 44 seconds, while llama.cpp peaks at 232 and KTransformers at 946.

it serves the OpenAI and Anthropic APIs under Apache 2.0, so Claude Code and Codex can point at it directly.

releasing weights publicly decides who can download a model, not who can afford to run one. frontier open models keep shipping, and running them still assumes a rented cluster.

meanwhile there are over a hundred million consumer machines with discrete GPUs sitting mostly idle. closing that gap was never a hardware problem, and work like this is what turns open weights into something you can actually use.

paper: https://t.co/bIOXIiBsXT

repo: https://t.co/uxWAP5PYx7

almost every idea in this post, from why memory bandwidth decides the outcome to why moving weights costs more than computing on them, comes straight out of how a GPU is built. I wrote a detailed primer on that.

the article is quoted below.

[CronicasVibeCodding @vibeCodder]

@GuliMoreno @tryholo Deja el
Gluten chico....

[CronicasVibeCodding @vibeCodder]

@barckcode Ya sabes que cuando quieras .... reducción del 75%

[Lorenzo Ramírez @lorenzoramirez]

Todos tienen alma de dictadores en cuanto se rasca un poco la superficie. Son el enemigo, hasta que no dejemos las tramposas ideologías a un lado seguiremos en las trincheras que ellos cavan para que nosotros luchemos mientras ellos avanzan en su agenda totalitaria y cleptócrata.

[CronicasVibeCodding @vibeCodder]

@felipepoloruiz Tener un router siempre tiene sentido.

Siempre.

El problema es poner el harness por cada modelo y caso.

[CronicasVibeCodding @vibeCodder]

@alvarombt Curucucucu!

[CronicasVibeCodding @vibeCodder]

@ContraInvest @JesusFerna7026 Muy buena Hugo,

Y ya que tú y yo tenemos hijos... tenemos que buscarnos las castañas para darles una posición decente y que puedan tener hijos z

[Rand @rand_longevity]

we are gonna simulate biology

[Alexander Stahel 🌻 @BurggrabenH]

The question for oil investors used to be: at what price will OPEC cut? The real question is: at what price will China buy?

Beijing can disappear from the crude market by millions of barrels a day, then return when the price suits it. Its crude import demand is now highly elastic.

China is the new OPEC.

https://t.co/iO1DYakqKD

[Paulo Macro @PauloMacro]

Brazil played this game once. I remember in the 90s at one point the sovereign was so tilted to bills that the govt was rolling over 98% of its entire debt overnight. Those were interesting times. Whenever I see this mix game I start to wonder…

And for people glad to see this move as part of some grand plan to get the interest bill, real rate, and term premium down to run it hot for the middle class — I just don’t think they have spent enough time in emerging markets.

[Enrique Martínez @QuiqueVal]

Federico García Lorca era republicano y anarquista. Y homosexual. Sí. Y también católico, taurino, muy tradicional y español y amigo de falangistas que lo escondieron y protegieron en su casa al estallar la guerra. Por poner en contexto y no lo que te vende la izquierda

[Arslan @_ai_jedi]

I've tasted the 9-to-5. I've also tasted being a founder.

I highly recommend being born into generational wealth.

[CronicasVibeCodding @vibeCodder]

@felipepoloruiz En ello estoy ;-)