@cocohearts i generally agree, prob won't ever be sota, but there's e.g. weird lookup things u should be able to try for local models, should be able to do better than "8B/27B dense"
very sad that we haven't made much progress on true edge/local models (ie models where inference throughput is ~optimal at batch size 1), seems like a very ripe direction for architecture creativity
a lot of the weird ds stuff recently seems like it might be useful idk
@jeffreygwang do you know if there is a relationship between architecture and CoT controllability? (i.e. do some architectures produce more CoT controllability at similar no-CoT capability than others?)
Most think vibecoding at scale is slop. Hereโs a counterexample.
Today we release a 16M-line formalization of the entire Stacks Project, critical infra underlying modern math, with optimized code quality and its highly interconnected structure preserved.
A mathematical city ๐
Ossoff: "Now our task is to reawaken that great national spirit that defeated fascism and established social security, that put men on the moon and passed civil rights laws in order to meet the challenges and obligations of our time. Because I refuse to accept that my two little girls will grow up in a world as dark and dangerous and cruel as this one threatens to become."
Sometimes I spend hours thinking about politics and stewing in frustration over the cynicism everyone seems to blindly hold onto. The fundamental story of civilization is one where goodness and cooperation has repeatedly triumphed, and the present is no different.
โThe core algorithm was found on October 16 by an internal Claude research model with 14M output tokens and no human input. Anthropic researchers worked with Claude to improve the presentation, resulting in this paper.โ
https://t.co/rZLajqbVH8
This sort of investigation is incomplete without adding in the novels of Min Jin Lee. I suspect her novels have far greater staying power than anything written about here, and it is worth pondering why.