britt

I love music, my dog and making things. building @cnvrsai

[britt @p3ery]

@nrol_ling @percyliang @JensenHuang datamix phase 2:

[Larry Dial @classiclarryd]

At Marin we’ve found that scaling laws let us simulate the entire training trajectory. This became clear in our last 67B MoE run. The run hit the loss target within 1%, but perhaps more interestingly, arbitrary points during training could also be predicted to within 1%. This model is finishing up long context and will soon enter RL. Based on this finding, we’ve used a scaling ladder to simulate the training trajectory of the 535B MoE we kicked off yesterday. Will see how it goes. Now if someone can figure out how to fit a scaling law to each parameter we can just predict our final model and skip this training business.

Something else I've learned from this process is that getting a training run off the ground is way more than just fiddling with the architecture. Lot of herculean efforts from teammates to get trillions of tokens curated, get hardware running on a new GPU stack, improve MFU, and propose great ideas that were tested and integrated.

Given that this is our largest run to date, I anticipate lots of learning and adapting along the way. But thats part of the fun. Details: https://t.co/N7ZxcgvkKs

[britt @p3ery]

relatable glimmer

[britt @p3ery]

@xhluca interesting! i wonder why not zcode..?

btw they advertise it as “llm gateway” rather than “use non-anthropic models” for hopefully obvious reasons, but they do support any /messages-compatible api providers

https://t.co/msDCiiZjwG

[Leonardo de Moura @Leonard41111588]

Here is our postmortem describing new Lean bugs found by OpenAI internal models. They are all fixed in Lean v4.33.1 Many thanks to Daniel Selsam from OpenAI for all the help.

https://t.co/lK8cKT8KAN

[thebes @voooooogel]

i wish more people in ai just found the models intrinsically interesting without an agenda.

i remember back when i was embedded in the linguistics crowd, i was very confused why nobody i talked to seemed to find deep learning or later GPT-3 interesting. don't these people care about language understanding and linguistic production? so why are they ignoring and dismissing this?

eventually i gave up and found tpot and the rationalists and mechinterp and everything i do now, and it's - to be clear - much better. but i still sometimes look around and see so many people doing research because they care about safety and x-risk, or steering capabilities because they want the singularity to go well, or posting blog posts about ai pause because they're worried about doom, or grinding gpu kernels or yelling at claude in a loop because they want to escape the permanent underclass...

those are all well and good, i can respect (at least some of) those motivations, and many of them intersect with finding the models at least instrumentally interesting - but i really just find the models interesting, terminally. it's what got me into this, and everything else is kind of a side project. i just want to know how and why the models do what they do.

i'm not without my Theories and my biases - it especially frustrates me how, per @norvid_studies, "people get used to anything in two weeks" and start kvetching about things that a few years ago would have been Fantastic Science Fiction - but ultimately, when i write, it's coming from a place of wonder about how these things work.

[britt @p3ery]

@Sandeep74757399 @k_leon7 @hadesdevs @thdxr from april:

[britt @p3ery]

@ElliotGlazer @ereliuer_eteer it’s a dumb independent classifier — the model is being censored

[britt @p3ery]

@snwy_me https://t.co/LSMz64H6f9

[Anthropic @AnthropicAI]

We’re pleased to have collaborated with AE Studio on this research.

Read more here: https://t.co/9exnFltK7A

[britt @p3ery]

@snwy_me they probably are doing similar to anthropic:

[AE Studio @AEStudioLA]

New research! Some AI capabilities are both helpful and dangerous. E.g., knowledge of virology can be used to create life-saving vaccines or deadly pathogens. We introduce GRAM, a training method that puts dual-use capabilities (like virology) into removable modules.

[britt @p3ery]

@max_spero_ seems like luna or sol are preferable to terra at all cost-intelligence optimizing points

[Artificial Analysis @ArtificialAnlys]

GPT-5.6 Sol and Luna are ahead of Terra at every point on the Intelligence vs Cost per Task chart. GPT-5.6 Luna stands out as a particularly cost efficient model

Charting the Artificial Analysis Intelligence Index shows the trade-off between intelligence and Cost per Intelligence Index Task. Across reasoning efforts, each GPT-5.6 model pushes past GPT-5.5 on the Pareto frontier (excluding non-reasoning).

However, Luna and Sol are always ahead of Terra. This means for any Terra effort level, there is a Luna or Sol effort level that is more intelligent at no extra cost, or as intelligent at lower cost.

[britt @p3ery]

@Lari_island looks like a total outage across multiple (all?) services

https://t.co/OSROkJ3rJw

[britt @p3ery]

@poetengineer__ shades of arthur koestler’s bisociation

[britt @p3ery]

@memecrashes incredible. and 22 months after this exam maxwell would apply stokes to formalize faraday’s lines of force

[britt @p3ery]

@meekaale lmao yeahhhhhhhh

[britt @p3ery]

@AnthropicAI what a joke

[britt @p3ery]

@antirez @sandst1 would definitely be curious to hear more about specific deficiencies relative to this approach as you see them

[Georgi Gerganov @ggerganov]

Highlighting the new WebGPU backend in llama.cpp/ggml

The work to bring full-fledged WebGPU support in llama.cpp started about an year and a half ago. It has been lead by @reeselevine and team at USCS.

For more information, checkout the interactive blog and paper in the quoted post. Here are 2 excerpts from the paper, summarizing the implemented software architecture.

[antirez @antirez]

Iterating... Evals take time and are boring: but are a fundamental validation step of sane LLM inference. Let's try to make them as easy and fun to run as possible.

[britt @p3ery]

differential admits differentiation

[TANSTACK @tan_stack]

@TansTack https://t.co/m1CuiGy30b