Frontier LLM

General reasoning, coding, and domain-tuned models — plus the agentic rails to run them in production. Your infrastructure, your data, your terms.

[Frontier LLM @frontierllm]

Model training was sufficient, however results are lackluster. Back to retraining and modifying the architecture a little bit.

[Frontier LLM @frontierllm]

We're evaluating results from our first prod training run!

[Frontier LLM @frontierllm]

75% done on production training!

[Frontier LLM @frontierllm]

Training is now taking place on a GPU cluster! Flying through the epoch's

[Frontier LLM @frontierllm]

Want to try EchoNest in alpha for free?

API access forms just dropped: https://t.co/S4WUzbT5OU

Apply today!

[Frontier LLM @frontierllm]

What if a 100M-param model could follow instructions efficiently on a plain CPU, no GPU, no massive pre-training stage? That's the idea behind EchoNest, our new architecture paper. Recurrence for local structure, attention for the global view.

https://t.co/qFywQIBJfy

[mmmatt @mmmatt]

training may be restarted on faster hardware tomo/the next day

will keep you bois posted

everything is looking very, very good though

[mmmatt @mmmatt]

the whole open-source LLM world quietly agreed on one architecture and nobody really talks about it.

crack open llama, mistral, gemma, qwen, phi- pick any of them, and you're basically staring at the same model. rmsnorm, rope, swiglu, grouped-query attention. pre-train on a few trillion tokens, instruction tune on top, ship it to a gpu. it works. it's genuinely a great recipe. but it's one recipe, and at this point everybody's just cooking it slightly differently.

and that recipe isn't neutral. it quietly assumes a bunch of stuff: billions of parameters, fat accelerator memory, and a two-stage life where you spend a fortune pre-training on raw text before you ever teach the thing to follow an instruction. all of that is completely reasonable if you're a frontier lab. it's just not the only point in the design space, and it's the only point anyone's building for.

so we built something for a different point. it's called echonest. it's a 102.8M param model, and the target is dumb: run well on a commodity cpu at the edge. no accelerator. tight memory. the kind of place you can't just drop a 7B model.

the usual move for "small model" is to take a big-model architecture and shrink it. we didn't want to do that. a tiny llama is still a llama wearing the assumptions of a thing 100x its size. if the deployment context is genuinely different, the architecture should be too.

the core idea is to stop pretending one flat stack of identical blocks will magically sort out local vs global processing on its own. big transformers get that separation to emerge by throwing depth and scale at it. we don't have depth and scale. so we just built the separation in, explicitly, with three nested tiers that each do one job:

echounit, the local tier: lstm style recurrence handling short range, order sensitive stuff. the place where sequence actually matters.

micronest, the middle: aggregates those local bits up into phrase and clause level features.

macronest, the top: attention. but only here. attention is quadratic and expensive, so you spend it once, at the level where global mixing actually earns its keep, instead of paying for it at every single layer.

bringing recurrence back is the part that'll make people twitch, because the whole point of "attention is all you need" was killing recurrence. OK. but recurrence lost for specific reasons: it blocks parallel training and it's bad at long range. now look at our setup: 512 token context, running on a cpu. the parallel-training penalty barely shows up, and in exchange you get a compact hidden state and positional order basically for free. the tradeoff that made recurrence a bad idea in 2017 quietly flips when you change the hardware and the context length.

the normie talk: no pre training at all. none. echonest trains straight on ~100k instruction–response pairs (alpaca, python code instructions, codesearchnet) from random init. loss only on the response tokens. adamw, lr 1e-4, ema 0.999, the whole run on cpu.

obviously you pay for that. you give up all the broad world knowledge that trillion-token pre-training buys you. what you get back is a training process that's small, cheap, fast, and actually controllable, tightly scoped to behavior instead of trying to absorb the entire internet. if what you want is a predictable, domain-focused assistant you can train and re-train without a pre-training budget, that's a real trade, not a downgrade.

now the part i have to be honest about, because everyone skips it: there are no results yet. zero. this is v0.1-alpha, training is literally still running, and i have no perplexity numbers, no benchmarks, no latency measurements to show you. right now the case for echonest is architectural and philosophical. it's a hypothesis with a model card, not a win.

and i know exactly what would actually settle it, so i'll say it out loud: echonest vs a parameter-matched vanilla transformer- same ~100M params, same 512 context, same instruction data- measured on cpu latency, memory, and task quality. that head-to-head is the whole ballgame. until it exists, treat all of this as a claim, not a result.

the bigger thing i actually care about, underneath the specific model: for a growing pile of edge and cpu deployments, the right model might not be a smaller frontier model. it might be a structurally different one. the monoculture is efficient but it's still a monoculture, and "make it smaller" is not the same as "design it for where it runs."

echonest is a first swing at designing for that context from scratch instead of shrinking down from the top.

roadmap, quickly: finish v0.1 and ship an eval suite, then v0.2 with inference-optimized weights, then v1.0 with a safety pass and an api. full technical report with reproducible specs and that parameter-matched bake-off comes with the real numbers.

[Frontier LLM @frontierllm]

We're running our first training job in the open.
Live train/val loss, perplexity, and grad norm for EchoNest-1 — all on the public status page. Building in public from epoch zero.
https://t.co/nKYOr5ZUB3

[Frontier LLM @frontierllm]

@mmmatt Followers will be the first to know about early access! <3

[Frontier LLM @frontierllm]

Introducing Frontier LLM — foundation models engineered for the enterprise.
General reasoning, coding, and domain-tuned models, with the agentic infrastructure to deploy them in production. Your infrastructure. Your data. Your terms.
https://t.co/2L7tPrn0aN