Lila

npm install life 🌷

[Lila @LilaRest]

Introducing 𝐆𝐞𝐦𝐦𝐚 𝟒 𝟑𝟏𝐁 𝐓𝐮𝐫𝐛𝐨 ⚡️

It runs on a 𝘴𝘪𝘯𝘨𝘭𝘦 RTX 5090, at 51 tok/s (single) and 1244 tok/s (batched). And prefills up to 15359 tok/s.

It's 𝟔𝟖% 𝐬𝐦𝐚𝐥𝐥𝐞𝐫 in GPU memory and ~𝟐.𝟓𝐱 𝐟𝐚𝐬𝐭𝐞𝐫 than the base model, and retains nearly 𝐢𝐝𝐞𝐧𝐭𝐢𝐜𝐚𝐥 𝐪𝐮𝐚𝐥𝐢𝐭𝐲 on benchmarks (1-3% loss).

Turbo is a derivative of the NVFP4 quant that NVIDIA released a few days ago. It fully leverages NVIDIA Blackwell FP4 tensor cores for ~𝟐× 𝐡𝐢𝐠𝐡𝐞𝐫 𝐜𝐨𝐧𝐜𝐮𝐫𝐫𝐞𝐧𝐭 𝐭𝐡𝐫𝐨𝐮𝐠𝐡𝐩𝐮𝐭 𝐭𝐡𝐚𝐧 𝐨𝐭𝐡𝐞𝐫 𝐪𝐮𝐚𝐧𝐭𝐬.

I'm using it for hard classification tasks — on internal benchmarks it showed 𝐒𝐨𝐧𝐧𝐞𝐭-𝟒.𝟓-𝐥𝐞𝐯𝐞𝐥 𝐢𝐧𝐭𝐞𝐥𝐥𝐢𝐠𝐞𝐧𝐜𝐞 (scored well above Haiku 4.5), at a 600𝘵𝘩 of the cost. A single RTX 5090 scales up to 18 req/s at 1000in/20out 🥵.

Model card and benchmark in comments 👇

I'd love to hear your use cases.

[Lila @LilaRest]

I see what you mean about solving skill issues at the framework-level, and in fact Langchain / Mastra are partially trying that by offering high-level APIs to even the simplest things, like managing the context window (which in most cases is a few hundreds lines of code). I'm talking about the opposite, instead of the framework support high-level features, it'd just offer low-level primitives you could compose to achieve any high-level feature. Like React for web development. React doesn't tell you how you should architect your app, nor how you should implement this or that features (and same for frameworks on top of React), instead it offers a set of primitives, that can be explained in 10mins, and allow you to compose from the simplest to the most complex applications.

[Lila @LilaRest]

Be honest, what's the worst part of building AI agents with frameworks like Langchain or Mastra?

[Lila @LilaRest]

Skill issue is partially the responsibility of the framework too. If skilled devs assisted with sota coding models are struggling to produce even the simplest agents, maybe the framework creates more complexity than it solves? Ultimately an agent is just a loop + tools + conversation history. It’s easy to call out skill issues, but in my opinion, if a framework was properly designed, even a junior dev with a coding assistant could scale a complex agentic app. Mastra and Langchain solve the « Can your framework do X? » question by pilling up features for years, instead of shrinking down to a set of primitives that are flexible and minimal enough to do anything.

[Lila @LilaRest]

@JE4NVRG @Rebecca49484009 Agreed

[Lila @LilaRest]

@JustJerry121 So basically being able to observe precisely why it failed right?

[Lila @LilaRest]

@adelbucetta but so that problems is not about the framework your using right? or the framework could help you in some way with that?

[Lila @LilaRest]

@Rebecca49484009 I feel you. It seems like the existing frameworks are failing pretty fast when you cross the boundaries of what their authors had in mind. And so we build layers of evals and observability tools to monior even the simplest agents instead of fixing the root cause

[Lila @LilaRest]

@shawntenam ahaha, that’s a great thank you

[Lila @LilaRest]

@bnjmn_marie Just perfect for nuanced zero-shot classifications

[Lila @LilaRest]

@TheAaryanKapoor lmao, you��re hired

[Lila @LilaRest]

Introducing Ghod-1, a model outperforming Mythos.

Ghod-1 excels at everything.

We released it to a handful of partners for safety and financial reasons.

Priced at $2K/M tokens in and $6K/M tokens out.

[Lila @LilaRest]

@momo_mattomo leave a star on the bible ^^

[Lila @LilaRest]

@momo_mattomo depends on whether you belive in Ghod

[Lila @LilaRest]

@Noor_Farayeh1 @X @grok that's a satire mate, and there is no link to click

[Lila @LilaRest]

@OmniScopeBio Ghod does

[Lila @LilaRest]

@nickshiva42 Confirmed

[Lila @LilaRest]

@ioannesesledieu It’s smart enough to refuse distillation attempts

[Lila @LilaRest]

@Mohamed_KhedrX Ghod-1 (high) scores 392, what do you say to that?
(love your model name though)

[Lila @LilaRest]

@Vertoxo_ai 🐍

[Lila @LilaRest]

@OmniScopeBio It is inaccessible, no worries

[Lila @LilaRest]

@aiseomastery Ghod is independent