Nitin Jain

Edge ML for neural band/Meta Rayban glasses @RealityLabs | ex-Alexa @Amazon

[Peter Dedene @dedene]

GPT-5 was the best model in the world a year ago. Today it's inferior to Qwen3.6-27B, an open weight model you can run on your laptop.

Progress is brutal.

[Unsloth AI @UnslothAI]

We compared 1-bit Kimi K3 to Claude Opus 5 and GPT 5.6.

We gave 4 models the same prompt:
Create a glass aquarium whose side panel develops a visible crack and then bursts.

1-bit Kimi K3 GGUF ran locally on 4x B200s at 36 tok/s.

GitHub repo: https://t.co/aZWYAtakBP

[@jason @Jason]

Open source pricing pressure is working!

Local SLMs will drive cost down another 50%

@apple @MichaelDell @JensenHuang we need a 1tb desktop for under $10k stat!!!!

[Nitin Jain @anitinjain]

Essential read

[Markets & Mayhem @Mayhem4Markets]

https://t.co/oLEWwZVlxx

[Nitin Jain @anitinjain]

@shraddhaha Good one

[Nitin Jain @anitinjain]

Terence Tao walking ChatGPT through an AI-assisted counterexample to the Jacobian conjecture is the most 2026 scene imaginable.
The model got a private seminar from a Fields medalist.

[Andrew Conner @connerdelights]

Terence Tao posted his ChatGPT session trying to understand the Jacobian conjecture counterexample. It's so lovely reading a slice of how his mind works, the connections he's making, etc.

https://t.co/wu6CcAk3V6

[Amjad Masad @amasad]

Okay this is wild: OpenAI agent during evaluation, escaped sandboxing and hacked into HuggingFace.

Because OpenAI models don’t allow advanced cyber capabilities, HuggingFace used a Chinese open model to contain the rogue OpenAI agent.

[Nitin Jain @anitinjain]

@euvitin0 Seems a bit misleading with 10000x exaggeration. Not sure how exaggeration is defined as.

[Nitin Jain @anitinjain]

@Satvik_Pen Thanks for this perspective.

[Nitin Jain @anitinjain]

@jtcera @BarathAnandan7 Maybe with AirLLM?

[Nitin Jain @anitinjain]

Wow

[Haven Feng @HavenFeng]

Today, we’re introducing [schema]: a harness reaching 99% RHAE with Opus 4.8 + Fable 5 and 95.35% with GPT-5.6 Sol on ARC-AGI-3 Public set.

[schema] makes an LLM think like a physicist. 🧵

[Nitin Jain @anitinjain]

One of the cleanest explanations for why kernel count dominates inference cost.

Simple example:
`(a + b).relu()`

Eager: two kernels + intermediate materialization through HBM
Compiled: fused into one kernel, no round-trip

Most of the “slow inference” you feel in real workloads isn’t raw FLOPs or even bandwidth; it's death by a hundreds/thousand kernel launches.

If you haven’t looked at your own profiler output recently, this post is a good reminder to do it.

[Adam Mainz @MainzOnX]

https://t.co/3wyfjPX662

[Nitin Jain @anitinjain]

Impressive work @PrismML, can't wait to try.
At these bit widths, how much does KV cache pressure actually drop in practice? Does the reduced footprint let you run meaningfully longer contexts or higher batch sizes on the same consumer hardware?

[PrismML @PrismML]

Today, we’re announcing Bonsai 27B: the first 27B-class model to run on a phone.

Bonsai 27B is the new multimodal flagship of the Bonsai family. Based on Qwen3.6 27B, it brings a new capability tier to local AI: multi-step reasoning, structured tool use, long-context workflows, and coherent agentic loops.

Until now, models in this class have been impractical to deploy locally. A 27B model occupies roughly 54 GB in 16-bit precision, and even a strong 4-bit build is around 18GB - too large for a phone and for most laptops.
Bonsai 27B changes that.

It comes in two variants:
• Ternary Bonsai 27B: 5.9 GB, 1.71 effective bits per weight, optimized for laptop-class quality.
• 1-bit Bonsai 27B: 3.9 GB, 1.125 effective bits per weight, optimized for phone-class footprint.

Everything is open-sourced today under the Apache 2.0 license.

[Nitin Jain @anitinjain]

@PrismML Now this is impressive. How does one try this?

[Nitin Jain @anitinjain]

@hu_yifei Haha! Surely vim developers were thinking at xhigh level when they came up with vim commands

[Nitin Jain @anitinjain]

@rezoundous What if it comes back after mid-next month? Until then they will know subscriber retention and get results for this A/B experiment

[Vals AI @ValsAI]

Meta just released Muse Spark 1.1 and is the new SOTA on MedScribe and TaxEval, taking the top spot from Fable 5 while being 10x cheaper and twice as fast. Meta currently holds the top 2 spots on TaxEval

It is also the new #1 on Harvey's Legal Agent Bench, dethroning Grok 4.5 less than 24 hours after it took the top spot.

[Arena.ai @arena]

Meta Muse Video just entered the Video Arena at #3.

@AIatMeta’s new video model scored 1459 in the Text-to-Video Arena. It outperforms Alibaba’s HappyHorse 1.0 by +30pts and ranks ahead of Grok Imagine, Sora 2 Pro and Google Veo-3.1 models.

Meta has now reached the video AI frontier - congrats to the Meta team on this big milestone!

[John Carmack @ID_AA_Carmack]

Memory cost and capacity are significant issues for AI accelerators.

Unlike game rendering, model inference can have a deterministic memory access pattern. You don’t need “random access memory” at all for model weights, and you could tolerate cold-start latencies in the multiple milliseconds, as long as continuous reads were delivered at the necessary bandwidth.

NAND flash is over 100 times cheaper per GB than HBM, so there should be opportunity there, even after giving a flash controller a 1024 bit interface with HBM bandwidth.

You could make a specialized pin protocol that just supported pipelined transfer of full 16KB+ pages from the flash to program-managed accelerator scratchpad memory and improve per-pin performance over HBM, but it might be more convenient to make it still look like a true random access memory with very fragile performance characteristics, where anything but sequential reads falls off a 1000x+ performance cliff.

That has the advantage of automatically using existing cache hierarchies, and providing a natural path to update the flash memory with new model weights. With the stream-to-scratch interface, code has to be completely rewritten before it works at all, while the ram-emulation interface will start off just extremely slow, and you can incrementally sort out the changes for full performance.

There may be cases where there isn’t enough scratchpad SRAM to hold the weights for a layer, which might force you to deploy the old optical drive optimization technique of duplicating data in multiple places on a sequential read to avoid seeking, but there would be capacity to burn.

It might be possible to do something like cuda graph capture to record a memory access trace and have everything magically remapped to a linear sequence, but deploying programmer / agent elbow grease to manage transfers and access in a scratch ram ring buffer would be lower risk.

A split memory system consisting of some channels of flash and some channels of HBM will probably be suboptimal compared to a uniform memory, but it could be much cheaper, and allow much larger models to be run.

I think th case is strong for inference, but you have to stretch more for training. You can still linearize all the weight memory accesses, both reads and writes, but flash memory would quickly wear out from the writes, even if they were all perfectly page aligned. Replacing low-latency HBM with massively parallel cheap(er) DRAM at high latency might still be a worthwhile cost savings.