Today we’re introducing Gemma 4 12B — our latest open model that brings advanced agentic reasoning, vision and audio directly to your laptop.
It delivers performance nearing our larger Gemma models with a much smaller total memory footprint, while being small enough to run locally with just 16GB of VRAM. It’s open and accessible for everyone to use under a permissive Apache 2.0 license.
This is all made possible by our new, unified architecture that removes separate multimodal encoders. Here’s how we did it 🧵
A guy in Karachi rebuilt GPT-4 in one Jupyter notebook.
OpenAI spent over $100 million to train the real one. He put the entire recipe on GitHub for free.
His README still says "I am looking for a PhD position in AI."
It's called Train LLM From Scratch. A working guide that walks you through building your own 2-billion-parameter language model on a single GPU.
OpenAI vs this repo:
- Training cost: $100M+ → Single A100 or RTX 4090 (you can rent for $1/hr)
- Code access: Closed → Open, MIT license
- Data: Secret → The Pile (open dataset, 825GB)
- Walkthrough: None → Every line of code explained, top to bottom
- Output quality: GPT-4 → A small model that writes broken English (but it's yours)
The whole thing fits in one notebook. No paid course. No paywall. No "Pro" tier.
What you actually learn:
→ How a transformer works, end to end
→ How to download and tokenize the Pile dataset
→ How to build multi-head attention from scratch in PyTorch
→ How to train on a single GPU without running out of memory
→ How to generate text from your trained model
→ How to scale from 13 million parameters to 2 billion
774 stars. 135 forks. MIT license. The full theory paper-to-code in one place.
One honest note: this is a learning repo, not a production model. Your output will be small and rough. But you will understand exactly how GPT-4 works after reading it.
Fareed Khan built this from Karachi, Pakistan. He has 1,780 GitHub followers. He's still looking for a PhD position. The recipe to billion-dollar AI is sitting on his profile, free.
This is what open AI was supposed to mean.
(Link in the comments)
MIT open-sourced an AI model that converts photos into fully editable CAD programs and it quietly kills the $150/hour CAD modeling industry.
Just upload a sketch or photo and it generates the full parametric 3D model. exportable as STL. ready for manufacturing.
→ no SolidWorks license
→ no weeks of modeling
→ no CAD engineer needed
100% Open Source
New article: a visual tour of recent LLM architecture advances, from Gemma 4 to DeepSeek V4.
I focus on long-context efficiency tweaks like KV sharing, per-layer embeddings, layer-wise attention budgets, compressed attention, and mHC.
Link: https://t.co/KO81y3kTH7
Neural networks might speak English, but they think in shapes.
Understanding their rich *neural geometry* is key to understanding how they work – and to debugging and controlling them with precision.
Starting today, we’re releasing a series of posts on this research agenda. 🧵
Maximal Brain Damage
Flipping just two sign bits reduces ResNet-50 accuracy by 99.8% and drops Qwen3-30B reasoning to 0%.
Researchers from IBM, Technion and NVIDIA introduce Deep Neural Lesion (DNL), a data-free method to locate critical parameters in neural networks.
Bir yapay zeka bilimsel makale yazdı, hakem incelemesini geçti, Nature'da yayımlandı.
Sakana AI'ın "AI Scientist-v2" sistemi baştan sona otonom çalışıyor. Hipotez kuruyor, deney tasarlıyor, verileri analiz ediyor, makaleyi LaTeX'te yazıyor. Üç makale göndermişler ICLR konferansına, biri geçmiş. Makale başına maliyet: 25 dolar. Bir araştırmacının yıllık maliyeti 200 bin dolar.
Fakat ilginç bir detay var. ICLR 2026'ya gelen 75 binden fazla hakem raporunun %21'i tamamen yapay zeka tarafından yazılmış. Yarısından fazlasında yapay zeka katkısı tespit edildi. Yani AI makaleyi yazıyor, AI hakemlik yapıyor, insan sadece onaylıyor.
25 Mart'ta Nature'da yayımlandı, 198 bin erişim aldı.
kaynak 1: https://t.co/tcVvchskFo
kaynak 2: https://t.co/uXwIlCyzo8
Mythos is a looped transformer!? 😳 Should be a Mixture-of-Recursions (MoR) — 2× faster, controlled effort.
Dense → sparse MoE was the efficiency unlock of 2023.
Uniform loops → MoR is the same move for recursive transformers.
Paper reading list below. 🧵
Introducing 𝑨𝒕𝒕𝒆𝒏𝒕𝒊𝒐𝒏 𝑹𝒆𝒔𝒊𝒅𝒖𝒂𝒍𝒔: Rethinking depth-wise aggregation.
Residual connections have long relied on fixed, uniform accumulation. Inspired by the duality of time and depth, we introduce Attention Residuals, replacing standard depth-wise recurrence with learned, input-dependent attention over preceding layers.
🔹 Enables networks to selectively retrieve past representations, naturally mitigating dilution and hidden-state growth.
🔹 Introduces Block AttnRes, partitioning layers into compressed blocks to make cross-layer attention practical at scale.
🔹 Serves as an efficient drop-in replacement, demonstrating a 1.25x compute advantage with negligible (<2%) inference latency overhead.
🔹 Validated on the Kimi Linear architecture (48B total, 3B activated parameters), delivering consistent downstream performance gains.
🔗Full report:
https://t.co/u3EHICG05h