Hao Hoang

I share daily insights on AI agents, LLMs, Data Science, Machine Learning | I help AI engineers crack top-tier interviews | 56K+ community |

[Hao Hoang @HaoHoangAI]

LLM System Design Interview #4 - The Gradient Highway

Why post-norm Transformers break at scale - and how one architectural swap enables stable training for 100B-parameter models.

Full breakdown on my Substack here: https://t.co/wklF8v9tBk

[Hao Hoang @HaoHoangAI]

LLM System Design Interview #4 - The Gradient Highway

Why post-norm Transformers break at scale - and how one architectural swap enables stable training for 100B-parameter models.

Full breakdown on my Substack here: https://t.co/wklF8v9tBk

[Hao Hoang @HaoHoangAI]

Human middleware 💀

[Hao Hoang @HaoHoangAI]

LLM System Design Interview #3 - The Scaling Law Playbook

Why elite AI engineers never guess model size - and how to use small-scale experiments to de-risk six-figure training runs.

Full breakdown on my Substack here:

https://t.co/Yku96zKchQ

[Hao Hoang @HaoHoangAI]

LLM System Design Interview #3 - The Scaling Law Playbook

Why elite AI engineers never guess model size - and how to use small-scale experiments to de-risk six-figure training runs.

Full breakdown on my Substack here:

https://t.co/Yku96zKchQ

[Hao Hoang @HaoHoangAI]

Github: https://t.co/yCZHo2QdbQ

[Hao Hoang @HaoHoangAI]

City2Graph turns buildings, streets, and GTFS feeds into spatial graphs.

It sits between GeoPandas, NetworkX, and PyTorch Geometric, so the same GeoDataFrame round-trips into HeteroData and back.

1.8k ⭐ stars BSD-3-Clause.
#GeospatialAI #GraphNeuralNetworks #PyTorchGeometric

[Hao Hoang @HaoHoangAI]

City2Graph turns buildings, streets, and GTFS feeds into spatial graphs.

It sits between GeoPandas, NetworkX, and PyTorch Geometric, so the same GeoDataFrame round-trips into HeteroData and back.

1.8k ⭐ stars BSD-3-Clause.
#GeospatialAI #GraphNeuralNetworks #PyTorchGeometric

[Hao Hoang @HaoHoangAI]

"just follow the pattern" prompt be like 💀

[Hao Hoang @HaoHoangAI]

OpenViking stores agent memories, resources, and skills as one filesystem.

Agents browse context with ls, tree, and find instead of querying a vector store. Every entry is written in three tiers, abstract, overview, details, and loaded only if the task needs.

32.1k stars ⭐

[Hao Hoang @HaoHoangAI]

Github: https://t.co/EnDEX030Ul

[Hao Hoang @HaoHoangAI]

OpenViking stores agent memories, resources, and skills as one filesystem.

Agents browse context with ls, tree, and find instead of querying a vector store. Every entry is written in three tiers, abstract, overview, details, and loaded only if the task needs.

32.1k stars ⭐

[Hao Hoang @HaoHoangAI]

LLM System Design Interview #2 - The Lossless Speedup Trick

How top AI teams double inference speed without quantization, pruning, or touching model weights - by exploiting a core asymmetry in Transformers.

Full breakdown on my Substack here:

https://t.co/VElvOShekL

[Hao Hoang @HaoHoangAI]

Stanford CS336 builds a language model from scratch across 18 recorded lectures. The course moves through data collection and cleaning, transformer construction, training, and evaluation before deployment. Nothing is imported that has not been built first.

Must watch

[Hao Hoang @HaoHoangAI]

Playlist: https://t.co/EXqi2NwUqc

Course details: https://t.co/F9Y8z9eVSf

Course website: https://t.co/fvbjsdeXlN

[Hao Hoang @HaoHoangAI]

Stanford CS336 builds a language model from scratch across 18 recorded lectures. The course moves through data collection and cleaning, transformer construction, training, and evaluation before deployment. Nothing is imported that has not been built first.

Must watch

[Hao Hoang @HaoHoangAI]

LLM System Design Interview #1 - The Tokenizer Trap

Why reusing a general tokenizer can quietly 16x your compute bill - and how to avoid it.

Full breakdown on my Substack here:

https://t.co/Zf3Bu0x9qQ

[Hao Hoang @HaoHoangAI]

RAG Interview Questions #8 - The IVF-PQ Compression Trick

Why blindly scaling HNSW is a scaling nightmare, and how to gracefully trade imperceptible coverage loss for massive infrastructure savings.

Full breakdown on my Substack:
https://t.co/YUjy0Etn68

[Hao Hoang @HaoHoangAI]

https://t.co/JRLSBAWygr

[Hao Hoang @HaoHoangAI]

I just gave my AI agent persistent memory using 30x fewer tokens than full context.

F1 score went UP, not down.

SimpleMem from aiming-lab, 3.3k 🌟 GitHub stars, MIT license.

Three design choices broke my model of how memory should work:

1️⃣ Compresses at write-time, not read-time

Raw dialogue gets rewritten into atomic facts with absolute timestamps BEFORE it hits storage. "He'll meet Bob tomorrow at 2pm" becomes "Alice will meet Bob at Starbucks on 2025-11-16T14:00:00." Disambiguation is a feature, not an afterthought.

2️⃣ Three indexes per memory, picked at query-time

Semantic (vectors), lexical (BM25), symbolic (entities + timestamps). The LLM generates a retrieval plan per query. No config tuning. No hybrid weighting magic numbers.

3️⃣ EvolveMem v3.0: the retriever rewrites its own architecture

An evaluate → diagnose → propose → guard loop mutates retriever config autonomously. After 7 rounds it discovered retrieval dimensions the original authors never designed, query decomposition, entity-swap, answer verification.

The numbers on LoCoMo:
1️⃣ 43.24% F1 at ~550 tokens (vs 30x full context)
2️⃣ +25.7% over best baseline on GPT-4o
3️⃣ +36.8% over best baseline on GPT-5.1
4️⃣ +64% over Claude-Mem on cross-session memory

Most memory frameworks compete on storage. SimpleMem competes on retrieval planning, then teaches the planner to evolve itself.

If you're building agents that need to remember anything beyond a single session, this is the new floor.