AI Engineering Interviews: Over 350 AI Interview Questions and Answers
📕 Get Now: https://t.co/sIrAL4pRYm
Generative AI is rapidly spreading across industries, and companies are actively hiring people who can design, build, and deploy these systems. But to land one of these roles, you'll have to get through the interview first. Generative AI Interviews walks you through every stage of the interview process, giving you an insider's perspective that will help you build confidence and stand out.
I created a Github repository to learn System Design, and it crossed 40k stars recently.
The repository contains a collection of FREE resources to study:
- System Design Core Concepts
- Networking and API Fundamentals
- Database and Caching Fundamentals
- Distributed Systems, Microservies and Architectural Patterns
- System Design Tradeoffs
- 40+ interview problems categorized by difficulty level
Check it out here: https://t.co/pkVpi6LxSV
If you find the repo valuable, consider giving it a ⭐️ and share with others.
Andrej Karpathy put his whole AI course on GitHub for free.
7 repos, in the order to learn them
→ nn-zero-to-hero, his video course that builds neural networks from scratch
https://t.co/bUbJjjJtup
→ micrograd, the math that makes AI learn, in a tiny amount of code
https://t.co/fa2H6dHRx2
→ makemore, a small model that learns to invent new names
https://t.co/ILWge5cDuv
→ minbpe, how AI chops your words into tokens
https://t.co/OnazNRnxZI
→ nanoGPT, train your own GPT model
https://t.co/a9aIZtuqw7
→ nanochat, "the best ChatGPT that $100 can buy"
https://t.co/Ws1gcb9J4l
→ autoresearch, AI agents that run AI experiments on one GPU by themselves
https://t.co/wEoFLqoL9d
284,000 GitHub stars between them.
Everything you need to understand how ChatGPT works is free and open source.
🚨Claude Code es un dolor de cabeza hasta que instalas esto
Hay un plugin oficial de Anthropic llamado claude-code-setup
Te dice que automatizaciones puedes montar (hooks, skills, MCP servers, subagentes) y como configurarlas paso a paso
Analiza tu proyecto y te recomienda que activar
Para instalarlo: /plugin install claude-code-setup@claude-plugins-official
Guardalo y luego me agradeces
This is f*cking insane. Andrej Karpathy, the man Elon called the #2 computer vision mind on Earth:
"Agents aren't magic. They're distillation at scale
99% of what your model knows is dead weight. A small model with the right tools and a closed loop beats it"
In 16 minutes, he lays out the stack that actually works, and why the people at the top stopped paying for the biggest model
Big Model → Small Model → Right Tools → Closed Loop → Self-Improving Agent
Most people keep upgrading the model. The loop is what makes a small one dangerous
Worth more than any $1,000 AI course this year
Bookmark and watch it today
Then read how to build that closed loop step by step in Harness, Loops, Graphs ↓
This question was asked in an AI Engineering interview at Anthropic.
Question: Walk me through what happens, tensor by tensor, in one forward pass of a decoder-only transformer.
Let's answer it step by step.
A decoder-only transformer turns token IDs into vectors, passes them through a stack of identical layers, where each layer has causal Self Attention and a feed-forward network on a residual stream, and finally turns the last vectors into a probability for every token in the vocabulary.
Let's use a model close to Llama 2 7B: d = 4096, 32 heads, head_dim = 128, 32 layers, FFN size 11,008, vocabulary V = 32,000. B is the batch size and T is the number of tokens.
Step 1: Token IDs
Shape: [B, T]. Just integers.
Step 2: Embedding lookup
The embedding table is [V, d]. Each ID picks one row.
x: [B, T, d]. This is the residual stream.
Step 3: Each of the 32 layers does the following.
Attention part:
- h = RMSNorm(x): [B, T, d]
- Q = h Wq, K = h Wk, V = h Wv, each weight [d, d]. Reshape to [B, 32, T, 128].
- Apply RoPE to Q and K, to add position.
- Scores = Q K^T / sqrt(128): [B, 32, T, T]
- Causal mask: set every score where a token looks at a future token to minus infinity.
- Softmax over the last dimension. Each row now adds up to 1.
- Out = weights times V: [B, 32, T, 128]. Merge the heads back: [B, T, 4096].
- Output projection Wo [d, d]: [B, T, d]
- Residual: x = x + out
Feed-forward part (SwiGLU):
- h = RMSNorm(x): [B, T, d]
- gate = h W_gate, up = h W_up: each [B, T, 11008]
- act = SiLU(gate) * up: [B, T, 11008]
- down = act W_down: [B, T, d]
- Residual: x = x + down
The shape of x stays [B, T, d] through all 32 layers. Each layer only adds to the residual stream.
Step 4: Final RMSNorm
x: [B, T, d]
Step 5: Output projection (LM head)
logits = x W_out, where W_out is [d, V].
logits: [B, T, 32000]
Step 6: What we do with the logits
- Training: use all T positions. The logits at position t predict token t + 1. Cross-entropy loss over all positions.
- Inference: take only the last position, [B, V], apply temperature and softmax, and sample the next token.
Now, the inference details:
- The K and V of every layer are saved in the KV Cache: [B, kv_heads, T, 128] per layer.
- Prefill runs this whole pass on the full prompt.
- In decode, T = 1. Only the new token goes through the layers, and its Query attends to the cached Keys and Values. The scores become [B, 32, 1, T_total].
With GQA, K and V have fewer heads, like 8 instead of 32, and each KV head is shared by a group of Query heads.
In the interview, the most important line is: the residual stream keeps the shape [B, T, d] from the embedding to the last layer, attention mixes information across tokens under a causal mask, the feed-forward network works on each token on its own, and only the LM head changes the shape to the vocabulary size.
this is f*cking crazy
karpathy's CLAUDE.md hit #1 on github trending.
228,000 stars. most devs still haven't read it.
it took my Claude agents accuracy from 65% to 94%.
the 6 rules inside:
• step 1 → surface assumptions
"State your assumptions explicitly. If uncertain, ask."
ambiguous task → list interpretations → ask → build.
• step 2 → cut the scope
"No abstractions for single-use code."
200 lines → refactor → 50 lines → ship.
• step 3 → keep the diff surgical
"Don't refactor things that aren't broken."
request → minimal diff. every changed line traces back.
• step 4 → clean only your orphans
"Remove imports/variables/functions that YOUR changes made unused."
old dead code → flag it, don't delete it.
• step 5 → turn tasks into tests
"Fix the bug" → "Write a test that reproduces it, then make it pass."
vague task → failing test → pass/fail signal.
• step 6 → run the verify loop
"[Step] → verify: [check]"
plan → code → verify → fail? loop → pass → ship.
that's it.
65 lines. 6 rules. 94% accuracy.
save this before everyone else does.
then, read the article on how to build a team of Claude agents with Graphs below ↓
Google Open Source Blog 是个巨大知识宝库 (从 2008-2026年)
虽然不少技术因为体量和规模你无法在现实中应用
但保持对新技术前沿科技的了解和感知也蛮重要的
https://t.co/ogRYymGlZW
同理,其它互联网大厂和组织的开源博客也可一同关注
Microsoft Open Source Blog
https://t.co/gWtKazEiUX
AWS Open Source Blog
https://t.co/zePienFdLc
Netflix TechBlog
https://t.co/UOBk5qmJIj
CNCF Blog (云原生计算基金会)
https://t.co/jIcFZ58VQx
Linux Foundation Blog
https://t.co/OoostPxcGx
还有很多 ... 感兴趣的朋友可自行收集正整理