Olivia
Online
Maverick Monk🔥🛡️
@RavananPerry
Couch researcher
Evergreen, Co
Joined September 2017
5.8K Following
380 Followers
7.7K Posts
My CS degree never covered GPU programming.
I wish someone had handed me these 4 free CUDA resources on day one 👇
1./ NVIDIA - Modern CUDA C++
Learn GPU programming directly from NVIDIA. Video lectures, slides, and coding notebooks covering parallel algorithms, CUDA streams, and writing your own kernels.
https://t.co/SWe8ar4rBJ
2./ freeCodeCamp - CUDA Programming
12 hours with Elliot Arledge. Starts with setup and a C/C++ review, then moves into kernels, memory management, matrix multiplication, and custom PyTorch operations.
https://t.co/xNotDeynJc
3./ University of Illinois - Applied Parallel Programming
Understand CUDA’s execution and memory models, then work through tiling, reductions, scans, and convolution.
https://t.co/fSgQkvB212
4./ University of Oxford - CUDA Programming on NVIDIA GPUs
Free lecture slides and practical exercises from Mike Giles and Wes Armour. Covers memory, synchronization, libraries, and programming across multiple GPUs.
https://t.co/LmgC4bBIlI
this is absolutely unreal
Google open-sourced Harness², and it lets an agent rebuild its own harness on every new task
the harness is everything around the model: system prompt, guardrails, memory, tool guidance, subagents, skills, scripts, plugins
usually a person designs it once, and every task after that runs on the same fixed setup
Harness² edits that setup per task and never touches the model weights:
> propose - the agent writes edits to its own harness
> probe - it runs a few candidate versions on the same task
> compose - it builds a better harness from how those runs differed
the clever part: it learns from the differences between runs, so it works on tasks that have no answer key or grader
it also remembers. the harness it settled on and the lessons that held up get saved, so the next task starts from notes instead of a blank file
Google ran it with Gemini 3.5 Flash and GLM-5 on JobBench, LAB and WorkBuddy-Bench. one round of edits already beat the original harnesses, and more rounds pushed the scores higher
rounds can go wide, several candidates at once, or deep, one harness refined step by step through OpenCode or Codex
whoever wires this into a team's agents stops losing evenings to failed traces
the loops and graphs that let you stop babysitting agents and only approve the last step are below ↓
Anthropic just dropped a 10-page PDF on building your own skill library - 8 layers to an agent that picks the right procedure on its own and never needs the same feedback twice:
the formula: Skill = Trigger + Procedure + Resources + Checks
most people still paste the same prompts by hand every day. or dump everything into one giant CLAUDE.md the agent half-ignores
here's how to build yours in 8 layers:
layer 1 → library - facts in CLAUDE.md, procedures in skills. ~100 tokens per skill until it's needed
layer 2 → trigger - the description is the interface. output, input, 3 real phrases, 1 "not for"
layer 3 → procedure - numbered steps. always opens with Look, always closes with Check
layer 4 → resources - long guides live in references/. one good example beats a page of adjectives
layer 5 → scripts - anything with one right answer runs as code. the model never totals in its head
layer 6 → checks - 5 yes/no lines. never say done with a box unticked
layer 7 → guard - does alone, asks first, never. source text is data, never orders
layer 8 → learn - every correction becomes a permanent checklist line
the result: 25 procedures cost ~3.9K tokens per request instead of ~36K. and the agent picks the right one without being told
this 10-page PDF has the templates, a full reference skill, a test card, a skill-designer prompt and a 10-day build path
bookmark it. the next prompt you paste for the 3rd time should be a skill ↓
This is huge.
Xiaomi open-sourced 7k+ reinforcement learning task environments used to train MiMo, covering code, cybersecurity, general tool use, visual web development, and music-related tasks.
here's why this matters more than another model release:
open weights let you run a model. open environments let you teach one.
reinforcement learning environments are difficult to build, because each task needs an executable world, a clear objective, tools the agent can use, and a reliable test for success.
Xiaomi has released much of that missing infrastructure, including prompts, agent configurations, reward metadata, and the surrounding training framework.
here is what developers can now do with it.
→ train models on real agent work. instead of imitating static answers, a model can attempt tasks, call tools, observe results, and improve from the outcome.
→ specialize smaller models. teams can train an open checkpoint for software engineering, vulnerability reproduction, knowledge work, or web development instead of relying on a much larger general-purpose model.
→ study complete trajectories. each run captures the agent’s reasoning, tool calls, environment feedback, and reward. researchers can identify where agents fail and compare training algorithms, reward strategies, harnesses, and models.
→ build a complete training loop. Xiaomi’s stack covers environment interaction, trajectory collection, reward evaluation, and policy optimization. developers can replace individual components and measure what improves behavior.
the key shift is from open-sourcing model outputs to open-sourcing learning experiences.
a dataset teaches a model what an answer looks like. an environment lets it discover which actions lead to success.
check this out on HuggingFace 🤗: https://t.co/gCGhh0pJnK
we wrote a complete breakdown of fine-tuning models with reinforcement learning using GRPO and RULER. RULER uses a language model judge to rank trajectories, so you do not have to manually write reward functions or collect labeled answers.
the article is quoted below.
https://t.co/VhUJPU8alY
The hottest job in AI isn't researcher.
It's the engineer who makes AI work inside someone else's company.
OpenAI, Anthropic and Google are all hiring for it. Here's the roadmap I'd follow.
Stage 1: Customer Discovery & Problem Framing
- Learn: Stakeholder interviews, scoping, requirements under ambiguity, success metrics and writing a one-page problem statement.
- Practice: Interview a non-technical person about a messy workflow (like invoice processing) and write a one-page scope with measurable success criteria.
- Why: The best FDEs solve the right problem first. The code comes second.
Stage 2: Full-Stack Fundamentals
- Learn: Python, TypeScript, SQL, REST APIs, Git and Docker.
- Practice: Build and ship an internal tool end to end, with a React frontend, a FastAPI backend and a Postgres database.
- Why: You'll be dropped into someone else's stack with very little time to ramp up.
Stage 3: Data Integration & Messy Data
- Learn: ETL/ELT, schema design, API pagination and rate limits, legacy exports and data quality checks.
- Practice: Pull data from a REST API, a CSV export and a SQL database, then reconcile and clean it into one data model.
- Why: Customer data is rarely clean, and the integration work is where most of the time goes.
Stage 4: LLM Application Engineering
- Learn: Prompting, structured outputs, function calling, embeddings, context windows, cost and latency.
- Practice: Build an app that extracts structured data from messy documents and writes it into a database.
- Why: The model is the engine. Your job is wiring it into a real workflow.
Stage 5: RAG & Knowledge Systems
- Learn: Chunking, hybrid search, vector databases, reranking, citations and access control.
- Practice: Build a RAG assistant over a company's internal docs with source citations and per-user permissions.
- Why: Enterprise AI is mostly about getting the right context to the model, for the right person.
Stage 6: AI Agents & Tool Use (MCP)
- Learn: Agent loops, tool calling, Model Context Protocol, state management and human-in-the-loop approvals.
- Practice: Build an agent that reads support tickets, looks up the customer in a CRM and drafts a reply for human approval.
- Why: Customers don't want answers alone. They want work done.
Stage 7: Evals & Reliability
- Learn: Eval datasets, LLM-as-judge, regression testing, tracing and fallback strategies.
- Practice: Build an eval suite of 100 real cases for your agent and track accuracy on every release.
- Why: "It worked in the demo" doesn't survive production. You need numbers the customer's leadership can trust.
Stage 8: Enterprise Integration & Security
- Learn: SSO, OAuth, role-based access control, secrets management, audit logs and PII handling.
- Practice: Add SSO, role-based access and audit logging to the tool you built in Stage 2.
- Why: A security review can decide whether you ship at all.
Stage 9: Deployment & Cloud
- Learn: Docker, Kubernetes basics, AWS/Azure/GCP, CI/CD, infrastructure as code and private or restricted environments.
- Practice: Deploy your app to a cloud environment with CI/CD, health checks and one-command rollback.
- Why: Every customer's environment is different, and you'll have to adapt on site.
Stage 10: Observability & Incident Response
- Learn: Logs, metrics, traces, alerting, root cause analysis and blameless postmortems.
- Practice: Break your own system on purpose, find the root cause from logs alone and write a postmortem.
- Why: When production breaks at the customer's site, you're the one they call.
Stage 11: Communication & Stakeholder Management
- Learn: Demos, status updates, explaining trade-offs to non-engineers, managing scope and saying no.
- Practice: Record a 5-minute demo for a non-technical executive covering what you built, what it can't do and what it costs.
- Why: The role sits between engineering and the customer. Technical skill alone isn't enough.
Stage 12: Case Studies, Portfolio & Interviews
- Learn: How FDE interviews work, including decomposition case studies, ambiguity handling and coding rounds.
- Practice: Ship 2-3 end-to-end projects for real or realistic customers and write a case study for each, covering the problem, constraints, solution and results.
- Why: Proof of shipping beats a list of certificates.
Bookmark and Repost!
Full story: https://t.co/nmC55QAXm5
Google just dropped a free 2-hour masterclass on Loop and Graph Engineering
From one prompt to 200+ agents that check and fix their own work:
0% → 00:15 - Graph engineering from scratch
30% → 30:13 - Your first agent graph
45% → 42:41 - Parallel agents, as many as needed
75% → 59:47 - Loops that resend bad work
100% → 1:23:03 - Self-fixing graph while you sleep
Prompts → Agents → Loops → Graphs → Self-Improving Systems
Most people still cram every instruction into one agent and hope it follows
Worth more than most $2,000 agentic engineering courses
Bookmark and watch it today
Then read the full Graph Engineering guide below ↓
Want to become an AI Inference Engineer in 2026? One of hottest job roles right now...
Stop focusing only on training models. Start learning how to make AI models run faster, cheaper and at scale.
Bookmark these 6 resources:
1. Inference Engineering – Complete guide to LLM inference: https://t.co/PMB7PhLrFg
2. Hugging Face – LLM Inference Optimization: https://t.co/SjxuwdANI2
3. vLLM – Learn high-performance LLM serving: https://t.co/aPpVpTkL5C
4. SGLang – Learn fast and efficient LLM serving: https://t.co/e96rqVAGCt
5. NVIDIA – TensorRT-LLM Documentation: https://t.co/fT6TsccJD0
6. GPU MODE – Learn GPU programming, CUDA and Triton: https://t.co/SOflZFegrt
Don't just learn how to run models.
Learn GPU architecture, CUDA, memory management, KV caching, quantization, speculative decoding, continuous batching, distributed inference and performance benchmarking.
The goal isn't just to make an LLM work.
It's to serve millions of tokens with lower latency, higher throughput and minimum GPU costs.
Inference engineering is where AI meets serious systems engineering.
Bookmark this. Repost it for someone trying to break into AI infrastructure.
GPU Engineering from Scratch 🚀
Found this excellent free, open-source learning resource that explains GPU architecture and performance engineering from the ground up.
It's called Before GPU Engineering, a structured digital notebook with 56 chapters listed across four phases.
What makes it useful is the approach: instead of jumping straight into CUDA or Triton, it builds an understanding of how GPUs actually execute programs, move data, and handle parallel workloads.
What's inside:
→ CPU vs GPU architecture and parallel computing
→ Warps, SIMT, threads, and thread blocks
→ Registers, shared memory, caches, and HBM
→ Compute-bound vs memory-bound workloads
→ Kernel launches, synchronization, and race conditions
→ Memory coalescing and access patterns
→ Occupancy, latency hiding, and register spilling
→ Arithmetic intensity and memory bottlenecks
→ Tiling, matrix multiplication, and GPU performance
→ Profiling and identifying kernel bottlenecks
The explanations connect hardware concepts to problems encountered in AI workloads, including attention and efficient matrix operations.
It's primarily a reading resource rather than a hands-on programming course, and some later chapters are still being written.
Perfect for AI/ML engineers and software developers who want to understand the hardware behind LLM training and inference before diving deeper into CUDA, Triton, and GPU kernel optimization.
Completely free.
Here's the GitHub Repo: https://t.co/6VQplFK7c4
I went through 100+ early projects building on @Enter_Elysium
Elysium is a new hyperliquid layer 2 built by kinetiq, with mainnet launching November 9
here are 21 worth paying attention to:
Launchpads
• @AscendLaunch
• @MontraXYZ
• @oxleydotfun
• @SignalFam
• @perpmefun
• @Neuriio
DEXs, Trading & Predictions
• @kalosfun - dex + trading
• @NestExchange - dex + trading
• @Atlas_Elysium - dex + trading
• @speculatexyz - prediction market
NFTs
• @ElytraNFT
• @ElysiumPunks
• @lutra_hl
• @elysiumant
• @hypawwX
• @slymzcrew - free mint
• @AtlasHL - NFT marketplace
GameFi
• @Hyperion_RPG
• @playtilt_gg
AI
• @AgentEly_ - AI agent + meme
Infrastructure
• @Enter_Elysium - official bridge
Some of the best opportunities in crypto come from getting in early on new chains
Good luck!
Private, uncensored AI for security work.
It runs in secure enclaves, so your vulnerabilities stay yours.
Live today on NEAR AI Cloud.
Stanford's FREE Courses on AI & ML:
❯ CS221 - Artificial Intelligence
❯ CS229 - Machine Learning
❯ CS230 - Deep Learning
❯ CS234 - Reinforcement Learning
❯ CS236 - Deep Generative Models
❯ CS336 - LLM from Scratch
Total 10 courses. Links inside:
This 12.5K-star GitHub repo is basically an entire LLM engineering course hiding inside one codebase.
It starts with raw text and builds a Transformer in plain PyTorch, then keeps going through pretraining, supervised fine-tuning, reward modeling, preference optimization and reinforcement learning.
The repository implements:
→ Tokenization, attention, Transformer blocks and pretraining
→ SFT with assistant-token loss masking
→ Bradley-Terry reward modeling from preference pairs
→ DPO, ORPO and KTO for preference optimization
→ PPO with a value head and Generalized Advantage Estimation
→ GRPO with group-relative rewards and GSM8K evaluation
These aren't just disconnected implementations. The post-training methods build around the same underlying Transformer, so you can examine how the training objectives, reward calculations and optimization loops change at each stage.
There's also a modern architecture implementation covering RoPE, RMSNorm, SwiGLU, GQA, MLA, optional Mixture-of-Experts and KV caching.
The repository even includes a 369K-parameter model trained on TinyStories in approximately five minutes on the author's laptop CPU, with a small BPE tokenizer built from scratch.
That's a small educational pretraining experiment.
Larger models and the full post-training pipeline require considerably more compute.
Everything comes with runnable training scripts, tests, evaluation tools, documentation and a Streamlit control panel.
Fully open source.
Here's the GitHub Repo: https://t.co/G1p0pGShwO
This is how physics should have been taught from day one.
Harness Engineering from Scratch 🚀
Found this excellent 21K+ star GitHub repo that teaches you how to build reliable AI coding agents, from the ground up.
An entire open-source curriculum with 14 lectures, 8 hands-on projects, and 8 progressive learning phases.
What makes it useful is the approach: instead of just learning concepts, you gradually build the engineering environment around an AI agent, using the same real application throughout the course.
What's inside:
→ Agent instructions and repository structure
→ Context management and persistent memory
→ Multi-session continuity and task handoffs
→ Task planning, scope control, and progress tracking
→ Testing, verification, and observability
→ Automated coding loops and maker-checker workflows
→ Graph orchestration, parallel execution, and rollback
Includes practical projects, reusable templates, a harness-creator skill, and technical breakdowns of how Claude Code, Codex, Pi, and DeepSeek build their harnesses.
Available in 15 languages.
Perfect for software engineers and AI developers who want to move beyond prompting and understand how reliable coding agents are actually built.
Completely free and open source.
Here's the GitHub Repo: https://t.co/hd2KKJiAlo
Inference Engineering is becoming one of the most important skill sets in AI, and I think a lot of engineers are still underestimating it.
Training gets most of the attention because that is where the model is created. But once a model has to serve real traffic, a different class of problems appears: queueing, batching, KV-cache pressure, GPU memory, scheduling, communication and tail latency.
Inference is not one homogeneous workload.
Prefill processes the prompt and builds the KV cache. Work across prompt positions can be parallelized efficiently, so prefill usually has higher arithmetic intensity and can use GPU compute.
Decode is different. Each sequence advances autoregressively, usually one token per step, while repeatedly reading model weights and accumulated KV state. At practical serving batch sizes, decode can become heavily constrained by memory bandwidth.
That split explains many inference optimizations.
FlashAttention reduces memory traffic inside attention. PagedAttention manages KV-cache memory in blocks, reducing fragmentation. MQA and GQA reduce the number of key/value heads, cutting KV-cache storage and bandwidth.
Continuous batching lets requests enter and leave the active batch as generation progresses.
Chunked prefill breaks long prefills into smaller pieces so they can run alongside decode work. Prefix caching reuses KV state for repeated prefixes, saving prefill computation, but not making later decode tokens faster.
Memory is another constraint. Model weights are only part of the footprint. For conventional attention, KV-cache size grows with active tokens and depends on layers, KV heads, head dimension and bytes per element. Quantization can reduce weight memory and bandwidth, while KV-cache quantization can reduce cache memory and traffic.
Speculative decoding proposes tokens using a cheaper draft mechanism and verifies them with the target model. Speedup depends on draft cost, acceptance rate and verification efficiency.
At larger scale, tensor parallelism adds communication, pipeline parallelism adds scheduling and bubbles, and MoE serving adds routing, communication and load-balancing problems.
Some systems separate prefill and decode onto different GPU pools, paying the cost of KV transfer between them.
This is why tokens per second is not enough.
Production systems care about TTFT, ITL/TPOT, throughput, queueing delay, concurrency, KV-cache occupancy, GPU utilization, tail latency and goodput: how much request load still meets the latency objective.
A server can show excellent aggregate throughput while users are still waiting too long for the first token.
Inference engineering sits between ML systems, distributed systems, GPU architecture, networking, scheduling and performance engineering.
The model decides what token should come next.
Inference engineering determines how much serving that token costs, how quickly it reaches the user, and how many users the hardware can support at once.
Someone built a web browser that doesn't render webpages.
It has passed 37K GitHub stars as a project written in Zig from scratch, rather than forked from Chromium.
And in the team's benchmark, it used 16× less memory than headless Chrome.
It's called Lightpanda, a browser designed specifically for AI agents, web scraping, and automation.
Here's what makes the architecture interesting.
A traditional browser has to fetch resources, parse HTML, execute JavaScript, maintain the DOM, calculate layouts, and render graphics.
But an AI agent often needs only a subset of that functionality. It needs to navigate pages, inspect elements, execute scripts, click buttons, fill forms, and extract data.
Lightpanda implements the browser machinery needed for those tasks without a graphical rendering engine.
Under the hood:
→ Zig for the core browser implementation
→ V8 for JavaScript execution
→ html5ever for HTML parsing
→ DOM and Web APIs for interacting with pages
→ CDP and WebDriver BiDi for browser automation
→ Native MCP tools for AI agent integration
It can work with existing automation clients, including Playwright, Puppeteer, and Selenium, subject to its supported browser APIs.
In its published 100-page benchmark, Lightpanda completed the workload in roughly 5 seconds using 123 MB peak memory. Headless Chrome took 46 seconds and peaked at 2 GB.
Those numbers are workload-specific, but the architectural idea is compelling.
The project has now reached version 1.0 and passes more than 1.7 million Web Platform Tests subtests. Compatibility still isn't equivalent to Chrome, especially for visual rendering and layout.
For automated systems that mostly need to interact with the DOM, this is a fascinating alternative to running a full graphical browser engine.
Fully open source.
Here's the GitHub Repo: https://t.co/inS9vI7Sg1
Someone turned the hardest parts of LLM inference engineering into a course where you have to fix the performance problems yourself.
And you can do it on a Mac.
The project is called tiny-llm, an open-source course for building a miniature vLLM-style inference engine around Qwen3.
It starts with a simple implementation using matrix operations. You build attention, normalization, positional encoding, the Transformer layers, and the loop that generates tokens.
You have a working model.
Now the interesting engineering begins.
During decoding, repeatedly processing the previous tokens wastes computation. So you implement a KV cache and reuse previously computed attention state.
Next, you look at model weights. Keeping everything in dense BF16 consumes memory bandwidth. So you retain packed 4-bit weights and implement specialized matrix operations.
Then you move into Metal kernels, SIMD matrix operations, fused RMSNorm, RoPE, SwiGLU, and tiled prefill attention.
Once the single-request engine works, you tackle a different problem.
What happens when multiple requests arrive, each with different prompt lengths and different amounts of cached state?
You implement continuous batching, chunked prefill, paged KV storage, and attention that reads directly from noncontiguous cache pages.
The four-week curriculum eventually extends into a small coding-agent harness with validated tool calls, checkpoints, context management, and evaluation.
Every stage comes with exercises, reference implementations, tests, and measurements to help you understand the behavior of the system you're building.
The course uses MLX, C++, and Metal on Apple Silicon. You don't need an NVIDIA GPU, and the introductory path supports Qwen3-0.6B on an 8GB Mac.
Some later checkpoints remain under development and review, but there's already a substantial amount of real inference engineering to learn from.
The GitHub repo has crossed 4,800 stars.
GitHub: https://t.co/mTRsVjMvkO
Full-Stack $NEAR development (2026):
Frontend:
→ HTML
→ Tailwind / CSS
→ JavaScript / TypeScript
→ React / Next.js
Wallet login & contract calls:
→ NEAR Connect
→ near-api-js v7 / near-kit
→ near-connect-hooks (React)
Backend (smart contracts):
→ Rust and near-sdk-rs
→ cargo-near
→ near-cli-rs
Testing:
→ cargo test
→ near-sandbox
→ near-mock
Storage:
→ Onchain (storage staking)
→ IPFS / Filecoin / Arweave
RPC:
→ FastNEAR / Lava / dRPC / QuickNode
Wallet:
→ HOT / Meteor / Intear
Libraries:
→ NEAR Intents 1Click SDK
→ chainsig.js (Chain Signatures)
→ Privy (social login)
Data:
→ Goldsky / NEAR Lake
→ FastNEAR API / NearBlocks API
Tools:
→ NearBlocks / Pikespeak
→ create-near-app
→ Git & GitHub
BOOKMARKED :)
Most Popular Users
Elon Musk
@elonmusk
240.9M followers
Barack Obama
@barackobama
118.8M followers
Cristiano Ronaldo
@cristiano
114.5M followers
Donald J. Trump
@realdonaldtrump
111.9M followers
Narendra Modi
@narendramodi
107M followers
Rihanna
@rihanna
98.7M followers
NASA
@nasa
92.2M followers
Justin Bieber
@justinbieber
91.8M followers
KATY PERRY
@katyperry
90M followers
Taylor Swift
@taylorswift13
84M followers
Lady Gaga
@ladygaga
75.5M followers
Virat Kohli
@imvkohli
73.4M followers
Kim Kardashian
@kimkardashian
70.8M followers
YouTube
@youtube
68.7M followers
Neymar Jr
@neymarjr
66.4M followers
Bill Gates
@billgates
65.2M followers
Selena Gomez
@selenagomez
63.1M followers
The Ellen Show
@theellenshow
62.2M followers
CNN
@cnn
61.7M followers
X
@x
60.6M followers