Rupesh Nehra

Getting 1% better everyday by learning AI, Finance & Startups | Follow to know more:)

[Arya Tschand @AryaTschand]

I’ve spent most of my internship at nvidia architecture research thinking about how to frontier llms should inference on our newest (heterogenous) systems

I wanted to share some of the things I’ve learned about inference from first principles

TLDR + mini experiments in thread!

[Austin Armstrong @SocialtyPro]

Don't pay for ElevenLabs, use Voxilica
Don't pay for Loom Pro, use ScreenDraft
Don't pay for Midjourney, use Glimpse Diffusion
Don't pay for Buffer, use PulseSocial
Don't pay for Notion AI, use ZenBrain
Don't pay for Zapier, use HookFlow
Don't pay for Typeform, use FormForge
Don't pay for Intercom, use ChatSprout
Don't pay for Crunchbase, use VentureMap
Don't pay for Viral Scripting & Video Research, use Syllaby
(SAVE THIS before it disappears)

[Rupesh Nehra @rupesh_nehra]

@Jason I'm in

[Rupesh Nehra @rupesh_nehra]

@boardyai 2x Ex-founder with exposure of scaling product from scratch. Solid 12 years experience in architecting AI & software systems for Fortune 500 in Healthcare, Pharma, Internet & Retail.

Looking for roles in Agentic AI, Evals, Harness Engineering and Causal Inference at scale.

[Rupesh Nehra @rupesh_nehra]

@systemdesignone Playbook

[anshuman @athletic_coder]

Tokenization is changing.

It has been the part of the stack nobody cared about until models started eating tons of text for breakfast.

Tokenization is the step that turns raw text into token IDs so a model can actually do math on it, deciding whether “Steve job's daughter btw” becomes two chunks, three chunks, or ten.

A “good” tokenizer squeezes your text into as few tokens as possible while still being reversible, which means more usable context for the same compute and fewer weird edge cases when you decode back.

In earlier transformer versions this whole process was basically a black box: slow Python vs fast Rust code paths, duplicated implementations, and checkpoint files that hid everything about how text was actually split. You often couldn’t answer simple questions like “Is this BPE or Unigram?” or “What does the normalizer do?” without spelunking through configs. Training a LLaMA‑style tokenizer on your own corpus meant re‑implementing half the pipeline by hand.

With the v5 release of Transformers v5 we have got proper, inspectable, debuggable architectures. You define the pipeline explicitly - normalizer, pre‑tokenizer, model (BPE/Unigram/WordPiece), post‑processor, decoder - and then load or train vocab + merges on top, similar to defining a neural net before loading weights. That means you can start from a blank llama tokenizer, feed in your own domain data, and get a custom tokenizer that behaves like LLaMA but speaks “legal”, “medical”, or “logs” fluently.

The library also cleans up the class hierarchy and workflow: one main Rust‑backed tokenizers backend instead of parallel slow/fast paths, a clear base class with chat templates and special‑token handling, and a straightforward loop of

“instantiate architecture → train on data → push to Hub / reuse.”

this removes a ton of subtle bugs around padding, truncation, and BOS/EOS handling, and makes it far easier to reason about why a model saw a particular sequence of IDs.

The result is that tokenization stops being an afterthought and becomes a real design lever. Teams that actually understand and customize their tokenizers will quietly win on context length, stability, and domain fit, while everyone else keeps losing quality in the first milliseconds of the pipeline - before a single attention head even wakes up.

[Ben Smith @bensmithlive]

A single night of sleep can now predict 130 future diseases with up to 93% accuracy.

Stanford researchers just released SleepFM.

This is a multimodal foundation model trained on 585,000 hours of physiological data from 65,000 participants.

And it's actually insane: (1/5)

[Dr.Sayajirao Gaikwad @DietDrsayajirao]

🫀 Key Cardiac Risk Markers You Shouldn’t Ignore
Heart attacks in 20s & 30s are becoming frighteningly common.
Fit on the outside ≠ Healthy on the inside.
Here’s how to investigate early & prevent silent cardiac risk 🧵 👇

[anshuman @athletic_coder]

Techniques I’d master if GPU memory was my real bottleneck:

Bookmark this.

1. KV-cache memory math
2. PagedAttention
3. Context length vs VRAM curves
4. INT8 / INT4 quantization
5. AWQ vs GPTQ tradeoffs
6. Activation offloading
7. CPU ↔ GPU memory swapping
8. FlashAttention memory benefits
9. Rope scaling side-effects
10. Weight tying across replicas
11. Prefill memory spikes
12. KV-cache eviction policies
13. Sliding-window inference
14. Mixed-precision decoding

[𝗿𝗮𝗺𝗮𝗸𝗿𝘂𝘀𝗵𝗻𝗮— 𝗲/𝗮𝗰𝗰 @techwith_ram]

Check out their blogs if you are into AI/ML.

1) Andrej Karpathy Neural networks & LLMs explained from first principles by one of the OGs of modern AI.
- https://t.co/rOFoQmoGDG

2) Sebastian Raschka, PhD Deep dives into LLM training and fine-tuning with super clear code examples.
- https://t.co/gxEb2jjs2S

3) Interconnects by Nathan Lambert AI alignment, open-source models, and ecosystem news. - https://t.co/clHxVsI9WS

4) Lil’Log by Lilian Weng Lessons from someone who worked on practical AI safety and alignment at OpenAI. - https://t.co/GtEOx18l5p

5) Chip Huyen Real-world MLOps and production ML systems design patterns.
- https://t.co/H11wqNsTlb

6) Eugene Yan Great writing on applied ML, data science, and working with recommender systems in production.
- https://t.co/UoGNYlHTFz

7) Philipp Schmid Tutorials on building and deploying LLM apps on AWS.
- https://t.co/9QGh9Yktsb

8) Jason Liu Learn from a consultant sharing real lessons on LLMs, data, and open-source tools.
- https://t.co/L8V830tQ1b

9) Hamel H. MLOps workflows, fine-tuning, and product strategy from an ML veteran.
- https://t.co/Rx5BvPT1FO

10) Berkeley Artificial Intelligence Research Blog Latest academic breakthroughs in computer vision, NLP, and robotics
- https://t.co/xLcvR2EmtR

11) Hugging Face Product updates, tutorials, and the latest from open-source AI.
- https://t.co/9oeyeKSUlv

12) Google DeepMind Google's premier AI research division.
- https://t.co/prmdJpRlcW

[Math Files @Math_files]

Mathematical Thinking
-For People Who Hate Math

[Elliot Arledge @elliotarledge]

someone kindly reached out and asked for systems engineering resources so I shared my top recommended repos and resources they should skim though and familiarize themselves with.

I figured more people could benefit from this list, so I explained and linked them below:

[PyQuant News 🐍 @pyquantnews]

Save your $90,000 and skip the quant finance degree.

Dive into 17 code repos that will teach you more than all your professors at school.

All without costing you $90,000:

[Travis Gatzemeier, CFP® @T_Gatzemeier]

JP Morgan's "Guide to the Markets" is a goldmine for illustrating investment topics.

Here are 8 charts that I think will help anyone become a better investor.

[Rupesh Nehra @rupesh_nehra]

@MyntraSupport Please check DM

[Rupesh Nehra @rupesh_nehra]

Hi @MyntraSupport @myntra,
For the past 48 hrs, the Exchange Item page shows “Something went wrong” whenever I try changing the product size on the Return page (screenshot attached).

Could you please fix this or extend the return window?

[Rupesh Nehra @rupesh_nehra]

@MyntraSupport Already shared the details on DM and discussed the issue with support team too. Even support helpline guy was facing issue when he tried to put exchange on my behalf. Please sort out the issue with tech team.

[Rupesh Nehra @rupesh_nehra]

@MyntraSupport Hi Team, I am on the latest app version and issue still persists, even after uninstall of the app and cache delete. This seems to be a technical bug and issue in the system. Please work with your tech team to resolve it. Sharing my details in the DM for sorting it out.

[pratham @prathammittal]

Elon Musk built what he calls the 'perfect school' for his 14 kids.

It's called Ad Astra and it doesn't have grades, majors or class ranks.

The school's curriculum is his answer to 'How do you make a school that produces more Elon Musks?'

Here's what's different: