Mahesh Kumar

Technology enthusiast AI ML, Humanity Before Religion, Fighting climatic changes for better Tommorow, Indian firstly and lastly

[Sebastian Raschka @rasbt]

Just saw that the LLMs-from-scratch repository passed 100,000 stars on GitHub!
This is super cool and motivating. I am really happy to see that this open-source repo has helped so many people.
Thanks also to everyone who shared ideas and opened PRs with improvements!

Of course, I plan to keep adding new material, including new attention variants and architectures (while bigger projects like RL and Reasoning From Scratch live in their separate repositories).

I am also currently working on a larger applied custom “small” LLM project. It has been keeping me super busy this month, but I will share more on that soon.

If you are new to it, some of the highlights include

1. Of course, the complete code path from tokenization and attention to pretraining, classification, and instruction fine-tuning, etc. All of it FROM SCRATCH, of course! (RL lives in a companion repo.)

2. From-scratch implementations of Llama, Qwen, Gemma, and Olmo (smaller variants that run locally and can be plugged into the training scripts).

3. From-scratch implementations of attention alternatives and other architecture components, such as GQA, MLA, sliding-window attention, Gated DeltaNet, DeepSeek Sparse Attention, cross-layer KV sharing, and mixture-of-experts

4. Materials on KV caching, training performance, memory-efficient weight loading, DPO, evaluation, and LoRA

So, if you don’t have any weekend plans yet, happy tinkering!

[Sanjay Ghemawat @Sanjay_Ghemawat]

I am excited to announce that I, along with my long-time colleagues (@JeffDean, @OriolVinyalsML, @quocleix), are founding Discovery Loop (@DiscoLoopAI), a public benefit corporation. We aim to automate the process of discovery and progress in machine learning, science, and engineering. The four of us have helped build some of the world’s most used products and infrastructure. We’re now excited to turn our attention to this ambitious endeavor. ♾

[Jeff Dean @JeffDean]

Announcing Discovery Loop!

I am very excited to announce that, along with my longtime friends and collaborators @Sanjay_Ghemawat, @OriolVinyalsML and @quocleix, we are founding Discovery Loop (@DiscoLoopAI), a Public Benefit Corporation whose mission is to automate machine learning, science, and engineering to accelerate discoveries and progress. The four of us have worked together for 14 to 30 years, and have helped build some of the world’s most used products, infrastructure and AI models, and we’re excited to turn our attention to this ambitious endeavor.

♾

Learn more at: https://t.co/Rv3LMdLluK

[Mahesh Kumar @MkMahe7]

Should we call it, end of an era, I have been following @JeffDean from initial Tensorflow days, what a genius! Then I came across this article https://t.co/W9MEcHdfA2
We don't appreciate the role of unsung developers enough in the success of a Company #Google #JeffDean #Sanjay

[Jeff Dean @JeffDean]

Tomorrow will be my last day at Google after 27 years, and watching it grow from 25 people to 190,000+ has been an amazing journey. Below is a note I shared with many people internally at Google today. An excerpt is:

It has been an absolute pleasure to work with you and to help build some of the most widely used and impactful products of all time. As a kid, I dreamed of helping build software that would be used by many people, and Google now has thirteen products used by more than a billion people (amazing!). Our work has had a tremendous impact in the world, and I have been lucky enough to collaborate and form friendships with many colleagues that I deeply admire, respect, and enjoy. It still brings me joy every time I see people out in the world using our products to find information, handle email, translate documents, watch videos, learn new things, navigate and understand the physical world, browse the web, use their phone, run large-scale computations on our infrastructure, ride in an autonomous vehicle, or perform complex tasks with the help of our AI systems. I hope you all share this sense of joy, because it is a shared accomplishment! Thank you to all of my colleagues at Google over many years!

Now I'm excited to go start @DiscoLoopAI with my longtime friends and colleagues @Sanjay_Ghemawat, @OriolVinyalsML, and @quocleix.

(Updated post: slightly redacted to not have some personal info)

[CyrilXBT @cyrilXBT]

This is F*cking gold

andrej karpathy joined anthropic to lead a team using claude to accelerate its own pretraining research, teaching claude to help build the next version of itself.

that's real, confirmed directly. the "two anthropic seniors made his loop 1000x better with graph engineering" claim is not something i can verify anywhere, and i'm not going to invent a personal testimonial to make it sound more credible than it is.

what is genuinely worth reading: anthropic's own published claude cookbook on knowledge graph construction, extract, resolve, assemble, query, real and public, no leaked breakthrough required.

read that, then check the article below.

[#YeThikKarkeDikhao @YTKDIndia]

Day 5 of patchwork on the Lucknow–Kanpur Expressway.

Not even a month has passed since it opened to the public.

This is one of the easiest ways to make money.

A perfect heist - no one gets caught or held accountable. And everyone walks away freely.

[#YeThikKarkeDikhao @YTKDIndia]

While the Health Minister is busy making reels of surprise inspections, RIMS Hospital in Ranchi is flooded with water.

Maybe the hospital knew the Health Minister wasn’t coming today.

Otherwise, the hospital would never have been flooded.

[KTR @KTRBRS]

A proud milestone for Hyderabad and India!

Congratulations to @AzadEngOfficial and @DRDO_India on the successful manufacturing and delivery of India's first indigenous 350 kg thrust-class expendable turbojet engine

Built in Hyderabad by @AzadEngOfficial, this engine significantly strengthens India's indigenous propulsion capabilities and defence self-reliance

Jet engine technology demands exceptional precision, advanced metallurgy and world-class manufacturing. This achievement reflects the strength of Hyderabad's aerospace and precision engineering ecosystem, built over decades and significantly strengthened over the last decade

Congratulations to the teams at @AzadEngOfficial, and @DRDO_India on this historic achievement.

Jai Hind!

[Mahesh Kumar @MkMahe7]

This is a pattern, isn't it? Firstly something terrible happens and then something will be done just to pacify few sections of society and then that'll be called remarkable, historical etc Paper Leaks shouldn't have happened in the first place. Anyways hope for the best #India

[Amit Shah @AmitShah]

The Modi govt today set a significant milestone in securing the future of our students by passing the Public Examinations (Prevention of Unfair Means) Amendment Bill, 2026, in Parliament.

The new law will emerge as the strongest deterrent against crimes that threaten the aspirations of our youth by diluting the credibility of our public examination system. By laying down the harshest provisions to punish anyone who attempts to tamper with the fairness of the merit-based framework, the new law will ensure that our youth fulfill the dreams they have envisioned for themselves.

[Sampath Reddy Mamidi @sampath6556]

DAY 574

2008: HYDERABAD METRO WAS APPROVED. SO WERE THE DREAMS OF OTHER CITIES WHO HAD ZERO KM THEN. LOOK WHERE THEY ARE NOW.

In 2008, Hyderabad Metro was sanctioned. That same year:

Bengaluru Metro: 0 km. Today: 96+ km operational, heading to 220 km by 2027.

Chennai Metro: 0 km. Today: 54 km operational, expanding to 173 km with Phase-II (₹63,246 Cr, approved 2024).

Mumbai Metro: 0 km. Today: 101 km operational, with Lines 2B, 4, 5, 6, 9, 12 under construction.

Delhi Metro: ~65 km. Today: ~400 km set to become the world's LARGEST single-city Metro network, surpassing New York.

HYDERABAD METRO: Built rapidly to 69 km by 2020. And then NOTHING. No new operational length since 2020. Not one kilometre added in 6 years and no hopes for next 5 years if Phase 2 gets approval today.

Cities that had ZERO Metro when Hyderabad started have since overtaken it by hundreds of kilometres. Cities that started later than us are finishing Phase-II while we are still waiting for Phase-II to be approved.

Since January 2024 alone, the Centre has sanctioned 328.63 km of new Metro nationwide, worth ₹1.59 lakh crore across Delhi, Bengaluru, Chennai, Thane, Jaipur, Pune, Lucknow, Noida, Ahmedabad. Hyderabad: 0 km of it.

Even Mauritius, a small island nation, built 13 km of Metro in 24 months funded by India.

Dear Prime Minister @narendramodi,

We don't need more words. Hyderabad needs helping hands, not moving lips.

After 2008, how many new Metro corridors have actually been sanctioned and built for Hyderabad? You don't need a search engine to check. The answer is zero.

RESOLVE THE IRFC ISSUE. APPROVE HYDERABAD METRO PHASE-II. CALL FOR TENDERS. START CONSTRUCTION.

574 days of continuous appeal. Our patience is not weakness. Our persistence is our strength. We will continue until justice is delivered.

STRONG METRO.
STRONG HYDERABAD.
STRONG INDIA.
#HyderabadMetro

@narendramodi
@PMOIndia
@AmitShah @AshwiniVaishnaw @mlkhattar @Secretary_MoHUA @kishanreddybjp @revanth_anumula @TelanganaCMO @HMDA_Gov @md_hmrl @HMRLHydmetro

@Asksindia_ @eenadulivenews @TOIIndiaNews @htTweets @abntelugutv @tv5 @NtvTeluguLive @V6Velugu @sakshitvdigital @etvtelangana @republic @THHyderabad

JOIN THE FIGHT.
RAISE THE VOICE.
STAND WITH HYDERABAD.

[Hi Hyderabad @HiHyderabad]

L😂L

[#YeThikKarkeDikhao @YTKDIndia]

What kind of protest is this?

Damaging public property built with taxpayers’ money is not protest, it’s vandalism.

[Mahesh Kumar @MkMahe7]

I am glad that someone finally speak up on generational reservations which is a threat not only for general category students but also the real oppressed and underprivileged ones. India needs educational reforms, reservation reforms #India #Education #Neet #Reservations

[Anuradha Tiwari @talk2anuradha]

General Category attending tomorrow's student protests- use these banners.

Reservation is one of the biggest issues affecting your future. Raise your voice.

Talk to the media. Share photos & videos on social media. Tag me- I will amplify them.

This is your moment. Use it.

[Mohandas Pai @TVMohandasPai]

PM @narendramodi Sir This is a major source of grievance by students. A GC with 90%+ does not get a seat but some others in categories with very low marks get in! This is very wrong a 10% lower marks is ok but not so low. This is creating huge anger amongst students who work very hard for years to qualify. Needs a solution.

[Mahesh Kumar @MkMahe7]

You should stop using the name 'HR' as they doesn't serve the purpose, I had huge respect for Tata Group, if anything you're just ruining the legacy @TCS @TataCompanies

[Mahesh Kumar @MkMahe7]

@TCS what's happening with TCS, one of my friends got an offer and his background verification is not initiated and portal shows error while uploading documents. HR doesn't respond and it's all chaos. What's this? @TataCompanies

[Anshul Saxena @AskAnshul]

NEET-PG 2025, cut-off:

A) Orthopaedics seat filled at:
4 marks out of 800.

B) Obstetrics and Gynaecology seat:
44 marks out of 800.

C) General Surgery seat filled at:
47 marks out of 800.

D) Biochemistry:
minus -8 marks out of 800.

E) Transfusion Medicine:
10 marks out of 800.

F) Anatomy:
11 marks out of 800.

G) MS Orthopaedics seat filled at:
1 mark out of 800 in a medical college in Hyderabad.

H) MD Forensic Medicine seat filled at:
12 marks out of 800 in a Govt medical college.

I) MD Pathology seat filled at:
24 marks out of 800 in a Govt medical college.

In NEET-PG 2025, candidates with scores as low as –40 out of 800 were allowed to participate in PG counselling. A negative score was considered qualifying.

These extremely low scores, at which seats were actually allotted, indicate a serious crisis in academic standards. A reasonable minimum qualifying merit must be established to maintain the integrity of medical education and the healthcare system.

Forget about dialogue or discussion. So far, neither the Government nor the Opposition has said a word about this because it does not suit their agenda, and it would affect their vote banks and political interests.

Even those who are protesting today in the name of concerns about the education system are not addressing this issue because they too have political ambitions and know that speaking about it could cost them support rather than gain it.

[Sampath Reddy Mamidi @sampath6556]

DAY 568

TELANGANA GOT ₹0. EVERY OTHER MAJOR STATE GOT CRORES.

The Ministry of Finance's official report, tabled in the Lok Sabha, reveals the state-wise break-up of the SASCI (Special Assistance to States for Capital Investment) scheme for the first 3½ months of FY27 — an opening pool of ₹44,544 crore.

Uttar Pradesh – ₹7,816 Cr
Madhya Pradesh – ₹3,780 Cr
Uttarakhand – ₹2,897 Cr
Nagaland – ₹2,804 Cr
Rajasthan – ₹2,720 Cr
Himachal Pradesh – ₹2,645 Cr
Assam – ₹2,454 Cr
Tripura – ₹2,310 Cr
Andhra Pradesh – ₹2,131 Cr
Karnataka – ₹2,088 Cr
Tamil Nadu – ₹2,070 Cr

TELANGANA: ₹0 CRORE.

Not a small share. Not a delayed share. ZERO.

This is the same Telangana that:

Contributes massively to India's GST and exports.
Hosts India's leading GCC, Pharma and IT hub.
Has ZERO kilometres of Metro Phase-II under construction — also confirmed absent from the Centre's own Rajya Sabha Annexure just days ago.
Still has no IIM, despite offering 200 acres of free land.

A pattern is now impossible to ignore.

Dear @nsitharamanoffc, @narendramodi,

Why does Telangana keep appearing at ₹0 while neighbouring and smaller states receive thousands of crores?

This is not a request for favouritism. This is a demand for fairness.

RELEASE TELANGANA'S RIGHTFUL SHARE UNDER SASCI.
RESOLVE THE IRFC ISSUE.
APPROVE HYDERABAD METRO PHASE-II.
START CONSTRUCTION.

A state that contributes cannot keep receiving ₹0.

STRONG METRO.
STRONG HYDERABAD.
STRONG TELANGANA.
STRONG INDIA.

@narendramodi @PMOIndia @AmitShah @AshwiniVaishnaw @mlkhattar @Secretary_MoHUA @kishanreddybjp @revanth_anumula @TelanganaCMO @HMDA_Gov @md_hmrl @HMRLHydmetro

@Asksindia_ @eenadulivenews @TOIIndiaNews @htTweets @abntelugutv @tv5 @NtvTeluguLive @V6Velugu @sakshitvdigital @etvtelangana @republic @THHyderabad

JOIN THE FIGHT.
RAISE THE VOICE.
STAND WITH HYDERABAD.

#HyderabadMetro

[Mahesh Kumar @MkMahe7]

What's the accountability taken while there was a leak earlier? This is not the first time sir. Why act like this is in first time? Own your mistakes #BJP #CJP #India #IndianEducation

[Raghunandan Rao Madhavaneni @RaghunandanraoM]

♦️ On what grounds should the Minister resign? Accountability means ensuring a fair investigation, bringing the guilty to justice, and protecting the interests of students—not demanding resignations before the facts are established.

♦️ If every criminal act committed by individuals automatically requires a Minister's resignation, did previous governments follow the same standard whenever examination irregularities or paper leak incidents occurred?

♦️ When the King is strong, all the thieves unite.

♦️ Even people above 57 years of age are calling themselves youth and staging protests at Jantar Mantar.

♦️ Those who oppose the nation's security have gathered and are protesting under the banner of the so-called "Cockroach Janata Party."

♦️ Since Independence, India has witnessed numerous accidents, examination irregularities, and paper leak incidents. Has any Minister in any government ever resigned over such incidents?

♦️ When invited to discuss the NEET issue in Parliament, the Opposition has chosen to walk away from the debate instead of participating in it.

♦️ If you truly have knowledge of the issue, come to Parliament and participate in the discussion instead of resorting to protests and slogans.

♦️ We are ready for a comprehensive discussion on the NEET issue in Parliament. Let the facts prevail, let accountability be established, and let the nation decide who is prepared to debate and who is trying to evade the discussion.

[Mahesh Kumar @MkMahe7]

Sir, where were you all when the NEET leak happened in the, why there's a delay in taking actions? #IndianEducation #CJP #BJP #IndiaNeedsChange

[Lokesh Nara @naralokesh]

Prime Minister Shri @narendramodi Ji’s decision to establish fast-track courts for swift trials and stringent punishment in paper leak cases marks an important step in protecting the future of our youth. It sends a clear message that those who compromise the aspirations of our students will face the full force of the law. The concerns and aspirations of our youth must be met with justice, accountability and decisive action - and that is exactly what this decision delivers.

I urge our students to keep engaging constructively, through dialogue, in shaping a stronger nation.

[Mahesh Kumar @MkMahe7]

The same Rahul Gandhi is not standing with the students of Telangana @CJP_for_India #CJP #Congress

[Cockroach Army @Coacroach_party]

Brave🫡🫡

बाद का पता नहीं पर अभी तक आप में एक सच्चे नेता की छवि दिखती हैं @RahulGandhi

#CJPMovement