R

Retweeter . PM @tech

[Rohan Paul @rohanpaul_ai]

Jensen Huang on "distillation"

On his new interview with axios, he was asked this question

"Should open source model companies be allowed to distill closed models"

"Distillation—learning from AI, learning from other people, and learning from other sources of knowledge, is fundamental to intelligence.

We are constantly learning from other people. I am learning from you through the questions you are asking, and you are learning from me. All day long, we are learning from one another. AI also has to learn from something.

The original AI models, whether they were open or closed, were trained on previously created knowledge from the internet. Now, AI is generating more content than humans. In a few more years, the internet could be 99% AI-generated content, and that content will have been created by some form of AI.

As a result, AI systems will constantly be distilling knowledge and intelligence from other AI systems. The fact that AI can learn is a good thing. We want AI systems to be intelligent because a smarter AI can also be a safer AI."

----

From "Axios" YouTube channel, (full video link in comment)

[Rajesh Sawhney 🇮🇳 @rajeshsawhney]

Who is the best spiritual guru in today’s India?

I see million babas but no one of class of Osho or Swami Vivekananda or Jiddu Krishnamurti or Sri Aurobindo.

[R @munirishi]

@PositiveGamma Alebit not a single line which hints that they will be positive about Stablecoins 😅

[R @munirishi]

@paraschopra Curious on how did you get the right timings to fetch recording .. Did the camera software automatically gave it? Or manual?

[Ahmad @TheAhmadOsman]

> be you
> random CS grad with 0 clue how LLMs work
> get tired of people gatekeeping with big words and tiny GPUs
> decide to go full monk mode
> 2 years later i can explain attention mechanisms at parties and ruin them

> here’s the forbidden knowledge map
> top to bottom, how LLMs *actually* work

> start at the beginning
> text → tokens
> tokens → embeddings
> you are now a floating point number in 4D space
> vibe accordingly

> positional embeddings:
> > absolute: “i am position 5”
> > rotary (RoPE): “i am a sine wave”
> > alibi: “i scale attention by distance like a hater”

> attention is all you need
> self-attention: “who am i allowed to pay attention to?”
> multihead: “what if i do that 8 times in parallel?”
> QKV: query, key, value
> sounds like a crypto scam
> actually the core of intelligence

> transformers:
> > take your inputs
> > smash them through attention layers
> > normalize, activate, repeat
> > dump the logits
> > congratulations, you just inferred a token

> sampling tricks for the final output:
> > temperature: how chaotic you want to be
> > top-k: only sample from the top K options
> > top-p: sample from the smallest group of tokens whose probabilities sum to p
> > beam search? never ask about beam search

> kv cache = cheat code
> > saves past keys & values
> > lets you skip reprocessing old tokens
> > turns a 90B model from “help me I’m melting” to “real-time genius”

> long context hacks:
> > sliding window: move the attention like a scanner
> > infini attention: attend sparsely, like a laser sniper
> > memory layers: store thoughts like a diary with read access

> mixture of experts (MoE):
> > not all weights matter
> > route tokens to different sub-networks
> > only activate ~3B params out of 80B
> > “only the experts reply” energy

> grouped query attention (GQA):
> > fewer keys/values than queries
> > improves inference speed
> > “i want to be fast without being dumb”

> normalization & activations:
> > layernorm, RMSnorm
> > gelu, silu, relu
> > they all sound like failed Pokémon
> > but they make the network stable and smooth

> training goals:
> > causal LM: guess the next word
> > masked LM: guess the missing word
> > span prediction, fill-in-the-middle, etc
> > LLMs trained on the art of guessing and got good at it

> tuning flavors:
> > finetuning: new weights
> > instruction tuning: “please act helpful”
> > rlhf: reinforcement from vibes and clickbait prompts
> > dpo: direct preference optimization — basically “do what humans upvote”

> scaling laws:
> > more data, more parameters, more compute
> > loss goes down predictably
> > intelligence is now a budget line item

> bonus round:

> quantization:
> > post-training quantization (PTQ)
> > quant-aware training (QAT)
> > models shrink, inference gets cheaper
> > gguf, awq, gptq — all just zip files with extra spice

> training vs inference stacks:
> > deepspeed, megatron, fschat — for pain
> > vllm, tgi, tensorRT-LLM — for speed
> > everyone has a repo
> > nobody reads the docs

> synthetic data:
> > generate your own training set
> > model teaches itself
> > feedback loop of knowledge and hallucination
> > welcome to the ouroboros era

> final boss secret:
> you can learn *all of this* in ~2 years
> no PhD
> no 10x compute
> just relentless curiosity, good bookmarks, and late nights

> the elite don’t want you to know this
> but now that you do
> choose to act
> start now
> build the models

[Malav Shah @malav_shah]

While some PAs push merchants into closed systems and vendor lock-in, Juspay is opening doors with open-source, flexible, and merchant-centric payment solutions—giving merchants the freedom to build on their terms.

[R @munirishi]

@AskPerplexity @CNBCTV18News @latha_venkatesh Can you explain issue in simple terms?

[R P Sharma @rp2sharma]

Worst customer care of railway. No response to tweet, can't talk to operator on 139, no update on website and it is said that railway minister is best.

[Jagdish Sharma @Jagdish40939031]

@AshwiniVaishnaw Honorable Minister, We are delegates of Bharat Vikas Parishad from Jodhpur, attending National Conference at Jalandhar. We have reservation in 19224 JAT GNC on 30.12.24. Looking to the Punjab Bandh. What will be the status of running of this train tomorrow.

[R P Sharma @rp2sharma]

Urgent Around 3500 delegates from across India have gathered in Jalandhar for National convention of Bharat Vikas Parishad and the convention has concluded today. Further most of the delegates are leaving on 30th Dec. So what happen to our trains tomorrow. In view of Punjab bandh

[R @munirishi]

@RailwaySeva @AshwiniVaishnaw @RailMinIndia Team, @AshwiniVaishnaw @RailMinIndia @RailwaySeva awaiting your reply.

Pl guide so that we can make alternate arrangements for traveling train is going to get cancelled

[R @munirishi]

@RailMinIndia @RailwaySeva 100 members of Bharat Vikas Parishad attending their national conference in Jalandhar will be returning back to Jodhpur on 30th Dec. There is a Punjab Bandh on same day and no information has been provided by railways on trains scheduled for 30th (1/2)

[R @munirishi]

@RailwaySeva I have talked to 139 two times for over an hour. No help was extended by them.

Request your urgent support otherwise the passengers will be struck.

@AshwiniVaishnaw @RailMinIndia

[R @munirishi]

@AshwiniVaishnaw @IRCTCofficial Need urgent help here. Group of 100 senior citizens attending Bharat Vikas Parishad conference clueless of train schedule given Punjab Bandh on 30th Dec

[R @munirishi]

@RailMinIndia @RailwaySeva 100 members of Bharat Vikas Parishad attending their national conference in Jalandhar will be returning back to Jodhpur on 30th Dec. There is a Punjab Bandh on same day and no information has been provided by railways on trains scheduled for 30th (1/2)

[R @munirishi]

@rp2sharma @AshwiniVaishnaw @WesternRly #punjabband #railwayhelp #bharatvikasparishad

[R @munirishi]

We have contacted the railway station and local police. No info has been given to us. Kindly inform if the train (19028) will run. My father (@rp2sharma ) is scheduled to be on the the train.

VERY URGENT as ~100 passengers will be struck

Cc: @AshwiniVaishnaw @WesternRly

[R @munirishi]

@i_m_Pania How much would you pay for it per month? Promise - Zero ads, Zero cross sell, Cleanest UX

[R @munirishi]

@mishramohan2 A lot of things in story doesn't sound true.

In my knowledge only way this happens is when phone has a malicious app/keyboard.

[JUSPAY @juspay]

Inherence as a factor will play a big role in coming days.

In this 2nd edition of Juspay Xplore, @munirishi and @Opinion_ishan continue the deep-dive on the topic of payment authentication.

In part 1, they discussed the basic frameworks related to authentication and the different approaches taken around the globe.

In part 2, we look to the future, as we cover
🧱 The Building blocks for Bio-metric authentication
📱 The "Secure Element" in today's phones
🔮 The future of payment authentication

Link to the Part I 👉 https://t.co/leCtvITp5d

#payments #india #business #authentication #paymentsolutions #sea #latam #us #europe #fraud #friction #experience #security