aadesh⚡

Somewhere, something incredible is waiting to be known!🌠🌌

[SpaceX @SpaceX]

Mechazilla has caught the Super Heavy booster!

[Tarun Mehta @tarunsmehta]

Entrepreneurs need to target an RnD of a million engineers in India.

Apparently BYD now has more people in R&D than the rest of the auto industry outside of China, COMBINED (https://t.co/XpCDgvqkeW)

The most important progress is technological progress and China, for all its faults, is doing a damn good job on this. BYD is the largest EV company and that's because they are all in. Just look at the freaking numbers:

An RnD team size approaching 100K people with some 20-30K engineers hired last year alone! Almost 10% of their revenue is spent on RnD, even at the scale of $40-50B in annual revenues.
Those are gargantuan numbers.

As a country we need to focus a lot on the numbers here. No one can otherwise beat a 10-100X numbers advantage.

Given the demographic advantage we are sitting on, our policy focus on RnD needs to go up in an immense way. Remember, suppliers come and setup capacities where OEMs produce. OEMs produce where their new product development (NPD) happens. And NPD happens where there are engineers. That's why China today wins the NPD game so that it can continue to maintain it's eventual manufacturing advantage.

Indian companies are often quite happy seeing a 3-4% RnD spend. If we have a 10-20X growth ahead of us for the next few decades then we need to target big numbers everywhere:
1. 10-15% RnD spend in all critical sectors.
2. A million engineers in all critical sectors.

In fact, a million engineers for an incredibly focused product company out of India itself won't be crazy and almost certain to happen in the future. That ought to translate into a $40-50B spend per year. That's roughly comparable with the RnD spends of the big tech players in US TODAY.

There are companies ALREADY spending more than that today. BYD is already at 10% of that headcount in China.

Throw enough engineering muscle at a problem. Eventually progress.

[Vipul Vaibhaw @vaibhaw_vipul]

"I was just an ordinary person who studied hard" -
https://t.co/bBQXqJRr1a

“Study hard what interests you the most in the most undisciplined, irreverent and original manner possible. - Richard Feynman

[aadesh⚡ @aadesh_gi]

@maaniblr @Conkreta1 @x_rahulraj @jogakhichudi Practical + Project which was mentioned in the original tweet is what happens at IIITH, without it, it is not possible to survive here.

[aadesh⚡ @aadesh_gi]

@LakshakarAyush @karpathy Time to code it now! kal aajao workspace🥸

[Andrej Karpathy @karpathy]

With many 🧩 dropping recently, a more complete picture is emerging of LLMs not as a chatbot, but the kernel process of a new Operating System. E.g. today it orchestrates:

- Input & Output across modalities (text, audio, vision)
- Code interpreter, ability to write & run programs
- Browser / internet access
- Embeddings database for files and internal memory storage & retrieval

A lot of computing concepts carry over. Currently we have single-threaded execution running at ~10Hz (tok/s) and enjoy looking at the assembly-level execution traces stream by. Concepts from computer security carry over, with attacks, defenses and emerging vulnerabilities.

I also like the nearest neighbor analogy of "Operating System" because the industry is starting to shape up similar:
Windows, OS X, and Linux <-> GPT, PaLM, Claude, and Llama/Mistral(?:)).
An OS comes with default apps but has an app store.
Most apps can be adapted to multiple platforms.

TLDR looking at LLMs as chatbots is the same as looking at early computers as calculators. We're seeing an emergence of a whole new computing paradigm, and it is very early.

[Andrej Karpathy @karpathy]

The Transformer is a magnificient neural network architecture because it is a general-purpose differentiable computer. It is simultaneously:
1) expressive (in the forward pass)
2) optimizable (via backpropagation+gradient descent)
3) efficient (high parallelism compute graph)

[aadesh⚡ @aadesh_gi]

@LakshakarAyush Here have a look at this amazing summary too!
https://t.co/3IbntmbGLC

[Trendulkar @Trendulkar]

Ashes &gt;&gt;&gt;

[aadesh⚡ @aadesh_gi]

The MLIR project for program understanding and RL4Real, a reinforcement learning approach to register allocation, were featured at the talk. #ML #compiler #optimization #LLVM #LLVMHyderabad

[aadesh⚡ @aadesh_gi]

Prof. U. Ramakrishna from @IITHyderabad spoke about ML-driven compiler optimizations with IR2Vec at a talk arranged by LLVM(@llvmorg ) Hyderabad at @iiit_hyderabad.

[aadesh⚡ @aadesh_gi]

Using distributed encodings and an agglomerative technique, IR2Vec generates application-independent program representations with potential applications in register allocation, voltage/frequency scaling, and algorithm recognition.

[Peter Nixey @peternixey]

I'm in the top 2% of users on StackOverflow. My content there has been viewed by over 1.7M people. And it's unlikely I'll ever write anything there again.

Which may be a much bigger problem than it seems. Because it may be the canary in the mine of our collective knowledge.

A canary that signals a change in the airflow of knowledge: from human-human via machine, to human-machine only. Don’t pass human, don’t collect 200 virtual internet points along the way.

StackOverflow is *the* repository for programming Q&A. It has 100M users & saves man-years of time & wig-factories-worth of grey hair every single day.

It is driven by people like me who ask questions that other developers answer. Or vice-versa. Over 10 years I've asked 217 questions & answered 77. Those questions have been read by millions of developers & had tens of millions of views.

But since GPT4 it looks less & less likely any of that will happen; at least for me. Which will be bad for StackOverflow. But if I'm representative of other knowledge-workers then it presents a larger & more alarming problem for us as humans.

What happens when we stop pooling our knowledge with each other & instead pour it straight into The Machine? Where will our libraries be? How can we avoid total dependency on The Machine? What content do we even feed the next version of The Machine to train on?

When it comes time to train GPTx it risks drinking from a dry riverbed. Because programmers won't be asking many questions on StackOverflow. GPT4 will have answered them in private. So while GPT4 was trained on all of the questions asked before 2021 what will GPT6 train on?

This raises a more profound question. If this pattern replicates elsewhere & the direction of our collective knowledge alters from outward to humanity to inward into the machine then we are dependent on it in a way that supercedes all of our prior machine-dependencies.

Whether or not it "wants" to take over, the change in the nature of where information goes will mean that it takes over by default.

Like a fast-growing Covid variant, AI will become the dominant source of knowledge simply by virtue of growth. If we take the example of StackOverflow, that pool of human knowledge that used to belong to us - may be reduced down to a mere weighting inside the transformer.

Or, perhaps even more alarmingly, if we trust that the current GPT doesn't learn from its inputs, it may be lost altogether. Because if it doesn't remember what we talk about & we don't share it then where does the knowledge even go?

We already have an irreversible dependency on machines to store our knowledge. But at least we control it. We can extract it, duplicate it, go & store it in a vault in the Arctic (as Github has done).

So what happens next? I don't know, I only have questions.

None of which you'll find on StackOverflow.

(I write on AI from a technical and product perspective. If you find that interesting then please do follow me for more)

[aadesh⚡ @aadesh_gi]

@jainsahab___ @RaviRanjanIn I still have it back home! Pure Nostalgia! ♥️ 🥹 Some things are meant to be stolen! ♥️ ♥️

[aadesh⚡ @aadesh_gi]

@RajaswaPatil Congratulations bro! 🎊 💐 and best wishes for future!

[aadesh⚡ @aadesh_gi]

@code Cool! Adding terminal support would make it much better!!

[Sumeet Jain @jainsahab___]

@indiantweeter If you could help one more... I am a Fresh Chemical Engineer graduate looking for job. Right now in my home town can shift anywhere in the country.
Thanks in Advance.

[Balaji @balajis]

If you’ve been fortunate enough to do well this year, consider joining me and @VitalikButerin by donating at the addresses below.

But if all you have is Twitter, help spread the word. For every RT, I’ll donate another $50 to fight COVID in India, up to $100k. #cryptovscovid

[Rahul Gandhi @RahulGandhi]

In view of the Covid situation, I am suspending all my public rallies in West Bengal.

I would advise all political leaders to think deeply about the consequences of holding large public rallies under the current circumstances.

[Trisha Shetty @TrishaBShetty]

These political party apologists make it sound like if you dare complain as a citizen - you’re lazy/ entitled and don’t know how to be part of the solution - actually NO.
We pay taxes, we elect you to do your job. Not shame us for daring to question your ineptitude, latency.