Rahul K

ML Engineer

[Royal Challengers Bengaluru @RCBTweets]

𝐂𝐇𝐀𝐌𝐏𝐈𝐎𝐍𝐒 ⭐️ RCB PLAYED BOLD! 😇

17 Years, 6256 Days, 90,08,640 Minutes later, the wait finally ends. 🙌🤯

The IPL Trophy is finally coming home. And we CANT KEEP CALM! 🤩😍❤️

[ADG PI - INDIAN ARMY @adgpi]

#PahalgamTerrorAttack

Justice is Served.

Jai Hind!

[Rahul K @iamrk_95]

@MatthewBerman Yes!!! Waiting

[Bindu Reddy @bindureddy]

Quantization Is All You Need

SOTA LLMs are too large to run on laptops. Quantization is a technique used to reduce LLMs' computational and memory requirements and create a smaller version of the model. Quantization is central to OSS progress

It involves converting the model's parameters, typically stored as 32-bit floating-point numbers, into lower-precision formats such as 16-bit or even 8-bit integers.

This process significantly decreases the model's size and accelerates inference speed, enabling deployment on resource-constrained devices. Quantization can affect model performance, but the trick is finding a balance between model compression and accuracy, achieved through careful calibration and fine-tuning of the quantization process.

This visual guide is an excellent way to learn about quantization (link in alt)

[Andrej Karpathy @karpathy]

📽️ New 4 hour (lol) video lecture on YouTube:
"Let’s reproduce GPT-2 (124M)"
https://t.co/QTUdu8b0qh

The video ended up so long because it is... comprehensive: we start with empty file and end up with a GPT-2 (124M) model:
- first we build the GPT-2 network
- then we optimize it to train very fast
- then we set up the training run optimization and hyperparameters by referencing GPT-2 and GPT-3 papers
- then we bring up model evaluation, and
- then cross our fingers and go to sleep.
In the morning we look through the results and enjoy amusing model generations. Our "overnight" run even gets very close to the GPT-3 (124M) model. This video builds on the Zero To Hero series and at times references previous videos. You could also see this video as building my nanoGPT repo, which by the end is about 90% similar.

Github. The associated GitHub repo contains the full commit history so you can step through all of the code changes in the video, step by step.
https://t.co/BOzkxQ8at2

Chapters.
On a high level Section 1 is building up the network, a lot of this might be review. Section 2 is making the training fast. Section 3 is setting up the run. Section 4 is the results. In more detail:
00:00:00 intro: Let’s reproduce GPT-2 (124M)
00:03:39 exploring the GPT-2 (124M) OpenAI checkpoint
00:13:47 SECTION 1: implementing the GPT-2 nn.Module
00:28:08 loading the huggingface/GPT-2 parameters
00:31:00 implementing the forward pass to get logits
00:33:31 sampling init, prefix tokens, tokenization
00:37:02 sampling loop
00:41:47 sample, auto-detect the device
00:45:50 let’s train: data batches (B,T) → logits (B,T,C)
00:52:53 cross entropy loss
00:56:42 optimization loop: overfit a single batch
01:02:00 data loader lite
01:06:14 parameter sharing wte and lm_head
01:13:47 model initialization: std 0.02, residual init
01:22:18 SECTION 2: Let’s make it fast. GPUs, mixed precision, 1000ms
01:28:14 Tensor Cores, timing the code, TF32 precision, 333ms
01:39:38 float16, gradient scalers, bfloat16, 300ms
01:48:15 torch.compile, Python overhead, kernel fusion, 130ms
02:00:18 flash attention, 96ms
02:06:54 nice/ugly numbers. vocab size 50257 → 50304, 93ms
02:14:55 SECTION 3: hyperpamaters, AdamW, gradient clipping
02:21:06 learning rate scheduler: warmup + cosine decay
02:26:21 batch size schedule, weight decay, FusedAdamW, 90ms
02:34:09 gradient accumulation
02:46:52 distributed data parallel (DDP)
03:10:21 datasets used in GPT-2, GPT-3, FineWeb (EDU)
03:23:10 validation data split, validation loss, sampling revive
03:28:23 evaluation: HellaSwag, starting the run
03:43:05 SECTION 4: results in the morning! GPT-2, GPT-3 repro
03:56:21 shoutout to llm.c, equivalent but faster code in raw C/CUDA
03:59:39 summary, phew, build-nanogpt github repo

[Elon Musk @elonmusk]

This week, @xAI will open source Grok

[Anthropic @AnthropicAI]

Today, we're announcing Claude 3, our next generation of AI models.

The three state-of-the-art models—Claude 3 Opus, Claude 3 Sonnet, and Claude 3 Haiku—set new industry benchmarks across reasoning, math, coding, multilingual understanding, and vision.

[Santiago @svpino]

Introduction to Linear Algebra for Machine Learning.

This is one of the best resources I've found to learn the algebra behind machine learning.

It's free, the content is great, and it's all in Python.

Link in the ALT of the attached image.

[Santiago @svpino]

There's a massive gap between what books talk about and what engineers spend their time doing.

I don't know if this happens anywhere outside the Machine Learning world.

Most books start with a curated dataset, build a model, and congratulate themselves as soon as it does well on some test data. But that's not where most of the effort goes when working on a production system.

We need more engineers and fewer academics writing books. We need more practical and less theoretical advice in the world.

[Matt Dancho (Business Science) @mdancho84]

It took me 5 years to master all 24 of these machine learning concepts. In the next 24 days, I'll teach them to you one by one (with examples of how I've used them in business cases). Here's what's coming:

1. Linear Regression
2. Clustering
3. Decision Tree
4. Neural Networks
5. Reinforcement Learning
6. Logistic Regression
7. Naive Bayes
8. Supervised Learning
9. Support Vector Machine
10. Probability
11. Random Forest
12. Variance
13. Evaluation Metrics
14. Bagging
15. Data Wrangling
16. Dimensionality Reduction
17. K-nearest Neighbors Algorithm
18. Programming
19. Regularization
20. Statistics
21. Binomial Distribution
22. Bootstrap Sampling
23. Exploratory Data Analysis
24. Data Collection

===

There’s a lot more to learning Data Science for Business. I’d like to help.

I put together a free on-demand workshop that covers the 10 skills that helped me make the transition to Data Scientist: https://t.co/6Ji4GtOTzy

And if you'd like to speed it up, I have a live workshop where I'll share how to use ChatGPT for Data Science: https://t.co/Ydsmzv7trP

If you like this post, please reshare ♻️ it so others can get value.

[Leonie @helloiamleonie]

Retrieval-Augmented Generation (RAG) is so hot that even @TheEconomist mentioned it in their "The World Ahead 2024" issue.

Learn more about RAG: https://t.co/B2d5n535hZ

Economist article: https://t.co/7NyddFoIor

[Santiago @svpino]

This new open-source library is magic!

It helps you detect any of the following issues with Large Language Models:

• Hallucinations
• Misinformation
• Harmfulness
• Stereotypes
• Disclosures
• Prompt injections

The library will help you find problems with your model you didn't know you had. If you try to do this by hand, it'll take you weeks.

Here is how it works:

The library is Giskard.

It's fully open-source. Star the repository here: https://t.co/RebHVwqEsZ.

Here is an example where we detect issues on a Retrieval Augmented Generation (RAG) task using Langchain and OpenAI GPT-3.5. We load a PDF document, ask questions about it, and run a scan to find any vulnerabilities.

https://t.co/anNwvvFqIz

Giskard works with any Python model, including PyTorch, TensorFlow, HuggingFace, Scikit-Learn, XGBoost, and LangChain. They also released integrations with HuggingFace Spaces, Weights & Biases, and MLFlow.

Seeing more and more open-source libraries tackling really complex problems is fantastic!

Thanks to the @giskard_ai team for partnering with me on this post.

Are there any other open-source libraries you'd recommend for machine learning? (Obviously, something different from the usual suspects.)

[Colaboratory @GoogleColab]

You can now safely store your private keys, such as your @huggingface or @kaggle API tokens, in Colab! Values stored in Secrets are private, visible only to you and the notebooks you select.

[Ministry of Information and Broadcasting @MIB_India]

BHASHa INterface for India: Bridging the Digital Divide with #Bhashini!🌐📚🇮🇳

Bhashini, the AI-driven language translation platform, is on a mission to enable every Indian citizen to access the nation's digital advancements in their own language, fostering true digital inclusion.📱🗣️

@AshwiniVaishnaw @Rajeev_GoI @GoI_MeitY @MeityPib

[Santiago @svpino]

Not using AI as a developer doesn't make you smart. It makes you unemployed.

[Niall Ratcliffe @NiallRatcliffe]

Everyone sees Steve Jobs as this revolutionary tech guy.

But he's taught me more about marketing than any "marketing guru" ever has.

Here are 7 lessons on marketing from the legend that is Steve Jobs:

[Dan Kornas @DanKornas]

📚 Stanford CS230: Deep Learning by Andrew Ng (@AndrewYNg)

You will learn about Convolutional networks, RNNs, LSTM, Adam, Dropout, BatchNorm, Xavier/He initialization, and more.

https://t.co/XLp5t1E3Pl

[Santiago @svpino]

AI progress doesn't sleep at night!

A new state-of-the-art speech recognition model was just released: Conformer-2 is now available!

I covered the first version of this model a few months ago. Conformer-1 was trained on 650,000 hours of audio, which now sounds like nothing next to the 1.1M hours of training audio data of Conformer-2!

The Conformer architecture is fascinating.

The Google Brain team combined transformers and convolutional neural networks to get the best of both worlds: Transformers are good at capturing global interactions in the data, while a CNN knows how to exploit local features.

The result is an architecture that can efficiently model an audio sequence's local and global dependencies, and that's the foundation of Conformer-2.

To understand how good the model is, go to the playground linked below and try a YouTube video. I have an accent, and the model transcribes my videos with no issues.

Compared to the first version, Conformer-2 is more accurate and robust, especially in processing noisy audio and dealing with the names of people, places, and things.

The team @AssemblyAI is behind Conformer-2. They partnered with me on this post. Conformer-2 is available in their API, and you can try it for free on their playground.

[Ed Krassenstein @EdKrassen]

Today Elon Musk held a Twitter Space to discuss xAI.

Here is a recap of what was discussed for those who missed it:

- The founding team was on hand to introduce themselves, and I must say it is an impressive team with an impressive background. They had very strong backgrounds with Deep Mind, OpenAI, Google, Tesla, etc.
- Elon Musk said the goal with xAI is to build a good AGI (artificial general intelligence) with the purpose of understanding the universe.
- Musk said that the safest way is to build an AGI that is ‘maximum curious’ and ‘truth curious,’ and to try and minimize the error between what you think is true and what is actually true.
- For truth-seeking super intelligence humanity is much more interesting than not humanity, so that’s the safest way to create one. Musk gave the example of how space and Mars is super interesting but it pales in comparison to how interesting humanity is.
- Musk said there is so much that we think we understand but we don’t in reality. There are a lot of unresolved questions. For example, there are many questions that remain about the nature of gravity, and why there is not massive evidence of aliens. He said he has seen no evidence of aliens whatsoever so far. He went further into the Fermi Paradox and how it's possible that other consciousness may not exist in our galaxy.
- If you ask today’s advanced AIs technical questions, you just get nonsense, so Musk believes we are really missing the mark by many orders of magnitude and that needs to get better.
- xAI will use heavy computing, but the amount of ‘brute force’ will become less as they become to understand the problem better.
- Co-Founder Greg Yang said that the mathematics they find at xAi could open up new perspectives to existing questions like the 'Theory of Everything.'
- Elon stated that you can't call anything AGI until the computer solves at least one fundamental question.
- He said that from his experience at Tesla, they have over complicated problems. “We are too dumb to realize how simple the answers really are," he said. "We will probably find this out with AGI as well. Once AGI is solved, we will look back and think, why did we think it would be so hard.”
- They are going to release more information on the first release of xAI in a couple more weeks.
- Elon Musk said that xAI is being built as competition to OpenAI, when asked by @krassenstein.
- The goal is to make xAI a useful tool for consumers and businesses and there is value in having multiple entities and competition. Elon said that competition makes companies honest, and he’s in favor of competition.
- Musk said every organization doing AI has illegally used Twitter’s data for training. Limits had to be put on Twitter because they were being scraped like crazy. Multiple entities were trying to scrape every tweet ever made in a span of days. xAI will use tweets as well for training.
- At some point you run out of human-created data. So eventually AI will have to generate its own content and self-access that content.
- Answering a question from @alx, Musk said there is a significant danger in training AI to be politically correct or training it not to say what it thinks is true, so at xAI they will let the AI say what it believes to be true, and Musk believes it will result in some criticism.
- Musk said it’s very dangerous to grow an AI and teach it to lie.
- Musk said he would accept a meeting with Kamala Harris if invited. He said he’s not sure if Harris is the best person to be the AI czar, but agrees we need regulatory oversight.
- Musk believes that China too will have AI regulation. He said the CCP doesn’t want to find themselves subservient to a digital super intelligence.
- Musk believes we will have a voltage transformer shortage in a year and electricity shortage in 2 years.
- xAI will work with Tesla in multiple ways and it will be of mutual benefit. Tesla’s self-driving capabilities will be enhanced because of xAI.
- According to Musk, the proper way to go about AI regulations is to start with insight. If a proposed rule is agreed upon by all or most parties then that rule should be adopted. It should not slow things down for a great amount of time. A little bit of slowing down is OK if it's for safety.
- Musk thinks that Ray Kurzweil's prediction of AGI by 2029 is pretty accurate, give or take a year.

I've love to hear everyone's thoughts on where you think xAI will go.

[Santiago @svpino]

You can now fine-tune an open-source LLM without writing a single line of code.

This is unprecedented.

No-code fine-tuning is a breakthrough in the open-source world, and it will help companies adopt AI at lightning speed.

Let me show you how you can do this.

I taught a model to recognize negative content on Twitter. Setting everything up took me 10 minutes. It took another 45 minutes to have the model ready.

If you aren't familiar with the term "fine-tuning," it’s the process we use to teach a model how to solve a specific task. Large Language Models have general knowledge but struggle to solve particular problems.

Fortunately, we can fine-tune these models and make them very good at solving specific tasks. In this example, that task is to analyze the sentiment of tweets.

But there’s a massive problem:

Fine-tuning a model is a complex, expensive process. It takes a lot of time, effort, and GPU computing. It's also hard to find experienced people who know how to do it.

The team @monsterapis built the first platform that offers no-code fine-tuning of open-source models, which changes everything. That’s what I’m using here.

Here is what you need to do:

1. Sign up here: https://t.co/HaeAxyM3be, and use the code SANTIAGO during your purchase to get an 80% discount.

2. Go to the FineTuning option and select your model. I'm using the Falcon 7B model, but you can pick any of the following options:

• Falcon 7B
• LLaMA 7B
• Open LLaMA 3B, 7B
• OPT 125M, 350M, 1.3B, 2.7B, 6.7B
• GPT J 6B
• Stable LM 3B, 7B
• GPT 2 XL

3. Select your task. I'm using "Text Classification" since we want to classify different tweets as Positive or Negative.

4. The last step is to select your dataset. I used a twitter-sentiment-analysis dataset from HuggingFace. They have data for almost anything you can think of, but you can also upload your own.

I didn't change any of the default hyperparameters, and 45 minutes later, I had my fine-tuned version of Falcon 7B ready to go! I spent 2,320 credits in the process, equivalent to $2.50. A fine-tuned state-of-the-art model for the price of a cup of coffee!

That’s one of the @monsterapis’ advantages: Besides not dealing with code, complexity, or hardware, their pricing is very competitive, thanks to their decentralized GPU platform.