Fiaz Riaz

Data Scientist, Machine Learning and Deep Learning practitioner.

[Fiaz Riaz @fiazriaz]

وه که گر من بازبینم روئے یار خویش را
تا قیامت شکر گویم کردگار خویش را

[Yann LeCun @ylecun]

I've made that point before:
- LLM: 1E13 tokens x 0.75 word/token x 2 bytes/token = 1E13 bytes.
- 4 year old child: 16k wake hours x 3600 s/hour x 1E6 optical nerve fibers x 2 eyes x 10 bytes/s = 1E15 bytes.

In 4 years, a child has seen 50 times more data than the biggest LLMs.

1E13 tokens is pretty much all the quality text publicly available on the Internet. It would take 170k years for a human to read (8 h/day, 250 word/minute).

Text is simply too low bandwidth and too scarce a modality to learn how the world works.

Video is more redundant, but redundancy is precisely what you need for Self-Supervised Learning to work well.

Incidentally, 16k hours of video is about 30 minutes of YouTube uploads.

[Fiaz Riaz @fiazriaz]

The emotional intensity inside must burn hot enough to protect you against the chilling effects of doubt, uncertainty, criticism, and failure. Only deep desire can generate such heat.

#FridayMotivation

[Andrew Ng @AndrewYNg]

After reading the @nytimes lawsuit against @OpenAI and @Microsoft, I find my sympathies more with OpenAI and Microsoft than with the NYT.

The suit:
(1) Claims, among other things, that OpenAI and Microsoft used millions of copyrighted NYT articles to train their models
(2) Gives examples in which OpenAI models regurgitated NYT articles almost verbatim

But the presentation muddies (1) and (2), and I saw a lot of commentary on social media that -- because of what I believe is a muddied presentation -- draws a link between them that I'm not sure is what people think it is.

On (1): I understand why media companies don't like people training on their documents, but believe that just as humans are allowed to read documents on the open internet, learn from them, and synthesize brand new ideas, AI should be allowed to do so too. I would like to see training on the public internet covered under fair use -- society will be better off this way -- though whether it actually is will ultimately be up to legislators and the courts.

On (2): I suspect a lot of the examples of ChatGPT regurgitating articles nearly verbatim were due to a RAG-like mechanism where the user prompt causes the system to browse the web, retrieve a specific article and then print it out. (If my statement here isn't accurate, I would love to see the @nytimes clarify this.) If this is the case, then (i) To OpenAI's credit, they seem to have already updated their software to make this much less likely, and (ii) This is also a much easier problem to fix than if an LLM were to regurgitate text using only the pre-trained weights, which AFAIK very rarely happens (and which, given its rarity, also raises the question of how much harm to NYT this has actually caused).

To be clear, I believe independent media is important for democracy and must be protected. I also sympathize with media businesses worried about Generative AI disrupting their businesses. But I'm not convinced the NYT lawsuit is the right way to do this.

Usual caveat: I am not a lawyer and am not giving legal advice or any other form of advice here.

You can also read more details of my take on this below. https://t.co/wkZSMHsvNA

[Google DeepMind @GoogleDeepMind]

A common problem faced by researchers is finding and using information from scientific papers. 🔍

Gemini's capabilities mean it can accurately uncover knowledge within thousands of files including dense text & complex images.

How could this help deliver breakthroughs faster? ↓

[Google DeepMind @GoogleDeepMind]

We’re excited to announce 𝗚𝗲𝗺𝗶𝗻𝗶: @Google’s largest and most capable AI model.

Built to be natively multimodal, it can understand and operate across text, code, audio, image and video - and achieves state-of-the-art performance across many tasks. 🧵 https://t.co/mwHZTDTBuG

[Rumi @Rumi_Quote]

"Travel brings power and love back into your life."

[Fiaz Riaz @fiazriaz]

So often people are working hard at the wrong thing. Working on the right thing is probably more important than working hard.

#Entrepreneurship #leadership #startup

[Fiaz Riaz @fiazriaz]

“I shall pass this way but once; any good, therefore, that I can do or any kindness that I can show to any human being, let me do it now. Let me not defer nor neglect it, for I shall not pass this way again.”

#Gratitude #LifeLessons

[Fiaz Riaz @fiazriaz]

Sharp criticisms and rebukes almost invariably end in futility.

#LeadershipDevelopment

[Tony Robbins @TonyRobbins]

Love is the oxygen of the soul. ❤️

[Fiaz Riaz @fiazriaz]

@philipvollet @weaviate_io @aestheticedwar1 Good luck 🤞

[OpenAI @OpenAI]

GPT-4 and GPT-3.5 Turbo models in the API now support calling your custom functions, allowing the model to use tools you design for it. Also — reduced pricing & new model versions (including 16k context for 3.5 Turbo): https://t.co/dalfgEQ9k2

[Stanford Online @StanfordOnline]

Prof @chrmanning talks w/ @AndrewYNg re how #NLP may evolve for better understanding of subtleties of human language. Curious about #AI? See the updated @Coursera Machine Learning Specialization 👉 https://t.co/Y1a8BR9jHJ @DeepLearningAI_ @StanfordAILab @StanfordHAI @stanfordnlp

[Fiaz Riaz @fiazriaz]

Love machines but love human beings more. Be grateful.

#fridaythoughts #humanity #gratitude

[Fiaz Riaz @fiazriaz]

🚀 Check out the full article for detailed steps and resources to guide you on your data science journey:

https://t.co/QSJw9bfoxP

#DataScience #DataScienceHelp #MachineLearning

[Fiaz Riaz @fiazriaz]

📢 Attention aspiring data scientists! Ready to become a data scientist? Follow this roadmap 🗺️:
1. Lay the Foundation
2. Master Data Analysis
3. Learn Machine Learning
4. Dive into Deep Learning
.
.
.
👇🧵

[Fiaz Riaz @fiazriaz]

📊 Remember, the choice of activation function depends on your specific task and network architecture.

Experimentation and careful consideration of the task at hand will guide you towards the optimal choice. 🧪

Have any questions or thoughts, drop them in the comments.🤗💬

[Fiaz Riaz @fiazriaz]

📣 The selection of Activation Functions:
Sigmoid, Tanh, ReLU, and Leaky ReLU💥

Let's see when and why you should prefer each of them! 💪

Thread🧵

#ActivationFunctions #DeepLearning #NeuralNetworks

[Fiaz Riaz @fiazriaz]

In summary:
- Sigmoid: Suitable for binary classification and when you need values between 0 and 1.
- Tanh: Useful when you need values between -1 and 1.
- ReLU: Ideal for deep networks, due to its computational efficiency.
- Leaky ReLU: Effective when dealing with dead neurons.