Dr. H. Al-Jamimi

اكاديمي وباحث
حساب شخصي يهتم بمستجدات البحث العلمي وقياس الناتج و الاثر
A personal account concerned with the research impact & latest developments

[Dr. H. Al-Jamimi @DrJamimi]

Nice shot 😀

SV1118 @Saudi_Airlines

[Ajit kumar @ajitcodes]

Learn AI for free directly from top companies.

1 - Anthropic:
https://t.co/1M15GC7cOY

2 - Google:
https://t.co/AIkLM6XjVL

3 - Meta:
https://t.co/UWdSCK5daz

4 - NVIDIA:
https://t.co/B458g8uF1u

5 - Microsoft:
https://t.co/UChXyK2ZQh

6 - OpenAI:
https://t.co/Do088drBz6

7 - IBM:
https://t.co/4XKRjVNMKA

8 - AWS:
https://t.co/36WQ7H0vfT

9 - https://t.co/saSYDpvXFf:
https://t.co/mtt2yYxm6a

10 - Hugging Face:
https://t.co/YYHwWDpwPF

👇Comment "Learning" if you find this helpful.

Repost so others can take help.

Must bookmark for future reference.

[Avi Chawla @_avichawla]

I have been fine-tuning LLMs for over 2 years now!

Here are the top 5 LLM fine-tuning techniques, explained with visuals:

First of all, what's so different about LLM finetuning?

Traditional fine‑tuning is impractical for LLMs (billions of params; 100s GB).

Since this kind of compute isn't accessible to everyone, parameter-efficient finetuning (PEFT) came into existence.

Before we go into details of each technique, here's some background that will help you better understand these techniques:

LLM weights are matrices of numbers adjusted during finetuning.

Most PEFT techniques involve finding a lower-rank adaptation of these matrices, a smaller-dimensional matrix that can still represent the information stored in the original.

Now with a basic understanding of the rank of a matrix, we're in a good position to understand the different finetuning techniques.

(refer to the image below for a visual explanation of each technique)

1) LoRA

- Add two low-rank trainable matrices, A and B, alongside weight matrices.
- Instead of fine-tuning W, adjust the updates in these low-rank matrices.

Even for the largest of LLMs, LoRA matrices take up a few MBs of memory.

2) LoRA-FA

While LoRA significantly decreases the total trainable parameters, it requires substantial activation memory to update the low-rank weights.

LoRA-FA (FA stands for Frozen-A) freezes matrix A and only updates matrix B.

3) VeRA

- In LoRA, low-rank matrices A and B are unique for each layer.
- In VeRA, A and B are frozen, random, and shared across all layers.
- Instead, it learns layer-specific scaling VECTORS (b and d) instead.

4) Delta-LoRA

- It tunes the matrix W as well, but not in the traditional way.
- Here, the difference (or delta) between the product of matrices A and B in two consecutive training steps is added to W.

5) LoRA+

- In LoRA, both matrices A and B are updated with the same learning rate.
- Authors of LoRA+ found that setting a higher learning rate for matrix B results in better convergence.
____
Find me → @_avichawla
Every day, I share tutorials and insights on DS, ML, LLMs, and RAGs.

[Kirk Borne @KirkDBorne]

#MachineLearning for #TimeSeries with #Python — Forecast trends, Predict the future, Detect anomalies with state-of-the-art #ML methods: https://t.co/LWJuB7Zc3M by @benji1a
——————
#DataScience #AI #Forecasting #PredictiveAnaytics #AnomalyDetection #IoT #IIoT #DataScientist

[Kirk Borne @KirkDBorne]

Unlocking Data with Generative AI and RAG — Enhance Generative #AI systems by integrating internal data with large language models using RAG: https://t.co/IVsKOlEMP1 v/ @PacktDataML
—————
#GenAI #LLMs #MachineLearning #KnowledgeGraph #DataScience #DataScientist

[Dr. Ganapathi Pulipaka 🇺🇸 @gp_pulipaka]

Graph Neural Networks in Action! #BigData #Analytics #DataScience #AI #MachineLearning #IoT #IIoT #PyTorch #Python #RStats #TensorFlow #Java #JavaScript #ReactJS #GoLang #CloudComputing #Serverless #DataScientist #Linux #Books #Programming #Coding #100DaysofCode
https://t.co/m7aByxyf4n

[Kirk Borne @KirkDBorne]

The Artificial Intelligence of Things #AIoT: https://t.co/HTYhOMjZuc

AIoT book v/ @PacktDataML → Hands-On #AI for #IoT — Expert #MachineLearning & #DeepLearning techniques for developing smarter IoT systems [2nd Ed.]: https://t.co/36PCOv9cLc

𝓚𝓮𝔂 𝓕𝓮𝓪𝓽𝓾𝓻𝓮𝓼:
🔴Leverage the power of Python libraries such as TensorFlow and Keras to work with real-time IoT data

🔵Enhance your IoT solutions with advanced AI techniques, including deep learning, optimization, and generative adversarial networks

🟢Gain practical insights through industry-specific IoT case studies in manufacturing, smart cities, and automation

🔴Purchase of the print or Kindle book includes a free PDF eBook

[Kirk Borne @KirkDBorne]

💥Hot💥New Release from @PacktPublishing @PacktDataML

"The AI Optimization Playbook: Drive business success with proven AI strategies, best practices, and responsible innovation"

See it at https://t.co/PLHC63UK1M

𝕋𝕒𝕓𝕝𝕖 𝕆𝕗 ℂ𝕠𝕟𝕥𝕖𝕟𝕥𝕤:
🔷Understanding the Perils of AI Products
🔶Building the Enterprise AI Strategy
♦️Selecting High-Impact AI Projects
🔷Beyond the Build: Gaining Leadership Support for AI Initiatives
🔶Building an AI Proof of Concept and Measuring Your Solution
♦️Beyond Accuracy: A Guide to Defining Metrics for Adoption
🔷From Model to Market: Operationalizing ML Systems
🔶From Metrics to Measurement: Experimentation and Causal Inference
♦️Generative AI in the Enterprise: Unlocking New Opportunities
🔷Understanding GenAI Operations
🔶AI Agents Explained
♦️Introduction to Responsible AI
🔷Implementing RAI Frameworks, Metrics, and Best Practices
🔶Building Trustworthy LLMs and Generative AI
♦️Regulatory and Legal Frameworks for Responsible AI
🔷The Future of AI Optimization: Trends, Vision, and Responsible Implementation

[Oliver Prompts @oliviscusAI]

Microsoft killed the GPU mafia 🤯

They finally open-sourced their 1-bit LLM inference framework called bitnet.cpp. It lets you run 100B parameter models on your local CPU without GPUs.

- 6.17x faster inference
- 82.2% less energy on CPUs

100% Open Source.

[Dr. H. Al-Jamimi @DrJamimi]

While PostgreSQL has thrived in recent years, MySQL has languished. MySQL backers are coming together to change that. https://t.co/slEPWGdCkZ

[Kirk Borne @KirkDBorne]

<<💡Best Seller🚀>>
Build a Large Language Model (From Scratch): https://t.co/LaARZ6CjFF by @rasbt v/ @ManningBooks
—————
#DataScience #DataScientist #MachineLearning #ML #DeepLearning #LLMs #AI #GenAI

[Dr. H. Al-Jamimi @DrJamimi]

From fine-tuning open source models to building agentic frameworks on top of them, the open source world is ripe with projects that support AI development.

https://t.co/Hmq20APPeJ

[Dr. H. Al-Jamimi @DrJamimi]

How to lead without burnout: strategies for middle leaders

https://t.co/gZ85fUJpZL

[Dr. H. Al-Jamimi @DrJamimi]

A quantum computer goes to space

#quantum_quantum

https://t.co/45OwfsqFG8

[Dr. H. Al-Jamimi @DrJamimi]

@Dr_Ahmad_Mugbil @KSGAFAL ربنا يحفظك وينفع بك
بالتوفيق ان شاء الله تعالى

[Dr. H. Al-Jamimi @DrJamimi]

@Dr_Ahmad_Mugbil @KSGAFAL ما شاء الله تبارك الرحمن

[Dr. H. Al-Jamimi @DrJamimi]

Do AI models produce more original ideas than researchers? https://t.co/yZx9Ltb9QZ

[Dr. Mohamed Qasem د. محمد قاسم @mqasem]

واو!!! شركة OpenAI تطلق ChatGPT o1، وهو متقدم جدا، يبدو أنه قادر على أن يتحدى حتى البشر في أسئلة بمستوى الدكتوراة. الذكاء الاصطناعي الجديد يفكر قبل أن يجيب على السؤال، ويأخذ وقتا طويلا.

قبل قليل دخلت على حسابي وكان موجودا لدي.

[Dr. H. Al-Jamimi @DrJamimi]

The citation black market: schemes selling fake references alarm scientists https://t.co/p1iNWRQEbz

[Dr. H. Al-Jamimi @DrJamimi]

Future of AI key focus of top global summit in Riyadh https://t.co/aDKFuDaVLz

[Dr. H. Al-Jamimi @DrJamimi]

https://t.co/z6hUGUUqQa