Here's my beginner's lecture series for RAG, Vector Database, Agent, and Multi-Agents: Download slides: 👇
* RAG: https://t.co/O3OvTXVWUI
* Agents: https://t.co/BeJedAKwC9
* Vector Database: https://t.co/UZw5Lieszp
* Multi-Agents: https://t.co/iEMJKeMYAE
---
100% original, made by hand ✍️
Join 47K+ readers of my newsletter: https://t.co/fFt8roc8D9
الخبير في الذكاء الاصطناعي عبد المنعم درويش يحذّر: نماذج ChatGPT لا توفّر الخصوصية الكافية مقارنة بأدوات ذكاء اصطناعي أخرى أكثر أمانًا للمستخدمين
#صباح_العربية
انضم إلى لقاء مميز بالتعاون مع @TheGarageKSA لاكتشاف أثر تقنيات الثورة الصناعية الرابعة على تحسين وتطوير العمليات التصنيعية، من خلال استعراض استراتيجيات التصنيع والتحديات المرتبطة بتطبيق هذه التقنيات في المصانع.
#تنمية_بالتقنية
الخميس | 01 مايو
7:00 - 8:00 مساءً
الرياض - الكراج
للتسجيل: https://t.co/XxPWau48Ed
الشيف خوسيه أندريس مؤسس المطبخ المركزي العالمي بشن هجوما كاسحا على اسرائيل ويتهمها بقتل عمال الإغاثة السبعة عمداً"ويقول إن الصراع في غزة "لم يعد حرباً ضد الإرهاب" بل هي "حرب ضد الإنسانية نفسها" تصريحات خوسيه أصابت الصهاينة في مقتل لأنها بثت في انحاء العالم، جرئم اسرائيل تجاوزت الفلسطينيين وأصبحت ضد الانسانية #غزه_تقاوم_وتنتصر #رمضان
I want to show you a clever trick you didn't know before.
Imagine you have six months' worth of data. You want to build a model, so you take the first five months to train it. Then, you use the last month to test it.
This is a common approach for building machine learning models.
Unfortunately, you may find out your model works well with the train data but sucks on the test data.
Overfitting is not weird. We've all been there. But often, the worst you can do is try and fix it before understanding why it’s happening.
Ask anyone about this, and they will give you their favorite step-by-step guide on regularizing a model. They will jump right in and try to fix overfitting. Don’t do this.
There's a different way. A better way.
Here is the question I want you to answer before you start racking your brain trying to fix a model:
Do your test and training data come from the same distribution?
When building a model, we assume the train and test come from the same place. Unfortunately, this is not always the case.
Here is where the trick I promised comes in:
1. Put your train and test set together.
2. Get rid of the target column.
3. Create a new binary feature, and set every sample from your train set to 0 and every sample from the test set to 1. This feature will be the new target.
Now, train a simple binary classification model on this new dataset. The goal of this model is to predict whether a sample comes from the train or the test split.
The intuition behind this idea is simple:
If all your data comes from the same distribution, this model won't work. But if the data comes from different distributions, the model will learn to separate it.
After you build a model, you can use the ROC-AUC to evaluate it. If the AUC is close to 0.5, your model can't separate the samples. This means your training and test data come from the same distribution. If the AUC is closer to 1.0, your model learned to differentiate the samples. Your training and test data come from different distributions.
This technique is called Adversarial Validation. It's a clever, fast way to determine whether two datasets come from the same source.
If your splits come from different distributions, you won't get anywhere. You can't out-train bad data.
But there's more!
You can also use Adversarial Validation to identify where the problem is coming from:
1. Compute the importance of each feature.
2. Remove the most important one from the data.
3. Rebuild the adversarial model.
4. Recompute the ROC-AUC again.
You can repeat this process until the ROC-AUC is close to 0.5 and the model can’t differentiate between training and test samples.
Adversarial Validation is especially useful in production applications to identify distribution shifts.
Low investment with a high return.
حلقتنا في #تيك_ناو غداً
الذكاء الاصطناعي التوليدي: الإبداع الآلي في عصر التقنية الرقمية
نناقشها مع
م. عبدالمنعم درويش، رائد أعمال و متخصص في الذكاء الاصطناعي
انتظرونا في 9 صباحاً
للتواصل 0115100100
#العربيةFM#بالصوت_يصلكم_الخبر_وأكثر
في #تيك_ناو
التحليل السلوكي والتنبؤ بالمستقبل باستخدام الذكاء الاصطناعي
مع..
م. عبدالمنعم درويش، رائد أعمال و متخصص في الذكاء الاصطناعي
انتظرونا في 9 صباحاً
للتواصل 0115100100
#العربيةFM#بالصوت_يصلكم_الخبر_وأكثر
LoRA is a genius idea.
To understand the fine-tuning of Large Language Models, you must understand how LoRA works.
By the end of this post, you'll know everything important about how it works.
Large Language Models are good generalists, but they have little specialization. We train them in many different tasks, so they know a bit about everything but not enough about anything.
Think of a kid who can play three different sports at a high level. While he can be proficient across the board, he won't get a scholarship unless he specializes. That's how the kid can reach his full potential.
We can do the same with these large models. We can train them to solve a particular task and nothing else.
We call this process "fine-tuning." We start with everything the model knows and adjust its knowledge to help it improve on the task we care about.
Fine-tuning is revolutionary, but it's not free.
Fine-tuning a large model takes time, care, and lots of money. Many companies can't afford the process. Some can't pay for the hardware. Some can't hire people who know how to do it. Most companies can't do either.
That's where LoRA comes in.
We realized we could approximate a large matrix of parameters using the product of two smaller matrices. There was a lot of wasted space within these large models. What would happen if we find a new, more optimal representation?
Did you ever buy a map at a gas station? Giant pages showing every small road, path, and lake around you. They were exhaustive but hard to navigate. These are like parameters in a large model.
LoRA turns a gas station map into a cartoon treasure map. Every useless parameter is gone. Only two roads, a palm tree, and a cross pointing at the treasure. We don't need to fine-tune the entire model anymore. We can only focus on the small treasure map that LoRA gives us.
It's a mind-blowing trick.
We can train the small approximation matrices from LoRA instead of fine-tuning the entire model. LoRA is cheaper, faster, and uses less memory and storage space.
You can also merge the approximation matrices with the model during deployment time. They work like simple adapters. You load up the one you need to solve a problem and use a different one for the next task.
Then, we have QLoRA, which makes the process much more efficient by adding 4-bit quantization. QLoRA deserves its own separate post.
The team at @monsterapis has created an efficient no-code LoRA/QLoRA-powered LLM fine-tuner.
What they do is pretty smart:
They automatically configure your GPU environment and fine-tuning pipeline for your specific model. For example, if you want to fine-tune Mixtral 8x7B on a smaller GPU, they will automatically use QLoRA to keep your costs down and prevent memory issues.
The @monsterapis platform specializes in no-code LoRA-powered fine-tuning. It's the fastest and most affordable offering for fine-tuning models in the market. They sponsored me and gave me 10,000 free credits for anyone who uses the code "SANTIAGO" in their dashboard:
https://t.co/DBsB9EXgYR
If you want to read their latest updates, get free credits and special offers, join their Discord server: https://t.co/oecoxEec14
TL;DR:
• Traditional fine-tuning trains the entire model. It requires a complex setup, higher memory, and expensive hardware.
• LoRA: Trains a small portion of the model. It's faster, requires much less memory, and affordable hardware.
• QLoRA: Much more efficient than LoRA, but it requires a more complex setup.
• No-code fine-tuning with LoRA/QLoRA: The best of both worlds. Low cost and easy setup.
Sam Altman has just been fired at OpenAI.
As a result, co-founder Greg Brockman and three senior AI researchers have quit following the departure (so far).
While most details unknown, here's EVERYTHING we know so far:
What is being perpetrated by the vicious Zionist occupation in Gaza, in the form of intense bombing, killing, cutting off electricity and internet, destroying all aspects of life, and blocking information about the scale of massacres and war crimes taking place, constitutes blind terrorism and a blatant violation of legal and humanitarian conventions and norms. The world must condemn and take decisive action to stop this at once. History will never forgive those who failed to defend innocent Palestinians or supported this continued Zionist terrorism. It is the duty of the Arab and Islamic nations, along with all free people of the world, to unite and find an immediate solution and rescue the oppressed Palestinian people, who are facing an unprecedented humanitarian catastrophe.