Kobi Barac

Principal Group Engineering Manager @ Microsoft, previously co-Founder & CTO at (Technology acquired by TASE:DANE).
Opinions are my own.

[Kobi Barac @KobiBarac]

@Eran_Efrat היי ערן, כתבתי לך בפרטי.
אשמח לדעת אם רלוונטי למאמץ.

כל הכבוד על הפעילות.

[Eran Efrat @Eran_Efrat]

רחפנים:
עשינו סיבוב בשוק. ראינו שאין פתרון טוב (רק קשקושים) ולכן החלטנו לפתח פתרון.
אנחנו משתפים פעולה עם כמה חברות מהתחום ועם אנשים מבריקים ואני משוכנע שנגיע לפתרון מעולה לפני כולם.
אנחנו צריכים עוד אנשים לצוות עם רקע ב- SDR, זיהוי אותות RF, כיוון מקור אות (Direction Finding) ו-SIGINT בכלל.

קחו בחשבון שכרגע הכל בהתנדבות ונשתדל לא לשגע אתכם מעבר למה שצריך.
מי שעונה על הקריטריונים ורוצה לקחת חלק, שישלח לי הודעה.

[Santiago @svpino]

LoRA is a genius idea.

To understand the fine-tuning of Large Language Models, you must understand how LoRA works.

By the end of this post, you'll know everything important about how it works.

Large Language Models are good generalists, but they have little specialization. We train them in many different tasks, so they know a bit about everything but not enough about anything.

Think of a kid who can play three different sports at a high level. While he can be proficient across the board, he won't get a scholarship unless he specializes. That's how the kid can reach his full potential.

We can do the same with these large models. We can train them to solve a particular task and nothing else.

We call this process "fine-tuning." We start with everything the model knows and adjust its knowledge to help it improve on the task we care about.

Fine-tuning is revolutionary, but it's not free.

Fine-tuning a large model takes time, care, and lots of money. Many companies can't afford the process. Some can't pay for the hardware. Some can't hire people who know how to do it. Most companies can't do either.

That's where LoRA comes in.

We realized we could approximate a large matrix of parameters using the product of two smaller matrices. There was a lot of wasted space within these large models. What would happen if we find a new, more optimal representation?

Did you ever buy a map at a gas station? Giant pages showing every small road, path, and lake around you. They were exhaustive but hard to navigate. These are like parameters in a large model.

LoRA turns a gas station map into a cartoon treasure map. Every useless parameter is gone. Only two roads, a palm tree, and a cross pointing at the treasure. We don't need to fine-tune the entire model anymore. We can only focus on the small treasure map that LoRA gives us.

It's a mind-blowing trick.

We can train the small approximation matrices from LoRA instead of fine-tuning the entire model. LoRA is cheaper, faster, and uses less memory and storage space.

You can also merge the approximation matrices with the model during deployment time. They work like simple adapters. You load up the one you need to solve a problem and use a different one for the next task.

Then, we have QLoRA, which makes the process much more efficient by adding 4-bit quantization. QLoRA deserves its own separate post.

The team at @monsterapis has created an efficient no-code LoRA/QLoRA-powered LLM fine-tuner.

What they do is pretty smart:

They automatically configure your GPU environment and fine-tuning pipeline for your specific model. For example, if you want to fine-tune Mixtral 8x7B on a smaller GPU, they will automatically use QLoRA to keep your costs down and prevent memory issues.

The @monsterapis platform specializes in no-code LoRA-powered fine-tuning. It's the fastest and most affordable offering for fine-tuning models in the market. They sponsored me and gave me 10,000 free credits for anyone who uses the code "SANTIAGO" in their dashboard:

https://t.co/DBsB9EXgYR

If you want to read their latest updates, get free credits and special offers, join their Discord server: https://t.co/oecoxEec14

TL;DR:

• Traditional fine-tuning trains the entire model. It requires a complex setup, higher memory, and expensive hardware.

• LoRA: Trains a small portion of the model. It's faster, requires much less memory, and affordable hardware.

• QLoRA: Much more efficient than LoRA, but it requires a more complex setup.

• No-code fine-tuning with LoRA/QLoRA: The best of both worlds. Low cost and easy setup.

[OpenAI Developers @OpenAIDevs]

Announcing a new generation of embedding models:

• text-embedding-3-small: 5x cheaper and stronger performance compared to the previous generation
• text-embedding-3-large: our best performing model, creating embeddings with up to 3072 dimensions
https://t.co/2J6rq1Sz5e

[John Spencer @SpencerGuard]

I just watched the roughly 45-minute footage of October 7th at a private screening in NYC. Here are my immediate thoughts minutes after watching it. 🧵

[Michael W @Michael_Wgd]

Our baby Kfir was not released

[OpenAI @OpenAI]

OpenAI announces leadership transition https://t.co/fFYDLwGXQz

[Mr.r @BthCount]

@glick_sh https://t.co/aL9dIutn26

[Under Maintenance @__maintenance]

🚨In this eye-opening interview, senior Hamas official Musa Abu Marzouk blatantly admits: the extensive network of tunnels in Gaza, a staggering 500 km of underground terror infrastructure, was not built for the protection of Gaza’s civilians. Instead, it stands as a chilling testament to Hamas’s priorities—terror over people🛑.

Despite receiving vast sums of money intended to better the lives of Gaza's people, Hamas chose to construct an underground city dedicated to furthering violence and chaos🕳️. These funds could have provided shelters and safety for civilians during attacks, yet the choice was made to divert resources, even to the extent of stealing civilians’ water pipes to manufacture rockets🚀.

As we've seen in the aftermath of the harrowing massacre on 10/7, and the relentless rocket attacks on Israeli cities, the imperative is clear: dismantling Hamas is crucial for the liberation and safety of both Israel and Gaza🛡️. This battle is not just about territory; it's an ideological war against Western civilization, a war that is spreading terror and antisemitism across the globe🌍.

Hamas, backed by Qatar, Iran, and Russia, is a cornerstone in this nefarious alliance against democracy and freedom. The West is next in line unless we act now. We must stand united, defend our values, and safeguard our societies from this violent strain of extremism🤝.

Israel's efforts to minimize casualties, even in the face of such brutality, showcase a stark contrast between democratic values and the inhumanity of Hamas. It is high time the world recognizes this evil for what it is and stands united against radicalism that threatens global harmony🕊️. #TheWestIsNext #FreeGazaFromHamas #HamasisISIS

[Amjad Taha أمجد طه @amjadt25]

Breaking News: Just now, #Hamas snipers have reportedly killed dozens of children and women on the streets, targeting those attempting to travel from north to south #Gaza and those displaying white flags as a sign of peace. Similar acts have previously been attributed to Palestinian and #Hezbollah terrorists in Syria. They do not want civilians to leave; they want to use them as human shields and kill anyone who attempts to leave. Hamas terrorists in Gaza will, as usual, blame #Israel because it is easy and there is media that accepts this propaganda.
#HamasisISIS #HamasTerrorrists

[Amir Barkol @BarkolAmir]

Hi @GretaThunberg 🌿 quick question: This Hamas terrorist confessed today during his interrogation that Hamas instructed them to rape dead bodies after killing them. I wanted to ask if you have any recommendations for an environmentally friendly body soap that Hamas terrorists can use after such an act. Thanks! 😇🌍🙌🏻

[Shai Davidai @ShaiDavidai]

An Open Letter to Every Parent In America

Please help me get this message out to every parent in the United States.

Please help me protect your children.

I am BEGGING you - "liking" this is not enough.

Please retweet right NOW.
https://t.co/a6asiL5HKZ

[Kobi Barac @KobiBarac]

@cichuck @hwchase17 @yoheinakajima @langchain This is a great, and the capabilities lights the imagination, this can very easily become scalable by referring to smaller/local llms (say Alpaca) trained for specific specific tasks.

[Ben Caspit בן כספית @BenCaspit]

מדהים. רפי מיארה, הגיס של היועמ"שית, עונה לדודי אמסלם

[ינון מגל Yinon Magal @YinonMagal]

אחרי 30 קילומטר של מסע צוות בסיירת גולני, מתחת לאלונקות, ולפני העליה למוחרקה,
זה מה שאמר מפקד הצוות לחייליו:

[Sahil Bloom @SahilBloom]

11 mindset shifts that changed my life (and may change yours):

[Kobi Barac @KobiBarac]

השגיאה הממוצעת שלי בהערכת שכר דירה היא 19.43%
🥉🥇🥇🥇🥉
https://t.co/JiD42AqBkf

[Kobi Barac @KobiBarac]

@therealnirs סרט ולא סדרה וגם עוד טרם הספקתי לבחון את התגובה של הדור הצעיר אליו, אבל כשהבן שלי יגיע לגיל רלוונטי לגמרי כל סרטי בחזרה לעתיד!

ואם בכל זאת סדרות אז משהו כמו גמביט המלכה - יש שם איזה ערך מוסף על התמדה, התמודדות עם כישלונות וכו'

[Nir Zacks @nirzacks]

סיימתי תפקיד מ"פ במילואים באמצע חודש מילואים (שבא בהפתעה!) בדרום הר חברון, ואיך אני אגיד לך את זה בעדינות אומה יקרה, שבי רגע, אנחנו צריכים לדבר. בפעם הראשונה מזה כמעט עשור אין לי חיילים. (1/26)
https://t.co/1iM7S1ZEiD