ms802x

Electrical Engineer with passion,

Ai Engineer @humainai

[عبدالله عامر السواحهAbdullah Alswaha @aalswaha]

لكل من يقلل من جهود رواد الأعمال والتقنيين 🇸🇦

1985: أُقيل ستيف جوبز من Apple، ثم عاد قصة نجاح

1996: كادت NVIDIA تنهار، واليوم أغلى شركة

2008: فشلت أول 3 إطلاقات لـSpaceX وكادت تفلس، واليوم أكبر اكتتاب

2026: بدعم سمو سيدي ٩ شركات مليارية 🇸🇦

ركّزوا على الرسائل، واتركوا الضوضاء

[Jahir Sheikh @jahirsheikh8]

Me trying to defend my O(n³) solution during the coding interview

[Fei-Fei Li @drfeifei]

The world is not just made of words, and spatial intelligence was never just about perceiving and generating worlds. It's about interacting with them.

Today, SceniX is joining World Labs. 🌎🤖👇

[سلمان بن عبدالعزيز @KingSalman]

نشكر المولى عز وجل أن شرّفنا بخدمة الحرمين الشريفين، ورعاية حجاج بيته الحرام، سائلين الله أن يتقبل من الحجاج حجهم ونسكهم وطاعاتهم.
ومع حلول عيد الأضحى المبارك، نهنئ شعبنا في هذا الوطن المبارك وأمتنا الإسلامية بهذه المناسبة، وندعوه سبحانه أن يجعله عيد خير وسلام واستقرار على أمتنا والعالم أجمع.
وكل عام وأنتم بخير.

[Tareq Amin @TareqAmin_]

Extremely proud of the team behind @HUMAIN’s #ALLAM 34B, our Arabic-first large language model (LLM), which has ranked #2 just behind GPT-5.2 on the BALSAM Leaderboard, a benchmark measuring global Arabic AI maturity.

[KAUST @KAUST_News]

KAUST PhD student Kaja Gruntkowska has been awarded a @Google PhD Fellowship, becoming the first-ever recipient from the GCC countries.

Recognized for her work in Algorithms and Optimization, her research advances both the theory and practice of optimization for machine learning, making AI training faster, more cost-effective, and more resource-efficient.

[Akshit @akshitwt]

this is a great read by jack morris. gives such a fresh perspective on what really matters.
TLDR: with every "new" architecture, we unlocked a new source of data to use at scale. it was the large amount of data we unlocked that boosted performance not the architecture itself, and videos are the next big thing to harness.

2012: alexnet unlocked the entire imagenet dataset
2017: transformers unlocked the entire internet (as text)
2022: RLHF unlocked learning from humans
2024: reasoning unlocked learning from verifiers
2026: ???

when you look at progress this way, it becomes very very clear what the next pillar to unlock is: videos. or more specifically, youtube.

youtube stores an insane amount of video data. people upload 720.000 hours of videos to the platform every single day. thats 4.3 Petabytes of new data every day that need to be stored. for comparison, currently models are trained on a few terabytes of text.
this means that the data uploaded to youtube daily is 1000x the data used to train a typical LLM.

once we come up with an architecture that can harness videos at scale, we will see the next big jump in our quest to AGI

[FFmpeg @FFmpeg]

No

[ms802x @ms802x]

@xichen_pan Nice work! Would it be possible to share the code for inference and training?

[Ostris @ostrisai]

I do not get "vibe coding". Maybe people are just doing much more llm friendly tasks than I am, but no matter what llm I use, 99% of the time, I spend 100x more time fighting with it to keep it from doing stupid things, and then eventually resort to just doing it myself.

[ℏεsam @Hesamation]

the only way you can learn how to work SMART, is by working really HARD

[يوسف القوس @y_algoos]

قصة يوسف، قصة وطن🇸🇦أُسس على المطايا🐪
(وأصبح وطن يحلم جميع العالم بالعيش فيه).

إحتفاءً ب #عام_الإبل_2024 (سطرت قصتي من الصحراء لعالم #أشباه_الموصلات في كلمة خريجي🎓 @KAUST_NewsAR).

(حلمنا بالأمس أصبح واقع اليوم) بفضل الله ثم بإستثمار وطنا بعقول أبناءه ليكونوا حراك رؤية 2030 🇸🇦

[SDAIA @SDAIA_SA]

نسأل الله العظيم رب العرش العظيم أن يمد #خادم_الحرمين_الشريفين بموفور الصحة والعافية.

[Sebastian Raschka @rasbt]

The Llama 3.2 1B and 3B models are my favorite LLMs -- small but very capable.
If you want to understand how the architectures look like under the hood, I implemented them from scratch (one of the best ways to learn): https://t.co/ODlwRfONOz

[ms802x @ms802x]

@turingpi 🫡

[Andrej Karpathy @karpathy]

Huge congrats to @AIatMeta on the Llama 3.1 release!
Few notes:

Today, with the 405B model release, is the first time that a frontier-capability LLM is available to everyone to work with and build on. The model appears to be GPT-4 / Claude 3.5 Sonnet grade and the weights are open and permissively licensed, including commercial use, synthetic data generation, distillation and finetuning. This is an actual, open, frontier-capability LLM release from Meta. The release includes a lot more, e.g. including a 92-page PDF with a lot of detail about the model:
https://t.co/48e3YJ8Sg9

The philosophy underlying this release is in this longread from Zuck, well worth reading as it nicely covers all the major points and arguments in favor of the open AI ecosystem worldview:
"Open Source AI is the Path Forward"
https://t.co/AdmpadCRM0
I like to say that it is still very early days, that we are back in the ~1980s of computing all over again, that LLMs are a next major computing paradigm, and Meta is clearly positioning itself to be the open ecosystem leader of it.

- People will prompt and RAG the models.
- People will finetune the models.
- People will distill them into smaller expert models for narrow tasks and applications.
- People will study, benchmark, optimize.

Open ecosystems also self-organize in modular ways into products apps and services, where each party can contribute their own unique expertise. One example from this morning is @GroqInc , who built a new chip that inferences LLMs *really fast*. They've already integrated Llama 3.1 models and appear to be able to inference the 8B model ~instantly:
https://t.co/b2kdSsz0fH
And (I can't seem to try it due to server pressure) the 405B running on Groq is probably the highest capability, fastest LLM today (?).

Early model evaluations look good:
https://t.co/RLR5YBpmks https://t.co/ipT4x4wCvy
Pending still is the "vibe check", look out for that on X / r/LocalLlama over the next few days (hours?).

I expect the closed model players (which imo have a role in the ecosystem too) to give chase soon, and I'm looking forward to that.

There's a lot to like on the technical side too, w.r.t. multilingual, context lengths, function calling, multimodal, etc. I'll post about some of the technical notes a bit later, once I make it through all the 92 pages of the paper :)

[ms802x @ms802x]

@XQ55 عظم الله أجركم وأحسن عزاكم وغفر الله لوالدكم وأسكنه فسيح جناته.

[Edward Beeching @edwardbeeching]

Imitation Learning support has been added to Godot RL Agents, you can now learn complex behaviours from player demonstrations and then fine-tune with RL. Check out the trained agent (a Neural Network) from our example game.

[AK @_akhaliq]

ViTAR

Vision Transformer with Any Resolution

This paper tackles a significant challenge faced by Vision Transformers (ViTs): their constrained scalability across different image resolutions. Typically, ViTs experience a performance decline when processing resolutions

[KAUST @KAUST_News]

The first #SaudiGreenInitiative Day highlights our effort to work together to create a greener future and improve the quality of life for future generations.

#ForAGreenerSaudi #KAUST