Mazen — sa/acc

LOCKED IN (prev: sudomaze)

agentic & evals @najdresearch

prev: @cursor_ai ambassador; @pytorch award winner; @deepforai @malaa_tech @oregonstate

[Mazen — sa/acc @ma7dev]

yay!

thanks @marksaroufim 🙏🙏

[Mazen — sa/acc @ma7dev]

to give them the benefits of the doubt

they do good stuff but they sometimes ruin the models themselves like the model in english text is acting normally but when trying with arabic text, it just pretend it doesnt comprehend language compared to actual oss model

so better eval models to know if they still work for ur use cases cuz fast + bad isnt a good combo especially for systems that should rely on reliability

[Mazen — sa/acc @ma7dev]

@Sal7one .@GroqLLC is famous for modifying the heck out of oss models

quantization and distillation to get "speed"

https://t.co/SBbhK7mC6f

[DeepSeek @deepseek_ai]

🚀 Introducing DeepInfer, our in-house inference framework, now fully open source!

Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable.

🔗 https://t.co/JGAik1kgk0

1/4

[Mercity AI @MercityAI]

We REMOVED upto 10 WHOLE layers from Qwen3-8B-Base using LaCo (Layer Collapse).

Upon deleting only 6 layers, the model got 16.7% smaller, keeping 78-94% of its reasoning scores at a 1.68x perplexity ratio.

Code and pruned weights are up on our Hugging Face.

Here are insights from our research :

[tender @tenderizzation]

>vLLM is now ~2x faster than vLLM

[Kian Kyars @neuralkian]

BIG NEWS!

My ~8 hour course on AI data centers is now live on @freeCodeCamp!

From Watts to Tokens: 16 chapters covering grid power, 800V DC racks, liquid cooling, networking and GPU cloud economics.

https://t.co/Bolpv7M1se

[Mazen — sa/acc @ma7dev]

@lillyguisnet prompt optimization through our fav lib, @DSPyOSS?

[Mazen — sa/acc @ma7dev]

لما كنت شغال ابني arabic OCR

بديت اسوي eval لأغلبية النماذج، منهم VLMs وtraditional CV

لاحظت ان كل VLMs يسوون نفس الأغلاط مهما كانت احجامهم ومن سواهم

> يحطون حركات من عندهم
> يترجمون الكلام المكتوب
> والأغرب، كانوا ما يشوفون الكلمات العشوائية حتى لو كانت واضحه!

هنا عرفت شيء جديد ان VLMs يمكن رفعوا جودة مخرجات OCR ولكن كونوا مشاكل جديدة نا كانت موجوده من قبل...

[Vik Paruchuri @VikParuchuri]

Why do VLMs sometimes output 罪 instead of "vain"?

An investigation into script confusions, and how we fixed them 🧵

[Vik Paruchuri @VikParuchuri]

Why do VLMs sometimes output 罪 instead of "vain"?

An investigation into script confusions, and how we fixed them 🧵

[Mazen — sa/acc @ma7dev]

الله يكون بعون عبدالرحمن الفوزان

[Brian Armstrong @brian_armstrong]

We were able to fine tune an open weight model (Qwen3.5-9B) on proprietary internal data at @coinbase, and it out performed frontier models, at much lower cost and speed, for a specialized task (fraud prevention in this case).

I think we'll see more of this pattern inside companies. The frontier models are amazing generalists. But increasingly you'll see specialist models out perform on more narrow tasks, either working on teams, or by themselves.

We're making it easy to fine tune models internally for our employees. We even have a skill for it in our internal harness that makes it easy to spin these up and start training them:

"/fine-tune-and-serve-llm I want to fine-tune Qwen for payments risk review. help me get started"

I imagine we'll have some backend services that could be replaced entirely by this new architecture.

[dax @thdxr]

it's crazy how easy it is to spot 10x engineers these days

for a certain set of people LLMs totally unlocked their potential

to the point where imagining their life before this seems totally depressing

[Mazen — sa/acc @ma7dev]

say/do <<< 1/2

[Mazen — sa/acc @ma7dev]

https://t.co/7VYd4Nr9jj

[Amit Shekhar @amitiitbhu]

LLM Inference Engineering - Problem and Solution

Problem: LLMs are slow
Solution: KV Cache - Avoid recomputing previous tokens.

Problem: KV Cache consumes huge memory
Solution: PagedAttention - Manage KV memory efficiently.

Problem: GPU is underutilized
Solution: Continuous Batching - Dynamically add/remove requests.

Problem: Token generation is sequential
Solution: Speculative Decoding - Generate and verify multiple tokens together.

Problem: Attention requires large memory usage and expensive memory movement
Solution: FlashAttention - Reduce memory usage and memory movement during attention.

Problem: KV Cache is still large
Solution: MQA / GQA - Share K/V across multiple query heads.

Problem: Many optimizations need to work together
Solution: vLLM - Build an efficient inference engine.

Problem: Applications repeatedly process the same prefixes
Solution: RadixAttention / Prefix Caching - Reuse previously computed work.

Problem: Models are too large
Solution: Quantization - Reduce model memory and computation.

Problem: Large models are expensive
Solution: Distillation / SLMs / MoE - Make inference itself cheaper.

Keep Learning, Keep Sharing, and Keep Growing.

[Amit Shekhar @amitiitbhu]

In agentic AI, log every step.

Every LLM call, tool call, trace, span, decision, input, and output.

Log the reasoning signals wherever possible.

If you can’t observe what your agent is doing, you can’t reliably debug it.

[Mazen — sa/acc @ma7dev]

@YazeedShalani @DalData_sa @HRSD_SA @Ahmed_S_Alrajhi ما شاء الله تبارك الله 😍😍😍

جهود جباره نشوفها بارض الواقع في تمكين شباب اليوم والمستقبل 🙏

الله يبارك في علمكم وعملكم 🙏🙏

[Yazeed @YazeedShalani]

سعداء بتكريم @DalData_sa في مملكة البحرين، ضمن تكريم الكفاءات والمشاريع الرائدة في العمل الاجتماعي على مستوى دول مجلس التعاون الخليجي.

كل الشكر ل @HRSD_SA على ترشيح الجمعية، ولمعالي المهندس @Ahmed_S_Alrajhi على دعمه وتقديره لجهود دال.

تكريم نفخر به، ودافع لنا لمواصلة الأثر في مجال البيانات والذكاء الاصطناعي.

[mousa alzahrani @imousart]

@ma7dev المال قد يأتيك من أبواب كثيرة بعد توفيق الله..

لكن الأثر عزيز، لا يُمنح لمجرد السعي إليه، بل لعظمة النية التي تقف خلفه.

فالخير والشر كلاهما قد يترك أثرًا، لكن ما يصنع الفرق هو صدق النية وعمقها.

والتاريخ خير دليل