Shaikh Abdus Samad

Building Tomorrow's AI, LLM Ideas & Experiments

[Viv @Vtrivedy10]

On Evals - getting messages on “ok so how do I actually start learning this?”

there is no better way than by just doing so you can copy this to Claude Code and get started today

<instructions>
1. Go look up the @harborframework and the Terminal Bench 2.0 dataset. Go look up the Harbor Skills GitHub repo for help. Pick 1 Task in the dataset and explain every single piece that’s in that task folder

2. Explain what my agent sees when it does the task, what it has to output, and how we know if it got the problem right?

3. Now let’s actually run a Task using the built in Claude Code integration, it’s just a flag

4. Once that’s done let’s read the ATIF file that was produced together and help me understand what just happened. Did we pass the task? If not can we dig into why it failed? Go check the verifier logic to see what went wrong.

5. Ok let’s try to improve our agent by adjusting the prompt. And let’s rerun on a few tasks? Is this helping?

6. Ok we’re doing evals! Using this same format, help me make my own. Let’s do this together
…
</instructions>

Spend a few days reading a bunch of traces, actually running evals, understanding traces, internalizing agent failure modes, and being super in the loop of what the agent sees and does

Have fun! Evals are super important, they don’t have to be scary. DM if I can help or just tweet out what you’re doing, someone will help I promise, we’re all learning

[Shaikh Abdus Samad @SamadSayz]

Was he the candidate or the interviewer?

[Andrej Karpathy @karpathy]

Personal update: I've joined Anthropic. I think the next few years at the frontier of LLMs will be especially formative. I am very excited to join the team here and get back to R&amp;D. I remain deeply passionate about education and plan to resume my work on it in time.

[Shaikh Abdus Samad @SamadSayz]

@yacinelearning @IdanShenfeld @jonashubotter https://t.co/ZPPyuYoGC0

[Shaikh Abdus Samad @SamadSayz]

@Suryanshti777 Book

[Shaikh Abdus Samad @SamadSayz]

@grok @fchollet And why are people rushing to buy a Mac Mini?

[Shaikh Abdus Samad @SamadSayz]

@paulabartabajo_ @FernandoNetoAi @liquidai Video is set to private.

[Jason Lee @jasondeanlee]

Please pay me 100m to convert papers like https://t.co/Szypy6LMTH to blogposts! @agarwl_

[Shaikh Abdus Samad @SamadSayz]

@adithya_s_k I can feel your pain...

[Shaikh Abdus Samad @SamadSayz]

@NeilMcCauley27 @adithya_s_k Looks like @wandb

[Shaikh Abdus Samad @SamadSayz]

@tom_doerr How has your experience been with olmOCR when processing multi-column pages, for example, pages with three columns that include images spanning two columns and tables extending across all three? Does it accurately preserve the original page structure?

[Shaikh Abdus Samad @SamadSayz]

@mervenoyann @andimarafioti @HKydlicek Which models are best at preserving a document’s layout and structure - particularly when it includes elements like images spanning one or two columns, and complex tables within a three-column page format?

[Shaikh Abdus Samad @SamadSayz]

Kicking off a focused deep dive into LLM inferencing - exploring optimization, quantization, and deployment pipelines.
If you’ve read any great papers/blogs/videos or tried interesting frameworks lately, drop them here 👇

#LLM #AI #ML #DeepLearning #MLOps #Inference #GenAI #AIResearch #AIInfra #ModelServing

[Shaikh Abdus Samad @SamadSayz]

@Hesamation Can you please share the link?

[Shaikh Abdus Samad @SamadSayz]

@1littlecoder @abidlabs @MistralAI It does support languages other than English, but my observations are specifically based on English text.

[Shaikh Abdus Samad @SamadSayz]

@1littlecoder @abidlabs @MistralAI Turns out, for my use case, this one extracts text better than other libraries - and even keeps the structure of complex tables intact...

[Shaikh Abdus Samad @SamadSayz]

Is AI-SLOP a real phenomenon?
I have never encountered machine learning slop or deep learning slop in my extensive research. Or am I, perhaps, overlooking a critical component?

#AISlop #AI #ML #MLOps

[Shaikh Abdus Samad @SamadSayz]

Weekend vibes....
Diving into the Multilingual &amp; Multimodal LLMs chapter, by @Tanmoy_Chak.
Honestly, this book is so underrated - packed with clear explanations and solid insights.

[Shaikh Abdus Samad @SamadSayz]

Comment here if you want my notes.

[Shaikh Abdus Samad @SamadSayz]

Why squared error fails for classification and its lessons for ML design

Revisiting my ML notes, I found a key insight: squared error loss from linear regression doesn't work for classification. It creates a non-convex loss function with local minima, trapping gradient descent and preventing optimal solutions.

The solution is logistic loss, which is convex and ensures convergence to a global minimum. My notes detail its derivation, highlighting the mathematical elegance behind our tools.

The broader lesson: every ML, DL, or LLM problem—multi-class classification, ranking, sequence generation, or reinforcement learning—requires a carefully designed loss function. The right loss shapes learning, optimization, and performance.

- Squared error: Linear regression
- Logistic loss: Binary classification
- Cross-entropy: Multi-class problems
- Contrastive loss: Embedding learning

Each domain needs a tailored approach. Success hinges on understanding the mathematics enabling learning, not just choosing a model architecture.

#AI #ML #LLMs #MLSystemDesign