Ankit Raj

AI Engineer | 20+ Production AI Products | 80K+ Users | Product Hunt Top 20 | Multi-Agent Systems | LLM Monitoring

[Ankit Raj @ankit1478]

Great opportunity to work with @prasad_pilla

[Prasad Pilla @prasad_pilla]

We’re hiring an SDE Intern to join the founding team at @tryAlanAI

You’ll work directly on the product across full stack features, coding agents, orchestration, cloud sandboxes, and developer infrastructure.

> We don’t care much about experience.
> We care if you can build.

If you’ve built something you’re genuinely proud of, apply!

Link in the description 👇

[jinjingliang @JinjingLiang]

Jev

[Abhinandan @abhibuilds]

What happens between your prompt and the tokens you see on screen?

𝗔𝗻 𝗟𝗟𝗠 𝗱𝗼𝗲𝘀𝗻'𝘁 𝗴𝗲𝗻𝗲𝗿𝗮𝘁𝗲 𝗮𝗻 𝗮𝗻𝘀𝘄𝗲𝗿 𝗮𝗹𝗹 𝗮𝘁 𝗼𝗻𝗰𝗲. It predicts one token, appends that token to the sequence, runs the model again, predicts the next token. And keeps repeating this process.

Most of us know this concept, but I wanted to understand what the inference loop actually looks like I build it myself.

So I have started building a mini LLM inference engine from scratch.

In this Part 1, I implemented greedy autoregressive generation:

input_ids [B,T]
→ model forward pass
→ logits [B,T,V]
→ take the last position [B,V]
→ choose the highest-scoring token
→ append it
→ repeat

Building such a generation loop, makes you to properly understand
- what logits represent,
- why the output has shape [B,T,V],
- why generation only uses the last position,
- and how the model's own prediction becomes the input for the next step.

All this question I have tried to answer in this video in easy to understand way.

This is the first part of a series Building Inference Engine where I'm progressively adding:
𝘀𝗮𝗺𝗽𝗹𝗶𝗻𝗴 → 𝗞𝗩 𝗰𝗮𝗰𝗵𝗲 → 𝗯𝗮𝘁𝗰𝗵𝗶𝗻𝗴 → 𝗽𝗮𝗱𝗱𝗶𝗻𝗴/𝗺𝗮𝘀𝗸𝘀 → 𝗧𝗧𝗙𝗧/𝗜𝗧𝗟

[Ankit Raj @ankit1478]

@saksham782 Jev is basically a fast probabilistic decision/selection model, while an LLM is a generative reasoning model.

LLM = language + reasoning + generation
Jev = statistical/probabilistic decision making over predefined possibilities

[Ankit Raj @ankit1478]

I got access to Jev!

[Ankit Raj @ankit1478]

Kill more chance

[Raj Shamani @rajshamani]

Your potential ends where your fear of judgment begins.

[Ankit Raj @ankit1478]

@shubhstwt Dario Amodei: Agree
Sam Altman: agree
Elon Musk: Agree.
What is cooking up ?

[Ankit Raj @ankit1478]

Very imp to read.

[Dario Amodei @DarioAmodei]

We Must Pace the Frontier: I’ve written a new essay on why the AI industry should slow down, with a three-part plan for doing so.

Anthropic is unilaterally committing to the first of these steps. We’ll provide third-party evaluators with permanent, employee-level access to our systems, so that they can verify adherence to our safety measures, report on incidents, and assess models’ alignment during training.

You can read the full post here: https://t.co/OGyPb7yaYt

[Ankit Raj @ankit1478]

@telegram CEO just announced that Telegram is sponsoring Codeforces.

[Pavel Durov @durov]

🤝 Telegram has become the sponsor of https://t.co/79NDWg9zsJ — the largest competitive programming platform in the world.

[Ankit Raj @ankit1478]

Never deleting this app #AppleEvent

[Ankit Raj @ankit1478]

@shubhstwt Bro I just got concerned and you turned it into a marketing campaign analysis.

[Ankit Raj @ankit1478]

This is really concerning.

[Jacob Coxon @hilbertspaess]

I resigned from Anthropic today. I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives. More thoughts below.

[Mark Zuckerberg @finkd]

Introducing Muse, the personal agent that understands your goals and works 24/7 to get things done for you.

[Ankit Raj @ankit1478]

Looking for a good whey protein under ₹3,200

Please suggest the best option you’ve personally used.

[Ankit Raj @ankit1478]

@mktpavlenko Good point. A judge can score well against the rubric and still be unreliable in production.

[Ankit Raj @ankit1478]

Read the LLM-as-a-Judge paper.

Then built the concepts step by step:
• Rubrics
• Scoring
• Bias checks
• Eval pipeline

GitHub: https://t.co/jickiYS8Ps

[Ankit Raj @ankit1478]

@coderandcreator Yes

[Ankit Raj @ankit1478]

Monsoon turns my village into a living painting.

[Ankit Raj @ankit1478]

@codeanuragg congratulations

[Akshay Saini @akshaymarch7]

One thing I understood quite early in my software engineering career:

Being good at coding can only get you noticed initially.

But after a point, your growth depends on how well you understand business, communicate with people, take ownership and make decisions when things are unclear.

And these are the core skills that later helped me build my own startup which is running successfully since 4 years now!