Sidhant

Agents and Inference | prev @Binoloop | Open to Opportunities

[Sidhant @sidmanale643]

>started reading about LLM inference
>accidentally built an inference engine
>added paged attention
>added prefix caching
>added chunked prefill
>added continuous batching
> achieve 14% speedup vs vLLM

wrote the whole journey here ↓ https://t.co/UfzzVpqwAd

[Sidhant @sidmanale643]

@Mahir28M That is so 2021

[Sidhant @sidmanale643]

Scaling laws have struck again

[Claude @claudeai]

Introducing Claude Opus 5.5, the first model in our new Claude 5.5 family.

It performs at the level of Claude Fable 5.1 for most tasks, and costs 40% less to run than Opus 5.

[Sidhant @sidmanale643]

Had no idea bro was chill like that

[Alexandr Wang @alexandr_wang]

state of the art motherfuckers

[Sidhant @sidmanale643]

@BhosalePratim Agreed, wrote the application's "anything else you would like us to know" section myself and just wrote my thoughts
Worked out for me

[Sidhant @sidmanale643]

@dextertwts 51*

[Sidhant @sidmanale643]

about time the non autoregressive text generation models start booming

[Sidhant @sidmanale643]

@ItsRiyasat Thanks man

[Sidhant @sidmanale643]

This is where I tweet from btw

[Sidhant @sidmanale643]

Jev this. Jev that.

Here’s Jev playing Pac-Man 👻

The harness sends it the board, legal moves, food distances, and ghost positions each turn. Jev picks a move; a small route rule keeps it heading for food when the map gets sparse.

[Sidhant @sidmanale643]

they're making me work on the android app for now, at this rate I might become a full stack engineer by the end of this year

[Sidhant @sidmanale643]

@AbhinavAKAblade @thsottiaux He's a good kid @thsottiaux

[Sidhant @sidmanale643]

@SergeStack Deterministic checks test for things like for eg max_num_tool_calls or max_cost

Llm as judges are used for quality checks like correctness, user goal following etc

Both things test different things and are treated separately

[Sidhant @sidmanale643]

Adding another Jev post to your already Jev filled TL

Evalon is an eval + observability harness I built a while ago

it tracks LLM calls, latency, tokens, cost, exceptions, and runs evals over datasets with deterministic checks or LLM judges

now I’m adding Jev as an evaluator

a surprising number of evals are basically binary decisions:

-> did the agent finish the task?
-> is this answer grounded?
-> was the tool call correct?
-> does this output satisfy the rubric?

Jev is a pretty natural fit for these

you give it the context + rubric, it gives you the verdict

and at Jev’s speed + price, this gets especially interesting at scale

Github:
https://t.co/bC0TnZH4Hi

[Sidhant @sidmanale643]

We just hit 16 stars on the Helios repo

Most starred repo I’ve ever built

Github gave me the Starstruck badge

WE ARE SO BACK

https://t.co/OZCVo97v1E

[Sidhant @sidmanale643]

>started reading about LLM inference
>accidentally built an inference engine
>added paged attention
>added prefix caching
>added chunked prefill
>added continuous batching
> achieve 14% speedup vs vLLM

wrote the whole journey here ↓ https://t.co/UfzzVpqwAd

[Sidhant @sidmanale643]

@devops_nk the choice >>>>
https://t.co/qf6d56Zwf2

[Sidhant @sidmanale643]

good day to own a motorcycle

[Sidhant @sidmanale643]

@asmah2107 The mini-sglang repo or this one :D
https://t.co/oDM1A20Fow

[Sidhant @sidmanale643]

>started reading about LLM inference
>accidentally built an inference engine
>added paged attention
>added prefix caching
>added chunked prefill
>added continuous batching
> achieve 14% speedup vs vLLM

wrote the whole journey here ↓ https://t.co/UfzzVpqwAd

[Sidhant @sidmanale643]

Career update soon?

[Kartik @code_kartik]

saw this and thought: why isn’t there an open-source version?

So I built one in a few hours. It uses your AirPods to blur your Mac when you look away no camera, no recording.

try it out now link in the comments ⬇️

[Sidhant @sidmanale643]

No wayyyyyy

[Eren Bali @erenbali]

I probably watched every single @3blue1brown video from Grant Sanderson (big fan) and I am noticing this for the first time 🤯

[Sidhant @sidmanale643]

@gordic_aleksa RLMs are great
https://t.co/gG9QDkmLUO

[Sidhant @sidmanale643]

A friend gave me a brutal benchmark for Ares, my self-improving RLM harness:

-> build an ML/DL library from scratch in C
-> then implement + train XGBoost from scratch on Boston Housing dataset

Ares did it with GPT-5.6 Sol for <$0.20 and ~1% context usage.

I compared the implementation against python XGBoost using the same dataset split and hyperparameters

result difference: ~0.35%

I have found this to be a great benchmark for both agent accuracy and efficiency

checkout Ares:
https://t.co/Bj51ucXTjA