Krish Modi

@uwaterloo se, incoming @palantirtech, prev @bloomberg, @Huawei, ISEF

[Krish Modi @krishmodi404]

I made AgentIR: a scheduler for distributed LLM serving that makes agent workloads run much faster.

41.3% lower E2E latency, and up to 70% higher throughput

[Rishi Shah @Rishi_Shah99]

I took OSDN, a brand-new linear-attention model that learns to tune its own memory updates as it reads (think AdaGrad for the architectures trying to replace the transformer), rebuilt it from scratch in pure C++ with my own autograd engine, and ran it on a $4 microcontroller to predict hypoglycemia 60 minutes before it hits.
No PyTorch. No JAX. No TensorFlow. No ML library at all. Straight C++ standard library.

[Krish Modi @krishmodi404]

@Rishi_Shah99 this is so fucking cool

[Krish Modi @krishmodi404]

@Rishi_Shah99 thanks bro

[Krish Modi @krishmodi404]

launching AgentIR Blackbox https://t.co/P7GC38xM5w

an llm request router for agent system

Blackbox finds which llm calls are on your workflow’s critical path, sends them to faster providers, and routes less urgent calls cheaper to maintain your selected cost-latency constraint

it uses your workflow stats and real-time provider latency profiles to reroute before throttling or slowdowns hit the full workflow

setup is simple too. connect your app, and blackbox handles the workflow annotations for you

use it for free!

[Krish Modi @krishmodi404]

@bmptrsn @datacurve so tuff

[Krish Modi @krishmodi404]

@hamostaf04 thanks hamza!!

[Krish Modi @krishmodi404]

@CMNM50660490 thanks!

[Krish Modi @krishmodi404]

@pahu2353 @IKorovinsky thanks pahu

[Krish Modi @krishmodi404]

@VishnuSatish_ thanks!

[Krish Modi @krishmodi404]

@argupta_ thanks boss

[Krish Modi @krishmodi404]

@waterloo_intern thanks!

[Krish Modi @krishmodi404]

@sunniekapar thanks sunnie!

[Krish Modi @krishmodi404]

@IKorovinsky thanks ian

[Krish Modi @krishmodi404]

@notakki_ UI GOAT

[Krish Modi @krishmodi404]

@fahmi___omer thanks boss

[Krish Modi @krishmodi404]

@ishaandey_ thanks bro

[Krish Modi @krishmodi404]

@_wilsonchenn thanks bro

[Krish Modi @krishmodi404]

@alexabelonix thanks!

[Krish Modi @krishmodi404]

@_rajanagarwal THANK YOU

[Krish Modi @krishmodi404]

@_rajanagarwal thanks ragarwal

[Krish Modi @krishmodi404]

@shamsharoonn 🤩🤩