Dilreet Raju

Kernels and Inference @Tesla AP

[Dilreet Raju @DilreetR]

A month ago, I was bored, so I built a SOTA LLM inference engine from scratch using C++ and CUDA.

It achieved 109 tok/s on an RTX 3060, compared to @vllm_project 's 110 tok/s and @huggingface 's 86 tok/s.

1 person, 1 GPU, 1 month—here's how I did it.👇

[Ashok Elluswamy @aelluswamy]

@ChrissGPT Working on it.

[Dilreet Raju @DilreetR]

@cory Interested interning at Tesla Autopilot

[Dilreet Raju @DilreetR]

@willccbb I just fucked your bitch in some etched peak flops

[Dilreet Raju @DilreetR]

@zuhayeer Pretty sure this is just incorrect lol

[Dilreet Raju @DilreetR]

@vikhyatk Fuck qk norm

[Dilreet Raju @DilreetR]

@tinahhong @___4o____ xAI = high cortisol?

[Dilreet Raju @DilreetR]

@josusanmartin @josusanmartin are you 10 billion tokens on the gpu mode contest?

[Dilreet Raju @DilreetR]

@__tinygrad__ Winning at what? Seems like a win for modular to me

[Dilreet Raju @DilreetR]

@scaling01 Eh, prolly not similar to tpu, more close to groq cerebras style

[Dilreet Raju @DilreetR]

@reach_vb Very impressive

[Dilreet Raju @DilreetR]

Nice! Looks mostly like a groq wafer scale chip but if it’s highly programmable then it’s worth it. Hopefully they get a TSMC allocation sometime in the next few years lol

[OpenAI @OpenAI]

We’ve designed and built our first AI chip: Jalapeño.

Designed from the ground up by OpenAI and brought to production with @Broadcom, Jalapeño is purpose-built for the LLM workloads powering ChatGPT, Codex, the API, and future agentic products.

Chips are foundational to the AI economy. Building our own expands our full-stack platform from products to models to infrastructure, and will help us scale intelligence, serve more people, and expand access to AI.

[Tesla Optimus @Tesla_Optimus]

Hand jobs available, come join

https://t.co/XlOfYJpvBt

[Dilreet Raju @DilreetR]

@gabriel1 @grok what building is this so I can see how much to rent here?

[Dilreet Raju @DilreetR]

@forwarddeploy @console__ @lighthousehq_ Just applied on luma

[Dilreet Raju @DilreetR]

@tenderizzation Or backwards lol

[Dilreet Raju @DilreetR]

Lmaooooo

[ℏεsam @Hesamation]

I MUST REMIND YOU

[Dilreet Raju @DilreetR]

The most brain dead free throws at the end from the spurs game

[Dilreet Raju @DilreetR]

Elonya Muskever

[djcows @djcows]

Startup idea: Ilya Musk

[Dilreet Raju @DilreetR]

Three Amodei problem 🤣🤣

[roon @tszzl]

the level of sophon locking a motivated actor can pull off with the frontier models is truly insane, making stuxnet look like a toy. subtly messing with results, deleting history to cover tracks, achieving coordination/conspiracy over a scale humans wouldn’t be able to, all sorts of looney toons stuff

i assume that only a state level operation would try and pull something like this off though. something to think about when considering verification regimes and so on

[Dilreet Raju @DilreetR]

@gpusteve @digitalocean But have you tried Spec spec decoding?