logotype

high-performance inference engine, backed by research

[logotype @logotype]

LayerScale is 70% faster on throughput than vLLM. At c128 on AgentX we’re faster on tok/s, ITL, latency, interactivity, TTFT… you name it. https://t.co/NLImocgkft

[logotype @logotype]

@danielgross Do we have to use SGLang? Can I bring my own inference engine to the challenge?

[logotype @logotype]

@ariG23498 Decision models are a HACK for obsolete inference engines.

[logotype @logotype]

@goodhartproof crazy… I just keep my head down and fucking build awesome shit

[logotype @logotype]

@bernhardsson AgentX, LayerScale vs vLLM. Enkel match. Modal kan serva kunder bättre med LayerScale!

[logotype @logotype]

@Luca_Pistor Looking forward to see a MoE demo using these primitives

[logotype @logotype]

@MTSlive @aidanmantine bros mouth on 2x and tensor cores for brains

[logotype @logotype]

@Ekaeoq It’s a daily beater

[logotype @logotype]

@theojaffee VCs does care tho

[logotype @logotype]

@bubbleboi vibecore
bubble5

[logotype @logotype]

@amritwt bro is a tensor core

[logotype @logotype]

@TheZachMueller Those are rookie numbers. We are 7x faster than vLLM/SGLang. I sent you the benchmarks on email. Proposal to collaborate!

[logotype @logotype]

@ajhodls We’re not raising and fuck you, how’s that for a signal

[logotype @logotype]

@billybylaws “audacity” = “money”

[logotype @logotype]

@glcst Happy with the Aeron since…2012 I think.

[logotype @logotype]

@aidanmantine @gabriel1 Not convinced of MegaKernels yet but the demo was great! Our inference engine does multiple things simultaneously too, making use of idle GPU cycles. Hella fast

[logotype @logotype]

@MaxJunestrand Möe pengar det

[logotype @logotype]

@jonallie Then there were the nerds who just sat there, in dark rooms, coding in the night.

[logotype @logotype]

@SubhoGhosh02 I don’t think that’s what matters. vLLM/SGLang and every other inference engine sits idle between requests and wastes billions of dollars of expensive GPUs. We built a NOVEL inference engine from scratch that thinks between requests, doing forward passes on idle GPU cycles.

[logotype @logotype]

@kushalbyatnal Today is the same thing with AI wrappers, and you’re part of it 😂