We moved DeepSeek V4.1 Flash on AMD MI355X from vLLM to SGLang.
Median time to first token on live traffic went from 388 ms to 193 ms. No failed requests during the switch, and nothing changed for anyone calling the API.
Dedicated endpoints: https://t.co/nhl8cdc8ZS
Excited to share that we're leading @OpenRelayInc's $10M seed round!
OpenRelay builds inference-native infrastructure, matching each workload to the hardware that suits it best. In just five months, they've grown from $0 to $10M in annualized run rate, processing over 5 trillion tokens a month across 20+ regions and 10 hardware types.
I first met @jadenswang and @prashant8124 this summer during @ycombinator's S26 batch and was immediately struck by how complementary their backgrounds were. Jaden started a data center business in college before joining Voltage Park, while Prashant was one of the founding engineers on AWS Bedrock.
As open models become increasingly capable and the hardware landscape diversifies, developers have more opportunities to improve the performance and economics of inference, but few have the infrastructure or expertise to take full advantage of them.
We're thrilled to be partnering with the OpenRelay team, alongside @EniacVC and @Standard_Cap.
More on why we invested: https://t.co/pLyoAA7mPa
The future of inference isn't everyone renting the same GPUs from neocloud training clusters. We're building a vertically integrated inference delivery network to change that.
Just raised $10M led by @uncorkcap with participation from @EniacVC and @Standard_Cap . Time to build.
Call one provider and you get their prices, their rate limits and their 3 a.m. outage.
OpenRelay puts one endpoint in front of GPUs across providers and routes every request to the best silicon. Provider down? We reroute.
Change one line in your SDK.
https://t.co/BgtC3C1UMo
OpenRelay is a delivery network for AI inference.
We pool GPUs across providers and locations and route every request to the best silicon. Provider down? We reroute.
We bill flat rate, not per token. Just create an API key and change one line.
https://t.co/BgtC3C1UMo
Reserved GPUs sit idle most of the month.
Bring Your Own Capacity: onboard your fleet with one command, no open ports. We serve inference on the idle hours and pay you for them. You want a node back, it's yours again.
And we run it for free.
https://t.co/wd5wUPWe6k
Your agent can manage its own compute on OpenRelay.
Ask for an H100 and it deploys the VM. Ask for a batch and it runs it overnight. It watches what it spends, live, and stops at the budget you set.
Give your agent an OpenRelay key, then let it cook.
https://t.co/mbpTImFuRA
Laya makes decisions instead of writing text. Use a few hundred labeled examples to set thresholds before relying on it. The playground has four presets.
Docs: https://t.co/COo7Ko3pHK
Model by Convai Innovations: https://t.co/TbRj0TFvPi
Decision models are a new category: Jev from TypeSafe launched last week, typed answers with probabilities instead of text.
Laya is the open-source one. It is live on OpenRelay: POST /v1/decisions, $0.02 per 1M input tokens, nothing else billed.
Same three primitives: choice, score, yes/no. Up to 64 questions per request, one forward pass, a probability distribution and a confidence on every answer.
A four-question ticket triage is 345 tokens, or $0.000007.
we've partnered with @OpenRelayInc as a compute partner to provide private AI inference
hmu if you need provably end-to-end encrypted tokens
register for capacity here: https://t.co/Ixotrgs5OU
The essays that shaped how I think were written by someone I assumed I would never meet. Today he spent an afternoon helping two founders (@jadenswang ) get better. Thank you @paulg and @jesslivingston for having us! This was the most memorable afternoon of my career