Avian.io

Worlds Fastest Inference for DeepSeek R1 at 351 tokens per second. Dedicated endpoints + Serverless APIs available.

[Avian.io @avian_io]

4X DeepSeek R1 671B Inference: As we prepare for DeepSeek R2, Avian presents 351 output tokens per second on DeepSeek R1 per user on NVIDIA B200 at FP4 precision.

Presented to you in collaboration with @Vultr Cloud GPU, who provided the NVIDIA HGX B200 hardware.

Keep your eyes out for more results to come out for blazing Llama 4 Maverick inference soon...

[Vultr @Vultr]

💥 GenAI inference just hit warp speed: @nvidia Blackwell hits 351 tokens/sec on DeepSeek R1 (FP4), and @avian_io hits 351/sec per user on Vultr Cloud GPU with the B200. High-throughput inference is here – and Llama 4 Maverick is next.
#GenAI #AIinference #NVIDIA #CloudGPU

[Avian.io @avian_io]

@NVIDIAAIDev @kristileilani you should cover this :)

[NVIDIA AI Developer @NVIDIAAIDev]

NVIDIA Blackwell sets a new benchmark with 303 Tokens/s for DeepSeek R1 in FP4 precision. 👀

🎉 Huge congrats to NVIDIA Inception partner @avian_io on this impressive achievement which showcases major ecosystem breakthrough, leveraging NVIDIA Blackwell and our open accelerated stack with TensorRT-LLM.

[Artificial Analysis @ArtificialAnlys]

NVIDIA Blackwell can achieve 303 output tokens/s for DeepSeek R1 in FP4 precision, per our benchmarking of an Avian API endpoint

Artificial Analysis benchmarked DeepSeek R1 on an @avian_io private API endpoint. Running DeepSeek R1 in FP4 precision on NVIDIA Blackwell, their endpoint achieved 303 output tokens/s - the fastest speed we have measured yet for DeepSeek R1.

The FP4 version of DeepSeek R1 maintained accuracy across our evaluation suite as compared to the native FP8 version.

Inference speed is especially critical for reasoning models that ‘think’ before they answer - we look forward to wider availability of NVIDIA Blackwell hardware in the coming months!

[Avian.io @avian_io]

@elonmusk @BowhuntressInMI https://t.co/hZeVXLWnBn is the fastest inference provider. Would love to host Grok.

[Avian.io @avian_io]

@andykonwinski invest in https://t.co/38ZoqQBKRk (fastest ai inference) instead.

[Avian.io @avian_io]

@Ahmad_Al_Dahle @cerebras You forgot https://t.co/hZeVXLVPLP achieved 142 tps on Llama 405B. https://t.co/pgOa3HJMfH

Check it out!

[Avian.io @avian_io]

@kristileilani How many is a lot?

[Avian.io @avian_io]

@OfficialLoganK Unfortunately it's literally just a parrot right now

[Avian.io @avian_io]

@OpenAI https://t.co/oBbtiFwQOM

[Avian.io @avian_io]

@OfficialLoganK https://t.co/oBbtiFwQOM

[Avian.io @avian_io]

@OfficialLoganK @OpenAI 1106 production model when?

[Avian.io @avian_io]

@OfficialLoganK Will the 32k API be released?

[Avian.io @avian_io]

@Jacobsebotter @OfficialLoganK We have changed some things in the description of the plugin and are waiting to get re-verified in the store. We should be back shortly if the OpenAI team is swift.

[Avian.io @avian_io]

@digitalconner I suppose if you use a Mac yes, however Notion is probably the best Note taking app for windows. I don't get the hype about Superhuman either...

[Avian.io @avian_io]

@OfficialLoganK I believe it should have the capabilities as the JSON generation / function calling of the plugins model.

[Avian.io @avian_io]

@OfficialLoganK Is this sthe same API that the plugins use? How can we access it?

[Avian.io @avian_io]

@GoodMarketingHQ Gambling should be illegal