Bryan

GMT+8. Personal rants.
Priors and opinions may change overtime.
Powered by coffee.

[Bryan @Bryan_1269]

@deepseek_ai All I want for Christmas is Deepseek v5 Flash (1225) & Deepseek v5 Pro (1225)

[Bryan @Bryan_1269]

In very rase use cases do I need more than 300toks/sec.

Wall clock for my normal workflow is bottlenecked more by tool calls (at the very least a couple hundred ms) compared to raw model toks/sec.

Also, consider the full e2e request latency spent on queue, not just tok/sec.

[Bryan @Bryan_1269]

Shout out to @PrimeIntellect serverless inference service!

Super reliable, models not quantized to the groud, high cache rate, good speed, and super reasonably priced!

Much better than the official Deepseek endpoint (Prime's cheaper) and other 3rd party providers!

[Bryan @Bryan_1269]

Jensen RESIGNS??!!

[The Spectator Index @spectatorindex]

BREAKING: NVIDIA CEO Jensen Huang resigns

[Bryan @Bryan_1269]

Muse Ember vs Deepseek V4.1 Flash 1017 cache hit rate.
How does Deepseek do it?

Super consistent high cache hit rate, same small model (would guess somewhat same ballpark), roughly same pricing.

[Bryan @Bryan_1269]

@ivanleomk What in the devrel is this? Loving it! Haha!

[Bryan @Bryan_1269]

Muse Ember not as good. I'd be surprised if the model is larger than 200B, won't be as surprised if it's less than 100B. Feels like using GPT OSS 120B / Gemini Flash. Super fast, super lazy, super confident, and super dumb.

[Bryan @Bryan_1269]

Update: A FULL week since their last reply, no further response. The 6 USD "refund" not even processed. Turns out, you get at most 1 reply from their support.

What a horrible company.

[Bryan @Bryan_1269]

Update: OpenRouter replied. They said it's not their fault, my fault. They acknowledged the anomalous requests and they did their accounting. They offered to refund $6.02 USD as a one-time goodwill gesture. But they can't lift the OpenAI ban on my OpenRouter account in any way.

[Bryan @Bryan_1269]

@RandallKanna "I don't want to live in a world where someone else makes the world a better place better than we do."

- Gavin Belson

[Bryan @Bryan_1269]

@MainzOnX I agree. But given it's only their first model, can't wait for Beam v2!

[Bryan @Bryan_1269]

@ThePrimeagen Basically the Accenture of LinkedIn

[Bryan @Bryan_1269]

@thinkymachines
Inkling-2-Small uses flintstone reasoning!

Me love small model think fast good.

[Bryan @Bryan_1269]

Does anyone here actually route to Deepinfra for actual inference workload? I need to compare notes.

[Bryan @Bryan_1269]

UPDATE: To stress my point that OpenRouter has terrible support. 4 days since their "goodwill" email.

They did their own accounting, I did not even question how they come up with that figure.

Just to show, refund not yet issued nor heard back from them. For $6!

[Bryan @Bryan_1269]

Update: OpenRouter replied. They said it's not their fault, my fault. They acknowledged the anomalous requests and they did their accounting. They offered to refund $6.02 USD as a one-time goodwill gesture. But they can't lift the OpenAI ban on my OpenRouter account in any way.

[Bryan @Bryan_1269]

@msdev Microsoft

[Bryan @Bryan_1269]

"Stealing IP is bad", says the company that steals everyone else's IP.

[Bryan @Bryan_1269]

See... this is why opensource models are DANGEROUS! Use OUR hosted models! Do NOT run your OWN models!

We buy DDR good, you buy DDR bad.
We run model GOOD. You run model BAD.
China communist. China run model BAD.
You run model = You china = You communist. Bad.

[Brendan Dolan-Gavitt @moyix]

Confirmed, and yeah this seems like a horrible idea

[Bryan @Bryan_1269]

@factory Next: Drop the 'y'. Just 'Factor', it's cleaner.
Next: Drop the 'Or'. Just 'Fact', it's cleaner.

[Bryan @Bryan_1269]

@OpenAI I don't want to do BYOK etc. The entire goal of me using OpenRouter so that I can top up once and use credits across different models and providers.

[Bryan @Bryan_1269]

Update: OpenRouter replied. They said it's not their fault, my fault. They acknowledged the anomalous requests and they did their accounting. They offered to refund $6.02 USD as a one-time goodwill gesture. But they can't lift the OpenAI ban on my OpenRouter account in any way.

[Bryan @Bryan_1269]

24h since I reached out to OpenRouter support, no response. My point.

[Bryan @Bryan_1269]

Matter closed.

New problem, I can't reach out to @OpenAI certainly to lift the ban, right? And OpenRouter can't lift the ban.
My OpenAI dev account, under the same email, still works fine. So not linked by email addr.