Dr. Daniel Bender

I'm building my assistant in Hermes. No, I won't blindly trust it ◆ I teach you the way of responsible AI: your data, your rules ◆ PhD in computer science ◆ Dad

[Dr. Daniel Bender @drdanielbender]

Ever wondered how model quantization (FP16, Q8, Q4) *really* affects performance?

There's an analogy that makes the trade-offs crystal clear... and it involves something you might drink. 😉🍺

Kudos to @jtdavies for this brilliant comparison. 🙏

See the image for the full explanation! 👇

[Dr. Daniel Bender @drdanielbender]

@Tim_Denning The well-known playbook.

Sounds good and it's still hard at the same time, especially to be consistent and stick to it, because it might take longer than you think to get the success you're looking for.

But nevertheless, I agree.

[Dr. Daniel Bender @drdanielbender]

@markfenner Exactly. A source alone is not enough. You have to look at its origin, and that's really a pity.

[Dr. Daniel Bender @drdanielbender]

A source link doesn’t guarantee an AI answer is right.

I asked ChatGPT and Hermes Agent a simple question:
What's a soccer club’s annual membership fee?

They gave different answers. Both cited sources, but only one was correct.

Check the origin of the sources yourself!

[Dr. Daniel Bender @drdanielbender]

@NorthstarBrain How sure is it that the AI tax will be realized in Germany? @grok

[Dr. Daniel Bender @drdanielbender]

WTF and thanks to OpenAI for making my digital life a bit safer by spending an equivalent of $1.2M to secure the Linux kernel🙏

Yes, less costs than the stated $1.2M for OpenAI as they don't need to pay the margin of customers, BUT compute is a rare resource for everyone. 👏

[OpenAI @OpenAI]

Our security researchers built an autonomous cyber vulnerability discovery, exploitation, and patching pipeline.

In a single run with an upcoming cyber model, it validated and generated patches for 948 local privilege escalation vulnerabilities in the Linux kernel at a total API cost of $1.2 million.

[Dr. Daniel Bender @drdanielbender]

The cause is likely that the account that created the calendar does not exist anymore.

> secondary calendars will now follow the lifecycle of their owner, meaning that when a user account is deleted, any secondary calendars owned by that user will be deleted as well. Similarly, when a user is suspended, their owned secondary calendars will be suspended as well.

https://t.co/sfvVWTlmEB

[Simon Høiberg @SimonHoiberg]

I'm so happy I'm self-hosting everything now! If you're not, please stick with the big providers: AWS, Google Cloud, Azure.

I wouldn't trust any of these small PaaS - Railway, Supabase, Netlify, Vercel, etc - for ONE second.

Get off the cloud.
(Or stick with the really big ones).

[Dr. Daniel Bender @drdanielbender]

Codex security plugin lets you scan your repos on GitHub with Daybreak Blue from OpenAI with less cyber guardrails:
https://t.co/eHcLPnF9VI

[Dr. Daniel Bender @drdanielbender]

Two weeks did pass super fast again and it is time to discuss AI news and lessons learned / best practices in a space today. 👇 https://t.co/iGFkcZ6id8

[Dr. Daniel Bender @drdanielbender]

Prepare yourself for the intenet cutoff with a offline version of Wikipedia and more:
https://t.co/rkbeaqLBAo

[Dr. Daniel Bender @drdanielbender]

Super cool, thanks for the share @jtdavies!

Most interesting remaining questions:

Which of the Qwen 4 models will be released as open-weight?

What will be the size of the Qwen 4 Flash model?

Previous generations:

Qwen3.6-Flash: Based on the open Qwen3.6-35B-A3B (35B total parameters, ~3B active).

Qwen3.7-Flash: Closed model; exact parameter count not publicly disclosed.

Qwen3.8-Flash-Next (Qwen4 architecture preview): 125B backbone with ~6B active parameters, plus extra components (e.g., large N-gram embedding table).

[Dr. Daniel Bender @drdanielbender]

Open-weight music generation YuE2 on the level of Suno:
https://t.co/fgB1VN5Edc

[Dr. Daniel Bender @drdanielbender]

Looking forward to you joining for listening or participating in the conversation. 👇 https://t.co/LJjZHcegxU

[Dr. Daniel Bender @drdanielbender]

Take the @opencode hack where API keys were stolen and many $$$ were spent as a warning to check your API keys, even if you don't use OpenCode.

Open your OpenAI, Anthropic, and other provider accounts:
• Revoke old keys you no longer use.
• Rotate any keys that may have been exposed.
• For active keys, review your spending limits.

For OpenAI, make sure the spending limit is actually enforced, not just an email alert. The screenshot shows where to enable this.

An alert is useful, but if someone abuses your key overnight, you may wake up to the bill before you can stop it.

[Vishal Singh @vishalsingh2972]

@opencode FUCK OpenCode got hacked 😳

how will they compensate us for the losses?

[Dr. Daniel Bender @drdanielbender]

@krech565 @thsottiaux Cool. Thanks for letting me know. 👌

[Dr. Daniel Bender @drdanielbender]

@stlhood Another happy Strix Halo user here saying "Hi" ✌

I die buy mine with 128 GB last year for $1.8k, a price not available anymore due to the increased memory prices.

Nevertheless, still the currently best setup to run small MOE models.

[Dr. Daniel Bender @drdanielbender]

Hi Alex, ich habe deinen Post under mit diesem auch deinen X account auf dem Weg mit der DB zur Buchmesse gefunden.

Würde mich über ein Rezensionsexemplar, bevorzugt als ebook, freuen. Im Austausch biete ich eine Amazon Rezension und werde etwas in meinem bald wieder startenden Newsletter über dein Buch schreiben.

[Dr. Daniel Bender @drdanielbender]

Well spotted, and thanks for your reply!

Both sides count chunks during streaming and use the server-reported completion token count for the final tokens/sec calculation, which is shown in the end.

Chunk rates are not used to decide the winner because, as you stated, chunks can vary in size depending on the framework used.

Fair point that the changing labels make the video confusing.

[Dr. Daniel Bender @drdanielbender]

Side-by-side test of qwen3.8-flash-next running locally:

• AMD Strix Halo 128GB with Halogen by @high52weeks
• RTX 3090 desktop with 94 GB of memory, using Strata by @coldniko

Both at ~50 tok/s and it feels good to power my Hermes agent with it for privacy-sensitive tasks. 👇

[Dr. Daniel Bender @drdanielbender]

@Father_TimeLine Nice! Impressive numbers for a 4090. Which quant are you running?

I used Unsloth's UD-IQ4_XS in my experiments so far.

[Dr. Daniel Bender @drdanielbender]

Wait, OpenAI only delivered 30 token per second?!

And now 50 TPS means my latest local AI is experiments with Qwen3.8-flash-next is now on par with the speed I get from OpenAI. 😳

[Tibo @thsottiaux]

Reaching 50 TPS instead of 30TPS, with also the most optimized tokenizer out there. And the models are quite the efficient ones in terms of number of tokens needed to get things done!

[Dr. Daniel Bender @drdanielbender]

@sorrydudelmao You are right, but ultimately, many tasks we give the frontier models are not complicated at all.