squawksquack

[squawksquack @blahsquak]

@jasoncwarner It's spot on is what it is. You've given us a model that works well in this middle weight class for ~128GB machines & Hermes harness is loving it. Would rather there be one model for agent & harness work, others for specalized work like web frontend vs. backend etc.

[squawksquack @blahsquak]

@sudoingX posting for completeness (90 day expire on links)
runner: https://t.co/2FelJYyg8M
findings: https://t.co/Rc5tC9eHVk

am aware I'm pushing it close to the OOM line, 5.556GB free with a few Hermes sessions running

[squawksquack @blahsquak]

@sudoingX Single DGX Spark. Laguna S-2.1 NVFP4.
DFlash set to 7x. 256K context.

29.26->93.40 tok/s from c=1 to 8
32.1ms->77.7ms TPOT mean (~20% faster)

w/o DFlash
18.85->81.31 tok/s from c=1 to 8
51.4ms->95.7ms TPOT mean

Hermes handling it well, some overthinking loops at deep context

[squawksquack @blahsquak]

@sudoingX @poolsideai try vllm-gb10:v0.25.1 for docker container, it has a pre-built correctly targeted kernel cache. I try to use your aeon-vllm-ultimate image where possible, but since there's no prebuilt flashinfer-jit-cache, it was too easy to take the spark down

https://t.co/Rc5tC9eHVk

[squawksquack @blahsquak]

@sudoingX hermes seems to love it. might need reasoning off for some agentic work, but we might have a really good winner here.

[squawksquack @blahsquak]

@StefanMaier @NVIDIAAI @AMD 2x sparks is handy but also where i draw the line.

got a RTX PRO 4500 in one host & 2x R9700’s in another. Sparks for experimentation and medium size model work. RTX/R9700’s for throughput. alternative is start renting from runpod etc.

[squawksquack @blahsquak]

@geogristle can be a bit pedantic though. have heard feedback that it prohibits use of greunds which can be offputting for some readers.

Worth looking at things like Caterpillar Technical English, IBM Easy English, and ISO 24620 as a generic standard, for different flavors to feed a LLM

[Matt Slotnick @matt_slotnick]

the most confusing part of the odyssey is how this took 10 years

[spencer @techspence]

If your 2026 security strategy is “we have EDR,” you’ve seriously fallen behind.

In 1,000+ hours of pentesting in 2025, I saw this pattern all the time. The orgs who invested in security tools beyond just EDR were able to identify our attack activity sooner and mitigate privilege escalation and lateral movement opportunities faster.

This was the same no matter the industry and size of the org.

I've seen small teams perform well and big teams perform bad. In some cases 3rd party SOC performed well and in other cases they performed bad. (like real bad :( )

The one commonality was how many “layers” of defense these organizations had. Eg defense in depth. Such as…

- Endpoint
EDR
App control
Deception
Hardened images
Regular pathing of 3rd party products

- Network
NDR
ITDR
Deception
Content filtering

- Identity
AD monitoring/auditing (which is unfortunately all too often neglected :( )
Logon restrictions
AD security features like protected users group & FGPP
Deception

To break this down further, we can look at it like this:

Prevent: app control, logon restrictions, Protected Users, content filtering, (and pentesting of course :O) )

Detect: EDR, NDR, ITDR, AD auditing, deception

Respond: EDR and ITDR automation, account lock, isolate host, block egress, disable risky auth paths

Contain: segmentation, least privilege, tiering model, admin workstation strategy, logon boundaries

Recover: backups, recovery plan, tested restores, incident runbooks

This of course is only scratching the surface, but it's a good start.

It’s not getting any easier to defend organizations. And of course many organizations don’t have the resources for "premium" products. Looking at you Universities. A problem for another post.

But to keep pace, your defensive strategy must move beyond 1 or 2 products/controls. Tools will fail, defenses will be circumvented, there will be holes. The way you mitigate that is by ensuring you’ve got multiple ways to defend at any given point.

TLDR;
One security tool is not enough. Need more layers. Good for winter and good for stopping bad guys.

[solst/ICE of Astarte @IceSolst]

Just saw a pentest firm report “no DNSSEC” as a finding, and maybe it IS time we replace all pentest firms with ai bots

[Grant Slatton @GrantSlatton]

have a rat getting into my attic somehow

calling exterminators to get quotes

one guy sounds like rats slept with his wife or something, guy was getting angry just talking about rats

found my contractor

[spencer @techspence]

It's the "boring" IT/Security work that prevents breaches

[squawksquack @blahsquak]

@MiaAI_lab @NVIDIAAI we’ve barely scratched the surface

it’s time to start democratizing fine-tuning models, specialized models, the tools are there but easing the learning curve and adoption will unlock some crazy, amazing things

[squawksquack @blahsquak]

@sudoingX Doing testing w/TensorRT-LLM & non-NVFP4 model now, given NVFP4+TP=2 over ConnectX produces garbage output due to vLLM 0.21.0 limitations, but the throughput measurement is still real.

[squawksquack @blahsquak]

@sudoingX Some fade at the deepest contexts. 7.61 → 6.86 → 6.57 at 100K→131K→200K. KV cache growth during gen starting to stress NCCL all-reduce? fade rate is mild (~3% across 100k->200k range), not end of world.

[squawksquack @blahsquak]

@0xSero what is that interface?

[Teknium 🪽 @Teknium]

Introducing Mixture of Agents 2.0 in Hermes Agent.

Combine any provider's models into a mixture of your own. Access your presets as if it were a normal model in Hermes.

Big improvement in our soon-to-release HermesBench against opus and gpt-5.5 with MoA using Opus & GPT together.