Andy Ritchie

Marketing Automation @ Cerity Partners.
On a mission to clone myself with @NousResearch

[Andy Ritchie @imandyritchie]

Last year, my beauty salon generated over 100K, and I worked less than 30 minutes a day on it! How? Well, I automated nearly every aspect of the business. Check out how

https://t.co/ydNLILwF1b

#business #automation #marketing #homeassistant #hass #systems #entrepeneurship

[Teknium 🪽 @Teknium]

Qwen 3.8 Max and a new local 27B Qwen 3.8 is coming!

Thanks @Alibaba_Qwen for featuring Hermes Agent in the release video!

[Teknium 🪽 @Teknium]

So much in the Herald Release of Hermes Agent

- Voice Activated Chats (Replace your google home and alexas!)
- Plugins and Kanban for the Desktop GUI App
- Agent2Agent Protocol
- Outbound Webhooks
- Massive performance optimizations and token efficiency milestones reached
- Buzz Integration
- Nous Portal's 20% Discount on all models
- Way more reliable local and open models
- New office/productivity skills
- Grounded research skill
- Secrets lockdown with Ironproxy

and like 500 other things. Check them all out!

[Andy Ritchie @imandyritchie]

@Teknium Dude the biggest update you have to add is multiplayer. Have multiple people be able to message the agent with multiple sessions unlocks huge unlimited scaling in corporations

[Andy Ritchie @imandyritchie]

@SeijinJung Yeah man, i dont want to hook it up to live environments but its clever how youve got it to create the design docs, strategy, market research and profile, I think these data stores are going to want to be internal, how are you getting around that?

[Seijin Jung @SeijinJung]

introducing Helena, the world's first self-improving AI marketer.

$10M in sales driven for 20,000 businesses already.

everyone's shipping an AI that just makes more slop:

so we went further.

Helena grades her own work, and rebuilds the ones that miss - including the ones she built.

what she does:
➤ runs loops to optimize Meta/Google ads & social
➤ drafts SEO content that ranks, straight to WordPress/Webflow/Framer
➤ optimizes emails in Klaviyo/Mailchimp/Brevo
➤ sends you a daily brief: what worked, what she's changing

real numbers:
➤ grew organic traffic 35%+ for hundreds of brands
➤ 2x-ed a DTC brand's sales in 5 weeks
➤ scaled a dead ad account from $0 to $10k/mo
➤ drove $40k email revenue in 6 weeks

100+ skills. 3,000+ integrations.

purpose-built by marketers who actually scaled hundreds of businesses.

just your url. all done in less than 3 minutes.

try it now 👇

https://t.co/LbKm7d2Fjk

[Andy Ritchie @imandyritchie]

@NousResearch is genuinely the crushing it; they're really keeping up with all the latest model releases, and their marketing is so different from everything else. One area I suggest you look at is home assistant. Its got great piping for turning the home into jarvis with hermes

[Andy Ritchie @imandyritchie]

@KinggTrades @AlexFinn I watch his videos but it's really hard to find the value it's 90% nonsense. It youre interested in demoing what you've done id love to connect, I work in finance and I've got Hermes set up pretty good for marketing automation agents from design to ops to ads, lmk

[Andy Ritchie @imandyritchie]

@KinggTrades @AlexFinn I think you're right

[Andy Ritchie @imandyritchie]

@MiaAI_lab Can we get a group show and tell going? I know people have made some awesome stuff and I wanna see it all

[Light Foundry Notes @light_foundry]

Full recipe: DeepSeek-V4-Flash-0731 + DSpark on 2× RTX PRO 6000 Blackwell (SM120)

Several people asked for the exact setup. Everything below is measured on my own system.

## MODEL

Official deepseek-ai/DeepSeek-V4-Flash-0731 (HF)

• FP8, ~167 GB
• Fits in 2×96 GB VRAM as-is
• No requantization required
• The DSpark draft model ships in the same repository

One small tip: downloading individual files with curl was much more reliable than the HF CLI, which repeatedly stalled for us.

---

## ENGINE

Current upstream vLLM and SGLang do not support this model.

You'll need a community vLLM build with:

• SM120 sparse kernels
• DSpark speculative decoding
• Built after Jul 31

The Jul 31 release introduced a new MTP norm layout. Older builds fail during weight loading with:

KeyError: ...mtp_block.main_norm.weight

I'm intentionally not naming the image publicly to avoid sending everyone to the maintainer. Happy to share via DM.

---

## LAUNCH (working configuration)

--tensor-parallel-size 2
--max-model-len 524288
--max-num-seqs 4
--gpu-memory-utilization 0.94
--kv-cache-dtype fp8
--speculative-config '{"method":"dspark","num_speculative_tokens":5}'
--tokenizer-mode deepseek_v4
--tool-call-parser deepseek_v4
--enable-auto-tool-choice
--reasoning-parser deepseek_v4
--block-size 256
--max-num-batched-tokens 2048
--disable-custom-all-reduce
--trust-remote-code

(prefix caching disabled)

num_speculative_tokens must be >= 5 (the DSpark block size). Anything lower is rejected during config validation before CUDA is even initialized.

---

## THE max_num_seqs LIMIT

max_num_seqs > 4 deterministically fails on the first request:

cannot reshape tensor of 0 elements

Root cause:

A hardcoded prefill chunk size (4) feeds an empty padding slice into the sparse MLA kernel.

Confirmed on both community builds we tested.

Cap max_num_seqs at 4.

Additional requests simply queue instead of crashing.

---

## WHAT TO EXPECT

Measured on this setup.

Code generation (temp=0)

• 235-246 tok/s single stream

General prose

• 130-145 tok/s

Aggregate throughput

• c2 ~300 tok/s
• c4 ~400 tok/s

Other numbers

• Prefill: 5-6K tok/s
• 384K needle retrieval: ~80 s
• KV pool: ~988K tokens

Why the spread?

The engine runs at roughly 50 verification steps/s.

Throughput is approximately:

tok/s = 50 × (1 + accepted draft tokens per step)

Acceptance depends heavily on workload:

• Code (temp=0): ~73%
• Prose: ~33%

Don't estimate throughput from SSE delta counts.

Those measure verification steps, not generated tokens.

---

## SANITY LADDER

1. Health check
2. Deterministic text
3. Usage-metered throughput
4. Needle (128K to 384K)
5. Tool calling
6. Confirm the max_num_seqs > 4 signature

If all six pass, you should have essentially the same environment I do.

Questions welcome.

[Andy Ritchie @imandyritchie]

@Abobsterina Who's video is this?

[kartiseira @Abobsterina]

HE BUILT AN ENTIRE AI COMPANY INSIDE A MAC MINI.

The creator calls it Bennett OS. At the center is Conductor, a super-agent that routes work to 43 specialist agents across sales, marketing, finance, client delivery and tech.

Drop in a document or voice note and it becomes part of a visual company brain. The system audits its own knowledge, tracks social growth across five platforms, monitors payments and shows which agent handled every task.

Estimated monthly economics:

AI models and cloud tools: $250
Comparable five-person team: $25,000+
Potential savings: $24,750 per month

The founder still makes the decisions. His AI departments handle the coordination.

This is what the next one-person company could look like.

[Andy Ritchie @imandyritchie]

https://t.co/F7GQuVPsoF

[Jordan Ross @jordan_ross_8F]

Turn your agency ops AI native:

https://t.co/HB6PmKBPjM

[Andy Ritchie @imandyritchie]

@Teknium Amazing. I have something I've custom made for this same thing with the voice pe from home assistant

[Andy Ritchie @imandyritchie]

@NousResearch Hell yes, now make an android app plz

[Andy Ritchie @imandyritchie]

@jordan_ross_8F I might join you