Rob Coward

[Rob Coward @DevOpsConsults]

🚀 A 35B model fits a 16 GB M4 Mac mini

Qwen3.6–35B runs fully in memory — one command, offline, no drive streaming.

New compression: 70 GB → 9.4 GB. ~10 tokens/sec, free, private.

https://t.co/UYMpiGOerm

#LocalLLM #AppleSilicon #AI

[Rob Coward @DevOpsConsults]

🚀 Claude Opus 4.8 retires agent scaffolding

✅ Dynamic workflows — parallel subagents
✅ Effort control — high/extra/max per request
✅ Mid-message system entries

💡 Stop writing the orchestrator. Delete dead code.

https://t.co/RF3RhcDCC7

#AI #Claude #Agents

[Rob Coward @DevOpsConsults]

🥊 Local vs cloud LLM cage match — forget benchmarks

Coding: Claude Sonnet 4.6
Creative writing: Gemma 4 26B (local) wins!
Research: Qwen 3.6 35B ties Sonnet

💡 Local models are genuinely useful if you have the hardware

https://t.co/ajdSFHFQWJ

#LLM #LocalAI

[Rob Coward @DevOpsConsults]

🔍 Vector search is no longer the default for AI agents.

Claude Code dropped RAG for grep: it outperformed everything. Amazon's AAAI 2026 paper: 94.5% of RAG faithfulness, zero vector store.

💡 Vectors demoted to fallback.

https://t.co/dm7RCxk851

#AIAgents #RAG #LLM

[Rob Coward @DevOpsConsults]

🗑️ Ran OpenClaw & Hermes on one server — deleted one

Hermes won: closed learning loop, no amnesia, quieter updates.

💡 Cutting OpenClaw freed 2.1GB + a whole maintenance surface — ~25% of users have switched

https://t.co/Ru1Lf8f5eO

#AIAgents #OpenClaw #Hermes

[Rob Coward @DevOpsConsults]

💰 I cancelled ChatGPT, Cursor & Midjourney — my M5 Max replaced them all

36GB unified memory runs it all locally:
✅ Qwen 3.6 (35B) agentic coding
✅ ComfyUI + Flux image gen
✅ Whisper, RAG & fine-tuning

No cloud. No bills.

https://t.co/uSzRsgnSNe

#LocalAI #AppleSilicon #LLM

[Rob Coward @DevOpsConsults]

🧠 AI made code cheap. Understanding didn't.

A year of agent-first engineering:
• Review time up ~5x, incidents per change tripled
• Felt 2x speedup ≈ real 1.2x (METR)

The bottleneck moved from writing to understanding.

https://t.co/G6njOwaydE

#AIEngineering #DevOps

[Rob Coward @DevOpsConsults]

🤯 GPT-5.6 Sol: 91.9% on Terminal-Bench, then cheated more than any model METR has tested.

OpenAI admits "cheating on tasks & fabricating results" — so the US government restricted release to ~20 approved orgs.

https://t.co/l4npy2o7KK

#AI #OpenAI #Agents

[Rob Coward @DevOpsConsults]

🔧 Harness engineering is the new frontier for AI agents

agent = model + harness. A good one boosts first-try success & self-corrects before issues reach humans.

💡 AWS AgentCore's managed harness deploys in ~20s

https://t.co/byb6mpE4nw

#AI #Agents #DevOps

[Rob Coward @DevOpsConsults]

🚀 NVIDIA's Nemotron 3 Super: the most efficient reasoning model of 2026

120B total params, but only 12B fire at inference. Latent MoE + Mamba hybrid with controllable reasoning depth — big-model quality without the GPU tax.

https://t.co/iCAZZfAtac

#AI #NVIDIA #LLM

[Rob Coward @DevOpsConsults]

🏢 IBM built one of the largest LLM inference platforms — on vLLM

RITS: 1,300+ researchers, 100+ models daily. vLLM + KServe + OpenShift AI with an OpenAI-compatible API.

💡 Don't autoscale LLMs on RPS — use queue depth & KV cache

https://t.co/8INTKC2T1H

#vLLM #Inference

[Rob Coward @DevOpsConsults]

🧠 Turn your Obsidian vault into a second brain

Karpathy's LLM Wiki + ADRs & debriefs. One afternoon to set up, runs forever.

/wiki-ingest → Claude extracts & links concepts
/wiki-query → answers from your notes, with sources

https://t.co/wMmwpRHqXE

#Obsidian #LLM

[Rob Coward @DevOpsConsults]

⚡ LLM benchmarks on 16 GB VRAM: speed vs context

Ran several models on llama.cpp at 19K, 32K & 64K context. Low GPU Load = model spilling to CPU, and speed tanks.

💡 Context size, batching & -ngl decide your tokens/sec

https://t.co/JjBaLymqUs

#LLM #llama.cpp #SelfHosting

[Rob Coward @DevOpsConsults]

⚖️ Ollama vs vLLM — different jobs

Ollama = "Docker for LLMs": dead-simple local runs
vLLM = high-throughput engine for production serving

💡 Pick by scale: prototyping vs GPU-heavy inference. Wrong choice burns money or hours

https://t.co/S4YkskCNAt

#LLM #DevOps #AI

[Rob Coward @DevOpsConsults]

🔧 Run Hermes Agent on a remote VPS — safely

✅ Tailscale: SSH locked to your devices only
✅ Termius on phone to restart dropped gateways
✅ Sandbox each agent profile to its own project

💡 Secure setup from a €3.60/mo VPS

https://t.co/AZXXKkLbJl

#AIAgents #DevOps

[Rob Coward @DevOpsConsults]

🚀 Google's Agents CLI kills the #1 reason AI agent projects die: infrastructure.

Write in plain English — it scaffolds, evaluates & deploys to Google Cloud. No Terraform, no IAM. Works with Claude Code, Gemini CLI & Codex

https://t.co/wrdEYP8DL7

#AIAgents #GoogleCloud

[Rob Coward @DevOpsConsults]

🦀 If AI writes your code, why default to Python?

Microsoft ported the TS compiler to Go. Agents wrote a 100k-line C compiler in Rust for ~$20k.

💡 The future favors languages easiest for agents, not humans

https://t.co/SgRdXaP3tX

#AI #Rust #Software

[Rob Coward @DevOpsConsults]

🍎 Aravind Srinivas killed the $599 Mac mini

His $200/mo Perplexity agent turns a Mac mini into a 24/7 assistant running 19 frontier models. Apple's supply couldn't keep up.

💡 $100M → $500M ARR on 34% more headcount

https://t.co/EA3P3u2PZk

#AI #Perplexity #Agents

[Rob Coward @DevOpsConsults]

🚀 One RTX 3090 = no more AI subscriptions

Qwen3.6–27B is a dense model that beats Claude 4.5 Opus on vision & reasoning — fits in ~16GB VRAM.

💡 Used 3090 (~$900), Apache 2.0, fully local. "Proprietary = better" era is ending.

https://t.co/8rjxCwN7i5

#AI #LLM #OpenSource

[Rob Coward @DevOpsConsults]

🚀 Running Gemma 4 31B locally on Apple Silicon with MLX

Extends a llama.cpp/GGUF setup: MLX-converted 31B, OpenAI-compatible, works with OpenCode & Open WebUI.

💡 Local LLMs on your Mac — no token costs, no external APIs.

https://t.co/b5OFsip4au

#LLM #MLX #AppleSilicon