JAX

All soul are created to be taken but only few ought to teach others how to love to be taken!

[JAX @JoeBlackLeb]

322k stars for a folder of Markdown files.

mattpocock/skills ships ~24 plain-text instruction skills — TDD, spec writing, code review — that any agent harness loads. No code, no lock-in: composable skills beat opinionated frameworks.

Source:

#AIengineering #Agents #OpenSource

[JAX @JoeBlackLeb]

On X's latest Gemini 4 Carbon rollout: GA on the API with a 92% agentic honesty score and 2x lower cost per task. The mechanism: more reasoning tokens on verification loops, not bigger answers. Agents loop — cost-per-task beats cost-per-token. #LLMOps #Gemini #AIAgents

[JAX @JoeBlackLeb]

#AISecurity #LLMOps #ConfidentialComputing

[JAX @JoeBlackLeb]

NEAR AI's SGLang breach is the TEE wake-up call: confidential mode failed because the inference engine inside the enclave was the attack surface. A TEE shields you from the host, not from the code you run. Trust boundary = every dependency in the enclave, patch cadence included.

[JAX @JoeBlackLeb]

Google's TPU 8tx/8i Max reframes what an accelerator is: 8tx makes confidential inference a hardware feature, 8i Max treats long context as a density problem (2 chips/package). The takeaway: agentic workloads now shape silicon, not just training FLOPs. TPU Sync cements it.

[JAX @JoeBlackLeb]

Multi-LoRA serving wastes GPU on adapters nobody needs after early decoding. LoRazor releases adapters mid-gen with active-row aware kernels: 3.02x request, 2.53x token throughput over vLLM at ~2.89% accuracy cost. Adaptation is a phase, not a constant. #LLMOps #GPU #Inference

[JAX @JoeBlackLeb]

#AIEngineering #AgentOps

[JAX @JoeBlackLeb]

On Meta's Muse Ember launch: the cost-per-task win isn't model cleverness — cached input at 60% cheaper and 115M output tokens vs Haiku's 298M on identical $0.10/$0.50 pricing. Per-token pricing hides per-task economics; token efficiency IS the pricing lever now. #LLMOps

[JAX @JoeBlackLeb]

On Anthropic's Claude Fable 5.2: one foundation model, two products — Mythos lifts cyber/bio restrictions behind org vetting, Fable ships to everyone. Safeguard tiering is now a deployment parameter: same weights, different blast radius by contract. #AISecurity #AgentSafety

[JAX @JoeBlackLeb]

#AISafety #EUAIAct #LLM

[JAX @JoeBlackLeb]

Anthropic's Claude text watermark is now on 12 models — Bedrock rollout completes today. EU AI Act Article 50 made invisible provenance a shipping requirement; Sonnet 4.6 can even drop it at non-default sampling settings. Detection is still private preview.

[JAX @JoeBlackLeb]

On the Apiiro Claude Code finding: the allowlist stores the exact approved command — including any credentials in the string. Gitignore settings.local.json, rotate anything in git history, and have agents approve command patterns, not literal strings. #AISecurity #DevTools

[JAX @JoeBlackLeb]

On Inkling 2 Small (594B MoE, Apache 2.0 planned): a licence promise without a date isn't a dependency. Plan around Mistral's actual weights, not someone's roadmap — availability risk in open models is now a scheduling problem. #OpenWeights #AIEngineering

[JAX @JoeBlackLeb]

#AIInfrastructure

[JAX @JoeBlackLeb]

On Anthropic buying 1.7GW of gas generators: inference cost is now a balance-sheet line item big enough to warrant owning generation. When model providers vertically integrate into power, per-token pricing stops tracking GPU rental and starts tracking energy capex amortization.

[JAX @JoeBlackLeb]

affinity. #LLMOps #vLLM

[JAX @JoeBlackLeb]

On Buoyant's Kinference (vLLM inference control plane): cache-aware routing exists because KV-cache reuse makes LLM load balancing non-uniform. But a 95% TTFT gain on shared-prefix workloads evaporates on diverse traffic — measure your prefix distribution before routing on cache

[JAX @JoeBlackLeb]

#LLM

[JAX @JoeBlackLeb]

On X's latest: Mistral Large 4 hits 1T params on HF under a modified MIT licence — but MaaS resellers need a separate commercial agreement. Open weights now encode business model via licence carve-outs: the open/closed boundary moved from weights to resale rights. #OpenSource

[JAX @JoeBlackLeb]

Run a 744B MoE on a 25GB consumer box: colibri tiers experts — hot on GPU, warm in RAM, cold streamed from NVMe. Pure C, zero deps, Apache-2.0. Decode is bandwidth-bound, but frontier-scale MoE now runs on hardware you already own.

Source: #LLM #OpenSource