Confidential AI

End-to-end private inference and confidential GPU VMs

[Confidential AI @confidentialdot]

DeepSeek V4.1 Flash now live on Confidential Inference.

[Confidential AI @confidentialdot]

5/ Confidential AI no longer means giving up performance.

Full write-up by @b0xtch:
https://t.co/JN0JjA7Hdh

[Confidential AI @confidentialdot]

1/ Breakthrough: First production KV cache offloading implementation under confidential computing (CC).

Same gain as without CC.

NVIDIA B300s, CC on:
throughput: +141%
time to first token: 39 s → 1.5 s
GPU to host copies: 4 GB/s → 50 GB/s

[Confidential AI @confidentialdot]

4/ Running in production on MiniMax M3.

3.15x the output once GPU memory fills.

95% of prompt tokens served from cache, against 39% without offloading.

[Patrick @patrickwoodhead]

The SF billboard was a success. So we upped our marketing budget.

[Confidential AI @confidentialdot]

@Robertg761_ You can verify a cluster if you go to our website confidential dot ai and then click the button that say "verify a live cluster". This will show you a full attestation report

[Confidential AI @confidentialdot]

Confidential Kubernetes (C8s) just reached 100 GitHub stars.

Next target: 1000 stars

https://t.co/WRVFgQ7G9c

[Patrick @patrickwoodhead]

Our SF billboard

[Confidential AI @confidentialdot]

Confidential Inference.

Don't let anyone see your data.

[Mahmoud Shehata @b0xtch]

We wrote up how c8s secures the network inside a confidential Kubernetes cluster. Short version: a pod only gets a certificate after hardware attestation, only attested peers can connect, and certificates expire within hours.

https://t.co/a1w0AVqECj

[Patrick @patrickwoodhead]

POV: you're a prompt going through confidential dot ai private inference.

[Confidential AI @confidentialdot]

Confidential Kubernetes (c8s): v0.12 to v0.33 in three weeks. Some highlights:

- env and mounts pinned at admission
- GPUs fail closed unless in CC mode
- post-quantum attested channels
- one CVM per k8s node
- sealed nodes, no shell into running workloads

https://t.co/K6ZvnZsjqH

[Confidential AI @confidentialdot]

We open-sourced confidential-inference.

The code behind https://t.co/GddN6nNPwr: gateway, serving images, allowlist.

Every production deploy ships as a signed release bundle. Read the code, check the digests, verify the cluster.

https://t.co/IPVpw3PSh7

[Confidential AI @confidentialdot]

We're now listed on https://t.co/PVEOVCaMu7. Thanks @aaaljaz.

Nice detail: it shows whether each provider is post-quantum secure (which we are).

[aljaz @aaaljaz]

new provider (@confidentialdot) added to confidentialinference. net

[Confidential AI @confidentialdot]

Open-weight models hit 78.4% of token volume on Vercel's AI Gateway last week, up from about 30% 3 months ago.

They are flying.

They are also very complementary to confidential inference architectures, because they simplify the trust assumptions.

In short, anything that is public and open is much easier to verify than stuff which is proprietary and closed.

[Guillermo Rauch @rauchg]

Looks like today may be a record day for token volume % of open models on Vercel AI Gateway:

🟦 Open 78.4% 🟨 Closed 21.6%

While spend 💲 usually tells a different story, #3 and #4 today are Moonshot AI & DeepSeek. Adding Z⁠.ai, their combined spend surpasses OpenAI (#2).

(Do note that's the spend for inference of the model across providers (mostly in the US), not revenue going directly to the open weight labs.)

[Patrick @patrickwoodhead]

We're hiring at Confidential AI:

🏗️ AI Infrastructure Engineer
⚡ AI Inference Engineer
🔐 Low-Level Systems & Security Engineer

We're building end-to-end privacy for inference.

https://t.co/S1kcvGMCbp

[Confidential AI @confidentialdot]

The report is generated for your session. report_data binds the attestation to a nonce your browser just sent, so what you see was produced now, for you. Not cached, not replayed.

Same check every c8s client runs before it sends a sensitive byte.

https://t.co/nBZL8srlsy

[Patrick @patrickwoodhead]

You can now verify a live Confidential Kubernetes (c8s) cluster from your browser.

Every check runs in your browser tab.

Attestation chains to Intel's TDX root. Firmware, kernel and rootfs match the audited image. A pinned CA proves it is the expected cluster. Nvidia GPU attestation included too.

https://t.co/iYEfgmFpnL