friendliai

The Frontier Inference Cloud for Agents

[friendliai @friendliai]

Our new blog covers how @kilocode , @friendliai, and @NVIDIAAI fit together for production AI coding agents, with Kilo seeing 7x faster inference and up to 72% lower costs routing to Nemotron 3 Ultra.

Read it: https://t.co/t9iMdt1foR

[friendliai @friendliai]

GLM-5.2 and Gemma 4 both hit #1 for throughput on @openrouter this week.

📊 GLM-5.2: #1 throughput, all providers
📊 Gemma 4 31B: #1 throughput, all providers

Run frontier open-weight models on FriendliAI:
⚡ https://t.co/PXQduA2Y1E
⚡ https://t.co/XIt1UIRSxX

[friendliai @friendliai]

▶️ Full Model APIs Quickstart here: https://t.co/9BfsSmHgYk

[friendliai @friendliai]

The hardest part of using FriendliAI Model APIs? Honestly, picking which model to use.

Our new quickstart tutorial covers the rest — here, we showcase switching from @Zai_org's GLM to @deepseek_ai's DeepSeek-V3.2 in seconds by pasting one identifier.

[friendliai @friendliai]

Provider A beats Provider B on the benchmark. Pick A?

Not so fast.

A single benchmark point = one operating condition. Production workloads don't run under one condition.

Compare Pareto fronts instead — the full set of latency-throughput trade-offs a provider can actually deliver.

👉https://t.co/A1pK1CRU3Z

[friendliai @friendliai]

Our CBO Brian Yoo is speaking at @Ai4Conferences 🎤

The Frontier AI Inference Cloud for Agents
📅 Aug 5, 11:20 AM | 📍 Delfino Ballroom 4104

Building agents? Listen in.

[friendliai @friendliai]

https://t.co/4hJRzrCzad

[Vercel @vercel]

https://t.co/GHeZXAQ6gl

[friendliai @friendliai]

Welcome to FriendliAI, Brian Cho! 🎉

Joining as Director of Engineering with 6 years at @Meta and 4 at @AptosLabs under his belt — right as our engineering team scales to match the pace of the agent era.🚀

[friendliai @friendliai]

NVIDIA Nemotron 3 Embed available day 0 on FriendliAI

Nemotron-3-Embed-1B-BF16 👉 https://t.co/hkrXmZz7bB
Nemotron-3-Embed-8B-BF16 👉 https://t.co/VCAscF5EY4
Nemotron-3-Embed-1B-NVFP4 👉 https://t.co/hz6TFAyW9v

[friendliai @friendliai]

Frontier retrieval accuracy AND topping the RTEB leaderboard...

We're in!

[NVIDIA AI @NVIDIAAI]

Today we released Nemotron 3 Embed 8B and it reached #1 overall on RTEB 🏆

RTEB benchmarks retrieval accuracy across real-world tasks. Better retrieval gives agents more relevant context, helping improve response accuracy.

[friendliai @friendliai]

NVIDIA Nemotron 3 Embed is now on FriendliAI.

Open embedding models built for agentic retrieval:

→ 8B: frontier accuracy, #1 on the RTEB leaderboard
→ 1B: designed to retain 95%+ of the 8B's accuracy, built for scale
→ 16K context, text + code, fully open weights, datasets, recipes, and fine-tuning guidance

Deploy via Friendli Dedicated Endpoints, OpenAI-compatible API.
👉https://t.co/q77AX3ZcHm
👉https://t.co/oPQgZZKNmE
👉https://t.co/G67AcO3Bn8

Read full blog 🔗https://t.co/ce5J8ELfe1

[friendliai @friendliai]

Your structured output fills with endless whitespace instead of JSON. Engine bug?

Probably not.

Constrained decoding masks invalid tokens, but it can't pick which valid token comes next. Whitespace is valid JSON grammar. If your prompt doesn't signal the expected fields, the model can stall on newlines until the token limit.

🧑‍🔧The fix: put a concrete example output in your prompt. Bonus: on FriendliAI Endpoints, response_format is up to 21% faster — narrowing the token set boosts draft acceptance in speculative decoding.

Prompt guidance + constrained decoding. You need both.

🔗 https://t.co/3CvmXp0CGj

[friendliai @friendliai]

FriendliAI will be at @Ai4Conferences 2026!

Swing by booth #401 to see how FriendliAI helps you build faster agents.

See you there 👋

[friendliai @friendliai]

Diagnose first, then tune. Full guide: https://t.co/6zMWeAfiFs

[friendliai @friendliai]

When your inference benchmarks fall short of latency or throughput targets, the fix depends on the problem.

Reduce latency:
→ enable speculative decoding
→ upgrade your GPU
→ increase tensor parallelism

Increase throughput:
→ enable Host KV Cache + raise max batch size
→ upgrade your GPU
→ scale out with replicas

[friendliai @friendliai]

How does @kilocode run 7x faster?

The stack:
→ Kilo Code orchestrates
→ FriendliAI serves
→ @nvidia Nemotron 3 Ultra reasons

Kilo's internal evals: up to 7x faster inference, 72% lower costs on complex agentic tasks, fewer tool-call errors.

Full breakdown: https://t.co/P8KIZIDBNm

[friendliai @friendliai]

We built a tower defense game with Claude Opus 4.8 and GLM-5.2 on FriendliAI.

For the same task, GLM-5.2 was:

→ 5.6x cheaper overall
→ 7.2x cheaper per request

Try GLM-5.2 on FriendliAI: https://t.co/p9OApVK2wG

[friendliai @friendliai]

Choosing an operating point is a business decision.

Define your SLA first — any config that breaks it is invalid
Build a Pareto frontier — keep only configs not beaten on both speed + throughput
Pick your point: latency-first, cost-first, or the knee point (usually best)

The knee point is your real ceiling — where latency starts climbing faster than throughput improves. Not where the curve flattens.

Full guide: https://t.co/SWpuZTA9lf

[friendliai @friendliai]

Day 1 of @icmlconf 2026 is here! 🎉

The FriendliAI team is at Booth B115 all week — come say hi.

We're the team behind continuous batching, and we spend our days making frontier open-weight model inference faster, cheaper, and more reliable. If that sounds like a problem you'd want to work on, we'd love to talk.

We're actively hiring across engineering and beyond. Stop by the booth to meet the team, ask us anything about inference, or just to have a chat.

👉 Open roles: https://t.co/862HK4fDIn

See you at B115!

#ICML2026 #Hiring #AIInference #FriendliAI