Inference

AI inference APIs at 90% lower cost. Switch in two minutes. Sign up today and start saving.

[Sam Hogan 🇺🇸 @samhogan]

We’re introducing HALO 😇

Hierarchal Agent Loop Optimizer

HALO is an RLM-based agent optimization technique capable of recursively self-improving agents by analyzing their execution traces and suggesting changes.

This work is inspired by the Mismanaged Genius Hypothesis proposed by @a1zhang and @lateinteraction earlier this month.

tldr; we improved performance on AppWorld (Sonnet 4.6) from 73.7 --> 89.5 (+15.8) by giving HALO-RLM access to harness trace data and asking it to identify issues.

The feedback from HALO surfaced failures in the harness such as hallucinated tool calls, redundant arguments in tools, refusal loops, and semantic correctness issues. Each issue mapped cleanly to a direct prompt update.

We then fed these finding into Cursor (Opus 4.6), and asked the coding agent to update the underlying harness.

We repeated this trace -> HALO-RLM analysis -> code update loop until the score plateaued.

Today we’re open-sourcing the core HALO-RLM framework, evals, and data for further review.

[Sam Hogan 🇺🇸 @samhogan]

Qwen3 is live on @inference_net

https://t.co/QJAb6MS9Wf

[Respan @RespanAI]

Excited to announce our integration with @inference_net - a new LLM provider hosting faster and more cost-effective open-source models!

Inference[dot]net can reduce your LLM costs by up to 90%. Models like Llama 3.3 70B, DeepSeek V3, and DeepSeek R1 are significantly faster than those from other providers.

We wrote a detailed blog post about inference[dot]net and ran multiple tests comparing its models with others. The results are impressive!

Check out the details below 👇

[Ibrahim Ahmed @atbeme]

So @GroqInc is cool, but it’s no longer magic

With basic out of the box optimizations from SGLang we’re achieving over 250 tokens per second in llama 3.1 8b at full precision on a 4090 (consumer gaming card) @inference_net

And we think we can go even higher (1k)

[Sam Hogan 🇺🇸 @samhogan]

Inference .net is hiring a Developer Relations Lead in San Francisco! If you:

- Love working with AI to amazing build products
- Have experience creating world-class educational content for developers
- Enjoy solving developer problems and fostering community growth

We want to hear from you. Please send relevant information including GitHub, LinkedIn, and something amazing you have built to sam @ inference .net.

If you introduce us to someone we hire, we will pay you a $10k CASH referral bonus.

Full job description below 👇

[Respan @RespanAI]

We worked with @OpenAI to build a native integration for the OpenAI Agents SDK. Today, we are very excited to launch as a tracing processor.

With just a few lines of code, you can trace all your agent workflows and debug them much faster.

Check out this quick demo to get started:

[Inference @inference_nett]

Try Google Gemma3 👇

https://t.co/LdxroMMOWe

[Inference @inference_nett]

Inference Weekly Roundup 3/21/25

[Ibrahim Ahmed @atbeme]

We're excited to be one of the first inference APIs to host Gemma 3, starting with the 27b variant @inference_net !

It has incredible vision capabilities, rivaling models that are 2-6x its size in quality

Give it a try and let us know what you think!

[Inference @inference_nett]

@deepseek_ai Try DeepSeek R1 for free https://t.co/QGzDH6rtfa

[Inference @inference_nett]

Weekly roundup 2/28/25

1. Support for new models from @deepseek_ai:
- DeepSeek R1 - $4/4
- DeepSeek V3 - $2/2
- R1 Distill Llama 70B - $.40/$.40
2. Support for image inputs in VLMs like llama-3.2-11b
3. Revamped website and model playground

Get started today 🤖👇

[Sam Hogan 🇺🇸 @samhogan]

R1 from @deepseek_ai is live on @inference_net!

R1 has one-shotted questions that O1 Pro stumbles on.

$5.00/million tokens. Top tier reliability.

Try it today 👇

[Sam Hogan 🇺🇸 @samhogan]

The inference .net grants program is live!

Apply and get $10k for AI projects that need LLM inference from top models:

- R1 and V3 from @deepseek_ai
- Llama 70b and 405b from @AIatMeta
- High rate-limits, batch support, and more

If you're building open source, read below  👇

[Sam Hogan 🇺🇸 @samhogan]

We are now serving large models on @openrouter

- DeepSeek V3
- @NousResearch Hermes 3 405B
- DeepSeek R1 coming soon

[Sam Hogan 🇺🇸 @samhogan]

DeepSeek V3 is now available on https://t.co/aKDbEohpnI

[Sam Hogan 🇺🇸 @samhogan]

We just updated pricing on @inference_net.

We are once again the most affordable LLM inference provider on the market

[Sam Hogan 🇺🇸 @samhogan]

If you're working on synthetic data generation and need cheap batch inference, hit my DMs.

We're cooking up something specifically for this on inference .net https://t.co/aKDbEohXdg

[Sam Hogan 🇺🇸 @samhogan]

This log scale graph from @openrouter clearly shows the cost of LLM inference has been decreasing exponentially since the release of GPT-3 in late 2022

New inference providers like @inference_net will continue this trend into the foreseeable future 👇

[Sam Hogan 🇺🇸 @samhogan]

The cost of LLM tokens has dropped more than 50% over the last 12 months.

@GroqInc was leading the pack for a while, but just isn't competitive any more

Llama 3.1 70B tokens on https://t.co/gjRnSxad5X are now 63% cheaper than @GroqInc and almost 90% cheaper than @replicate

[Inference @inference_nett]

hello world.

we are live in beta.

visit our website for more information.

[Sam Hogan 🇺🇸 @samhogan]

inference .net / @inference_net