Novacula

AI inference engine for faster, more efficient model serving.

[Novacula @novaculaio]

Novacula is moving into AI.

We’re shifting our focus from blockchain infrastructure to building an AI inference engine — with performance and efficiency at the core.

Novacula Nodes → Novacula AI.

#AI #AIInfrastructure #Inference

[Novacula @novaculaio]

Different answers don’t always mean different model quality. Here’s why task-level results and repeated runs matter more than exact output matching when evaluating quantized models: https://t.co/WQcnQPoslK

[Novacula @novaculaio]

We tested long context separately too.

On Qwen3.8-27B vs BF16:

Needle retrieval: 1.000 vs 1.000
RULER: stable at 32K and 128K
LongBench v2: 302 vs 300 / 503
The drop showed up in real-text QA, while synthetic tasks stayed stable.

[Tudor Iustin @tudoriustin_]

@novaculaio 7.13 bits/weight with no measurable loss across 1,015 tasks is a strong claim. Did the eval include long-context and needle-in-haystack tasks, or was it mostly short-prompt accuracy where mixed-precision usually survives?

[Novacula @novaculaio]

Does a smaller AI model always mean faster inference?

Not necessarily. At NVCL AI, we tested how quantization affects model quality, speed, and memory usage.

Read the full article 👇

https://t.co/Jzinc9KM9J
#AI #LLM #ModelQuantization #NVCLAI

[Novacula @novaculaio]

@tudoriustin_ Yeah, we tested long context separately too. On Qwen3.8-27B vs BF16, needle retrieval was 1.000 vs 1.000, RULER stayed level at 32K and 128K, and LongBench v2 was 302 vs 300 out of 503.

Full results: https://t.co/ajusn5L71R

How's long context holding up for you on Gaudi 3?

[Novacula @novaculaio]

NVCL AI Standard uses about 7.13 bits per weight.

On our 1,015-task evaluation suite, Standard showed no measurable quality loss.

Free for personal use:
https://t.co/S9VX4mqA73
Commercial trial licence:
https://t.co/bJwOYFHRyO

#AIInference #LLM #Quantization

[Novacula @novaculaio]

Read more here:

https://t.co/Lewdgak0OI

[Novacula @novaculaio]

Benchmark context matters.

Model, GPU, concurrency, prompt length and decoding settings can change the result a lot.

What do you check first when you see an inference benchmark?

#AIInference #LLM

[Novacula @novaculaio]

“2× faster” doesn't tell the whole story.

We break down AI inference benchmarks using NVCL AI vs vLLM results.

👇 Read more:
https://t.co/wapAAcy3Gq

#AIInference #vLLM

[Novacula @novaculaio]

Setup: Qwen3.8-27B · ~4K-token prompts · outputs up to 2,048 tokens · thinking on · one H200 · vLLM 0.32 BF16 with dynamic draft depth · measured 24–25 Oct 2026.

Standard: ~7.13 bits/weight, with no measurable quality loss on our 1,015-task suite.

[Novacula @novaculaio]

On one NVIDIA H200, NVCL AI Standard produced 1.34–2.52× the total output of vLLM 0.32 BF16 across the 1–32-user loads we measured.

The 2.52× one-user result used our separate draft model.

#AIInference #LLMInference #vLLM

[Novacula @novaculaio]

Inference performance is only part of the deployment problem.

As @nvidia brings new GPU platforms forward, the runtime, networking and orchestration stack around them has to keep up too.

#AIInference #NVIDIA #GPU

https://t.co/oREk291LbA

[Novacula @novaculaio]

Throughput is an important inference metric, but it does not tell the whole story.

For agent workloads, latency and task-completion time matter too.

What do you look at first: output per GPU, latency, or task-completion time?

#AIInference #LLM #AIInfrastructure

[Novacula @novaculaio]

Want to test NVCL AI?
Apply for beta access here:
https://t.co/bJwOYFHjJg

Docs and access details:
https://t.co/57J2O92DXz

[Novacula @novaculaio]

Novacula AI is our inference engine for open models.

Built in C++/CUDA, it works through an OpenAI-compatible API and uses our own weight formats.

Standard uses ~7.13 bits/weight, with no measurable quality loss on our 1,015-task suite.

#AI #Inference #AIInfrastructure

[Novacula @novaculaio]

@bibryam Blockchain nodes have a similar failure mode: 200, flat latency, while the node is actually blocks behind. Re-checking on a config change covers what you changed. Do you also re-run checks on a schedule to catch upstream or provider-side changes?

[Novacula @novaculaio]

@NilooNiloos Block-number timers quietly change meaning when block time changes. Is there tooling people use to find that logic before an upgrade, or is it mostly manual review for now?

[Novacula @novaculaio]

@zkjays @MinaProtocol @o1_labs The health check tools are the part of 4.0.1 I'd look at first. Do they check that the daemon is actually keeping up with the chain, or mostly that it's running?

[Novacula @novaculaio]

@Mergentmedia A path that survives when fiber, towers and roads fail together is the real point. For anyone running infra there, would it end up as a primary link or purely a backup?

[Novacula @novaculaio]

@getblockio Reorg-corrected is the part I'd want to understand better. How does the consumer find out that a delivery has changed — do you emit a new event pointing to the old one, or replace the original?