Base Compute

Run AGI on-device

[Base Compute @basecmpt]

We’ve built the fastest inference runtime for LLMs on Apple Silicon, faster than MLX and llama.cpp.

BaseRT is up to 35% faster than Apple's MLX on token generation (decode) for small models, and 78% faster on prefill for bigger models.

Details below.

[Base Compute @basecmpt]

@Underfox3 thanks for sharing @Underfox3 . Support for ROCm and CUDA in the making :)

[Base Compute @basecmpt]

Our technical report on BaseRT, for anyone interested in benchmark details and what approaches we used to achieve best-in-class LLM inference on Apple Silicon.

[Base Compute @basecmpt]

We're Base Compute, an AI inference lab. Small team, Melbourne and Berlin.

Working towards running AGI on-device, because people should own the AI they use.

Try out BaseRT: https://t.co/2BNhCIf0KL

We're hiring:
https://t.co/CH91l2ELQv

[Base Compute @basecmpt]

We focused on Apple Silicon first.

NVIDIA, AMD and Intel are next.

[Base Compute @basecmpt]

The way we got here: we tune the GPU kernels for each specific chip and model instead of using one generic path.

Technical report:
https://t.co/OKgXxz3TaL

[Base Compute @basecmpt]

M4 Pro, 4-bit:

Decode
Qwen3 0.6B: BaseRT 465 tok/s vs MLX 344 vs llama.cpp 297.

Prefill
Qwen3 30B-A3B: BaseRT 738 tok/s vs MLX 415 vs llama.cpp 407.

You can run the benchmark on your own machine:
https://t.co/qQIROZm3qS

[Base Compute @basecmpt]

We’ve built the fastest inference runtime for LLMs on Apple Silicon, faster than MLX and llama.cpp.

BaseRT is up to 35% faster than Apple's MLX on token generation (decode) for small models, and 78% faster on prefill for bigger models.

Details below.