LocalAI

OpenAI Open Source alternative. LocalAI is a community, drop-in replacement API compatible with OpenAI for local CPU/GPU inferencing

[LocalAI @LocalAI_API]

LocalAI 4.8 is out. We just crossed 48.3k stars!

- `local-ai chat` is a TUI agent harness now.
- image to 3D, audio-driven talking avatars
- voice cloning first-class in the UI
- vLLM's serving architecture in C++20, no Python

386 PRs, 22 days, 25 contributors, 11 new👇

[LocalAI @LocalAI_API]

@gr3yg00 Thanks @gr3yg00 🫶

[LocalAI @LocalAI_API]

@ivanfioravanti oooor

[LocalAI @LocalAI_API]

@Teknium @iamlukethedev we got tons of support for 3d model generation here !

[LocalAI @LocalAI_API]

@ivanfioravanti @mudler_it 🫶

[Ivan Fioravanti @ivanfioravanti]

If you want a Local AI super star to follow here he is @mudler_it 🚀 He is the creator of LocalAI project that has ONLY 50K stars on github.

Keep pushing Ettore!

Link: https://t.co/5hlR9HutC9

[Geekus @GeekusFr]

LocalAI fait tourner texte, voix, image et vidéo en local, sans carte graphique obligatoire.

Une seule API compatible OpenAI, Anthropic et ElevenLabs.
Le lancement tient en une commande Docker.

cc @LocalAI_API
https://t.co/UlawvUFE1P

[Ettore Di Giacinto @mudler_it]

Runs out of the box on parakeet.cpp! Trying it out right now and adding it to @LocalAI_API !

[Ettore Di Giacinto @mudler_it]

Wow the locate-anything models just reached almost 11M download, and are in the most downloaded models of @huggingface , in the first page !

Thanks @fabiolalli for telling me, I didn't notice 😅

Now, I don't know about the download stats and how that's calculated, but, in any case, happy to see community is using my locate anything port to @ggml_org ! you can also use it in @LocalAI_API in a couple of clicks away, published for all platforms we support.

In any case, I'm happy to contribute to the Open source community! Let's push Open source Local AI!

[Ettore Di Giacinto @mudler_it]

A huge thanks to @niels_gron for supporting @LocalAI_API with a brand new M5 Max MacBook Pro!

Generous support like this helps drive open-source forward.

Excited to start optimizing and making all our projects work natively for Apple Silicon!

[LocalAI @LocalAI_API]

Release blog post: https://t.co/VaGR21nXL9

Audio understanding: https://t.co/MYy4daPJUe

Decisions API: https://t.co/bKfhH5JqWn

[LocalAI @LocalAI_API]

LocalAI 4.11.0 is out

- You can now transcribe recordings with speaker labels and detect sounds alongside speech
- New Decisions API
- Model failover
- text-to-animation with Kimodo
- Much more, check links below

Thanks to everyone who contributed, tested, and reported issues!

[Ettore Di Giacinto @mudler_it]

parakeet-redux is a really interesting beast. You can extract the VAD head and then, you discover that has better recall than Silero. Despite the footprint compared to it, offline might have other uses.

@vikhyatk cooked hard here, congrats for the release, and thanks for working in the open!

By the way, all Parakeet-redux models are now in parakeet.cpp (links below). And you can use parakeet now as standalone VAD with both.

[Ettore Di Giacinto @mudler_it]

Your local agent can now hear what was said, who said it, and what happened around them.

parakeet.cpp transcribes audio, separates speakers, detects 527 sound classes, and now enrolls and remembers named voices across recordings. One pass, local C++.

[Ettore Di Giacinto @mudler_it]

Nemotron-3-Diarization was just released by @NVIDIAAI , and it runs in parakeet.cpp now, next to Parakeet speech recognition. It runs already on Apple with Metal, CUDA, @AMD and with anything that Vulkan supports.

We (@LocalAI_API) also added sound detection in: @Xiaomi's CED models (527 sound classes), ported to C++ sitting alongside it. You can run it with a couple of click via API, all Local and open source.

Parakeet.cpp now runs all three in one pass and you get the transcript, who is speaking, and what else you can hear. This scene from Sprite Fright took 2 seconds on a CPU.

[Richard Palethorpe @jichiep]

I added full-body-control to my simulation using gem-x.cpp and GEAR SONIC (motion-bricks.cpp). The physically simulated character is controlled by my body movements and SONIC which tries to balance it.

Inference is done on @LocalAI_API which takes in video frames and produces reference movements.

Note that I don't think I'm getting the best performance here. There are probably some issues to iron between gem-x.cpp and motion-bricks.cpp.

[Ettore Di Giacinto @mudler_it]

A small update on what's going on in vllm.cpp:

- Minimax and LTX 2.5 got Lora support. Now you can Load fused-model, or either make lora load in runtime with prompts in vllm.cpp
- Support for Kev (@jaredpalmer), Laya and GLiNER2.5 (@fastinoAI ). Same SystemOne api, so you can switch quickly and have a fast Jev replacement, running locally
- Mia-AiLab/Qwen3.8-27B-EXL3-3.5bpw runs at ~53 tok/s at high concurrency (up to c32) with DGX Spark

I'm working now on the docs and making this a bit more easy to go, you'll also see all these models popping up in @LocalAI_API for a one-click install.

A big shout out our friends at @regolo_ai , for sponsoring us as part of their Builder Program and giving access to their API, thank you 🫶! I've also added them to my agent harness, nib, so you can /login and use their API directly, links👇

[Richard Palethorpe @jichiep]

I added kimodo.cpp animation gen & playback in https://t.co/gqJOZZ0O3E. So you can describe a motion, then it'll generate it using a @LocalAI_API instance and play it back on the physically simulated character.

Of course there are issues with finding animations that are physically possible, but that is part of the fun... maybe.

[Randy Webb @scottymicfree]

https://t.co/aHfIzG7Axt
Lucy's coming together, 100+ setting configurations, multiple subsystems, 4 different operator modes, and cloud integration is still safe. Norton and Microsoft security systems were not bypassed.
@LocalAI_API @GeoStl @WashUengineers

[Grey G. 🇦🇶 @gr3yg00]

🚨🚨🚨BRAKING NEWS!!!🚨🚨🚨

I recorded how alien artificial super intelligent agents could communicate just using speakers and microphones!

[Richard Palethorpe @jichiep]

You can make any LLM into a Jev style classifier using @LocalAI_API .

In our implementation you can use hybrid inference as well to do small chunks using autoregressive generation. This allows quickly choosing a function and filling in a parameter.

One disclaimer though, calculating confidence scores is not well defined on a regular LLM, or at least I didn't find a way to do it. Practically speaking it chooses "I don't know" fairly reliably though. I'm not sure what Jev does on that front.