@psychip @Inst_decay @GithubProjects I understand, let me know when you finish! I would love to test it. That earthquake feature will be really useful for countries like Peru.
ollama launch is a new command in Ollama 0.15 to run Claude Code, Codex, Droid and OpenCode with Ollama!
GLM 4.7 Flash is now optimized to use much less memory for longer context lengths (64k+).
Need additional hardware? Ollama's cloud offers GLM 4.7 with full precision and context length.
You can now run 70B LLMs on a 4GB GPU.
AirLLM just made massive models usable on low-memory hardware.
๐ช๐ต๐ฎ๐ ๐ท๐๐๐ ๐ต๐ฎ๐ฝ๐ฝ๐ฒ๐ป๐ฒ๐ฑ
AirLLM released memory-optimized inference for large language models.
It runs 70B models on 4GB VRAM.
It can even run 405B Llama 3.1 on 8GB VRAM.
๐๐ผ๐ ๐ถ๐ ๐๐ผ๐ฟ๐ธ๐
AirLLM loads models one layer at a time.
Instead of loading everything:
โ Load a layer
โ Run computation
โ Free memory
โ Load the next layer
This keeps GPU memory usage extremely low.
๐๐ฒ๐ ๐ฑ๐ฒ๐๐ฎ๐ถ๐น๐
โข No quantization required by default
โข Optional 4-bit or 8-bit weight compression
โข Same API as Hugging Face Transformers
โข Supports CPU and GPU inference
โข Works on Linux and macOS Apple Silicon
๐ช๐ต๐ฎ๐ ๐๐ผ๐ ๐ฐ๐ฎ๐ป ๐ฑ๐ผ
โข Run Llama, Qwen, Mistral, Mixtral locally
โข Test large models without cloud GPUs
โข Prototype agents on cheap hardware
NVIDIA just removed one of the biggest friction points in Voice AI.
PersonaPlex-7B is an open-source, full-duplex conversational model.
Free, open source (MIT), with open model weights on @huggingface ๐ค
Links to repo and weights in ๐งตโ
The traditional ASR โ LLM โ TTS pipeline forces rigid turn-taking.
Itโs efficient, but it never feels natural.
PersonaPlex-7B changes that.
This @nvidia model can listen and speak at the same time.
It runs directly on continuous audio tokens with a dual-stream transformer, generating text and audio in parallel instead of passing control between components.
That unlocks:
โ instant back-channel responses
โ interruptions that feel human
โ real conversational rhythm
Persona control is fully zero-shot!
If youโre building low-latency assistants or support agents, this is a big step forward ๐ฅ