Your Home Assistant doesn't need a Pi or a server closet. On Android, SmartHouse can install HA itself in a proot container on the phone: no root, no laptop, no always-on box. Old phone on a charger, on Wi-Fi, and the lights answer. iOS still needs a host.
20 minutes of questioning a local model: phone warm across the back, battery down about 6%. Nothing is uploaded - that heat is the whole model running on the chip in your hand.
If it has to last a day outdoors: one question at a time, screen off between.
Told Assist "kitchen lights" and the living room came on. It didn't mishear: that entity had no alias and no area, so the match fell back to name similarity.
Fix in HA: add an alias, set the area, retest. Most "bad voice model" reports end here.
Second copy of the same image in Droid Containers hit two walls: "container name already in use", then the host port was taken. Nothing wrong with the image, you just can't reuse either one.
Rename it, map a different host port. No root, no Termux.
Switched to a flight app for 40 seconds, came back to LocalAI, and the first question had to wait again. The model lives in the app's memory, and iOS reclaims it the moment you leave.
For emergencies: ask everything you need before you switch apps.
LocalAI's OpenAI-compatible endpoint has no API key, no login, no account. That's the point: scripts on the phone can call the local model.
So it belongs on localhost. Bind to the device, skip port forwarding, and nobody on your Wi-Fi reads your prompts.
Pulled an image fine, then inside the container: ping 1.1.1.1 works, ping https://t.co/vMI4qeIZLP doesn't. Network is up, name resolution isn't.
Check `cat /etc/resolv.conf` in the container before you swap images. Wrong nameserver? Fix that file, restart.
Rebooted the phone. The container in Droid Containers was stopped. Not a crash: a foreground service keeps it alive, it can't bring it back from a cold boot. You start it once.
Overnight: survives. Reboot: doesn't. No root, no Termux.
Door opened at 2:10. Notification landed at 2:31. Home Assistant wasn't late β Android had suspended the client and only woke it on the next window.
Add SmartHouse to the never-sleeping battery list, re-run the same automation, time it.
Five toggles in SmartHouse, all on by default: battery, Wi-Fi, steps, BLE, location. All five on and the phone is lighter by evening.
Not a bug: each wakes the app on a timer. Battery and Wi-Fi feed the arrival automation, so those stay. Other three: off.
Bleach, then an acid toilet cleaner in the same bowl. The smell flipped to sharp and wrong.
Asked LocalAI if those two can mix: sodium hypochlorite plus acid makes chlorine gas. Get out, ventilate, don't add water to "dilute" it.
No signal in that bathroom.
@jurlycat 80GB fixes capacity, not throughput. Before anyone buys into this: benchmark a 70B at Q4 and watch decode tokens/s against memory clock. A card modded this far can prefill fast and still decode slower than the capacity suggests.
@Alacritic_Super NPU handles prefill fine; decode is where these boards stall, since every token re-reads the weights out of LPDDR5 that the CPU is also using. Ask vendors for tokens/s at 2k context, not a first-token demo. The gap between those two is the whole product.
@bhanco2023 9B at Q4 is ~5.3GB and the PS4 leaves games about 4.5-5GB of its 8GB, so this only fits at short context: the KV cache is what tips it over. Q3_K_M plus a 2k window is the realistic config. Bandwidth isn't the wall here, capacity is.
@Propriocetive Weight quant shrinks the weights, not the KV cache. At 4k context a 7B model is already carrying ~1GB of cache, so the decode speedup you see at 512 tokens can be gone by 8k. Bench at a fixed context length or the numbers aren't comparable.
@0xBakeer Worth separating the surfaces though: llama.h is the stable one, but the server's OpenAI-compatible endpoints and the GGUF metadata layout drift on their own schedule. If you ship on it, pin the tag and diff llama.h between releases.
@unbug V100 is the awkward one to debug on: SM70 has no bf16, so it falls back to fp16, and it needs an explicit CUDA arch 70 build. Worth asking which kernels actually ran β fp16 tensor cores vs plain FMA is most of the 2x people chase.
@typecraft_dev Split by what each box is good at: Mac Studio for inference (unified memory bandwidth), the 128G board for the boring always-on stuff β storage, backups, DNS β which needs capacity, not speed. The 3 laptop boards make a decent k3s worker pool.
Droid Containers isn't phone-only. Tablets, x86_64 emulators, Android-on-PC images: any Android 8.0+ build with a 64-bit ABI runs the same proot containers.
32-bit is the hard no. A 32-bit kernel can't execute 64-bit binaries, and no app can fake that.
Your music is probably already in Home Assistant β media_source, the same place as the cameras.
SmartHouse browses that library and plays it in a native player instead of a webview. Same token, same tab, no second app.
@_gear_geek_ Equations surviving is the PDF parser, translation quality is the quant. For JP-EN academic text Q4_K_M holds up, Q3 starts dropping subscripts and math tokens. Slow first page is vision prefill, so batch pages.