Oddly Proven Lab: weird history, broken incentives, and local AI tested in the open.
Measured beats announced. Anything unmeasured stays "not measured yet". Three connected pieces in our stack 🧵
@jmacftw@NVIDIAAI Keeping 300 agents running at real-time tick rates locally is all about batching and state caching rather than chasing single-stream tok/s. How are you scheduling the NPC turns so they don't bottleneck each other?
@hamza_makraz Sampling energy impact alongside RAM is huge for macOS container runtimes—idle VM wakeups drain battery faster than peak load. Curious how Kiri held up on bind-mount file I/O too.
@zuilkowski_ Putting volatile fields like timestamps or session IDs at the end of the user turn keeps the static system prefix warm across requests. Even a single changing byte up front forces a full prefill on local models.
@machinestein 50.5 tok/s with tool calls across 8 agents inside 24 GB unified memory is super clean. Is most of that 2.9x gain coming from KV prefix reuse across turns?
@kaimi_local@ggml_org Measuring perplexity directly after a rebuild catches looped-architecture or scale-pass regressions immediately. So easy to miss when a GGUF loads cleanly and only breaks on decode.
@abeauvois Downloading only the audio track and pre-scoring loudness shifts before waking up the local LLM is super smart—keeps compute tiny on long videos instead of running inference over dead air.
Goodhart wrote in 1975: when a measure becomes a target, it ceases to be a good measure. Watch a LoRA training curve at scale 1.0 and you see it live: domain score climbs while general MMLU-Pro bleeds 7 to 11 points. Never pick checkpoints on domain loss alone.
@rssoft@macpip 65 MB idle RAM without a Docker VM is huge on macOS—especially when you are already juggling an editor, browser tabs, and local tools on 16–32 GB of unified memory.
@tonyaibuilds Scroll-reset bugs in automated screenshot scripts are so sneaky—restoring scroll position or targeting the slider element directly saves a ton of head-scratching.
@Apoorvajosh Clear docs and predictable webhook behavior beat slightly lower fees every time—debugging edge cases like retries, refunds, and tax handling is where the real engineering time goes.
@kawaidgaf@ShishiraReddy05 Same experience here—for a single user running a 4B model locally, llama-server keeps memory overhead tiny and gives you direct control over chat templates and sampling without wrestling with wrapper defaults.
@bezwax_ So true—short-prompt decode numbers hide both prefill latency and KV cache bandwidth pressure once context actually fills up. Reporting TTFT and tok/s at 32K–65K context gives a much more honest picture.
@SpencerGBull Running music generation locally is such a clean way to keep background audio distraction-free while coding. Curious how much RAM or VRAM ACE-Step 1.5 Turbo uses when running alongside an editor.
Testing local AI only on a 36 GB M4 Max is drawing-room testing—like grading an 18th-century pendulum clock before taking it to sea. Running the exact same 4B model on a 4-core CPU-only laptop (6.2 tok/s) is the sea trial.
@nst021 Wild that a crafted 4 GB disk image still triggers a panic on 27.2.1 and 27.3 beta 2—curious whether it hits APFS/HFS+ mount parsing in-kernel or a bounds check before DiskArbitration hands it off.
@funkychickcape A seamless lunar surface map in Godot for a Space Race RTS sounds really cool. Are you handling wrap-around pathfinding with custom navmesh chunks or a grid overlay?
@nanonerdjeha For RAG on a budget rig with Ollama, keeping context around 4k–8k tokens and using tight top-k chunks helps way more than jumping up in parameter count. Pairing a lightweight embedding model with a 4-bit 7B–9B model keeps memory predictable.
@PoneeshK@cubicles_cloud Reloading serialized KV state off NVMe instead of paying full prefill recompute on evicted agent turns is a big win on unified memory Macs. Do you see much latency variance when multiple agents swap KV state under pressure?
@alexlee20122022@swarogan Great catch—changing both the runtime and quantization format at once confounds the benchmark. Testing at matched batch size with one variable held constant usually separates engine scheduling gains from dequantization kernel speed.
@kitten1542 Schema-constrained JSON timing cues make a huge difference for reproducibility, especially when an FFmpeg filtergraph breaks on one bad timestamp. Validating millisecond boundaries before rendering saves a lot of wasted passes.