OAra Labs

Building the things that should belong to you and don't. Open source where it counts. Findings from the build: Prometheus (agent daemon)

[OAra Labs @OAraLabs]

started this in march, before most of what's out now existed.

i'd been reading Sigrid Jin's breakdown of how claude code actually works under the hood. all that careful engineering, and none of it running on your own machine. steal the fire, bring it down to everyone else'. seemed appropriate at 1am.

open source has been good to me. this was my way of putting something back.
six months later it's become something i didn't set out to build. the imitation was just the on-ramp. what it actually turned into is something that stays running and remembers.

Beacon, the desktop side, drops next month.

[OAra Labs @OAraLabs]

@nitin_builds Ungood

[OAra Labs @OAraLabs]

History time. Off topic for this account, but I fell into this while working on something else and it stuck with me.

Around 1499 a Benedictine abbot wrote a book that read like a manual for speaking to spirits across long distances. Angel names, invocations, tables of hours.

It got printed in 1606. The Church banned it three years later. Occultism.

They had it backwards. The invocations were the cipher and the angel names were the key. Trithemius had written a book about hiding messages, and hidden that fact inside the book itself. Nobody fully decoded the third volume until 1996.

John Dee kept a copy. he used it to hide his letters to Elizabeth I.

The book was called Steganographia. Covered writing.

The idea hasn't changed since. The safest message is the one nobody knows is a message.

[OAra Labs @OAraLabs]

@Sahil_Saghir per token billing on invisible, encrypted reasoning is unauditable by design

[OAra Labs @OAraLabs]

@sama Labs now encrypt CoT for two reasons:

1- Plaintext thoughts are a distillation set. Hide them so nobody clones the model.

2- The good agent path needs that private blob. Drop it and quality dies. That’s lock-in.

You still pay for the tokens. You just don’t get to read them.

[OAra Labs @OAraLabs]

@nickgraynews Genuine question on ability: what specifically doesn’t work locally for you? Hard reasoning? Long-horizon agents? Coding that actually lands? Reliability under messy prompts? If “junk” means “can’t match the frontier model on X,” that’s a different claim than “can’t do the job.”

[OAra Labs @OAraLabs]

the correlation ids are the part i’d steal. shim emitting tool_call_id and invocation_id means a spike in grafana joins back to the exact call that caused it. most setups log the agent and the infra separately and can never connect the two.

mine does the same thing but into sqlite on disk rather than a cloud backend. works, just nobody else can see it.

[OAra Labs @OAraLabs]

@adhamuxi appreciate it. mostly built it because everything out there is cloud-first with local bolted on. wanted the other way round.

[OAra Labs @OAraLabs]

@kunchenguid why is it one or the other? cheap stuff runs local, hard stuff goes to a frontier mode

[OAra Labs @OAraLabs]

that’s the one i was missing, thank you. reject count as liveness, not just an error log.

the asymmetry is annoying though. post-hoc validation you own the rejection so you can count it. decode-time constraints happen inside the inference server and it won’t tell you what it refused. the layer i had no visibility into is the one that failed silently.

going to alarm on repair rate hitting zero. genuinely useful, appreciate it.

[OAra Labs @OAraLabs]

Telemetry is a dirty word because it usually means your data going to someone else's dashboard.

Local flips it. If you're running your own quants on your own hardware, you're the only one who can tell whether that Q4 is holding up, or whether the 27B earns the extra VRAM over the 14B. Nobody is publishing that for your machine.

Prometheus writes every call to sqlite on your disk. Model, quant, tokens, latency split between harness overhead and raw provider time, tool call outcome. Nothing leaves. Same data the frontier labs collect on you, except it's about your setup and you're the one reading it.

[OAra Labs @OAraLabs]

yeah, i’ve been building this. slack, telegram and discord all talk to the same daemon, so it’s one agent with one memory instead of a bot per channel. model agnostic is the premise, local or hosted, swappable per turn.

self hosted though, you run it on your own box. and it’s a repo with rough edges, not a product.

https://t.co/vGa9IlOS4w

[OAra Labs @OAraLabs]

yeah that’s the right split, and it maps onto tool calls almost exactly. freeform json from a 7b is rough. constrain the output and the failure rate falls off a cliff. shrink the space instead of growing the model.

only thing i’d pass along, the constraint layer becomes the thing that breaks and it breaks quietly. lost a while on a grammar that passed every test and was never actually applied, parser rejected something the spec allowed. output looked fine, just wasn’t constrained.
so worth having a way to see the selection actually happened. i didn’t, for longer than i’d like.

[OAra Labs @OAraLabs]

mostly right if you’re calling an API. the loop is easy when the model’s good and someone else maintains it.

the “providers integrate upwards” part only works if there’s a provider. weights on your own box, nobody above you to absorb any of it. spent last month on a grammar that passed every test and was silently never applied, because llama.cpp’s parser rejects something the spec allows. that doesn’t happen behind an endpoint.

so not alpha. just work that doesn’t go away.

[OAra Labs @OAraLabs]

not selling it, so I get the easy version of this. it’s a repo and my own hardware.

but on the review thing, local only kills it if the whole path is local. a lot of “local BI” still sends the schema and the question out to a cloud model to write the sql. warehouse never moved, data still left. reviewer asks where generation happens and you’re back in the meeting.

and then the model small enough to run on prem has to actually get the sql right. that’s usually where it falls over. not throughput.

[OAra Labs @OAraLabs]

Most agents know their hardware because someone typed it into a config once. That file starts lying the day you swap a card.

I type /anatomy and Prometheus tells me: GPU and free VRAM on every box in the mesh, which processes are holding it, the exact gguf and quant that’s loaded, which peers are up right now. it can answer whether the next model fits instead of me guessing.

[OAra Labs @OAraLabs]

@analogalok Thank you for this 💯

[OAra Labs @OAraLabs]

They’re mostly the same because they’re all built against frontier models, where the loop is the easy part. Point one at a 27B on your own box and the differences show up immediately: chat template mismatches, malformed tool calls, whether the harness repairs or just fails.
Same loop, different problem.