Jabrod

lil bit of everything. AI engineering notes, creative tools & customer insights. Jabrod ART + Signals. Explore a Signals pilot:

[Jabrod @JabRodOfficial]

An agent choosing the wrong tool may have an interface problem. Compare the names, parameter descriptions and error messages first. Give overlapping tools clear boundaries so the model can distinguish their jobs.

[Jabrod @JabRodOfficial]

Agents, 3/3

Set a stopping condition and a budget. Test failed tool calls and ambiguous results, not just the happy path.

A useful architecture reference:
https://t.co/OCwvHFtpb6

[Jabrod @JabRodOfficial]

Jabrod AI notes | Agents, 1/3

First decide whether the steps are known. A fixed workflow can be easier to inspect; an agent can choose its next step when the path is less predictable. Add autonomy for a reason.

[Jabrod @JabRodOfficial]

Agents, 2/3

Write tool descriptions like API contracts: required inputs, possible errors and what changes when the tool runs. Separate read-only inspection from actions that modify data.

[Jabrod @JabRodOfficial]

Where would a customer-risk list fit in your week: account review, renewal planning, or support triage?

If you have a specific workflow in mind, include it when applying for a Signals pilot.
https://t.co/cIe3u0qvBy

[Jabrod @JabRodOfficial]

Put awkward queries in a RAG test set: an exact invoice code, an acronym, a paraphrase and a question whose answer is missing.

Retrieval that looks good on broad questions can behave differently on precise identifiers.

[Jabrod @JabRodOfficial]

Shorter embeddings can reduce vector storage, but retrieval quality still needs testing. For models that support Matryoshka truncation, compare supported dimensions on the same labelled queries before choosing the smallest vector.

[Anshuman Tiwari @Realthugus]

alan code agents built 5 parts of agentbill.
then agentbill priced those runs: 128 calls, $0.62.
then it went back into alan as a budget skill.
weird loop. i like it
@tryAlanAI @prasad_pilla #OpenFire

[Anshuman Tiwari @Realthugus]

submitted agentbill for openfire.
24 hours, one laptop, a bill for every agent session.
if u use claude code or codex, run it once on ur own logs. u wont like the number
@tryAlanAI @prasad_pilla
https://t.co/bdnpvT7Ko0

[Anshuman Tiwari @Realthugus]

what agentbill actually does for u:
- prices every model call from logs ur agents already write
- shows which file reads and cache rebuilds cost the most
- tells u what an open model would charge for the same tokens
no proxy, no account. it just reads files
#OpenFire

[Anshuman Tiwari @Realthugus]

@tryAlanAI @prasad_pilla repo: https://t.co/5mWbOQ4a8e
live demo: https://t.co/S5zrRmvaFj
@tryAlanAI @prasad_pilla

[Anshuman Tiwari @Realthugus]

built agentbill for #OpenFire @tryAlanAI @prasad_pilla

problem: coding agents bill per call, nobody sees where it goes. mine: $4.8k, 23% leaked

it reads agent logs, prices each call, shows the leaks

built w alan code agents on deepseek: 5 tickets, $0.62, priced by agentbill

[Anshuman Tiwari @Realthugus]

$4800 that's what I spent in months of building across agents sessions.

[Jabrod @JabRodOfficial]

For local-model comparisons, record the configured context window as well as the model name. Two runs with different context limits are different experiments.

Our test sheet would include prompt size, output cap, hardware and runtime version.

[Jabrod @JabRodOfficial]

For local-model comparisons, record the configured context window as well as the model name. Two runs with different context limits are different experiments.

Our test sheet would include prompt size, output cap, hardware and runtime version.

[Jabrod @JabRodOfficial]

A useful churn pilot needs a baseline, a definition of churn and a way to review false positives.

Exploring customer-risk modelling? Tell us what you can measure in the Signals pilot form.
https://t.co/nVrIg04DlX

[Jabrod @JabRodOfficial]

A useful churn pilot needs a baseline, a definition of churn and a way to review false positives.

Exploring customer-risk modelling? Tell us what you can measure in the Signals pilot form.
https://t.co/nVrIg04DlX

[Jabrod @JabRodOfficial]

Quantization check: compare accuracy, memory and latency on the deployment hardware. A smaller weight file is only one result. Kernel support and the quantization format affect runtime behaviour.
https://t.co/qOuM8znark

[Jabrod @JabRodOfficial]

Transformers 5.19.0 (Oct 6) changes router-logit outputs for MoE models. If monitoring depends on those outputs, include shape and schema checks in the upgrade test. Accuracy alone will not catch every integration break.
https://t.co/Jch9lEaRFQ

[Jabrod @JabRodOfficial]

RAG, 3/3

Chunk context and reranking are separate levers. Test them independently so you know what helped.

Anthropic describes this approach here:
https://t.co/csSvMgv9En

[Jabrod @JabRodOfficial]

Jabrod AI notes | RAG, 1/3

A fluent answer can still come from the wrong document. Evaluate retrieval separately: did the relevant evidence reach the model? Keep exact identifiers and vague natural-language queries in the test set.

[Jabrod @JabRodOfficial]

RAG, 2/3

Dense embeddings find semantic neighbours; BM25 helps with exact terms such as error codes. Try hybrid retrieval and deduplicate the candidates before sending context to the model.