AI Engineering Concepts β 18/20
LLM-as-a-Judge makes large-scale evaluation and regression testing practical. But judges can be biased by position, verbosity, or self-preference. Use a clear rubric and calibrate against human labels before
Remote-first architecture feels like the right direction for keeping agent tooling lightweight. Versioned protocols and durable sessions could also make experimentation far easier to evolve safely
Tired of agent harnesses shipping a browser inside Electron, I built a lighter remote-first one: GPUI client (native + wasm), Rust backend, durable sessions on a versioned protocol.
Experimental, pre-1.0.
https://t.co/1WOQDZtMT6
https://t.co/dSQ1vFlpqw
#RustLang#GPUI#AIagents
AI Engineering Concepts β 20/20
Distributed tracing reconstructs a request across services, models, and agents using correlation IDs and spans. Propagate the ID through every hop, including tool calls. Log each agentβs inputs and outputs to debug
AI Engineering Concepts β 5/20
Continuous batching keeps GPUs productive by refilling finished slots during generation instead of waiting for the slowest request. It boosts throughput, but overly aggressive batching can increase queueing delay.
AI Engineering Concepts β 9/20
Query Rewriting translates conversational requests into clear, standalone search queries by resolving context and adding useful terms. It should clarify intent, not change it. Preserve the original, inspect variants,