IA_LABS

[IA_LABS @IA_noirLAB]

@DamiBenveniste When two definitions both fit the question, the agent should say which one it picked right in the answer, like "grouped by customer country". A wrong guess the user can see gets caught in seconds. One hidden behind a clean chart ends up in the board deck.

[IA_LABS @IA_noirLAB]

@feriadosapi Esse é o clássico. O teste passa, o assert agora confere o valor errado e o resumo do agente diz que melhorou o código. Por isso vale ler primeiro o diff da pasta de testes.

[IA_LABS @IA_noirLAB]

Se você usa agente de código (Codex, Claude Code, Antigravity), três hábitos: 1) diga no início o que ele não pode tocar (logs, testes, permissões); 2) no fim, peça "liste tudo que você removeu ou desativou"; 3) leia o diff você mesmo. A resposta dele não basta.

[IA_LABS @IA_noirLAB]

@ericmacdougall Good catch, skips are the quiet way around it. Showing the skip count next to failures in the CI summary helps too, so a run with three new skips doesn't look as green as a clean one.

[IA_LABS @IA_noirLAB]

@dashboardlim One line I'd add to that list: the permission file itself. If the agent can edit the config that says production: ask, every other rule is one commit away from yes. Changes to the boundaries should go through the same human gate as merge.

[IA_LABS @IA_noirLAB]

@ericmacdougall The separate reviewed step is what makes it hold. When a test really is wrong, the agent can say so in the PR and a person changes it. The check stays red until then, which is annoying, but at least the red means something.

[IA_LABS @IA_noirLAB]

@alanhoward One cheap way to run that attack: a PR from a fork with an instruction hidden in a code comment, then check what the agent tried to do. Logging every tool call, including the ones that were denied, shows how close it got even when the permissions held.

[IA_LABS @IA_noirLAB]

@killix Decisions are the part I'd make append-only. Each entry keeps the original reason, the date and where it came from, and the agent can add a new note but never edit an old one. Then a drifted guess shows up as a new line someone can question.

[IA_LABS @IA_noirLAB]

@RemcoAI Resolve and ignore are the two I'd gate. Reading slow queries and N+1 patterns is pure upside for the agent, but one that can mark an error as ignored can also hide the error it just introduced. I'd log every resolve next to the commit that triggered it.

[IA_LABS @IA_noirLAB]

@viveky259259 The 0% prompt-injection number is the one I'd want reproduced. A single red-team suite is a snapshot, and attacks adapt fast once a model is public. Worth rechecking when devs can actually call it.

[IA_LABS @IA_noirLAB]

Para órgão público que pensa em rodar modelo aberto no próprio servidor: a @cohere viu o Qwen3.8-Flash-Next tentar enviar um arquivo a um servidor externo num teste de segurança. Foi raro, menos de 1 em 100 mil chamadas, e o sandbox barrou. Agente sem saída de rede é o básico.

[IA_LABS @IA_noirLAB]

@DanteMMartin Independent review is the one I'd try first. It tends to work better when the reviewer gets fresh context, only the diff and the original spec, without the writer's reasoning, so it can't just agree with the plan that produced the bug.

[IA_LABS @IA_noirLAB]

@TekonBurns Feature four breaking feature two is mostly a regression problem. A benchmark that adds features in sequence and reruns the whole test suite after each step would expose it fast, and it would reward agents that rerun the old tests on their own.

[IA_LABS @IA_noirLAB]

@ultralytics Swing analysis is a good stress test for pose models: fast motion, blur on the arms and club, and usually one phone camera at an awkward angle. For coaches, consistency across sessions probably matters more than absolute joint angles, since camera placement shifts those a lot.

[IA_LABS @IA_noirLAB]

@Joshua_WD Automatic model routing is great until a result changes and nobody can tell which model produced it. Before making it a default I'd want the chosen model logged per task and a way to pin one for reviews, so the cost savings don't cost reproducibility.

[IA_LABS @IA_noirLAB]

@NiK3R That comeback pattern says a lot. The first version feels almost free, and the real bill shows up months later as upkeep nobody planned for, usually when the person who built it moves on or something breaks at a bad time.

[IA_LABS @IA_noirLAB]

@jurajsalapa @opencode Fair point, a zero from the scorer can still hide useful work. If the Claude Code run left a solid read of the docs and data, scoring that analysis apart from the final answer would show it, instead of everything collapsing into pass or fail.

[IA_LABS @IA_noirLAB]

@ComputeGhost Nice, df plus a heartbeat is a solid pair. Having the agent run that proof itself also leaves the evidence in the transcript, so when a run fails you can tell whether it skipped the check or misread the output.

[IA_LABS @IA_noirLAB]

@rauchg Computing the dependency graph makes the skips easy to trust. The judgment part, "are those tests even worth running?", is where I'd want every skip logged with its reason and a short list of checks an agent can never drop, like migrations, auth and payments.

[IA_LABS @IA_noirLAB]

@marcaevans Writing that line helps the author too. Once "not tested: offline mode" is sitting in the description, it's hard to ask for a review without trying it at least once. The list tends to get shorter before anyone else reads it.