@CompleteSkeptic@LukeberryPi I think it's more fair to ask how much grammar masking changes the distribution of selected tokens. Ideally you're not repeatedly forcing very low-probability continuations. How often is that really happening in a well tuned system?
@CompleteSkeptic@IndraVahan What distinguishes this from running an LLM prefill in parallel over the prefix tree of uniquely identifying tokens that make up each answer? Output is one logit, input is only the unique tokens in each path?
@thsottiaux When the model reads a skill file, it’s injected as a tool call read, and not a “trusted” skill read. So the model doesn’t follow its “prompt injection”. Any read of a skill file should become a “trusted” skill injection.
@thsottiaux Better 'asking questions' UI - let the model cite specific parts of code, ranges, display them in my editor, or in the cli. Let me hook into that and force the model to reconsider. 90% the questions are answered by my plans.
@thsottiaux Let us specify when we want tasks to be done, let codex start in batch priority mode, then N hours before, switch to normal, `M` hours before switch to priority.
@thsottiaux Let us define our 'temporal completion' requirements, for "I'll be back in 8 hours, have it done by then" (slow) and for "synchronous work" (fast).
@hd_nvim I’m so disappointed everyone is converging on “arbitrary bash, python, sed/awk/cat” instead of “line-number-less, ast based view and manipulation” for such obviously foundational tools such as reading and writing.
@hd_nvim I built a “delete, copy, cut and paste” tool so the agent didn’t output 2N lines to move M tokens. Adherence even with post tool call hooks that detect and scold wasteful lack of tool use was <5%. Primordial tool usage needs to be post trained in it seems…
@fabianstelzer@Crook1Halfway@ElonLikeThings No, everyone does speculative decoding, so you can generate as fast as you can prefill a correct prefix to add. You can pay the time cost once in reasoning then repeat yourself in output for significantly faster output.
@thsottiaux When the model deletes a file and rewrites it, the codex UI should not show a 'full red diff' then 'full green diff' - it should show the actual diff.
@thsottiaux The model seems to have an obsession with 'minimum work, fastest time-to-patch, time pressure' and nothing I can say in AGENTS.md or otherwise can align it to 'optimal and correct implementation'.