Luke Gill

I sing and code sometimes

[Luke Gill @itslukethan]

@AdamHoltererer Would you be able to ask in T3 code as a workaround?

[Luke Gill @itslukethan]

@Willsquirrels TRUE.

[Luke Gill @itslukethan]

@eveporcello @johnlindquist

[Luke Gill @itslukethan]

I should clarify, this isn't with Claude's memory or Codex memory enabled. This is a markdown folder that's only job is to capture mistakes or decisions made for repos over time. Without the memories they made more mistakes more frequently without the pointer.

[Luke Gill @itslukethan]

Memory A/B follow-up: Claude and Codex used the notes differently.

Claude found them alone: opened the right note 8 of 10 runs, passed 9.
Codex opened it 4 of 10 times and passed 2.

Add a paragraph to AGENTS.md saying "check the notes first": Codex opened it 10 of 10, passed 6.

[Luke Gill @itslukethan]

With no memory, Codex passed 0 of 10, and every Codex run that skipped the note failed.

Reading it isn't the whole fix: with the pointer, Codex read it every time and still missed 4 of 10.

10 runs per model per setup, graded blind. Pointer vs no memory: p = 0.03 for each model.

[Luke Gill @itslukethan]

Memory A/B follow-up: Claude and Codex used the notes differently.

Claude found them alone: opened the right note 8 of 10 runs, passed 9.
Codex opened it 4 of 10 times and passed 2.

Add a paragraph to AGENTS.md saying "check the notes first": Codex opened it 10 of 10, passed 6.

[Luke Gill @itslukethan]

@AdamHoltererer What skills do you use for testing? I use impeccable myself

[Luke Gill @itslukethan]

Details: 5 tasks across two production codebases, Claude Opus 5.5 and gpt-6.1-sol, one run each per setup. Runs that opened the right note passed 13 of 16. Runs that didn't passed 1 of 4.

Small sample, and the tasks were picked where a note holds the answer.

[Luke Gill @itslukethan]

Ran a blind A/B on our coding agents. Same tasks, each with a trap the code doesn't reveal.

No memory: 2 of 10 passed
Memory notes in the repo: 6 of 10
Notes + one paragraph telling agents to read them first: 8 of 10

Writing decisions down is the cheapest upgrade I've found.

[Luke Gill @itslukethan]

@johnlindquist Lmaoo

[Luke Gill @itslukethan]

@maria_rcks Separate window for the preview browser

[Luke Gill @itslukethan]

@theo

[Luke Gill @itslukethan]

@maria_rcks @theo For sure! You guys at T3 are all Rockstars 😊

[Luke Gill @itslukethan]

@theo I said it before and I'll say it again, DRAMATIC difference using the method in the video. Thank you man!

[Luke Gill @itslukethan]

@theo Seriously, I used your method on the pros apps I build for my company and they were drastically better, thank you!

[Luke Gill @itslukethan]

@AdamHoltererer What skills for frontend do you use? Have you tested things like basic Claude frontends vs impeccable by @pbakaus ?

[Luke Gill @itslukethan]

@orcdev I truly wonder this. I've cut out all unit tests because I don't find them helpful. I think tests for critical infra are important. But everything else? I just don't see it

[Luke Gill @itslukethan]

@theo It is THE BEST. Seriously it's the thing I loved most about grok bot and I can't ever go back.

[Luke Gill @itslukethan]

@theo Whoa really? I mean I know people can make their own but still

[Luke Gill @itslukethan]

@AdamHoltererer dude congrats on the shoutout from @theo on his vid for OpenAI subs! Well deserved