I ran Sol Max top of the day. After a couple hours I was at 23% weekly remaining. Then I had to do other stuff, and a reset happened. Back to 100%. Then I ran Terra Extra High for about 2 hours, including a couple of agents for about 30 mins. Now I'm at 71% weekly. I spend a lot of time on crafting my prompts, and GPT spends a lot of time running and reading unit test results, so...I dunno...it's very hard to compare these stats...cached vs uncached tokens...actual model-run-time vs time in front of the computer... ๐คท๐ผโโ๏ธ
I haven't tried your skills docs, but I effectively do the same thing by making the AI write thorough task summaries, goal, and check lists for every item, with the check per item being very narrow, so I can start with a fresh conversation. No matter how you approach the problem, it's always a bit lossy...
@mattshumer_ I've yet to run any of these tools with full permissions; I always use manual approval so I can read the shell commands it executes. I do let GPT work outside the sandbox (have to), but I approve everything.
And I'll just add to this: in the era of development under which I was trained (late 80's early 90's and beyond), we had to comment literally -every single line- of code to pass tests. And even today, when I'm developing a new project that I know is going to scale, I think about the diagnostics and logging as much or more than the features themselves. If Codex/GPT intuitively built useful diagnostics and code graphs and comments as it generated code, I assume (???) it would be able to consume far fewer tokens downstream, during the debugging and refactoring phases (which are effectively perpetual).
@jack When you have enough compute to really let them spend on the first few drafts of code, I imagine it probably works pretty well. We Poors will have to keep writing long engineered prompts for a while longer. ๐