AI is meta. A dream within a dream, like inception the movie. There are always more levels.
Level 1 - directly using AI to achieve a goal
Level 2- Using Ai to help you use AI to achieve a goal.
Level 3- Using AI to help you build something to accomplish levels 1 & 2.
It’s a productized version of a fleet of agents working on SDLC. Right now tech people are reinventing the wheel because we like to build. However just like every stack. 1- it’s a lot of work to tinker and get setup right. (And often just for you). 2- Imagine going a step further such that non technical folks can use an out of the box software factory and software best practices (what we’re still deciding for the ai) is baked in..
@jnardiello@AnthropicAI Once they have public shareholders, they need to start focusing on profit. I assume it’ll be much harder to keep Dario’s stated trajectory of hitting AGI at all costs. So, has AGI already been hit? If you converse with Fable, it sure seems very close to hitting most criteria.
If you architect things right, it doesn’t need to be. Skills, agents can be leveraged across models and harnesses. Use you knowledge and docs to improve and refine said skills and allow agents to query your onprem docs as needed. The agent will behave differently depending on the model but the emerging self improvement capabilities may help with that
@itsalicesoul Uh. Of all code? Probably not, there will be holdouts. I do think adoption will be incredible, going vertical and then plateau. Many people code and My code - yes. Will we see artisan coders? Where the built by a human becomes a vanity thing for some? lol
It all depends on what you're trying to achieve.
Is the gap in the tooling....
Something you want to try to productize - That's a gamble that frontier companies won't address it.
Something that prevents you from achieving your goals whereas you'd be happy to adopt vendor's offering when it comes out.
My suggestion, point fable at investigating this.
From what I can tell, the hermetic products need to support your platform and can fail silently (not detecting a critical file is a dependency). They collapse work by detecting change based on dependencies. There are a few products (bazel, pants, buck2, please, nix) and they need to support your language/dependency management solution.
Your mileage may vary, This is a simplistic incomplete view from someone who hasn’t tried any.
All depends in what you include in your cicd workflow and what it depends on.
Self host your runners. Optimize your ci workflow for parellism, caching and skipping unnecessary steps. As someone else said, collapse new builds on same branch. Build a mechanism to have a high priority queue. Look at oss cicd options. You can also have dynamic scaling of runners (computer costs tho).
Claude can now help you build evaluations and hillclimb on them.
In this article, we share guidance on eval design & skills that Claude Code can use to improve your applications.
https://t.co/PgKFC2DWth
So genuinely wondering. Give each agent its own sandbox and have it figure out what needs verification? Codify the guardrails and have it detect what’s changed and what needs to be tested? Does this scale all the way to e2e? E2E is my pain point. I already parallelize and skip based on dependencies. But if you touch something at the heart of the system….
I asked fable, it came up with:
Affected set selection - done to some degree
Reviewers (done)
Speculative batch queue - this is interesting.
Post merge/nightly - yeah, well, then you hold deploys?
Buck2 does seem to solve this for large systems. I guess AI could figure out how to configure it.
@devops_prashant What if AI is watching and can fix it? People are wiring up AI to monitor and fix systems (more disk, spin up new node. Write new code, maybe even deploy). Eventually perhaps it’ll even be able to dispatch a robot to replace the Ethernet cable.
@chribjel They need observability of your SDLC cycle. How long tooling takes. How long changes take. Once that’s in place, they will become more accurate than us optimistic software devs lol.
Coming from an AI is eating everything perspective.
Develop a passion or interest which will prompt you to explore and be curious.
Work on your ability to learn, adapt, process and understand information and adeterming what questions to ask.
Once you find a problem, these skills/traits will serve you well when working with AI.
@paradite_ I like the feedback capability. Consider dynamic model selection depending on task and include cheaper models into the mix for appropriate tasks.
@BenjaminDEKR You can reduce that by putting in place guardrails for the AI generated codebase. Deterministic tooling ensuring your codebase has a certain shape. Complexity, code duplication (imperfect), best practices to reduce coupling, etc.