When you are sitting before a blank page, there are two options: either you can one-shot it or you have to put in some effort into defining the structure and workflow of the project. You can have the agents do the grunt work of bootstrapping a project but you need to put in the deliberation and decision making.
Is it worth fighting? Well slip communication is annoying in the best case and often very time consuming or misleading.
With code it is worth fighting, because codebases die by thousand paper cuts and this is an exponential process. If it's not getting better over time, it's doomed.