Are there any pitfalls to using a proxy API with Claude Code?
I'm mainly concerned about these issues:
· Streaming compatibility.
· Latency performance (especially for large models like Opus).
· Frequency of errors or failures.
How does it stack up against a direct connection?
Our GPT bill isn't high because of the model. It's high because every retry re-sends the same 40k tokens of context. We audited our last 50k calls. 31% were retries. That's 1.7k of a 1.7k of a 5.4k bill — a third of our spend bought nothing。
Here’s where most people go wrong with AI today: you simply say, “Write me a proposal,” and it might deliver a grand, “cosmic-scale strategy” with a budget you don’t have.😳