@pvncher@RepoPrompt What about the follow-up messages with the agent? Do I also need to build context for those and then send it back? When i am referring to multiple repos
It took a while for me to put my full thoughts into the OpenAI opensource models. I come away with a mixed opinion.
I LOVE that I can run the OSS20B on my computer at greater than 150 tps.
I HATE that they both versions suck at agentic workflows.
I also really dislike how censored the models are, but I understand why OpenAI is doing this.
1. I really did try to give it my best effort to make them usable in AI coding tools, but I just don't think its possible.
2. The variance in providers is nuts from openrouter per usual. Not only do we now need to figure out what provider is best but also I discovered that temperatures per providers can be different as well.
3. I learned Groq's temps run warmer than others.
4. So many failures I can't really show evals, but I did get some of them to complete at least once so below is the MAX scores I was able to get, take these at temperature data points not scoring datapoints.
5. I am very thankful OpenAI did enter the open source arena, and I hope we see more of this in the future, and i'm happy for anyone that finds a good use for these models.
I also learned Groq's models run warmer than others, I was so confused about this until someone in discord shared with me info about how its just the way Groq works.
Groq's best temp is around 0.7
Cerebras's best temp is around 1.0
Bizarre Right?