Phil

Writing about Vibe Coding and AI on X.

[Phil @thereturningdev]

Monday again with OpenAI.

- Sol 5.6 High effort till 40% of the weekly limit left
- Sol 5.6 Medium till 15% left
- Luna after?

Hoping for some reset along the way.

How do you use your models?

[Phil @thereturningdev]

@WisprFlow Does it mean I now speak some uncomprehensible gibberish like Claude?

Completely possible

[Phil @thereturningdev]

Unexpected gains from using @WisprFlow : speaking faster

I was stunned to see someone's Wispr Flow Insights showing he was speaking at ~190 words per minute (and I was just at 100wpm!!)

I focused on improving that, and a few weeks later I already speed up by 28%!

[Phil @thereturningdev]

One way I learnt to improve the quality of AI output are adversarial tests

- Ask Agent 1 to produce something
- Tell Agent 2 to assume the product of Agent 1 is an hallucination or prove that it is correct

This seems to work much better than "testing if Agent 1 work is correct"

[Phil @thereturningdev]

Example

- Agent 1: find all bugs due to incorrect validation of input in the API of this service
- Agent 2: given this list of bugs of the API, assume they are all hallucinations and prove if any of them is actually correctly reported

[Phil @thereturningdev]

One way I learnt to improve the quality of AI output are adversarial tests

- Ask Agent 1 to produce something
- Tell Agent 2 to assume the product of Agent 1 is an hallucination or prove that it is correct

This seems to work much better than "testing if Agent 1 work is correct"

[Phil @thereturningdev]

Read the code VS Test the code

In the 26+ years I worked in IT, we always reviewed the code. Every commit, in every company.

Testing, on the other hand, we could have always done more.
More adversarial testing, more negative test cases, more integration, more end to end, more frequent performance and load testing.
But there was never enough time.

That's how I see the read the code vs don't read the code debate.

AI allows us to create the most extensive test suites we have ever had; it allows us to modify tests quickly when we change the code. It allows us to create new test environments quickly

and ultimately allows us to not read the code

[Phil @thereturningdev]

I haven't had time to play video games for decades, but I recently realized the probably obvious thing of how AI can make video games modding and tinkering explode

Decompiling data files, save files, skins, and making all kind of changes

Is this actually happening in gaming communities?

[Phil @thereturningdev]

I mentioned I could switch from Claude Code to Codex within a day, but if I put back my CTO hat, that's not sustainable.

As a company, you can't chase every quarter the best model, and ask all engineers to switch to it all their customization and skills.

This is a harder problem to solve than it looks like. On the surface, /skills are easily transferrable, but the reality is that every model will interpret (or ignore!) instructions in a different way

Your AI provider choice is definitly more sticky then one could think

[Tibo @thsottiaux]

It's me again. I come bearing great news.

First of all, we have hit 20M active users for Codex some time this week. Second of all, this is cause for celebration and during the day we will credit every Codex and ChatGPT Work user with a BANKED reset that you can use at your own leisure. And we will have some other good news later too!

Now, on usage limits draining faster, while we're not seeing anything abnormal, we do take it incredibly seriously and there is an ongoing investigation. I will share if we do find anything and my below post is really a clarification on a specific pattern that we did see that I wanted to call out.

Go do something amazing today.

[Phil @thereturningdev]

A few months of intense AI coding in, how am I doing at the kind of aggressive, intensive handling of 4-8 Codex sessions simultaneusly?

I'm getting better honestly. A couple of tricks:

1. I scribble down, on paper usually, what each agent is doing, so I don't lose track

2. I alternate between coding mode and merging mode. I start 3-4 coding agent in parallel (coding mode), wait for all of them to finish, and the test each agent work and merge it (merge mode)

[Phil @thereturningdev]

Well, yes they overpaid.

However all the sector is overpriced, if you think that Wispr Flow, which is way simpler to replicate than OpenRouter (you really can vibe code Wispr Flow in a day, I did it) is apparently valued 2B$

I think there is a monetary value of time in play. Yes you can replicate OpenRouter, but it is more complex than other services.
By buying it, you buy the business and customers now and you are immediately in business.

If you decide to build it, you realistically start in 3+ months and with no customers

[Phil @thereturningdev]

If AI allows you to create way more tests than you ever had the luxury to create (unit tests, integration, e2e, load tests, performance tests, security)...

... does it matter if you don't read the code?

[Phil @thereturningdev]

@svpino It is truly great. I have built tools that I use every single day, with all the features I need.

[Phil @thereturningdev]

Today we are thanking Adobe for making their crap so hard to remove from your computer that you need to install an "Adobe Cleaner App"

[Phil @thereturningdev]

@petergyang In fact yes, that’s a sort of adversarial testing.

I used that in bug finding
- find bugs (within a certain category) in the codebase
- given the list of bugs found by the other agent, assume they are all hallucinations and find if there is any of them that isn’t

[Phil @thereturningdev]

That's all for me trying Hermes.

The UX is just terrible.

I tried it the first time a month or so ago, my cron jobs would work for a couple of days and then fail

This second time, after being able to install it *with a lot of erros and asking ChatGPT for scripts to fix it*, I got it working. Desktop App running on Mac, with remote gateway and agents running on a VPS:
- The remote agent would work the first time
- The second time can't load the session
- And the session is exactly 1 question

Known bug from a month ago, never fixed
https://t.co/L4M9GkSCvX

[Phil @thereturningdev]

@tdinh_me I’m hooked, but what’s the deal with level 15? It is never ending, right?

[Phil @thereturningdev]

2 hours to the reset line, I'm running Sol 5.6 on /fast ... but honestly I was expecting a more noticeable performance improvement... disappointed

[Phil @thereturningdev]

@AminTechs Yes that's what I did, I installed the full Desktop app on Mac OS and then configured the remote gateway.

[Phil @thereturningdev]

Hermes Bot finally set up

For people saying "How can people not use agents"?

- Hermes installation failed multiple times (on Ubuntu)
- I needed ChatGPT to create scripts to install Hermes
- I could not install the remote desktop app only on Mac. I had to install the full Hermes locally, and then ignore it and ask to connect to the remote agent

Successful apps do care about UX over everything else