Arthur Busser · [email protected]

Author of tfautomv, tftree, and murmur 🤫 SRE @gopigment building a scalable platform in the cloud ☁️ Coder at heart

[Brian Grant @bgrant0607]

Some more thoughts on Infrastructure as Code, and how little has changed in 30 years.
https://t.co/eHMmCQO0AZ

[Arthur Busser · [email protected] @ArthurBusser]

Do you think machines will soon write code just as well as us humans?

Do you think LLMs are how machines will learn to code well?

[Arthur Busser · [email protected] @ArthurBusser]

Copilot used a regex that didn't work, so it ended up missing a lot of words.

Even when told that the solution was wrong, Copilot never suspected that the issue was the regex it had written.

Pro tip: triple check your regular expressions; write tests for them!

[Arthur Busser · [email protected] @ArthurBusser]

It's very cool to see machines improve their ability to write code.

It seems like they have caught up with a significant portion of the world's programmers, but not yet with many more.

They still have a lot to learn.🧠

[Arthur Busser · [email protected] @ArthurBusser]

ChatGPT wouldn't find any digits written as words.

Its code iterated over each character in the line and looked them up in its word-to-number map; looking up individual letters never yielded any results... 🙈

[Arthur Busser · [email protected] @ArthurBusser]

Can AI replace software engineers? 🤖

Not yet.

LLMs get stuck on the first day of the #AdventOfCode!

After I solved today's problem myself, I decided to see how well ChatGPT 4 and GitHub Copilot Chat would do. Here's what happened. 🧵

[Arthur Busser · [email protected] @ArthurBusser]

But their implementations were buggy...

There were subtle mistakes that made their solutions yield results that were way off.

[Arthur Busser · [email protected] @ArthurBusser]

Once I gave them the second part of the problem statement, they started to design a solution.

Their solution was pretty much the same: find both numerical and alphabetical digits in each line, map them to numerical values, and then do the math.

So far so good! 👍

[Arthur Busser · [email protected] @ArthurBusser]

ChatGPT and Copilot both wrote working solutions to part 1 on the first try! 👏👏

This didn't surprise me. The first days of the #AdventOfCode are pretty simple and don't require complicated algorithms to solve.

What I didn't expect, was for AI to get stuck on part 2! 🤯

[Arthur Busser · [email protected] @ArthurBusser]

ChatGPT wrote in Python because I let it choose whatever language it wanted.

Copilot wrote in Go because I had a Go file open.

[Arthur Busser · [email protected] @ArthurBusser]

I gave them both the problem statement for today's first part, as it's written, with minor formatting changes to make it as clear as possible, as well as my own input.

They both fully understood what they were being asked to do.

[Arthur Busser · [email protected] @ArthurBusser]

I told ChatGPT and Copilot I wanted to do an experiment, to see how well they could write code to solve increasingly complex logic problems.

ChatGPT was really into it! 🤩

Copilot did not care at all... 😐

[React Day Berlin @reactdayberlin]

😎Welcome @ndubien, Author of https://t.co/SUKoP3ugOa & Software Engineer @GoPigment!

Can't wait to see you at React Day Berlin: https://t.co/bqqsFgHlrd