Frank DeLuca

Professional Googler, President @TheCTOAgency

[Dave Kennedy @HackingDave]

Introducing a new side project called Model Regression. It tests daily Claude, GPT, and Grok on various benchmark statistics to determine how well its performing and to identify model degrades over time.

@edskoudis had an idea for model testing before they conducted offensive testing to ensure the model was performing as expected, and @BlasikRandy pushed me down this road with actually going and doing it.

The main intent here is the frontier models will experience outages, issues, bugs, intentional/unintentional nerfing of the models without notice. You can't typically trust day to day activities in these models for stability, so leveraging this on your daily routine to see how well the model is performing for that day is something I'll be using everyday.

Runs every morning in my DGX sparks environment and automatically updates with how well its performing.

Enjoy!

https://t.co/1Pep6NyGoh

Also open-sourced the project, can run on your own server as well and look at the benchmarks and how they are calculated:

https://t.co/GFPigpRtUF

[Frank DeLuca @frdeluca]

@HackingDave Any idea how it compares with Function? I just had one of those tests done and almost fainted from all the blood they took lol

[Frank DeLuca @frdeluca]

@HackingDave My best friend and I said we'd do this once his daughters start dating 😂

[Frank DeLuca @frdeluca]

@HackingDave ...prompt issue

[Frank DeLuca @frdeluca]

@HackingDave You as well! 👊

[Frank DeLuca @frdeluca]

@HackingDave I just want to know what kind of animal that is sitting on the couch 😆

[Frank DeLuca @frdeluca]

@BentleyAudrey @gabsmashh I keep going back and forth on the Italy thing for myself. Need to do some actual research and see what I need

[Frank DeLuca @frdeluca]

@bankertobuilder I'd like to buy one just so I can be on the HOA. I'd love nothing more than walking around in my whitest of white New Balances, citing anyone with a door color even the slightest hue off of the approved color list. 🤤

[Frank DeLuca @frdeluca]

@levelsio Egg whites are a hack. I make breakfast shakes/smoothies with 8-16oz of egg whites.

[Frank DeLuca @frdeluca]

@Aimee_Knight I’m sorry you have to deal with this. I’ve been hearing more stories like this lately. I hope you get answers, and are recovering.

[Frank DeLuca @frdeluca]

@H3KTlC Have the bike - just got a different treadmill because I didn't think the Peloton one was worth the price. We just use an iPad w/ the Peloton app, and it still syncs to the treadmill to maintain stats

[Frank DeLuca @frdeluca]

👀

[Dave W Plummer @davepl1968]

It's time for Microsoft to have another XPSP2 moment. No more AI, no more features. Just fixes.

When I was working on Windows XP, Blaster hit. It was a big enough deal that we set aside all feature work.

For the next several months, all we did was improve security. We didn't add "security features"; we fixed bugs. Lots of bugs. Until there weren't security bugs to fix anymore.

Then we fixed the ones we didn't know about yet.

Put more simply, we stopped trying to "add value" to the product through features that PMs thought users would like, and instead we focused on the things that had been important for a long time, but overlooked.

Like performance and configurability today.

Rather than trying to improve and add value to the system through new AI features -now-, I argue it's time for Microsoft to stabilize, improve, and make the system more performant. And more usable for power users.

Just for one release. Just till it doesn't suck.

[Frank DeLuca @frdeluca]

Just spoke with a customer service rep at @ArcBestCorp to get some tracking details, and holy cow, that had to be the friendliest, quickest, best experience I've ever had with a shipper. 👏👏

[Frank DeLuca @frdeluca]

Who keeps breaking the internet???

[Frank DeLuca @frdeluca]

@castrovince You had that one queued up didn’t you? Lol

[Frank DeLuca @frdeluca]

@HackingDave Not sure how I feel about this being called “vintage” and “throwback” lol. Pretty sure I still have a jersey with that design 😭

[Frank DeLuca @frdeluca]

@HackingDave 👀👀 nice little upgrade there!

[Frank DeLuca @frdeluca]

@gabsmashh R8 is still on my list - just hate that they stopped making them 😢

[Frank DeLuca @frdeluca]

@ChipotleTweets Can I send a message to an actual person, or just the chatbot that didn’t understand the problem in the first place?