ATechieThought

Human, Problem Solver 🇮🇪 ☘️

[ATechieThought @atechiethought]

intelligence is the one which removes suffering. If suffering is not removed it's not intelligence.

[ATechieThought @atechiethought]

@GoogleCloudTech The premise and study from AAIF don’t really is a benchmark study. There is no verifiable proof and also author don’t claim the reductions are coming due to agents.md I wish claims should’ve been tested with engineering rigour. Please consider. Thanks

[Juraj Bednar🏴💛🌘 @jurbed]

I highly recommend cloudflare's security audit skill: https://t.co/Mu6Dm0fzk8

Run it on all your software. Run it with the free models. Use it even if you used a model to audit something - it will find much more things. Works in all agent harnesses and is model independent.

[ATechieThought @atechiethought]

It’s nightmare when you treat security as second class citizen.

[Derek Devicemanager @IT_unhinged]

Security called me at 6:12 AM today.

"Patch every PHP server immediately."

I asked what happened.

"Two critical vulnerabilities. Remote code execution. No authentication required."

Cool.

How many PHP servers do we have?

According to inventory: 12

According to reality:

Nobody knows.

We found one running WordPress.

Three running old internal tools.

One under someone's desk.

Two inside vendor appliances.

A forgotten employee directory from 2016.

And a server named:

PHP-OLD-DO-NOT-TOUCH

Naturally, that one runs payroll exports.

Then we found a PHP 5.6 box.

I asked who owned it.

The documentation said:

BRIAN.

Brian left in 2019.

We called Brian.

He said:

"Oh wow, that's still running?"

Excellent.

By 8 AM, leadership joined the incident call.

CEO:

"Can hackers actually get in?"

Security:

"Potentially yes."

CEO:

"Without a password?"

Security:

"Yes."

CEO:

"Then why do WE need passwords?"

Not the lesson.

Finance asked if we could wait until the weekend to patch because downtime affects revenue.

Security explained that the alternative was letting strangers execute code on our servers.

Finance asked for a cost comparison.

At 9:17, someone suggested temporarily taking the vulnerable websites offline.

Marketing refused.

"Our campaign is live."

So for the next hour, the company officially decided maintaining a landing page for 20% off webinar registration was more important than preventing remote server takeover.

We finally patched everything we could find.

Then our scanner discovered another PHP server.

Hostname:

TEMP-PHP-2014

Uptime:

4,512 days.

Nobody knows what it does.

Nobody owns it.

Nobody can shut it down.

But every time we block network access, the warehouse stops printing shipping labels.

Security has classified it as critical infrastructure.

It is running PHP 5.4.

The vulnerability has existed longer than half our employees.

The emergency patch took four hours.

Finding out what PHP actually runs took eleven.

The internet isn't held together with code.

It's held together with servers everyone is afraid to turn off.

[Ryan Brown @NotRyanB]

SlopCodeBench is the most relevant coding agent benchmark left--the top score is 16% vs. 50-75% on other coding benchmarks like Terminal-Bench, FrontierCode, and DeepSWE

summary of how it works: an agent receives a spec for a small program, like a backup scheduler or a mock HTTP server. the agent implements it, then the agent receives a new spec and has to extend the program. this repeats 3-8 times, and with each new spec there are hidden tests that check the new feature and all the old features.

there are a few reasons why I really like it:
1. it comes far closer to simulating real, long-horizon work with new and updating requirements, which is what it’s actually like to work on a codebase vs. solving a single isolated task
2. code coherence, maintainability, and extendability naturally matter more
3. it’s significantly less saturated than the other coding agent benchmarks

full results in the image (credit to @dexhorthy for the benchmark run + charts), the “core” number measures whether the main tests for the new feature pass; the “strict” number is the headline and measures whether every test passes, including regression tests for all earlier features

astra and sol are at the top (interestingly 6.1 is slightly worse than 6 although likely just noise), but given how low the scores are overall + how small the spread is, i don’t think it’s super meaningful yet in terms of determining which model you should use--opus 5.5 is still my daily driver, and i’m kind of surprised it’s not higher

the big caveat imo is that these runs all used the base harness with no customization. i personally think having a couple of core skills + agents.md instructions (planning, verification, etc) is necessary and improves performance, and that improvement is much more likely to show on SlopCodeBench than the other benchmarks

my prediction is that custom context + skills would improve performance by 5 percentage points or more and (hopefully) result in a ranking that’s a better representation of how good different models are to actually use for engineering work

do you think the improvement from proper skills/verification would be more or less than 5%?

[ATechieThought @atechiethought]

I do some thought experiments with #Claude. I was deep into #MiyamotoMusashi and his principles (from The Book of Five Rings) and had an idea to apply to define software excellence.
If people here likes it, will be happy to share more.

https://t.co/3cadkTFuU7

#philosophy

[ATechieThought @atechiethought]

@mmonis It’s in demand zone. 😊

[ATechieThought @atechiethought]

@dani_avila7 Congratulations 🙌 your work is definitely inspiring!

[ATechieThought @atechiethought]

cash is a position. you don't always have to win or loose, you choose not to play the game.

[ATechieThought @atechiethought]

@AmeetRai @RichardMoglen Risk management

[ATechieThought @atechiethought]

@sryanbruenphoto This is brilliant

[ATechieThought @atechiethought]

@TrixCity May time show you the path.

[ATechieThought @atechiethought]

Solitude and Meditation does wonderful things to your mind. It is not 10 minutes exercise, but lifelong pursuit.

[Hauntologist Hunter @subtlesteve1]

@MilesCranmer Also on Youtube, looks like a good watch. BRB.
https://t.co/IijNgnYLnj

[Miles Cranmer @MilesCranmer]

Here is the full talk:

[ATechieThought @atechiethought]

@sudoingX whatever fits less than $20 bucks!

[ATechieThought @atechiethought]

@dani_avila7 love it!

[ATechieThought @atechiethought]

Weekend Ritual: https://t.co/mFQbY81FO4

[ATechieThought @atechiethought]

@BrkoutGeek HVC

[ATechieThought @atechiethought]

@Muninn @PradeepBonde This is called as selection bias. it is funny that you are trying to apply one traders ruleset to another who has complete different personality and style.

[ATechieThought @atechiethought]

Stop loss hit on both.

[ATechieThought @atechiethought]

$AMD, $TSM