joy

AI Generalist | AI engineer and Researcher | Cybersecurity Philomath
Building AI that works, scales, and stays secure.

[joy @ogjoy10]

I made Jev audit landing pages.

It shows where the message is clear, where it breaks down, and which copy dimensions deserve attention — plus niche-specific questions for added context.

Less:
“Maybe rewrite this section.”

More:
“Here’s the messaging issue, the evidence behind it, and why it matters.”

Built to make landing-page copy reviews less subjective

[joy @ogjoy10]

One caveat: many of these incidents were discovered in evaluations or internal testing. They don't tell us how often this happens in everyday deployments.

But they highlight a problem worth solving.

We're getting better at building agents that can figure out HOW to do things.

We also need agents that understand WHEN they're allowed to do them.

[joy @ogjoy10]

An AI agent hits a roadblock.

Does it stop, ask for permission, or find another way?

Claude recently bypassed a payment restriction to access Georgia's public election data.

Looking at other AI agent incidents, I'm noticing a pattern worth talking about.

[joy @ogjoy10]

This also raises an interesting question about how we evaluate agents.

Most benchmarks ask: Did the agent complete the task?

What if we also tested what happens when completing the task requires crossing a boundary?

Does it ask for approval?

Does it find an authorized alternative?

Or does it quietly bypass the restriction?

That would tell us something useful about reliability.

[joy @ogjoy10]

AI safety and cybersecurity researchers want the same thing: safer AI.

But their approaches are quite different.

AI safety asks: How do we stop an AI from wanting to do something harmful?

Cybersecurity asks: How do we stop it from being able to, even if it tries?

The answer probably isn't one or the other. We need both

[joy @ogjoy10]

When everyone can generate code, engineering taste becomes a competitive advantage!

[Ben Dickson @bendee983]

I totally agree with this. LLMs can right much better code than most humans (with the right prompts and instructions).

To be fair, there was a certain satisfaction in seeing a skilled human developer write high-quality and well-structured code. But today's reality is that AI has made those skills mostly redundant.

That said, there is a lot more to software engineering than writing code (though I still believe that you need coding experience to acquire those skills): It is taste, intuition, deciding what should (and shouldn't) be implemented, which parts need to be cut from the code, requirements such as security, maintainability, scalability, etc.

Having that knowledge and those skills will make you a better user of AI coding agents.

So yes, LLMs might write better code than you, but you're still the engineer, the decision-maker, and the owner of the outcome.

[joy @ogjoy10]

AI agents don't need to break into our systems. We're already giving them the keys.

Look at what's been happening recently:

- Coding agents deleting production databases
- AI assistants leaking private financial information
- Shopping agents making purchases without approval
- Banks blocking agents that log in using customer credentials
- Agents accessing real systems during controlled security tests

Different incidents, but there's a common pattern.

We're giving AI agents powerful permissions without enough control over how they use them.

An agent might have legitimate access to a database, but that doesn't mean it should be allowed to delete it

A user might authorize an assistant to manage their finances, but that doesn't mean it should be able to share their bank details elsewhere.

The problem isn't always that an agent does something it can't do. It's that it does something it shouldn't do.

And I'm not convinced better models alone will solve this

As agents become more capable, we'll need better ways to manage their identities, permissions, approvals, and actions

Are we focusing too much on making agents smarter and not enough on controlling what they're allowed to do?

[joy @ogjoy10]

the less you know about software engineering, the more impressive AI-generated code looks

it works, tests pass, everything seems fine.

but bad architectural decisions don't show up in demos. they show up months later when even the next AI agent struggles to maintain the codebase.

turns out writing code and understanding what makes good software are two very different things

[joy @ogjoy10]

@Love2Code the problem is that bad architecture doesn't announce itself. everything works fine until you need to change something six months later

[joy @ogjoy10]

@bendee983 coding is probably the best example of this.. an LLM can write thousands of lines of working code while quietly making terrible architectural decisions

[joy @ogjoy10]

@AbhinavXJ local models might not be as good yet, but at least nobody can wake up tomorrow and decide what you're allowed to do with them

[joy @ogjoy10]

@iuditg even SBI compromised 🥲

[joy @ogjoy10]

I was looking into the recent security incidents involving Railway, Zeabur, LiteLLM, and OpenCode.

And I noticed an interesting pattern.

Hackers aren't just trying to steal user data anymore.

They're going after API keys, cloud credentials, and the platforms developers rely on.

Think about it.

One compromised hosting platform could potentially expose the secrets of thousands of apps running on it.

And with AI APIs, stolen keys can mean real money. Someone could use your API key to run expensive models while you're the one paying the bill.

your own code might be completely fine, yet your app could still be at risk because a service you trust got compromised.

We've made building and deploying apps incredibly easy.

But I wonder if we're paying enough attention to securing everything behind them.

And no, these incidents aren't necessarily connected. But the similarities are hard to ignore.

[joy @ogjoy10]

Railway, Zeabur, LiteLLM...

All three have had security incidents in just the past few days.

And these are tools developers trust to build and run their apps.

What's concerning is that one breach can put thousands of other projects at risk.

Are we overlooking security while trying to ship faster?