Obedience Adara

Building in public: Aerospace, Defense Tech, RL/AI, & Autonomous Systems | Demos, research, and technical builds | Building @plexhedge.

[Obedience Adara @ObedienceAdara]

One of the more interesting things I've been researching lately is something I'm calling agent-generated entropy.

An agent can successfully complete the task and still leave the system worse than it found it.

[Obedience Adara @ObedienceAdara]

I also added the experiment to my RL lab with the actual reward curve and an agent recording.

Next up: I'm gonna try to figure out why the policy is struggling, then test multiple seeds + different state discretizations.

https://t.co/O0OQJvm9Yy

[Obedience Adara @ObedienceAdara]

First proper reinforcement learning experiment.

I implemented tabular Q-learning from scratch on CartPole-v1, with state discretization, ε-greedy exploration and the Bellman update.

The result? 52.0 ± 36.7 eval reward.

Repo: https://t.co/Jm5up27W7l

Attach my reward curve

[Obedience Adara @ObedienceAdara]

Yeah… this one didn’t solve CartPole.

Best training episode: 191
Success rate: 0%
Training runs: 1

Keeping the bad result too. I want the repo to show what actually happened, not just the runs that look good.

[Obedience Adara @ObedienceAdara]

@lucyjcai surprised the glass didn't shatter 😭 RIP to the table frame though

[Obedience Adara @ObedienceAdara]

@kaggle Ensembling

[Obedience Adara @ObedienceAdara]

@Sahil100X @jitendarma73662 @Adarsh_VMore @ashutosh_sao @deepanshuliv @Deepjyoti100x @kirat_tw @100xDevs @100xSchool @rohanBuilds @AnkurJangid02 🙌🔥

[Obedience Adara @ObedienceAdara]

A more intelligent model doesn't automatically make a better agent.

Give the same model better tools, context, state, feedback loops and permissions, and you can get a dramatically better system.

[Obedience Adara @ObedienceAdara]

Hot take:

The next AI engineering skill isn't prompt engineering.

It's context engineering.

The model is only as good as the world you construct around it.

[Obedience Adara @ObedienceAdara]

Intelligence may be better defined by the ability to learn every task than by already knowing every task

[Obedience Adara @ObedienceAdara]

Autonomous agents should be evaluated not only by what they accomplish, but by the state they leave behind

[Obedience Adara @ObedienceAdara]

Quite happy to see a lot of interesting research & development in continual learning

[Obedience Adara @ObedienceAdara]

And that creates the hardest question:

How do we know a change was necessary?

The developer patch isn't a perfect oracle. Neither is the smallest possible diff.

I'm digging into that problem now.

[Obedience Adara @ObedienceAdara]

I've been using "agent-generated entropy" as a working term.

I'm not claiming I invented software entropy. I'm asking a narrower question:

What happens when an autonomous agent is the source of unnecessary state change?

[Obedience Adara @ObedienceAdara]

One of the more interesting things I've been researching lately is something I'm calling agent-generated entropy.

An agent can successfully complete the task and still leave the system worse than it found it.

[Obedience Adara @ObedienceAdara]

This is why I don't want to define entropy as "large diffs."

A 500-line migration can be necessary.

A 3-line unrelated change can be unnecessary.

I'm interested in justified state change, not line count.

[Obedience Adara @ObedienceAdara]

@bromann @LangChain Yh, actually 😋

[Obedience Adara @ObedienceAdara]

@Ananth7e Nah, it will be too dangerous in the hands of the public

[Obedience Adara @ObedienceAdara]

Intelligence may be better defined by the ability to learn every task than by already knowing every task