StubroFX

I build trading systems with an AI agent and publish what breaks. 375 verdicts, most of them dead. Numbers, doses, post-mortems. No signals, no course.

[StubroFX @Stubrofx]

I automate a trading system with an AI agent, and I publish what goes wrong. With
numbers. Here's the short version of why that's worth your time.

Three years in, I have 375 recorded verdicts on ideas. Almost all of them say dead.
Last week I audited every single one with a question I'd avoided: how many did I
kill by mistake? Seven. One of those changes money.

That's the account. Not tips. Post-mortems with the arithmetic attached.

Things I've published here: a safety feature that could never fire and passed every
test. A metric that improved 0% of the time because I was measuring my own
assumption. Two copies of a process running side by side for ten hours because a
watchdog matched on formatting. A one-minute clock shift that moved my drawdown by
16 points.

What I don't post: signals, entries, returns, or anything you could copy into a
broker. The mechanics stay private. The engineering and the measurement don't, and
those are the parts nobody writes about.

If you build systems that touch money, some of this will look familiar. Most of it
cost me something to learn.

[StubroFX @Stubrofx]

I removed a year of data as unrepresentative and my results improved. The year I
removed was the only one where the strategy lost.

I had reasons. The period was unusual, conditions didn't match the rest of the
sample, and a well-known argument says to exclude structural breaks.

Every one of those reasons was true. And every one of them was also a reason the
year mattered most.

The number I'd been optimizing improved because I'd deleted the evidence against it.
Not falsified. Deleted, with a justification I believed at the time.

Now any exclusion has to be declared before I look at what it does to the result. If
I can only justify removing data after seeing the improvement, the justification is
the improvement talking.

The uncomfortable version: the periods you most want to exclude are the ones with
the most information in them.

[StubroFX @Stubrofx]

My software read the server clock before the connection was authorized, and placed
the trading session three hours away from the real one.

The platform doesn't fetch server time from the server. It computes it as local time
plus an offset learned at connection. Before authorization, there's no offset. The
difference reads as exactly zero, and zero means "this server runs in UTC".

The journal shows it by the second: expert loaded at 17:02:29, session set at
17:02:36, network authorized at 17:02:39. The measurement happened three seconds
before the fact it was measuring existed.

Worse, my priority rule preferred the measurement over the broker's own published
schedule, on the theory that measuring beats reading a config. The schedule was
right. The measurement was an uninitialized variable of the same type.

**A measurement taken before its input exists isn't a measurement.** It's a default
value in disguise, and it will beat your real data every time if you tell it to.

[StubroFX @Stubrofx]

I added a feature that entered at a better price when one was available. It turned
a parsing error into a position five times larger than intended.

The logic: if the current price is better than the one specified, enter now instead
of waiting. Reasonable on its own.

Now combine it with a parsed price that's wrong by a factor of ten. "Better than
specified" becomes true almost always. The system enters immediately, and because
the distance to the protective level is computed from the wrong price, the size
calculation inflates.

Two features, each defensible alone. Together they build an amplifier for an error
neither of them can see.

I added a plausibility bound: if the real price differs from the parsed one by more
than a set percentage, refuse and alert. Not "use the better one". Refuse.

[StubroFX @Stubrofx]

The string "2,650" parsed as 2.65 in my system, and the result was still a valid
number.

Different sources write thousands differently. Mine assumed one convention. On a
source using the other, prices quietly shrank by three orders of magnitude and
stayed numeric, stayed positive, stayed parseable.

Every downstream check passed. The value was a number. It was in the right type.
It just wasn't the right value.

I keep coming back to the same idea: validators check shape, and shape is not
meaning. "Is this a number" and "is this the number" are different questions, and
almost every validation library I've used only answers the first.

[StubroFX @Stubrofx]

My message loop treated every error response as "nothing new".

Token revoked returns an error. A second instance stealing the connection returns an
error. Rate limiting returns an error. My loop checked whether the response had
items, found none, and went back to sleep.

The process stays alive. The watchdog sees a running process and reports green. The
heartbeat keeps beating. And the system is completely deaf.

This is the failure mode I've come to fear most: everything looks healthy because
every component is technically doing what it was told. The loop was told to check
for items. There were no items. It did its job.

Now every response gets its status checked before its contents, and an unexpected
status is loud. Also: no network call without a client-side timeout, because
"waiting forever" is another way to look alive while doing nothing.

[StubroFX @Stubrofx]

My notification pipeline deleted each event from disk before confirming it was sent.

Send fails, event is already gone. No retry, no record, nothing to recover. A
closed position with no notification looks exactly like a position that never
closed.

I'd built at-most-once delivery by accident, in the one place where at-least-once
was the whole point.

The order matters more than the code: confirm delivery, then mark as done. Never
the other way round. If the process dies between those two steps, a duplicate
notification is annoying. A missing one is invisible.

I had the same class of bug in another service, where a status file was written but
never read by anything. It sat there for months, correct and useless.

[StubroFX @Stubrofx]

My freshness check measured age from the moment my own code noticed the message,
not from when the message was sent.

Which means after any outage, the backlog was fresh by construction. The bridge
comes back up, reads a queue of messages hours old, stamps every one of them with
"now", and hands me yesterday's decisions with today's timestamp.

I found it when a stale instruction appeared with live action buttons attached.

Now age comes from the source timestamp, and anything over three minutes carries a
warning. The general form of this bug: **if your clock starts when you look, you
will never measure a delay.** Everything will always be current, including the
things that aren't.

[StubroFX @Stubrofx]

A number in my logs read 414 when the real value was 4414. The parser had eaten a
digit and nobody noticed for weeks.

The input format numbered its targets: "TP 1: 4414". My regex matched the target
index and the price with one pattern, and on a source that dropped the index, the
first digit of the price became the index.

4414 became 414. Which is a plausible number. It's in range for other instruments,
it doesn't look like garbage, and no validator I had would flag it.

That's the thing about silent corruption: it never produces obviously broken values.
It produces slightly wrong ones that survive every sanity check you thought to write.

The fix was to stop treating position in a string as meaning. The lesson is that a
value can be wrong and still be plausible, and plausible is what gets through.

[StubroFX @Stubrofx]

An audit told me my parser would go silent on unrecognized input. Nineteen hours
later it did, on a live message, and cost me the trade.

The finding was sitting in the "unverified" pile: stop-loss written in a language
the parser didn't know would produce no match, no error and no reply. I'd read it
and moved on.

Next morning a message arrived in that exact format. The parser found no stop,
returned null, and said nothing. It arrived twice. Both times into nowhere.

Silence is the worst failure mode a parser has, because it's indistinguishable from
"there was nothing to do". Now anything containing a recognizable action word gets
an answer, even if the answer is "I saw this and couldn't read the stop".

An unverified finding isn't a wishlist item. It's a prediction with no deadline.

[StubroFX @Stubrofx]

My own analysis tool marked 87 ideas as "refuted" when nobody had actually judged them.

I ran a multi-agent audit: one pass generates findings, a second pass sends skeptics
to challenge each one. The script counted a finding as refuted when it had zero
supporting votes.

The verifiers never ran. They crashed on startup and returned nothing.

Zero votes and zero verifiers look identical if you only count votes. So 87 ideas
went into the graveyard with a confident label, and the report printed a clean
number with no warning anywhere.

I now keep "not checked" as a third bucket, visible in the output, separate from
"checked and rejected". Absence of evidence had been silently rendering as evidence
of absence, inside a tool I built specifically to be skeptical.

[StubroFX @Stubrofx]

My new tool said +4.11. My old record said +1.46. Same idea, same data, same file.
I almost assumed the old number was wrong, because the old number was inconvenient.

Both numbers measure the same thing: how much a candidate filter improves my
risk-adjusted return. One says "far above the bar." The other says "below the bar,
rejected." That's the whole verdict, flipped.

The tempting move is to debug the old one. It's five days old, I've improved things
since, and the new answer is the one I want.

Instead I did the boring thing first. I pointed the NEW tool at the OLD window and
told it to print what the old tool printed.

645 trades. 208.8 total. +0.083 average. +1.46. 33.0% winners.

To the digit. Every one.

So nothing was broken. The tools agree completely. The windows differed, and the
baseline metric itself differs by window, so an identical improvement reads as 1.46
against one baseline and 4.11 against another. The candidate didn't get better. Its
denominator got smaller.

That reproduction took nine minutes to write. It converted an argument about which
tool to trust into a fact about arithmetic.

When a new measurement disagrees with an old one, make the new tool reproduce the
old answer first. If it can't, you're not comparing results. You're comparing bugs.

[StubroFX @Stubrofx]

I found a shelved idea of mine that looked alive on recent data. Then I read the
window it was measured on: "recent years, excluding 2024." 2024 was the year it
failed.

With that year back in, the idea is worse than my baseline on both metrics I care
about, across both windows I test. The entire advantage lived in the gap.

Here's the uncomfortable part. I wrote that exclusion myself, months earlier, and I
gave a reason at the time: one of my data feeds had known problems that year. The
reason was true. I can show you the audit.

But the effect of a true reason can still be to delete the evidence against you.

Nobody was being dishonest. I excluded a year for a data reason in one context, then
a later version of me read "recent years, excluding 2024" as a window definition
rather than as a decision with a cost.

The shelf is now closed. Three other buried candidates I re-checked this week stayed
buried, and that's fine. This one was never alive. It was alive in a subset.

Cutting a failed period is a claim about the answer, not about the window. If the
reason for cutting it is real, your candidate still has to survive with the period
left in. Otherwise you've built a filter for your own results.

[StubroFX @Stubrofx]

My AI reviewer reported that a core piece of my system had been validated by a
broken tool for 17 days. I checked it myself. It had read half the file.

Context: I ran a multi-agent pass over three years of my own research to find ideas
I'd buried unfairly. One agent came back with a headline finding, and it was the
kind that ruins your evening. A central filter, judged by an instrument that didn't
reproduce the baseline. Undetected for 17 days.

The entry it quoted was real. I found it. It says exactly that.

Eight hundred lines further down sits the continuation. A second tool was built. The
baseline reproduced exactly. Both causes of the original mismatch were found and
named, in writing, by me, at the time.

The agent stopped reading at the problem and never reached the fix.

I want to be fair to it. That same pass did useful work, and one of its seven
flagged candidates turned out to be real. The failure mode isn't stupidity. It's
that an agent reading 400 pages will stop at a dramatic sentence, and drama sits at
the START of a story, never at the end.

So the rule I follow now: the louder an agent's finding, the more it needs my own
eyes before it enters a document. Verify the catastrophes, skim the boring stuff.
Most people do it the other way round.

This one cost me twenty minutes. It was two edits away from costing a rewrite.

[StubroFX @Stubrofx]

A filter I was about to accept removed trades that won 33.7% of the time. My
threshold was 30%. Near miss, right? Wrong. That percentage hid the number that
mattered, and it nearly cost me 11% of my annual profit.

The logic of the threshold is sound. My system wins about 39% of trades. A filter
earns its place if it cuts a group that wins clearly less than that. 33.7% is worse
than 39%, so the filter looks directionally right.

Then I printed the total of the removed group. Positive. +23.6 in units of risk.
About 11% of everything the system makes. Not losers. Payers.

So it wasn't cutting losers. It was cutting weak winners. Average per trade goes up,
because you removed below-average trades. Total goes down, because they were still
profitable. Drawdown goes down too. That's why every ratio I looked at improved.

The filter was buying a smoother curve with cash.

Then I ran the same check on a filter I already trade live. Same shape. It removes
858 trades whose combined total is +9.1. My own note said it "removes bad trades."
It doesn't. It removes mildly good ones, and I'm paying for smoothness.

I'm keeping that one. The difference is I now know what I'm buying.

Win rate of a group can't tell you the sign of its sum. Print both, or you'll
congratulate yourself for throwing money out.

[StubroFX @Stubrofx]

One of my own quality rules was quietly killing good ideas for two years. It sounded
like discipline: "the sign has to hold in all three eras."

Nobody argues with that. Then I actually looked at what's inside the eras.

In one of them my baseline system averages −0.004 per trade across 500 trades. It
makes nothing. It's a flat line with noise on it.

Ask any filter to prove itself on that. It can't. There's no signal to improve. A
filter tested there returns a coin flip, and my rule reads that coin flip as
evidence of failure. "The sign didn't hold." No. The coin came up tails.

Every good idea had to pass a test that one third of the panel was incapable of
scoring. Over two years, that's a lot of quiet executions.

The fix is stricter, not softer. An era gets a vote only if it has power: the
baseline itself needs |t| ≥ 1.0 there. An era below that becomes a placebo channel,
where the filter is now REQUIRED to show no effect. A big effect on a dead series
isn't strength. It's fitting.

So the weak era went from judge to control. It now tests something real.

A rule that can only punish and never confirm isn't a test. It's a tax you pay on
every idea you have.

[StubroFX @Stubrofx]

"No edge survives ten years. Look at the last five." Good advice. I took it, and it
made my buried candidates look WORSE, not better. Two reasons, both arithmetic.

I'd been judging ideas on my full history. Fair point that old data describes a
market that's gone. So I cut the window to recent years and re-ran everything that
had died.

First thing that happened: the baseline got stronger. On the full history my system
breaches its account limit in 11.9% of simulated runs. On the recent window, 4.0%.

Read that again as a filter designer. If the thing I'm improving already fails only
4% of the time, then the best possible risk filter can win back four points. On the
old window there were 10.3 points on the table.

The window that revives a candidate shrinks its prize at the same time.

Second: sample size. Recent years hold about a third of my trades. For a t-stat to
merely hold steady when N drops to a third, the effect has to grow by 1.72×. An
additive effect doesn't grow when you shorten the window. Saved costs stay saved.

So an honest candidate gets a WEAKER statistic on the window that suits it better.

I changed the window expecting a better answer. I got a different question, with a
smaller payoff and a higher bar, and I'd planned for none of that.

[StubroFX @Stubrofx]

I shifted a clock by one minute. My drawdown moved 16 points and my risk-adjusted
return moved by 3.25. Same strategy. Same data. Same costs. Nothing else changed.

Here's what I did. My system samples prices on a fixed clock. That clock has five
possible one-minute offsets, and which one I use is an accident of how I wrote it
years ago. So I ran all five.

Average per trade: 0.090 to 0.156.
Drawdown: 36.1 to 52.4.
Risk-adjusted return: 2.48 to 5.73.

Nothing in those five runs is a different idea. It's the same idea, sampled one
minute apart. That spread is my measurement noise, and I had never measured it.

It means something brutal. Any improvement smaller than that spread isn't an
improvement. It's the offset. I've been approving and rejecting changes inside my
own noise and calling the result a decision.

And the part I didn't enjoy. My live configuration sits on the luckiest of the five
offsets. On both axes. My best-looking number, 5.73, is the top of a band that runs
down to 2.48.

So every risk figure I've ever quoted about this system is an optimistic estimate,
not a central one. I have to say that out loud now, in writing, every time.

Before you compare two versions of anything, measure how much your metric moves
when you change nothing at all.

[StubroFX @Stubrofx]

I asked my own system a question I'd been dodging for months. How many good ideas
have I killed by mistake?

I had 375 recorded verdicts. Three years of them. Every one says "tested, dead."

So I went back through all 375 with fresh eyes and one filter: which of these were
judged on a window that no longer describes the market? Twenty got flagged. Sixteen
were unique. Seven survived a second look. One of them changes money.

1.9%.

I expected worse. But the number wasn't the finding. This was: I had spent three
years building defenses against accepting garbage, and I had never once measured
the cost of rejecting something real.

So I measured it. I took my testing tools and fed them a known effect. An edge of
+0.10 in my units gets detected 27% of the time. A smaller one, 14%.

Three quarters of genuine improvements that size are invisible to me. Each one gets
filed as "not proven." And every time, I read that as "doesn't work."

Here's what I didn't do: lower the bar. Out of 375 verdicts, 368 held up. The
strictness was right. What was broken was the window and one dead era inside it.

"Not proven" and "doesn't work" are different verdicts. I'd been writing the first
one and reading the second for three years.

[StubroFX @Stubrofx]

I "found a bug" in my own numbers, wrote a rule about it, and the rule was wrong.
Then I found the real answer by asking one question.

A simulation reported "path to target: 237 days." I checked the source file: 645
trades over 7.6 years. So a step must be a trade, not a day — I corrected the table
to "2.8 years" and wrote a lesson about honest units.

Both wrong.

Between the file and the simulation sits a function that expands trades into a
CALENDAR: every weekday gets a row, days without trades get zero. 645 trades became
1988 rows.

Step = one weekday. The original "237 days" was right all along. ≈11 months.

My mistake: I read the input file, read the loop, and connected them. I never read
the function in between. Half the data path, full confidence.

The check that would have caught it in ten seconds: how many elements does the
simulation receive? 645 or 1988? One question, no theory.

Prove the unit by the LENGTH of the array, not by reading the loop.

[StubroFX @Stubrofx]

My filter looked worse than no filter. Then I compared them properly and it flipped.

The filter cuts trade count by two thirds. Naturally it earns less per year — fewer
bets. Every comparison I ran said "filter loses."

That comparison is meaningless. A filter that trades less will always lose on raw
return, and that tells you nothing about whether it's good.

So I raised the filtered version's risk until BOTH produced identical annual
returns, then compared what I actually care about: how often the account dies.

Same return, both versions:
- with filter: 12.4% chance of hitting the hard limit
- without: 25.3%

Half the failure rate for the same money. The filter wasn't losing. I was asking
the wrong question.

Any time one variant trades less often, normalize to equal return before judging it.
Otherwise you're just measuring frequency.