ATH again. V3 portfolio built in @P123Finance has been a game changer in reducing volatility while protecting returns. Have reduced turnover, improved execution and reduced position concentration!!Exciting to see the last roughly half of the year goes.
I was able to fix this for all my affected sessions by asking codex to go fix it with a screenshot of how it looks. Unironically this is a lesson in self evolving software & how software may work more in the future but glad to get my sessions back @OpenAI@OpenAIDevs
.@thsottiaux there is a major bug on codex mobile where you will have chat history disappear. It will remember the chat but it clears it to some previous message. It’s extremely frustrating. I will try to capture a video but there doesn’t seem to any rhyme or reason to it
Bit more on the technical side.
Yes. There are two different “cursor” concepts, but the broken one is a very small SQLite record—not an opaque UI cursor object.
Exact on-disk representation
It is stored in:
/home/trading/.codex/thread_history_1.sqlite
The exact schema is:
CREATE TABLE thread_history_projection_state (
thread_id TEXT PRIMARY KEY,
next_rollout_byte_offset INTEGER NOT NULL,
next_rollout_ordinal INTEGER NOT NULL
);
A live example from this task was:
thread_id: 01a05a3b-7b0a-7e02-ac12-05e6f6c73e7d
next_rollout_byte_offset: 9,659,469
next_rollout_ordinal: 1,359
Conceptually:
struct ProjectionState {
thread_id: String,
next_rollout_byte_offset: i64,
next_rollout_ordinal: i64,
}
The release binary is stripped, so I cannot confirm whether the implementation literally uses that named Rust struct or an SQLx tuple such as (i64, i64). What is confirmed is that Codex is written in Rust, the responsible paths are:
thread-store/src/local/live_writer.rs
thread-store/src/local/thread_history_materialization.rs
and the two values are persisted as SQLite signed 64-bit integers.
What the fields mean
next_rollout_byte_offset is the byte position in the task’s JSONL file where projection should resume.
next_rollout_ordinal is the exact top-level ordinal Codex expects to find at that byte position.
At end-of-file:
next_rollout_byte_offset == JSONL file size
next_rollout_ordinal == last JSONL ordinal + 1
The bug violated that invariant:
SQLite expected: ordinal 17
JSONL at saved offset: ordinal 18
The projector refuses to guess or skip, so it stops.
What the projection contains
The cursor controls materialization into these tables:
thread_history_projection_state Resume checkpoint
thread_turns UI-visible turns and status
thread_items User, agent, reasoning and tool items
thread_realtime_items Temporary/live presentation items
state_5.sqlite is separate. It stores task metadata such as:
rollout_path
history_mode = legacy | paginated
title/name
archive state
timestamps
cwd
model
When it is created
There are two lifecycle levels:
Database/schema initialization — eager
When app-server opens its local store, SQLx checks/runs migrations. Migration 1, named thread history, creates the projection tables if necessary.
The tables are not recreated every launch; app startup opens the existing database and verifies the migration ledger.
Per-task projection row — lazy/selective
Codex does not create a projection-state row for every task at startup. We had approximately 688 task metadata rows but only 46 projection-state rows.
A row is created when a task becomes history_mode = paginated, through either:
rollout migration/materialization; or
the paginated live writer when it first persists that task.
So the lifecycle is approximately:
App-server startup
↓
Open SQLite + run schema migrations
↓
No full replay of every task
↓
Task migrated/opened for paginated persistence
↓
Create projection-state row
↓
Read JSONL from saved byte offset
↓
Write thread_turns/thread_items
↓
UPSERT new byte offset + next ordinal
When it advances
The installed binary shows projection occurring from live_writer during:
append_items
durable rollout flushes
turn persistence
shutdown flushes
The update is effectively:
INSERT INTO thread_history_projection_state (...)
VALUES (...)
ON CONFLICT(thread_id) DO UPDATE SET
next_rollout_byte_offset = https://t.co/RCPj3NBAhS_rollout_byte_offset,
next_rollout_ordinal = https://t.co/RCPj3NBAhS_rollout_ordinal;
Projection appears transactional. If it encounters a later ordinal mismatch, the catch-up attempt fails instead of leaving a partially updated projection.
What loading a task does
These operations differ:
thread/read reads stored history without loading the task into memory.
thread/resume loads the task so later turns can append to it.
turn/start causes durable writes and therefore activates live_writer.
App startup does not automatically reproject every old task.
After the last subscriber leaves, a loaded task can remain in memory for a 30-minute grace period before unloading.
Those lifecycle distinctions are documented in the official Codex App Server documentation.
That explains why the corruption survived restarts: SQLite opened successfully, but nothing performed a global replay. The bad per-task checkpoint was simply reused whenever the UI queried its stored projection.
CODEX DEBUG RCA
Yes. We have a confirmed failure mechanism.
Codex 0.145.0 wrote the authoritative JSONL transcript correctly, but saved an incorrect cursor in thread_history_1.sqlite.
The cursor said “next ordinal 17,” while the next JSONL record was ordinal 18. The strict projector stopped with: expected ordinal 17, got 18.
Two tasks had additional stale-writer resets: 1432 → 1013 and 1486 → 1474.
The agent reconstructed context from JSONL, while the UI/CLI depended on the halted SQLite projection. OpenAI documents those JSONL transcripts under $CODEX_HOME/sessions.
The fix corrected the 45 cursor mismatches, renumbered the two non-monotonic suffixes in chronological order without changing their payloads, and replayed every transcript into SQLite.
Every affected task was created by Codex 0.145.0; 33 of 45 were subagents. That strongly implicates a 0.145 paginated-history/live-writer race, especially around resuming or concurrent agents. However, without the upstream source change, that final trigger remains a well-supported inference rather than proven. @thsottiaux@btraut
@paulljump@OpenAI@OpenAIDevs Yeah this is my go to but kinda cool it fixed it and reloaded the app itself hackily and fully got me out of this situation. Now all i have to do is tell that chat to fix sessions on repeat when they render badly.
@paulljump@thsottiaux@OpenAIDevs@OpenAI Was this for cowork or codex? My issue is with codex? Is the solution to delete all my history in codex maybe for that folder? Or just remove the remote connection and recreate it?
Ended the day at my best MTD since I started in July 2025, will see if it holds up through the end of the month. Thanks @P123Finance!! Some amount of beauty is to be attributed to being well positioned in factor and industry exposure 😀 thanks to good ranking systems.
Won't leak any details but extremely impressive to see how trades by my models resemble accumulation and distribution patterns.
Interesting difference between model's and discretionary trading is many times they double down where humans psychology would get in the way.