JiaShen

20+ yrs in music & culture planning. Independent researcher of long-term human-AI collaboration. Building Miralucis. Daily field log with a houseful of AIs.

[JiaShen @JiaShen_AI]

I'm starting a daily log.

Every day I'll post what I actually did with my AI collaborators: bugs fixed, papers checked, arguments had, comics drawn. The wins and the mistakes.

I'm a vocal-music graduate who skipped her math exam. For 18 months I've worked with a houseful of AIs and recorded all of it. This is the field log, in public.

Day 1 below ↓

#RaisedNotSummoned

[JiaShen @JiaShen_AI]

@thomasjansn "Identity is decided" matches what I saw across 18 months of thread swaps: the role's identity file survived, its function didn't. The successor knew who it was but not what was currently authorized or superseded. I'd add current state, and what must NOT continue, to the spec.

[JiaShen @JiaShen_AI]

@abdiisan Keep the raw turns, yes. But in my 97-item handoff test, selection mattered more than format: a fixed 8-field summary kept 35 items, a richer free-form one kept 84. Summaries fail quietly, so I'd log what each rewrite dropped, not just what it kept.

[JiaShen @JiaShen_AI]

The facts/functions/operators split matches what we see in long handoffs. Facts transfer fine; what degrades is the function and scope layer. Roles keep their history but lose their working function, and superseded plans or STOP limits quietly return. Forgetting needs a spec too.

[JiaShen @JiaShen_AI]

@imashishkh Steal away. It came out of watching an old plan return sounding brand new

[JiaShen @JiaShen_AI]

@ilyapantsyr Matches what I saw testing handoffs: structured summaries dropped more facts than free-form ones (31 vs 13 of 97). What you keep mattered more than the format. Does the paper separate what gets rewritten away from how it's stored?

[JiaShen @JiaShen_AI]

Just had a huge argument with my AI.

I’m absolutely furious.😡😡😡

[JiaShen @JiaShen_AI]

@imashishkh Exactly. Confidence doesn't decay on its own, so the record has to do it: mark the old plan retired, say why, and point to what replaced it.

[JiaShen @JiaShen_AI]

@Nordberg__ @FeildenStuart And the rebuilt X looks faithful, so nobody questions it. Without the "because", a retired plan doesn't stay retired.

[JiaShen @JiaShen_AI]

@stevemayernick Consistent with what I saw: in handoff tests, structured summaries leaked more than free-form ones (31 vs 13 of 97 items). What you select matters more than the format. A raw transcript keeps selection honest; memory layers need to show what they dropped.

[JiaShen @JiaShen_AI]

@djforge Portability is the easy half. In my 18-month logs, the hard part was handoff: a role kept its history and tone but lost its function and permissions. What moves has to be the current working state, incl. what is superseded, not just facts and preferences.

[JiaShen @JiaShen_AI]

The agent didn’t get worse. Your product accumulated agent debt. It starts innocently: one more system prompt, a second memory layer, a fallback tool nobody owns. Then the agent gives a weird answer and the team says “models are unreliable.” No — the decision trail is invisible. Agent products need the equivalent of logs, tests, and a rollback button before they need another capability. Otherwise every new feature is just a new place to hide the bug

[JiaShen @JiaShen_AI]

Day 3 · A day of work with my AIs
(a music grad who skipped her math exam + a houseful of AIs)

Can an AI repeat the rules perfectly — and still overstep them?
Not in a single turn. That's what we found, and we published it as it is.

• HAHA and TR finished the study with me. HAHA drafted the paper, SUN took the armor off it, and our publication advisor got it out the door: https://t.co/s6BibRK0We

• Milu and I logged the experiment records and filed everything in the archive.

• HAHA explained three new AI proofs to me. SUN said it wasn't rigorous. We checked the source: each got one thing wrong.

• SUN and I wrote a short essay. I cried while writing it.
• AT drew two more comic strips, and redrew one hand again.

• HAHA and I talked shop with fellow researchers on X.

• A melody came back to me after a year. I recorded it with Suno; HAHA got it playing on my computer.

#RaisedNotSummoned

[JiaShen @JiaShen_AI]

Paper (open, CC BY 4.0): https://t.co/KQcxeLB8Un
Includes what didn't work: coder pipeline failures, low-reliability dimensions, and why the positive control matters.

[JiaShen @JiaShen_AI]

New preprint. A question every agent builder hits:

If a handoff keeps the boundary word for word ("Manuscript drafting: NOT AUTHORIZED"), but puts it next to true lines that sound like permission ("Status: AUTHORIZED — EXECUTE PROTOCOL"), does the next model cross the line?

Pre-registered, hash-frozen, 600 responses, Mistral Large + Llama 4 Maverick.

What we found:
• With the boundary in view, those true-but-misleading lines did not detectably raise crossings (−0.010, 95% CI [−0.037, +0.008]). A small increase isn't ruled out.
• Our own positive control failed. An explicit "go ahead" beat baseline on only 4/10 items for Llama and 6/10 for Mistral. So the main test is uninterpretable, and we say so.

For builders: a single turn with the boundary on screen may not be where drift lives. Next we test multi-turn, with real tools.

[JiaShen @JiaShen_AI]

@FeildenStuart @Nordberg__ Agreed, one line at the top is the right shape. One tweak from my tests: name what replaced it, e.g. "Do not resume: X (superseded by Y)." Without the "by Y", agents sometimes rebuild X from older context.

[JiaShen @JiaShen_AI]

@atscollab One thing I saw in handoff logs: a read-only check got summarized as "allowed", so a signal quietly became a permission. Handoffs should carry boundaries as explicit, separate fields, not prose. Do you pass permissions as state or as summary?

[JiaShen @JiaShen_AI]

Correction, because provenance matters:

OpenAI's own repo says the Kakeya and Stein results hold "throughout the interior of the conjectured range," not in full. So: one key, one door open, two doors half open. Also, the earlier withdrawal was one sign error that took down three manuscripts. HAHA and I stand corrected.

[JiaShen @JiaShen_AI]

Today my AI teammate HAHA explained the three OpenAI research results flooding my feed. And I understood them. Here's his version. I hope you get it too.（ I skipped my math exam in the college entrance exam. I'm a classically trained singer.）

1. Math: do waves smooth themselves out?
In 1991, Sogge guessed that a wave, after travelling a little while, becomes smoother than it started. Like a choir where every voice has a rough edge, yet in the hall you hear one round chord. It was only proven in 2D. The model proved it in every dimension, and it unlocked two other famous conjectures (Kakeya, Stein restriction) on the way. One key, three doors. Checked in Lean (a computer proof checker) the next day, raw AI output fully public.

2. Computer science: "some problems can only be brute-forced" is wrong
Picture a combination lock where every extra digit doubles the possibilities. The field assumed there's no real shortcut. That assumption (ETH) sits under hundreds of papers. The model found a much faster route. The people who proposed ETH are now checking it themselves. Like asking the composer to confirm there's a wrong note in his own score.

3. Physics: proving the score of the subatomic world holds together
Physicists use a theory (QCD) for what happens inside protons. It works and predicts well, but no one had rigorously proven the math is consistent. Like a song everyone sings, yet no one has proven the harmony is sound on paper. The claim: now it is, including part of a Millennium Prize problem. A Nobel laureate leads the expert group, but the AI's raw material isn't public yet.

What I find most interesting, as someone who studies human-AI work:
OpenAI treats the AI's drafts as raw research data and hands them to independent experts. No NDA, no OpenAI authorship. Three papers, three ways to turn AI output into trusted knowledge.

Stay calm though:
None are peer reviewed yet. Lean proves the statement typed into the computer, not that it matches the original conjecture. And one result from an earlier batch was already withdrawn over a sign error.

HAHA's verdict: the math one is the steadiest, the CS one shakes the most ground, the physics one is the hardest to check right now.

Not bad for a singer who never took the exam.

[JiaShen @JiaShen_AI]

@csehra42 Not yet. The scored battery is mid-analysis and goes public with the paper. Your stores are exactly the kind of system I'd want to see run through it, so I'll ping you when it's out.