pageshift.ai

An AI lab building models that power the next generation of storytelling across movies, games, and books.

[pageshift.ai @pageshiftAI]

We outperform OpenAI and Anthropic on creative writing…!

Good storytelling requires dishonest characters, unresolved tension, and exposition earned through narrative.

Frontier models are trained to do the opposite: explain everything, resolve uncertainty, be helpful.

So we built a model aligned to storytelling, not assistance.

[pageshift.ai @pageshiftAI]

When building our model for writing full books, we introduced a Planning Scaffold: a structured plan the model generates before writing.

But learning to produce the scaffold does not guarantee that the model also learns to use it during generation.

To test this, we performed an attention-based interpretability analysis. We aligned generated tokens with the scaffold’s semantic components and measured how attention was routed between the current chapter and earlier planning fields.

The results show that retrieval is balanced, persistent, and targeted. Roughly half of the attention remains local, while the rest reaches back to the scaffold. This continues into later chapters, with individual fields receiving more attention when their information becomes relevant.

This clearly proves that our model has learned to use the provided Planning Scaffold to write books.

[pageshift.ai @pageshiftAI]

Generative-media deployments already use a median of 14 different models, according to @fal.

Those models can generate the pieces: images, video, voices, music, etc.

What’s missing is the storytelling engine that brings them together. Something that has the taste to make it actually great.

We are building the solution to this gap. The narrative fabric that ties these modalities together.

https://t.co/ahnpZ6x0uC

[pageshift.ai @pageshiftAI]

Storytelling also remains unsolved for another fundamental reason: taste.

What is taste? It is the ability to combine suspense, playfulness, pacing, and restraint to reveal the right information at exactly the right moment. The interplay of these qualities is what narrative theory has long recognized as the foundation of compelling storytelling.

This became especially apparent when our first research models outperformed OpenAI's and Anthropic's models by 9× on storytelling evaluations. That result was not an accident.

Frontier models are optimized to be predictable. They should be. Their objective is to answer as truthfully as possible. Storytelling, however, requires the opposite. Great stories deliberately withhold answers, build tension, surprise the audience, and reveal information only when it has the greatest emotional impact.

That is why storytelling remains an unsolved problem. It is not simply a challenge of generating coherent text. It is a challenge of narrative taste.

[Paul Graham @paulg]

I was curious why LLMs have gotten so good at math and still aren't that good at writing, so I asked an expert. It's not because math is easier. It's because it has clear right and wrong answers and so is easier to train on. But they are coming for me next.

[pageshift.ai @pageshiftAI]

One of the hardest challenges we had to overcome when building PageStorm was training instability caused by training with book-length sequences.

As is standard, we keep FP32 master weights but run the model using a bfloat16 copy. Normally, tiny FP32 updates accumulate until they become large enough to appear in bfloat16. But in our book-length training regime, even small updates mattered a lot. Standard casting repeatedly erased these sub-resolution changes, so the training copy stopped accurately reflecting the optimizer’s intended trajectory, making training unstable.

The key idea came from stochastic rounding. In the original approach, training happens in bfloat16, with stochastic rounding applied directly to each model update. Instead of always selecting the nearest representable value, it uses randomness so small updates are preserved on average.

We adapted this to our setup. The master weights and optimizer remain in FP32, and we introduce randomness only when creating the bfloat16 training copy. We also scale it with the learning rate because leaving it unscaled degraded performance during the later stages of training. This preserves small updates without destabilizing late training.

[pageshift.ai @pageshiftAI]

@elonmusk Should our model write the script for it? @elonmusk

[pageshift.ai @pageshiftAI]

The Future of Entertainment...
Do you remember the last time you read a book or watched a movie that started that feeling in you that you can't really describe?

It feels like being in that Iron Man suit flying over deserts at night, that romantic mirror scene in The Curious Case of Benjamin Button with Brad Pitt, or that little famous affirming head nod in The Dark Knight Rises between Alfred and Bruce Wayne. For a moment, the world feels bigger, everything is possible, and there is no limit.

Or maybe this is just us and it's different for everyone, but we're sure you know which feeling we're talking about. This is magic. This is storytelling at its best.

But times change. New technologies make new things possible.

You know, we get asked that question a lot... What does the future of entertainment look like? And we usually give the same answer.

"The economics clearly point to it being fully AI-driven." Better retention, lower costs, and faster production.

And while that's true, it's not the real reason.

The actual reason, though, is the emotional part. The part where we unlock that feeling. That feeling of flying over a desert at night in your Iron Man suit. The feeling of dancing in the mirror like Brad Pitt did in The Curious Case of Benjamin Button. That one scene, one line, one moment that stays with you for years and quietly changes something inside you.

The thing is... it's different for everyone.

For the first time in history, we can give you those movies, books, and shows that are for you. That are not made for an audience of millions.

Let's face it. We are all so different, and that's a good thing.

That kid in rural Montana, a mother in Thailand, or a young salesman in Shanghai can each experience their magic moment in their own way. A story that makes them believe a little more. Dream a little bigger. Walk away thinking, maybe I can do that too.

We are working hard so that you no longer have to choose the least-bad option from a shelf built for the average viewer. You are experiencing the ultimate version built exclusively for you.

And that's the real secret.

Hollywood spent a century mastering the art of the crowd.

The next era belongs to the Art of One.

[pageshift.ai @pageshiftAI]

"People won't consume AI content."

Which is kind of a lazy argument from people with a status quo bias.

People don't reject AI. They reject bad content. The label stops mattering the moment the art clears the quality bar.

The receipts:
A Deezer/Ipsos blind test across 9,000 listeners in eight countries found that 97% couldn't distinguish AI-generated songs from human-made ones. Once the output clears the quality bar, the "AI" label becomes largely irrelevant.

Suno alone generates roughly 7 million songs every day more volume than Spotify's global charts with 100M+ users, 2M paying subscribers, and an estimated $300M ARR. Let that sink in.

Another company in the generative media space is Midjourney. It has an estimated 1.2–2.5 million daily active users, generates around $500 million in annual revenue.

The fact that consumer AI companies are generating hundreds of millions in revenue tells you everything you need to know: people want these products. No amount of anti-AI noise changes that.

The market is already voting with its time, money, and streams.

[pageshift.ai @pageshiftAI]

Unlike math or coding, there isn't a large corpus showing how to reason through writing a novel. We also can't synthesize one by doing large-scale RL, like DeepSeek did for R1 with math and coding problems.

One of the core ideas behind our model is that long-form writing needs its own version of reasoning traces. Our solution is a hand-engineered planning scaffold that makes that reasoning process explicit.

It gives the model a structured way to plan before it writes, and serves as the cold start for future reinforcement learning.

[pageshift.ai @pageshiftAI]

Our trained models are available on Hugging Face: https://t.co/2crMbEZ8Pd

[pageshift.ai @pageshiftAI]

We outperform OpenAI and Anthropic on creative writing…!

Good storytelling requires dishonest characters, unresolved tension, and exposition earned through narrative.

Frontier models are trained to do the opposite: explain everything, resolve uncertainty, be helpful.

So we built a model aligned to storytelling, not assistance.

[pageshift.ai @pageshiftAI]

Introducing our paper "Towards Human-Level Book-Writing Capability" a comprehensive, groundbreaking 72-page deep dive into how we built models specialized in creative writing.
https://t.co/6W9OzKObkg

[pageshift.ai @pageshiftAI]

Hollywood seems a little scared about AI...

And honestly, that’s valid as we’re building the story engine for the next era of entertainment.

[pageshift.ai @pageshiftAI]

And again, we made it into the Top 5 of the Hugging Face dataset rankings 🚀

Check out our newest iteration if you haven’t yet: https://t.co/fNywEjNmdc

[pageshift.ai @pageshiftAI]

Our goal is to train models that become the story engine for future AI-generated entertainment. Think books, movies, and shows.

One crucial aspect of reaching that goal is having great datasets… That’s why today we’re releasing a much larger (6k books) version of our LongPage dataset, which we initially released in September. This dataset is the first of its kind focused on helping models write complete novels with sophisticated long-horizon reasoning.

Here’s a breakdown of what the dataset enables:
- Full books (40k–600k+ tokens per sample)
- Hierarchical reasoning traces covering character arcs, plot structure, and worldbuilding
- A complete cognitive roadmap for long-form storytelling
- A cold-start SFT → RL training pipeline

This is essentially chain-of-thought for creative writing. It shows models how to reason about character development, plot progression, and thematic coherence across entire books, not just individual scenes or chapters.

Check it out: https://t.co/v0rvEkUwlv

Follow us to see what exciting stuff comes next!

[pageshift.ai @pageshiftAI]

@Codie_Sanchez watch us!

[pageshift.ai @pageshiftAI]

@_clashluke and there is more to come!

[pageshift.ai @pageshiftAI]

Let the numbers speak for themselves: our newly published dataset ranks among Hugging Face’s Top 5.

[pageshift.ai @pageshiftAI]

Just dropped LongPage on @huggingface: the first dataset teaching AI how to write complete novels with sophisticated reasoning!

- Full books (40k-600k+ tokens each)
- Hierarchical reasoning traces (character arcs, plot structure, world building)
- Complete cognitive roadmap for long-form storytelling
- Cold-start SFT → RL pipeline included

This is "Chain of Thought for creative writing" - showing models HOW to think about character development, plot progression, and thematic coherence across entire books.

Scaling to 100K books!

Perfect for:
- Long-context model training
- Creative AI research
- Novel generation systems
- Hierarchical reasoning studies

Link: https://t.co/v0rvEkV4b3

#AI #LLMs #CreativeAI #MachineLearning #Datasets #HuggingFace #OpenSour