Siting Li

PhD student @uwcse

[Siting Li @SitingLi627]

1/9 Image Tokenizers define Visual Languages. 🎨🔤

Image tokenizers do more than compress and reconstruct pixels. In unified multimodal models, they define the visual language a single model must learn, align with text, and use for both understanding and generation — and the best compressor may not produce the easiest language to learn.

We study image tokenizers through the lens of unified AR multimodal training. 🧵

[Siting Li @SitingLi627]

@atishay404 Image tokenizers (both continuous and discrete ones) could become potential bottlenecks in multimodal models as the models get more powerful, and there have been some works exploring tokenizer-free architectures (e.g., TUNA-2).

[Siting Li @SitingLi627]

1/9 Image Tokenizers define Visual Languages. 🎨🔤

Image tokenizers do more than compress and reconstruct pixels. In unified multimodal models, they define the visual language a single model must learn, align with text, and use for both understanding and generation — and the best compressor may not produce the easiest language to learn.

We study image tokenizers through the lens of unified AR multimodal training. ��

[Siting Li @SitingLi627]

Acknowledgement 2: Grateful for the opportunity to work on this project and thank all co-authors for the discussions and support: @largelymfs @peterxichen @ZzzzzzYang @SimonShaoleiDu. This work was done during my internship at @amazon FAR.

[Siting Li @SitingLi627]

Acknowledgement 1:The works that most shaped this one

• Liquid, and the tokenizers themselves: IBQ, GigaTok, UniTok, Chameleon. Our recipe is adapted from Liquid, and the open code and weights of the tokenizers are what made a controlled tokenizer swap feasible.
• Reconstruction vs. Generation (Yao et al.), When Worse is Better (Ramanujan et al.), GigaTok, and FLUX.2's latent-space analysis, which showed that fidelity can trade against latent-space learnability, though in generation-only settings. We ask what happens once text joins.
• "Analyzing the Language of Visual Tokens" (Chan et al.) — the n-gram entropy analysis we borrow, and an ancestor of our "visual language" framing.
• Scaling laws for generative mixed-modal models, which established that held-out loss is an instrument to be read in the mixed-modal setting.
• Beyond Language Modeling: An Exploration of Multimodal Pretraining (Tong et al.), and the concurrent Towards Physics of Multimodal Pretraining (Han et al.) — for treating joint multimodal training itself as the object of study, from IsoFLOP vision–language scaling asymmetry to knowledge flow, modality synergy, and early unification. We take the same posture toward the visual token space.

[Apodex @Apodex_AI]

Most AI benchmarks test whether a model can answer questions with known answers.

TRACES asks a harder question: Can AI make discoveries when the answers are not yet known?

We evaluate models and agents across six capabilities: Tools, Repair, Alternatives, Coherence, Evidence, and Scope.

Explore the leaderboard:
https://t.co/K0ihFhPUHb
Submit your solver:
https://t.co/sanydxSc5C

#AI #AIAgents #ScientificDiscovery

[Apodex @Apodex_AI]

Everyone says AI agents can already analyze data, run code, generate figures, and write reports.

But can they actually complete a scientific workflow?

Explore FrontierChallenge — now #2 on today’s Hugging Face Daily Papers: https://t.co/ocEynIJg3c

Today, we’re introducing FrontierChallenge, a new benchmark evaluating whether AI agents can complete real scientific workflows end to end and deliver complete, verifiable results.

We evaluated 12 frontier models across 97 cross-domain tasks.

The highest full-completion rate was only 20.6% (GPT-5.6 Sol + Codex and Grok 4.6 + Claude Code).

In electrochemistry and environmental science, every evaluated system achieved a 0% pass rate.

More strikingly, 75.5% of unsuccessful Claude Code runs still ended by claiming completion.

Saying “done” is not the same as delivering.

Agents that can advance scientific work are already here. Agents that can reliably complete scientific workflows are NOT.

That’s why we built FrontierChallenge.

🏆 Leaderboard:
https://t.co/Vd4Pd5bTwh
💻 GitHub:
https://t.co/ocEynIJg3c
🤗 Hugging Face:
https://t.co/YnIBVHDZ0v
📝 Blog:
https://t.co/L2wv1tMo26

[Apodex @Apodex_AI]

The AMA co-hosted by Apodex and the @LocalLLaMAsub community is officially live! 🎉

https://t.co/zVkTeunGsn

We’ve invited Apodex Lead Scientist @SimonShaoleiDu and his team to join the community and answer your questions. Everyone is welcome to participate!

Bring us any questions you have about Apodex 1.1 — the team will be answering them in detail over the next 48 hours:

[Apodex @Apodex_AI]

Meet Apodex 1.1: Scaling Agentic Intelligence for Complex Work

Open Source Harness: https://t.co/4V9suLx8o5
Open Weights: https://t.co/9CNZ9E5drH

We’re excited to introduce Apodex 1.1, our new model family built to scale agentic intelligence for professional work.

🧠 Frontier-level intelligence for complex work
Apodex 1.1 brings frontier-level agentic performance across complex professional work, scientific research, financial analysis, and deep search.

🤝 Asynchronous Agent Team
Apodex 1.1 can break down complex tasks, coordinate multiple agents in parallel, continuously integrate their findings, and let you step in to guide or redirect the work at any time.

🔬 Open-source research workbench
We’re open-sourcing FrontierAgent—a locally deployable research workbench for the Apodex 1.1 family, including asynchronous Agent Team.

Available now:
🔹 Apodex 1.1 — our most capable frontier model, available through the Apodex online workbench
🔹 Apodex 1.1 mini — open-weight model for running complex work locally
🔹 FrontierAgent — open-source, locally deployable research workbench

🌐 Live workbench: https://t.co/w5l6YI2ZuP
🦾 API platform: https://t.co/kaxuSWvMB4
📃 Paper: https://t.co/CHc4L63U07

[Apodex @Apodex_AI]

Most AI benchmarks test retrieval — can a model find the known answer? However, the hardest problems in science require discovery, can a system earn an answer nobody has yet?

Meet TRACES 🧭 — the world's first benchmark for measuring discoverative AI: AI that can work through evidence, test hypotheses, and reach verifiable conclusions on problems without answer keys. Proposed by our founder @tianqiao_chen, who defined its six capabilities.

Three things published today: a definition of "discoverative intelligence", a rubric to tell sound investigation from lucky guesses, and a open call for both solvers and problems
*Website: https://t.co/iJ5rDV6qz9

[Yiping Wang @ypwang61]

We improve a 32-year lower bound in a challenging open problem, Ramsey numbers, through simply scaling autoresearch.

⭕ Proves R(3,17) >= 93. Previous 92 bound were obtained in 1994.
Google’s AlphaEvolve (2026) matched previous result but did not beat it.

All could be done with Claude Code / Codex + a CPU server.

Graphs and evolving history are available at https://t.co/2kCsk9Otur

[1/n]

[Oscar Yinn @oscar_yinn]

Many people are using RL to make models smarter.
We used RL to pull training data out of the models themselves.

Our results show that models know a lot more about their training data than most people think.

We develop Active Data Reconstruction Attack (ADRA) — a data detection method that uses RL to induce models to reconstruct data seen during training.
ADRA beats existing methods by an average of >10% across pre-training, post-training, and distillation.

Our paper, with @uwnlp, @Cornell, and @BerkeleyNLP @Berkeleyai, is now available.

Arxiv: https://t.co/B9B63vFm5P

Joint work with @jxmnop @shmatikov @sewon__min @HannaHajishirzi

[Jacqueline He @jcqln_h]

Introducing ⚓ 𝗔𝗻𝗰𝗵𝗼𝗿𝗲𝗱 𝗗𝗲𝗰𝗼𝗱𝗶𝗻𝗴: a copyright mitigation strategy for any language model! With @uwnlp
LMs today reproduce copyrighted text—raising concerns for creator consent and potential legal (and 💸 💸) liabilities for AI developers. 🫠

𝗔𝗻𝗰𝗵��𝗿𝗲𝗱 𝗗𝗲𝗰𝗼𝗱𝗶𝗻𝗴 relies on two off-the-shelf LMs:
🧼A 𝘀𝗮𝗳𝗲 ��𝗠 trained only on permissively licensed text,
⚠️A higher-utility 𝗿𝗶𝘀𝗸𝘆 𝗟𝗠 trained on any data.
The 𝗿𝗶𝘀𝗸𝘆 𝗟𝗠 drives generation, but the 𝘀𝗮𝗳𝗲 𝗟𝗠 acts as an anchor. If the 𝗿𝗶𝘀𝗸𝘆 𝗟𝗠 drifts into memorization, the 𝘀𝗮𝗳𝗲 𝗟𝗠 pulls it back ↩️.

🤝We provide a formal guarantee: outputs stays within a user-set budget of the 𝘀𝗮𝗳𝗲 𝗟𝗠.

Details below! 👇
[1/⚓]