Rujikorn (Tan) Charakorn

🇹🇭 based in 🇯🇵
Research @SakanaAILabs
Hypernet in the era of LLMs: @VISTEC_Thailand
B.Eng. @chulalongkornU

[Rujikorn (Tan) Charakorn @tan51616]

Doc-to-LoRA (📜 → 🧠): Hypernetworks that update LLMs to remember factual information!

Super excited to finally release the blog post to the public😚💖

It's a culmination of my work on hypernetworks since I was an intern @SakanaAILabs . I'm really proud of it!

[Sakana AI @SakanaAILabs]

We’re excited to introduce Doc-to-LoRA and Text-to-LoRA, two related research exploring how to make LLM customization faster and more accessible.

https://t.co/ApVzVsBuv1

By training a Hypernetwork to generate LoRA adapters on the fly, these methods allow models to instantly internalize new information or adapt to new tasks.

Biological systems naturally rely on two key cognitive abilities: durable long-term memory to store facts, and rapid adaptation to handle new tasks given limited sensory cues. While modern LLMs are highly capable, they still lack this flexibility. Traditionally, adding long-term memory or adapting an LLM to a specific downstream task requires an expensive and time-consuming model update, such as fine-tuning or context distillation, or relies on memory-intensive long prompts.

To bypass these limitations, our work focuses on the concept of cost amortization. We pay the meta-training cost once to train a hypernetwork capable of producing tasks or document specific LoRAs on demand. This turns what used to be a heavy engineering pipeline into a single, inexpensive forward pass. Instead of performing per-task optimization, the hypernetwork meta-learns update rules to instantly modify an LLM given a new task description or a long document.

In our experiments, Text-to-LoRA successfully specializes models to unseen tasks using just a natural language description. Building on this, Doc-to-LoRA is able to internalize factual documents. On a needle-in-a-haystack task, Doc-to-LoRA achieves near-perfect accuracy on instances five times longer than the base model's context window. It can even generalize to transfer visual information from a vision-language model into a text-only LLM, allowing it to classify images purely through internalized weights.

Importantly, both methods run with sub-second latency, enabling rapid experimentation while avoiding the overhead of traditional model updates. This approach is a step towards lowering the technical barriers of model customization, allowing end-users to specialize foundation models via simple text inputs. We have released our code and papers for the community to explore.

Doc-to-LoRA
Paper: https://t.co/87xEEpf0GN
Code: https://t.co/zBfQi2L9LW

Text-to-LoRA
Paper: https://t.co/emLRZ4Vdvo
Code: https://t.co/b9mrdoWWRB

[Rujikorn (Tan) Charakorn @tan51616]

@peterwisu Let’s grab a coffee together!

[Rujikorn (Tan) Charakorn @tan51616]

Doc-to-LoRA will be presented at #icml this year! Happy to talk and discuss the project throughout the conference. Feel free to reach out!

Come see our poster at Thu, Jul 9, 5:00 PM – 6:45 PM KST; HALL A #3803

See our blogpost https://t.co/ENSnoyqILM

[Sakana AI @SakanaAILabs]

"Doc-to-LoRA: Learning to Instantly Internalize Contexts" will be presented at #ICML2026

Paper: https://t.co/sZOMq0kKpj

Long-term memory is one of the most important cognitive capabilities that LLMs still lack today. Without long-term memory, users have to provide LLMs with relevant content at the start of every new session, creating friction, discontinuity, and longer time-to-response. Doc-to-LoRA tackles the long-term memory problem directly. Given a document, it generates a LoRA adapter that internalizes the document’s content into the base model’s weights. Subsequent questions can then be answered without the document ever appearing in the context window—cutting both latency and VRAM, and adding long-term memory well beyond the model’s native context limit.

https://t.co/tNLtfWghQG

[Sakana AI @SakanaAILabs]

Sakana AI is heading to #ICML2026 in Seoul (July 6–11)! 🐟🇰🇷

Our team will present 11 papers spanning multi-agent coordination, sparse and efficient LLMs, test-time scaling, long-term memory, and agent benchmarks. A thread of everything we're presenting:

[Sakana AI @SakanaAILabs]

Introducing Sakana Fugu: A full multi-agent orchestration system accessible via a single model API.

Our ‘Fugu Ultra’ model matches the performance of Fable and Mythos, delivering frontier capability without the risk of export controls.

Try it: https://t.co/hhO6qTawgb 🐡

[Rujikorn (Tan) Charakorn @tan51616]

Super happy to had a chance to give a talk on this series of work during my time @SakanaAILabs. This talk goes through the high-level idea and the theme of our research, meta-training cost amortization of model updates. Excited to keep working on hypernetworks in the age of LLMs

[Rujikorn (Tan) Charakorn @tan51616]

@SakanaAILabs Thank you @savvyRL @ml_collective for giving me a spot to spread the words and @RobertTLange for always supporting and helping out during the talk!

[Rujikorn (Tan) Charakorn @tan51616]

Super happy to had a chance to give a talk on this series of work during my time @SakanaAILabs. This talk goes through the high-level idea and the theme of our research, meta-training cost amortization of model updates. Excited to keep working on hypernetworks in the age of LLMs

[Sakana AI @SakanaAILabs]

Sakana AI research scientist Rujikorn (Tan) Charakorn recently presented Doc-to-LoRA at @MLCollective’s DLCT journal club, covering hypernetworks, cost amortization, and future directions. A very lively discussion followed. Many thanks to the organizers!

https://t.co/kAKLdNvcLL

[Sakana AI @SakanaAILabs]

Sakana AI research scientist Rujikorn (Tan) Charakorn recently presented Doc-to-LoRA at @MLCollective’s DLCT journal club, covering hypernetworks, cost amortization, and future directions. A very lively discussion followed. Many thanks to the organizers!

https://t.co/kAKLdNvcLL

[Robert Lange @RobertTLange]

Really excited to be part of this journey and team 🚀

Ever since @SakanaAILabs inception, we have been discovering stepping stones for a fundamental 'AI² paradigm shift': Leveraging AI systems to improve themself and discover knowledge. This was just the beginning.

Please reach out if you want to be part of this journey. I will be at ICML in Seoul and am happy to chat in person. We are recruiting 🤗

[Sebastian Risi @risi1979]

Let’s go 🚀

[Sakana AI @SakanaAILabs]

Building AI that Builds AI: Introducing the Sakana AI RSI Lab 🚀

https://t.co/AskX3J5oEJ

Today, we are announcing the Sakana AI Recursive Self-Improvement (RSI) Lab: a dedicated research group in Tokyo tasked with redesigning the AI development process itself using AI.

While the industry increasingly speculates about the theoretical potential of self-improving AI, we’ve spent the last two years actively laying the foundations to make it a reality:

▪ LLM²: AI models automating research to invent better preference optimization algorithms.
▪ Darwin Gödel Machine: Agents autonomously rewriting their own codebase to double software-engineering performance.
▪ ShinkaEvolve: Hyper-sample-efficient program evolution that builds novel loss functions for MoE models.
▪ ALE-Agent: Reinforcement agents outperforming hundreds of human experts via self-learning.
▪ Digital Red Queen: Open-ended adversarial coevolution laying the groundwork for RSI in cybersecurity.
▪ The AI Scientist: Towards end-to-end automation of AI research, recently published in Nature.

Now, we are unifying these breakthroughs. The Sakana AI RSI Lab is officially tasked with building open-ended, adaptive architectures that collectively self-improve.

Human intelligence did not emerge from limitless resources; it was forged through the open-ended, compounding process of evolution operating under strict constraints. We are applying this exact principle to AI.

We believe recursive self-improvement is achievable on modest, sample-efficient compute. It shouldn’t be a winner-take-all asset locked inside hyperscale clusters, but a democratized public good.

We’re scaling our team to execute this mission. We are looking for frontier scientists and engineers who are entirely unsatisfied with the brute-force status quo. If you are ready to break away from standard benchmarking and build the self-improving future in Japan, come build with us.

[Rujikorn (Tan) Charakorn @tan51616]

@shengranhu @Recursive_SI Congrats Shenran!!

[Rujikorn (Tan) Charakorn @tan51616]

@VictorTaelin Did that happen on xhigh?

[Sakana AI @SakanaAILabs]

We’re launching the beta for our new commercial AI product: Sakana Fugu 🐡, a multi-agent orchestration system!

Blog: https://t.co/36Ud311KCP

Fugu hits SOTA on SWE-Pro, GPQA-D, and ALE-Bench, and has been our internal secret weapon. It dynamically coordinates frontier models, autonomously selecting the optimal agent combinations and roles for each task.

Available as an OpenAI-compatible API, you can seamlessly integrate Fugu into your existing workflows with minimal changes.

🐟 Fugu Mini: High-speed orchestration optimized for latency
🐡 Fugu Ultra: Full model pool utilization for deep, complex reasoning

Apply for the beta test here: https://t.co/1fjuAha7ci

[Rujikorn (Tan) Charakorn @tan51616]

@badlogicgames is /fast turned on for codex? i think it's on by default

[Rujikorn (Tan) Charakorn @tan51616]

@lucasmeijer I built an "activity block" extension for my own use. This requires some patches to pi core so it's not super clean but works very well in that it removes a lot of toll call noise and you can jump back to the prompt fairly quickly in case you forget your prompt

[Rujikorn (Tan) Charakorn @tan51616]

@lucasmeijer @badlogicgames @mitsuhiko Made an issue requesting for "zen mode". Also post a diff there. Not sure if Mario is happy with this one though https://t.co/ot7XByYgio

[Rujikorn (Tan) Charakorn @tan51616]

@badlogicgames Or you can always go back i should say…

[Rujikorn (Tan) Charakorn @tan51616]

@badlogicgames /tree is such an amazing feature introduced in pi. After you use it you cant go back!

[Andrej Karpathy @karpathy]

Judging by my tl there is a growing gap in understanding of AI capability.

The first issue I think is around recency and tier of use. I think a lot of people tried the free tier of ChatGPT somewhere last year and allowed it to inform their views on AI a little too much. This is a group of reactions laughing at various quirks of the models, hallucinations, etc. Yes I also saw the viral videos of OpenAI's Advanced Voice mode fumbling simple queries like "should I drive or walk to the carwash". The thing is that these free and old/deprecated models don't reflect the capability in the latest round of state of the art agentic models of this year, especially OpenAI Codex and Claude Code.

But that brings me to the second issue. Even if people paid $200/month to use the state of the art models, a lot of the capabilities are relatively "peaky" in highly technical areas. Typical queries around search, writing, advice, etc. are *not* the domain that has made the most noticeable and dramatic strides in capability. Partly, this is due to the technical details of reinforcement learning and its use of verifiable rewards. But partly, it's also because these use cases are not sufficiently prioritized by the companies in their hillclimbing because they don't lead to as much $$$ value. The goldmines are elsewhere, and the focus comes along.

So that brings me to the second group of people, who *both* 1) pay for and use the state of the art frontier agentic models (OpenAI Codex / Claude Code) and 2) do so professionally in technical domains like programming, math and research. This group of people is subject to the highest amount of "AI Psychosis" because the recent improvements in these domains as of this year have been nothing short of staggering. When you hand a computer terminal to one of these models, you can now watch them melt programming problems that you'd normally expect to take days/weeks of work. It's this second group of people that assigns a much greater gravity to the capabilities, their slope, and various cyber-related repercussions.

TLDR the people in these two groups are speaking past each other. It really is simultaneously the case that OpenAI's free and I think slightly orphaned (?) "Advanced Voice Mode" will fumble the dumbest questions in your Instagram's reels and *at the same time*, OpenAI's highest-tier and paid Codex model will go off for 1 hour to coherently restructure an entire code base, or find and exploit vulnerabilities in computer systems. This part really works and has made dramatic strides because 2 properties: 1) these domains offer explicit reward functions that are verifiable meaning they are easily amenable to reinforcement learning training (e.g. unit tests passed yes or no, in contrast to writing, which is much harder to explicitly judge), but also 2) they are a lot more valuable in b2b settings, meaning that the biggest fraction of the team is focused on improving them. So here we are.