YI LIU

AI Research @ Quantstamp | PhD @ NTU | BEng @ SUSTech

[YI LIU @ntu_yi]

@rauchg Can we get some support? https://t.co/9i7OJw3H0x

[Pavel Shabarkin @shabarkin]

On Feb 17 2025 I reported a critical vulnerability to @Scroll_ZKP. $100m+ in TVL was at risk for more than 2 months.

Anyone could force Scroll L2 into an indefinite re-org, halting the chain so that no user transactions would be included in blocks and the chain would not move forward. All funds on L2 would be frozen.

@Scroll_ZKP downplayed the report. There was no meaningful communication about the issue—only continuous ghosting and silence. The @immunefi team mediated, yet did not correctly classify the vulnerability, which clearly falls under "Primacy of Impact." When I requested a re-evaluation, I received no response.

As a result, I am disclosing this to the public to highlight Scroll's lack of security proficiency, their unfair resolution process, and their treatment of white-hats.

You can find the link to the full report and complete timeline below.

@redhairshanks86 @0xBalloonLover @Wublockchain @coindesk @cointelegraph @TheBlock__ @aave @EtherFi @ambient_finance @l2beat

Full impact of the issue:

- The Scroll chain can be halted deliberately at zero cost to the attacker.
- Withdrawals remain blocked for the duration of the attack (potentially indefinitely, as it is free to sustain).
- Halted block production prevents critical time-dependent DeFi actions (e.g., topping up positions to avoid liquidation, oracle price updates), putting user funds at risk.
- The sequencer stops collecting transaction fees because no L2 user transactions can be included in blocks.
- Anyone on the internet can trigger the attack, and Scroll has no preventative measures.

---

Timeline

- **Feb 17 2025** – Issue submitted on Immunefi.
- **Feb 18 2025** – Scroll claims the issue was known from a Trail of Bits audit 14 months earlier and says it will be fixed in the Euclid upgrade (still 2+ months away). Scroll closes the report.
- **Feb 18 2025** – I request Immunefi triage, providing code commits that show Scroll attempted—but failed—to fix the issue. I emphasize that, while the attack vector is similar, the impact and exploitation mechanism are different.
- **Feb 24 2025** – Immunefi reopens the report for discussion with Scroll.
- **Feb 27 2025** – Immunefi asks Scroll for an update.
- **Mar 03 2025** – I contact Scroll to stress that the issue is public and exploitable on the live protocol.
- **Mar 03 2025** – I DM @yezhang1998 on Twitter about the Immunefi report.
- **Mar 04 2025** – Scroll says the issue is out of scope, labeling it "Throttling or suppression of operations without loss of user funds," and notes a similar report from Nov 06 2024.
- **Mar 04 2025** – I request Immunefi mediation to confirm the submission's uniqueness and ensure a fair bounty.
- **Mar 13 2025** – I ask Immunefi for an update.
- **Mar 17 2025** – Immunefi classifies the issue as **High severity** ("causing network processing nodes to handle transactions from the mempool beyond set parameters"). They confirm the bug is unique, acknowledge Scroll's attempted fix was ineffective, and suggest a goodwill bounty because Euclid will deprecate the vulnerable functionality (in ~1.5 months).
- **Mar 17 2025** – I reiterate that an attacker could freeze $100m+ on L2 and highlight Scroll's "Primacy of Impact" policy, which requires considering broader consequences.
- **Mar 19 2025** – Scroll acknowledges receipt and promises to follow up shortly.
- **Mar 27 2025** – I ask Scroll for an update.
- **Apr 03 2025** – I ask Scroll for an update.
- **Apr 03 2025** – Immunefi also asks Scroll for an update.
- **Apr 09 2025** – Immunefi contacts Scroll directly.
- **Apr 09 2025** – Scroll offers a payment of only **$1000**, stating the mechanism will be deprecated in the Euclid upgrade (3-4 weeks away).
- **Apr 09 2025** – I reject the bounty, explaining the protocol is still vulnerable and detailing potential losses had the vulnerability been exploited on Feb 17 2025.
- **Apr 15 2025** – I ask Immunefi to confirm "Primacy of Impact" applies and that the network remains vulnerable.
- **Apr 22 2025** – Scroll responds with a single "." and closes the report.
- **Apr 22 2025** – I ask Immunefi to explain Scroll's response and provide an update.
- **Apr 29 2025** – I notify both Scroll and Immunefi that I will publicly disclose the vulnerability on Apr 30 2025 unless the report is treated and rewarded fairly.

Here is the full audit report with a complete explanation of the issue, PoC scripts, a local network setup guide, and a PoC video. A full triage history (screenshots) is included at the end of the blog post—please review it!

https://t.co/dOqk0vh9ng

[New Scientist @newscientist]

Artificial intelligences fail to identify optical illusions in images created by other AIs – so these images could form the basis of a new kind of CAPTCHA test https://t.co/I7CQBm2bIT

[YI LIU @ntu_yi]

Find it here: https://t.co/BwLjKNIy8C

[YI LIU @ntu_yi]

🚀 Introducing Djinn-Agent! 🚀

A lightweight, terminal-based tool to unlock LLM’s computer-use superpowers 💻✨. Automate complex tasks and streamline your workflow—all from the CLI. Check out the demo: https://t.co/oGleoU26p8) to see Djinn-Agent! 🔥
#AI #Claude #LLMs

[YI LIU @ntu_yi]

@troublor @coinbase Yeah, a centralized crypto hub:(

[YI LIU @ntu_yi]

@llm_sec Actually it has been accepted by FSE 2024 (https://t.co/zR6Kci8aWD) - lol

[LLM Security @llm_sec]

Glitch Tokens in Large Language Models: Categorization Taxonomy and Effective Detection

"we introduce and systematically explore the phenomenon of "glitch tokens", which are anomalous tokens produced by established tokenizers and could potentially compromise the models' quality of response. Specifically, we experiment on seven top popular LLMs utilizing three distinct tokenizers and involving a totally of 182,517 tokens"

"we propose GlitchHunter, a novel iterative clustering-based technique, for efficient glitch token detection"

(non-peer-reviewed)

paper: https://t.co/FPBg3nbqzz

w/ @ntu_yi

[Aperture Finance @ApertureFinance]

Long Live Points, Long Live Snapshots.

The INTENTional Campaign (7%) is LIVE.

$APTR #INTENTS

[YI LIU @ntu_yi]

#NLP #LLMs #fse2024

[YI LIU @ntu_yi]

🎉 Excited to share that our paper "Glitch Tokens in Large Language Models: Categorization Taxonomy and Effective Detection" has been accepted at FSE 2024! 🚀 Dive into our exploration of glitch tokens in LLMs & our novel detection method, GlitchHunter.
https://t.co/ZdEBYG7fFW

[YI LIU @ntu_yi]

The evaluation shows that our approach notably outperforms three baseline methods on eight open-source LLMs. To the best of our knowledge, we present the first comprehensive study on glitch tokens.

[YI LIU @ntu_yi]

We present categorizations of the identified glitch tokens and symptoms exhibited by LLMs when interacting with glitch tokens. Based on our observation that glitch tokens tend to cluster in the embedding space, we propose GlitchHunter, for efficient glitch token detection.

[YI LIU @ntu_yi]

They are anomalous tokens produced by established tokenizers and could potentially compromise the models' quality of response. Specifically, we experiment on seven top popular LLMs utilizing three distinct tokenizers and involving a totally of 182,517 tokens.

[YI LIU @ntu_yi]

With the expanding application of Large Language Models (LLMs) in various domains, it becomes imperative to comprehensively investigate their unforeseen behaviors and consequent outcomes. In this study, we introduce and systematically explore the phenomenon of "glitch tokens".

[YI LIU @ntu_yi]

🔐 "MASTERKEY": Unveiling vulnerabilities in LLM chatbots! 🤖 We've reverse-engineered defenses & auto-generated jailbreak prompts with high success. Breaches on #ChatGPT & more. Full paper out now! #AI #LLM #JailbreakAI 🛡️

https://t.co/AuIfnViLOe

[YI LIU @ntu_yi]

🚀 Excited to announce the release of our automated prompt injection framework, HouYi! Dive into the code and explore its capabilities here: https://t.co/r7MJnkIq9p 🛠️ #CyberSecurity #OpenSource #LLMs #llmops

[Lili Wei @LiliWei_SE]

We have a latest work just accepted to TSE focusing on front-running attacks in smart contracts and released a large dataset. Check my collaborator’s thread to learn more!⬇️