Jing Xiong

Phd student in HKU. Research Direction: Efficient Natural Language Processing and Automated Theorem Proving

[Jing Xiong @_June1126]

@FengzhuoZhang @YuanC233 @Yunlong_Hou_ @ducx_du @duchao0726 @TianyuPang1 @axsun @zhuoran_yang Congratulations on your paper！I would also like to share our related work for your reference: DoPE: Denoising Rotary Position Embedding (https://t.co/ZzrWrcbLbw
). @YuanC233 @Yunlong_Hou_ @ducx_du @duchao0726 @TianyuPang1 @axsun @zhuoran_yang

[Jing Xiong @_June1126]

@PetarV_93 @FengzhuoZhang @YuanC233 @Yunlong_Hou_ @ducx_du @duchao0726 @TianyuPang1 @axsun @zhuoran_yang @Guangxuan_Xiao @songhan_mit @iofu728 @chengruidong @fedzbar I agree with your point. Similarly, I have a paper; the conical constraint induced by the low-frequency component of RoPE has already been studied in DoPE: Denoising Rotary Position Embedding (https://t.co/ZzrWrcbLbw
). I would like to know how the authors judge this paper.

[Jing Xiong @_June1126]

@FengzhuoZhang @YuanC233 @Yunlong_Hou_ @ducx_du @duchao0726 @TianyuPang1 @axsun @zhuoran_yang @Guangxuan_Xiao @songhan_mit @iofu728 @chengruidong @fedzbar @PetarV_93 The conical constraint induced by the low-frequency components of RoPE has already been studied in DoPE: Denoising Rotary Position Embedding (https://t.co/ZzrWrcbLbw
). May I ask for your thoughts on this?@Guangxuan_Xiao @songhan_mit @iofu728 @chengruidong @fedzbar @PetarV_93

[Jing Xiong @_June1126]

@FengzhuoZhang The conical constraint induced by the low-frequency components of RoPE has already been studied in DoPE: Denoising Rotary Position Embedding (https://t.co/ZzrWrcbLbw
). May I ask for your thoughts on this?

[Zhijiang Guo @ZhijiangG]

🤗Will present our #EMNLP2025 paper this morning! TLDR: Beyond KV Cache: New Insights on LLM Sparsity.
This paper offers not just an efficient inference framework, but a new theoretical lens to understand how information flows inside LLMs.
Come & talk to us if you are interested!

[Sumit @_reachsumit]

CTR-Sink: Attention Sink for Language Models in Click-Through Rate Prediction

Introduces behavior-level attention sinks to address semantic fragmentation in LM-based CTR prediction by inserting recommendation-specific tokens between user behaviors.

��https://t.co/UHLoK6aPDd

[Sumit @_reachsumit]

UncertaintyRAG: Span-Level Uncertainty Enhanced Long-Context Modeling for Retrieval-Augmented Generation

Uses SNR-based span uncertainty for improved chunk similarity estimation, offering flexible integration across various LLMs without fine-tuning.

📝https://t.co/4Okp2IezIs

[Jing Xiong @_June1126]

#ICML2025 #ParrallelComp #Long-context #Length Extrapolation #Memory-bound #Efficient-inference #KV cache Compression #128K Token

[Jing Xiong @_June1126]

🔬 The HKU team presents ParallelComp: a training-free technique for efficient context length extrapolation in LLMs—from 8K up to 128K tokens—on a single A100 GPU, with minimal performance loss.

📄 Paper: https://t.co/HbKsGN0eqX
💻 Code: https://t.co/T2Au0WEGY1

[Jing Xiong @_June1126]

🚀 Our 8B LLM achieves 91.17% of GPT-4's performance on ultra-long context reasoning, surpassing formidable models such as Claude-2 and Kimi-Chat—all with only 8K context training.

[Jing Xiong @_June1126]

🧠 A key contribution is our theoretical and empirical analysis of attention bias under parallel attention. We uncover how and why attention sinks emerge and provide effective calibration strategies.

[Jing Xiong @_June1126]

🔍 We tackle memory limitations in length extrapolation by introducing parallel attention, KV cache compression, and chunk eviction strategies that break the GPU memory bottleneck—without any retraining required.

[Jing Xiong @_June1126]

Our paper has been accepted to ICML 2025! 🎉

📢 In this paper, we propose ParallelComp, a training-free method to enable LLMs to extrapolate context length from 8K up to 128K tokens on a single A100 GPU, with minimal performance loss.

[Hui Shen @HuiShen_umich]

📷 New Benchmark Release: PhyX - Physical Reasoning for Multimodal Models

👉 Project Page: https://t.co/cKe4lehSsz
👉 Github: https://t.co/4ZtvWoBsaZ
👉 arXiv: https://t.co/IsxqqtZ1uL
👉 Huggingface Dataset: https://t.co/SU589GaF15

[Chunlin Tian @clin_tian]

🔥Thrilled to announce our Oral acceptance at #NeurIPS2024! 🚀HydraLoRA, an asymmetric LoRA architecture with a shared A matrix for common knowledge and multiple B matrices for specialized adaptations, enhancing model performance while maximizing efficiency with a reduced param.

[yuxuan_yy @cerana99x]

🌟Excited to share LeCo's acceptance at #COLM2024!
🤔Fed up with LLMs' self-correct struggles and endless prompts?
🪄LeCo uses logits for confidence scores, skipping tedious prompts and rethinking from the last correct step.
📖:https://t.co/RMh6f1qKEe
💻:https://t.co/EollTSJBZq

[Zhijiang Guo @ZhijiangG]

+👋LLMs work quite well on modeling/understanding long context.

What about generating long content 🤔

Check our ACL paper ProxyQA for evaluating Long-Form Generation (way longer🪘🪘

📝Paper: https://t.co/5HsVjNf17T
🐙Code: https://t.co/KhSXLin2yU

[Yinhong Liu @YinhongLiu2]

🔥New paper!📜
Struggle to align LLM evaluators with human judgements?🤔
Introducing PairS🌟: By exploiting transitivity, we push the potential of pairwise preference in efficient ranking evaluations that has better alignment!🧑‍⚖️
📖https://t.co/W4wSHQqdYc
💻https://t.co/q5ZMGkvaaj

[DiscreteSpace @space_discrete]

翻到一篇文章[ICLR'24]Understanding Addition in Transformers
回忆起在初学oi年代被老师问了一道题：怎么直接按从左到右的顺序直接做大整数加法，不允许读完再翻转。当时想了十分钟想到一个存9和进位链的做法，被老师夸了。但我觉得这题没现实意义
没想到LLM因为必须从左到右写，也悟出了一样的算法

[Jing Xiong @_June1126]

Excited to announce our paper's acceptance at ICLR 2024! 🌟 Our algorithm leverages CoT for enhanced in-context exemplar selection, optimizing it with a unique mapping function. Dive in: 📄[Paper](https://t.co/7LaZZwPlT1) 💻[Code](https://t.co/tSjPJuZaB2) #ICLR2024 #AI #NLP #LLMs