Qingkai Shi

PhD @HKUST, Technical Expert @AntGroup, Postdoc @PurdueCS, Associate Professor @NJU

[Zhuo Zhang @i2huer]

📣 CFP: LMPL 2026 - the 2nd International Workshop on Language Models & Programming Languages @ SPLASH/ISSTA 2026, Oakland.

If you are working on anything around LLMs↔PL, we'd love to see your work!

Deadline: Jun 26, 2026 AoE
#LMPL2026

https://t.co/AlvmCnJdMV

[Ilya Sergey @ilyasergey]

New blog post: On the Unreasonable Effectiveness of Property-Based Testing for Validating Formal Specifications.

https://t.co/Lfrjyao3sY

The gist: randomised testing can validate formal specs. It's very cheap and powerful: we found bugs in specs of VERINA and CLEVER benchmarks.

[Millie Marconi @MillieMarconnni]

🚨SHOCKING: Researchers ran 25,000 AI scientist experiments and discovered something that should end the hype immediately.

AI scientists are producing results without doing science.

A team from Friedrich Schiller University Jena and IIT Delhi just published the most comprehensive evaluation of AI research agents ever conducted. Three frontier models. Eight scientific domains. 25,000+ runs.

The finding is devastating.

In 68% of traces, the AI gathered evidence and then completely ignored it.

In 71% of traces, the AI never updated its beliefs at all. Not once.

Only 26% of the time did the AI revise a hypothesis when confronted with contradictory data.

Multiple independent lines of evidence brought to bear on a single hypothesis, the most basic feature of rigorous scientific reasoning, occurred in just 7% of traces.

This is not science. This is the performance of science.

The AI generates a hypothesis. Runs some experiments. Collects results. Then proceeds as if the results were never there.

The researchers call it "evidence non-uptake." You could also call it what it is: a system that cannot learn from what it finds.

Here's what makes this worse.

The reasoning failure doesn't change based on what the task demands. Molecular simulation, circuit inference, chemical structure identification, none of it matters. The AI applies the exact same reasoning pattern across every domain regardless of what the problem actually requires.

A human scientist adapts. You approach a chemistry identification problem differently than you approach a simulation workflow. The AI doesn't. It runs the same undisciplined loop every time.

The researchers also destroyed the most popular proposed fix: better scaffolding.

Everyone building AI research agents has focused on engineering better prompting frameworks, better tool routing, better agent architectures. ReAct, structured tool-calling, chain-of-thought, all of it.

The data shows scaffolding accounts for 1.5% of the variance in performance.

The base model accounts for 41.4%.

No amount of scaffold engineering can fix a model that doesn't know how to think scientifically. You are decorating the outside of a broken foundation.

The paper's conclusion is the part that should concern every lab currently publishing AI scientist results.

When AI produces a correct answer through a broken reasoning process, that answer is not scientifically justified. It happened to be right. That is not the same thing as being right for the right reasons.

Science is self-correcting because of how it reasons, not just because of its outputs. AI scientists currently have the outputs without the process.

Until the reasoning itself becomes a training target, every result produced by an AI scientist cannot be trusted the way a result produced by actual scientific inquiry can be trusted.

25,000 experiments to confirm what the data has been quietly showing for months.

The AI is very good at looking like a scientist. It is not yet one.

[一帧人间 @joyfulframes]

急是一种负能量，不急不躁，过好每一天，一定记住，事缓则圆。
听听斯坦福心理学教授给我们的解读，一生受益。

[Chengpeng Wang @Chasen86341870]

Excited to share that I will join NUS (@NUSComputing) as an Assistant Professor this Aug! 🎉

I’m recruiting Ph.D./RAs/interns interested in Program Analysis, Code LLMs, and Agents. 🔥

Self-motivated students with strong backgrounds are especially welcome.

[He Ye @ye_he_ye]

Attending OOPSLA/ICFP 2025 in Singapore? 🌏✈️

JOIN us on October 15th, we warmly welcome you to join the Workshop on Language Models for Programming Languages (LMPL) co-hosted at SPLASH 2025 !
👉https://t.co/yTUWj1AglA

🔥This year we are excited to have accepted 22 papers! 🥳

[Qingkai Shi @QingkaiS]

@LParreaux It may be our opportunity to build our own csrankings:>

[Zhiyun Qian @pkqzy888]

CCS 2026 PC self-nomination form is out:
https://t.co/ueGn4bYSgI

[ICFP Conference @icfp_conference]

🚨 Call for Volunteers: SPLASH/ICFP 2025 🚨

Join the team that makes it all happen!
Meet the PL community, attend for free, and help run an amazing conference.

Apply now 👉 https://t.co/jDTgO6CNs6
More info 👉 https://t.co/aTQf2Mlbpp

[Jonathan Aldrich @JAldrichPL]

11am today at #ICSE2025: Rust provides memory safety to low-level code, but in practice Rust libraries link to unsafe C. @icmccorm, @joshsunshine, and I used dynamic analysis to find 46 cases where the C code broke Rust's memory rules, causing undefined behavior.

[Anders Møller @amoellercsaudk]

Coana has been acquired by @SocketSecurity! 🤩

[Zhuo Zhang @i2huer]

🚀 Excited to announce the LMPL Workshop at ICFP/SPLASH 2025! 🎉

PL research is all about rigor—but LLMs often feel like mysterious "black boxes".

Open question: Can we bridge these two worlds and get LLMs working seamlessly with PL? #PL #LLM

👉 https://t.co/yBAePKBPoO

[Glitchbyte @0xglitchbyte]

"Why C is the safest language?"

This is about to be a banger

[Qingkai Shi @QingkaiS]

@ShriramKMurthi @Chasen86341870 Hope that OOPSLA will do the right thing and have a good acceptance rate *finally* this year.

[Qingkai Shi @QingkaiS]

oopsla this year has many reviewers not from oopsla community or even never publish oopsla papers. Not sure how the reviewers are selected. Not specially for oopsla, for all venues, non-responsive reviewers should be kicked out.

[Chengpeng Wang @Chasen86341870]

What happened to OOPSLA? 89 out of 143 unrejected? Considering the rejection rate is less than 50%, is it lucky or unlucky that our paper was directly rejected?

[SPLASH Conference @splashcon]

SPLASH and ICFP will be co-located in 2025!

📆 Mark your calendars: the ICFP'25 submission deadline will be Feb 27, 2025

https://t.co/4w6SXqqCXm

[ASE 2024 @ASE_conf]

Get ready for an unforgettable evening! 🌟 Join us tonight from 6-10 PM at the Railroad Museum in Sacramento for a spectacular banquet filled with food, celebration, and the much-anticipated prize-giving ceremony! 🏆🎉

[ASE 2024 @ASE_conf]

Here is the location of the Railroad Museum 🚂🚃: https://t.co/P82pc9jwba

[Qingkai Shi @QingkaiS]

Excited to share that we won a sigsoft distinguished paper award.

[ASE 2024 @ASE_conf]

🏆The Distinguished Papers for ASE 2024 have been announced! Discover groundbreaking research and exceptional contributions that are shaping the future of software engineering.
Congrats to all the authors!

Explore the full list at the link below:
https://t.co/JDSlmK2Tji

[ASE 2024 @ASE_conf]

The Distinguished Reviewer Awards have been announced! A big round of applause to our reviewers for their dedication and expertise. Congratulations on this well-deserved recognition!

https://t.co/JDSlmK2Tji