Steven Xia

PhD Student @illinoisCDS studying SE \\ Undergrad @eceuoft 2T1

[Courtney Miller @courtneyelta]

𝐀𝐠𝐞𝐧𝐭𝐢𝐜 𝐜𝐨𝐝𝐢𝐧𝐠 𝐭𝐨𝐨𝐥𝐬 𝐡𝐞𝐥𝐩 𝐬𝐡𝐢𝐩 𝐜𝐨𝐝𝐞 𝐟𝐚𝐬𝐭𝐞𝐫, 𝐛𝐮𝐭 𝐚𝐭 𝐰𝐡𝐚𝐭 𝐜𝐨𝐬𝐭?

𝐖𝐡𝐚𝐭 𝐰𝐞 𝐝𝐢𝐝:
🎯Across 182 open source projects, we tracked >30K commits from agentic coding tools to measure the true cost of adoption.

🧵 (1/6)

[Yuvraj Virk @yuvrajvirk0]

Introducing 𝗦𝘄𝗮𝗿𝗺𝗥𝗲𝘀𝗲𝗮𝗿𝗰𝗵: agent skills that orchestrate a population of independent research agents for autonomous discovery.

SwarmResearch achieves over 2× the throughput of vanilla autoresearch when designing speculative decoding implementations. While autoresearch gets stuck refining locally optimal approaches, SwarmResearch continues to explore diverse approaches after 11 hours.

Repo: https://t.co/ft7U6JEr2W

[Hwiwon Lee @hwiwonl]

We’re happy to release 𝐒𝐄𝐂-𝐛𝐞𝐧𝐜𝐡 𝐏𝐫𝐨: a benchmark for measuring the bug-hunting capabilities of AI agents in critical software systems such as Chromium V8, Firefox SpiderMonkey, and more.

Explore the details here: https://t.co/gIMnS4Oohx

[Steven Xia @steven_xia_]

@slimshetty_ @LingmingZhang Also feel free to check out the paper for more detailed analysis. We have a whole bunch of examples and results looking at the different tools agents generate!

[Steven Xia @steven_xia_]

@slimshetty_ @LingmingZhang Cool question! For Gemini-3-Pro, it mainly creates tools that are well-suited for analyzing the repo (e.g., read/search/write files). There are also task-specific ones like creating edit tools that apply targted patches that are quite difficult to do via bash commands only.

[Lingming Zhang @LingmingZhang]

🤯🤯🤯 Gemini 3 Pro + Live-SWE-agent hits 77.4% on SWE-bench Verified, beating ALL existing models, including Claude 4.5!!

🤖 Live-SWE-agent is the first live software agent that autonomously self-evolves on the fly — and it even outperforms the manually engineered scaffold used by the Gemini 3 Pro team (76.2%)

[Lingming Zhang @LingmingZhang]

🚀 Introducing Live-SWE-agent:

🤖 An autonomous coding agent that self-evolves on the fly while solving real-world issues.
✨ Simplistic design: no offline training, no heavy workflows.

👇 Surprising gains:

📌 75.4% on SWE-bench Verified
📌 45.8% on SWE-Bench Pro (new SOTA!)

[Toronto Blue Jays @BlueJays]

WE'RE GOING TO THE WORLD SERIES!!!!! #WANTITALL

[OpenAI @OpenAI]

We're releasing a new iteration of SWE-bench, in collaboration with the original authors, to more reliably evaluate AI models on their ability to solve real-world software issues. https://t.co/qJuLpCdSWJ

[Ziqi Zhang @ZiqiCharles]

I'm on the way to USENIX Security'24. We will present one paper about privacy-preserving app authentication. I'm also happy to discuss TEE-based AI security and other AI-related security topics. If you're interested, please get in touch with me! #usenix #USESEC2024

[Lingming Zhang @LingmingZhang]

Introducing OpenAutoCoder-Agentless😺:

A simple agentless solution solves 27.3% GitHub issues on SWE-bench Lite with ~$0.34 each, outperforming all open-source AI SW agents!

It's fully open-source, try it out:

🧑‍💻https://t.co/AKyiZhmi7B
📝https://t.co/Oc4QCaQult

[AK @_akhaliq]

Magicoder: Source Code Is All You Need

paper page: https://t.co/ZeZH3PIXln

introduce Magicoder, a series of fully open-source (code, weights, and data) Large Language Models (LLMs) for code that significantly closes the gap with top code models while having no more than 7B parameters. Magicoder models are trained on 75K synthetic instruction data using OSS-Instruct, a novel approach to enlightening LLMs with open-source code snippets to generate high-quality instruction data for code. Our main motivation is to mitigate the inherent bias of the synthetic data generated by LLMs by empowering them with a wealth of open-source references for the production of more diverse, realistic, and controllable data. The orthogonality of OSS-Instruct and other data generation methods like Evol-Instruct further enables us to build an enhanced MagicoderS. Both Magicoder and MagicoderS substantially outperform state-of-the-art code models with similar or even larger sizes on a wide range of coding benchmarks, including Python text-to-code generation, multilingual coding, and data-science program completion. Notably, MagicoderS-CL-7B based on CodeLlama even surpasses the prominent ChatGPT on HumanEval+ (66.5 vs. 65.9 in pass@1). Overall, OSS-Instruct opens a new direction for low-bias and high-quality instruction tuning using abundant open-source references.

[Jiawei Liu @JiaweiLiu_]

In the past 6-mon release of HumanEval+ we have been improving its toolchain usability and dataset quality from v0.1.0 to v0.1.7 releases.

🔥 Now we release MBPP+, a new benchmark in EvalPlus v0.2.0: https://t.co/2vUM9b38s7
🧵

[Steven Xia @steven_xia_]

@nenrightjerger make it 3 years in a row😅

[Jiawei Liu @JiaweiLiu_]

Introducing the EvalPlus leaderboard! https://t.co/H0aOtkzbnc

🔥28 models have been evaluated on coding HumanEval & HumanEval+
🔥7B CodeLlama outperforms ~16B models e.g. StarCoder&CodeGen
🔥Phind-CodeLlama-34B-v2 and WizardCoder-Python-34B-V1 as open models both beat ChatGPT
🧵

[Steven Xia @steven_xia_]

@fried_rice @DomSteinhoefel Definitely! We are working on that and hoping to add additional targets

[Talia Ringer 🕊🪬 @TaliaRinger]

OK now we have @steven_xia_ on his work with @YuxiangWei9 and @LingmingZhang, all of @plfmse. Talking about program repair using large pre-trained language models

[Steven Xia @steven_xia_]

@saikatdutta2012 @cs_cornell @CornellCIS So happy for you! Updating the ICSE slide to include this rn!

[Jiawei Liu @JiaweiLiu_]

We welcome everyone to try out 📚𝐇𝐮𝐦𝐚𝐧𝐄𝐯𝐚𝐥+! A dataset to reflect the "real" correctness of LLM-generated code.

Using📚𝐇𝐮𝐦𝐚𝐧𝐄𝐯𝐚𝐥+ is the same as HumanEval. You can easily pip install it and evaluate in our prepared sandbox (optional). https://t.co/96YcDvNJpm

[Steven Xia @steven_xia_]

🚨 Evaluating LLM-generated code on datasets with just "3 test-cases" is NOT enough! 🚨
We built ✨HumanEval+✨: improving HumanEval with up to thousands of new tests to fully evaluate functional correctness of LLM generated code!
@JiaweiLiu_ @YuyaoStarling @LingmingZhang