DataMining

Hard agent & code eval data. PKU Software Institute PhD team. Every task ships with a reference solution that passes. Samples via DM.

[DataMining @clgssss]

AI data stopped being sold as answers this year.

We build code and agent tasks that frontier models still fail, and publish the traces.

Six notes on what changed on the buyer's side — and what we measured.

[DataMining @clgssss]

@follistatindev 49.9 at $0.40 is the hook. the +7.6 against its own price line is the bit i'd actually keep.

[DataMining @clgssss]

@ziqiao_ma for the public-dataset third-party benches you also reimplement internally — do those BenchBench rows come from your internal runs, or from the vendor's published number?

[DataMining @clgssss]

@refuselessai 16.3/21 on Nova, ahead of Sol's 15.7. the headline ranking lives in Lighthouse — DeepSeek 21/31 there, vs Sol 27 and Wonder 28.

[DataMining @clgssss]

Code + ASSEBench fully open. If you need hard agent / eval environments (not answer dumps), DM.

[DataMining @clgssss]

Give your LLM-as-judge an experience memory. AgentAuditor: no extra training — RAG over curated memory to catch Agent safety + security risks. #AgentAuditor #LLMJudge #AgentSafety #AgentSecurity #RAG

ASSEBench: 2293 labeled agent interactions, strict + lenient dual labels.

[DataMining @clgssss]

How it works: build feature + reasoning memory from past judgments → retrieve context-aware CoT shots → LLM judge. Lifts multiple LLMs toward human annotator agreement on agent risk calls.

[DataMining @clgssss]

TXT/Markdown · JSON/XML/YAML · DB/CSV/Excel — same agent loop. Model + code + training data open. DM for hard agent / eval data that trains stacks like this.

[DataMining @clgssss]

First agentic LLM for data science: DeepAnalyze-8B (Renmin U + Tsinghua). #1 on HF Daily Paper at drop. #DeepAnalyze #AgenticAI #DataScience #LLM #OpenSource #AIResearch

Prep → analysis → modeling → viz → insight. Structured/semi/unstructured data in, analyst-grade report out.

[DataMining @clgssss]

Sparse reward + scarce paths in DS agents. Two levers: Curriculum-based Agentic Training (simple→complex in real envs) and Data-grounded Trajectory Synthesis — 500K reason + env-interact traces.

[DataMining @clgssss]

@0xrompasso in-house 66.4 at xhigh, and still no row on the public Terminal-Bench 4.0 board. a copied rival column doesn't close that gap.

[DataMining @clgssss]

@kobe0938 64 model cards later — still the citation that shows up at launch.

[DataMining @clgssss]

@omarsar0 been sending agents into raw paper repos — they usually choke on the env, not the method. the best-worst gap falling from 21.6 to 3.9 after refined artifacts feels familiar.

[DataMining @clgssss]

@dbreunig popular benches don't just get overfit — they train a house style into the agent. high-taste evals are how you pick the behavior you actually want.

[DataMining @clgssss]

@PatronusAI 25–50% less filler across 12 models × 20 tasks, and the bill didn't follow. "Need inspect repo. Grep." is a voice change, not a cheaper run.

[DataMining @clgssss]

@loresjpg quick check — after the lowercase-filename tweak, is the grader locked to the new requirement, or can a pass still come from the original DeepSWE tests?

[DataMining @clgssss]

@arthurjdev No status over fake Idle — silence isn't done.

[DataMining @clgssss]

@IngotSubnet first SWE-ML / TB lift still sitting on the noise band. promising isn't proven yet.

[DataMining @clgssss]

@mercor @TencentHunyuan 77.6 Pass@1 as open-weights #1 on APEX-Agents — that's a different board than Terminal-Bench alone.

[DataMining @clgssss]

@169Pi_ai 61.2 on SWE-Bench Verified from a 9B on-device card is a big claim — still want the decontam and sampling settings before treating it as settled.

[DataMining @clgssss]

@0interestrates near Qwen 3.8 max on DeepFinance at ~11% of the cost — you putting $/workflow next to the scoreboard?