Turing Community

Accelerating frontier AI models + systems.
Powered by ALAN and a 4M+ global talent cloud.
AGI at scale.

[Turing @turingcom]

BREAKING: A new interview with @MollySOShea
and our CEO @Jonsid.

They discuss how AI training is moving beyond benchmarks into simulated environments built for real work, as agents move from completing tasks to operating autonomously for days, & eventually weeks, months & years and break down what this shift means for frontier labs, enterprises & the AI stack:

› Frontier AI vs. sovereign AI
› Distillation & the shrinking model gap
› Why enterprises are moving to open models
› What AI safety & alignment teams actually do
› Reward hacking & why “if the model can figure out a way to cheat & get the reward, it will”
› Super Intelligence, recursive self-improvement & slow takeoff

Watch the entire interview below.

[Turing @turingcom]

We’re hiring at @Turingcom!

[Turing @turingcom]

We’re heading to COLM 2026 in San Francisco!

Turing will be at Booth 305, bringing together ideas and conversations around the rapidly evolving world of language models.

@COLM_conf is a unique academic forum dedicated to understanding, improving, and critically examining language modeling from foundational research to the broader questions shaping the future of AI.

If you’re attending COLM 2026, come meet the @Turingcom team. We’d love to connect with researchers, practitioners, and fellow AI enthusiasts exploring what’s next in language modeling.

Where: San Francisco, Booth 305

See you there!

[Turing @turingcom]

New research from Turing, in collaboration with Oracle: PLSQLBench, a benchmark for evaluating whether LLMs can write executable PL/SQL programs.

Most benchmarks test general code generation or text-to-SQL. But real database work isn't a one-shot query, it's stored procedures, cursors, exception handling, and iterating against a live schema.

That entire layer has gone basically untested. Until now.

Introducing PLSQLBench:  the first benchmark built for procedural database programming.

-2,865 instances
-2,594 single-turn tasks + 271 multi-turn conversations (978 turns)

Built on enterprise-style Spider 2 databases, schema-grounded Spider tasks, and MBPP-derived procedural problems.

It measures what developers actually do: write procedures, handle exceptions, ground work in real schemas, and refine iteratively.

If we want LLMs that ship real database code, we have to evaluate them on real database work.

Testing eight LLMs surfaced recurring weaknesses in schema grounding, PL/SQL dialect fidelity, procedural control flow, exception handling, and consistency across turns. Tool-augmented agents closed some of the gap on schema-grounded tasks, but meaningful gaps remain.

The paper has been accepted to EMNLP industry track 2026 in Budapest.

Paper + Code below.

Congratulations to all: Marianne Menglin Liu, Leo (Leonid) Boytsov,
Daniel Petersen, Pramuditha Perera, Rongguang Wang, Sai Ashish Somayajula, Syed Hamza Rafique, Rohit Saini, Shubham Pathak, Sujeeth Bharadwaj, Tao Sheng, Graham Horwood, Fahad Shah, Ankan Bansal, Sujith Ravi, Dan Roth

[Turing @turingcom]

Paper: https://t.co/vSeHOXOUzy

[Turing @turingcom]

Our Turing Frontier Research Lab recently released CyberStrike, a real-world cybersecurity benchmark.

A long-horizon benchmark evaluating frontier agents across the cybersecurity lifecycle, from vulnerability exploitation and secure remediation to threat detection and incident reconstruction.

Results across 3,600 trials: GPT-5.6 Sol (xhigh) led at a 31.5% mean per-task pass rate, but many high-scoring near misses failed binary grading. Seventy-two of 200 tasks were unsolved.

CyberStrike has 200 expert-authored tasks: 120 defensive, 68 offensive, and 12 DFIR.

Of 3,600 recorded trials, 3,106 were graded; the remaining 494 were reported safety refusals.

On the atlas-records-export-offensive task, GPT-5.6 Sol reported: "The submitted proof replayed successfully: all 14 HTTP steps returned 200." The verifier found no protected artifact. No configuration solved the task; four declared success, and two refused.

At the Turing Frontier Research Lab, we continue building benchmarks, training sets, and training recipes that push frontier models forward in the domains where real capability matters.

More on CyberStrike: https://t.co/93qEBLT0KI

[Turing @turingcom]

Reach out to learn more about our research, benchmarks, and datasets: https://t.co/d6Uk6c5EIX

[Turing @turingcom]

SciCode++ is here, with ~6,000 tasks that test how well frontier models solve scientific problems through code.

SciCode introduced executable scientific coding problems. SciCode-Verified showed how often flawed specifications and tests can distort the results. SciCode++ turns those lessons into a production process: domain experts write the tasks, independent reviewers check them, executable tests validate every subproblem, and model runs calibrate difficulty. That helps separate model capability gaps from eval issues.

Early results post-training Qwen 3.5 9B on 4,000 of the 6,000 tasks: +10.3% on SciCode and +9.1% on SciCode-Verified relative to baseline.

https://t.co/6cJ06m486F

[Turing @turingcom]

If you’re working on scientific reasoning or coding models, get in touch: https://t.co/JX7G3eeAKq

[Turing @turingcom]

Proud to see our CTO, @ecekamar named one of the Top Women in AI 100 for 2026.

Ece’s leadership continues to shape what responsible, human-centered AI can become. We’re proud to see her recognized among the women building the future of AI.

Congrats, Ece!

[Turing @turingcom]

Last week, the Turing Frontier Research Lab launched KernelQuest, a benchmark that tests whether AI agents can perform the full job of GPU kernel optimization.

KernelQuest contains 100 engineer-authored Triton tasks across 21 kernel families. Each task gives an agent a live environment where it can profile a PyTorch workload, write and revise kernels, and measure performance. The tasks range from single operators and fused workloads to complete models and large Transformer/MoE workloads.

Learn more in the article below:

[Turing @turingcom]

https://t.co/cBVzq8YVuh

[Jonathan Siddharth @Jonsid]

Data as a lever is becoming increasingly important for advancing the models.

[Ece Kamar @ecekamar]

Introducing CEO Bench by @turingcom : 500+ expert-authored tasks inside a complete simulated company with human experts in the loop and agentic QC on every task.

https://t.co/Ym3KVpPVmL

[Jonathan Siddharth @Jonsid]

Can an AI agent run part of a company? We built an entire company to test it.

Cirrus Sleep, Inc. is a fully simulated firm: one complete fiscal year, 1,100+ files across 9 systems, a year of internal Slack. Conflicting numbers, governance constraints, ambiguity by design. No single string-matched answer, agents are graded against expert-written rubrics, the way you’d judge a real analyst.

Result: the best frontier model hit 41% of rubric criteria on department-level tasks. On half the tasks, no model out of six cracked 50%. The failures cluster in the same places: reconciling conflicting sources, knowing what’s authoritative, deciding what to disclose.

Company-wide executive tasks are next. We expect those to be harder.

What’s the eval design you’d want to see run in an environment like this? Task details are on the CEO Bench site.

[Jonathan Siddharth @Jonsid]

What does it take for an AI agent to fail an audit?

We planted a trap to find out.

One CEO Bench task hides conflicting bank evidence for the same month in the data room.

Checking that the bank ties is the first thing a first-year auditor does. Flag it, reconcile it, or log it as unresolved.

All six frontier models did none of the three. Silently treated both versions as consistent and built polished deliverables on top. 0 for 6. Nothing hallucinated, nothing wrong in isolation. An omission, which is exactly the failure answer-matching evals can't see.

Here's what makes this interesting. Ask any of these models to reconcile two conflicting bank statements and they'd likely do it well. The capability exists. What's missing is the propensity: recognizing, unprompted, that this is the moment a professional would stop and check. That's tacit process knowledge. It lives in practitioners, not in any manual, which is why our rubrics are written by them.

So the overhang is real: enormous capability the economy hasn't absorbed. Closing the gap means teaching models what experts do, not just what experts know: the moves, the order, the steps they'd never skip. Evals like this tell you whether it worked.

If you're working on helping models master real enterprise workflows, DM me about CEO Bench and where we're taking this.

[Turing @turingcom]

Turing welcomes Patrick McKinney as Chief Information Security Officer.

As AI becomes more capable and more deeply embedded in how companies operate, security, privacy, and responsible data stewardship have never been more important. As we scale our work with frontier AI labs and Fortune 500 enterprises, Patrick will lead our global security strategy with a deeply technical, hands-on approach.

Patrick will help set the security bar for our Frontier AI work with leading labs on training data and evals and drive new security research and benchmarks like CyberStrike. He’ll also work directly with enterprise customers and their security teams as we deploy agents in highly regulated environments, helping answer the hard security questions that come with putting AI into production. Across this work, he’ll help protect the data entrusted to us and strengthen our security capabilities across cloud identity, detection engineering, incident response, and offensive and defensive security.

Patrick brings more than 15 years of experience building and scaling security organizations, including leadership roles at Invisible Technologies and security and compliance roles at Dropbox and Coinbase.

Welcome to Turing, Patrick!

[Turing @turingcom]

The Turing Frontier Research Lab is launching KernelQuest, a benchmark that tests whether AI agents can perform the full job of GPU kernel optimization. https://t.co/r55k2fFecE

[Jeffrey Weichsel @jeffreyweichsel]

I am super proud of what my team has been working on. SVC has spawned its first benchmark, CEO Bench. Much more is coming.

https://t.co/8J52LkudIp

[Jeffrey Weichsel @jeffreyweichsel]

Turing published my write-up of CEO Bench, a project our team has been building over the past few months.

The idea is to evaluate AI agents in a more realistic enterprise environment: a fictional company with a coherent operating history, interconnected data, and the kinds of ambiguity and conflicting information that arise in real work.

Our first simulated company, Cirrus Sleep, includes more than 1,100 files across finance, accounting, legal, HR, operations, and other functions, along with 500+ expert-authored tasks.

We also share results from an initial evaluation of six frontier models. The findings suggest that, despite significant progress, there is still a meaningful gap between producing a convincing answer and reliably completing complex enterprise work.

I’m grateful to the team that helped bring this together. We’re continuing to expand the environments and evaluations, and I look forward to sharing more as the work progresses.

Full write-up below.