ExpertCrowd

ExpertCrowd is a flexible platform for labeling, evaluating, and improving AI, powered by top professionals with expertise in any domain.

[ExpertCrowd @ExpertCrowd_]

ExpertCrowd, @toptal's platform for training and improving AI models and systems, is partnering with @hud_evals, a Y Combinator-backed startup that builds reinforcement learning environments, to advance AI training and evaluation. Our first joint release is ExpertCrowd-SWE, a software engineering benchmark for frontier-grade LLMs and coding agents.

Engineers from Toptal’s talent network authored tasks and grading rubrics, while HUD’s infrastructure hosted task-run environments and verified the benchmarks. The results highlight performance gaps that other benchmarks have missed: On ExpertCrowd-SWE, GPT-6 Astra Max’s performance drops by 8.3% points when compared to DeepSWE, and Fable 5.1 Max’s performance drops by 17.1% points when compared to APEX-SWE.

This is the first release of an ongoing collaboration, with more coming soon.

Learn more at https://t.co/zOJoWNiGfA

#AI #SoftwareEngineering #LLMEvaluation

[ExpertCrowd @ExpertCrowd_]

Today, we’re launching ExpertCrowd: the next generation of datasets, RL environments, and benchmarks. This platform gives frontier labs and innovative enterprises the expertise, tooling, and talent they need to generate high-quality data and tasks for training and improving their AI models and agents.

As part of this launch, we’re publishing the ExpertCrowd-SWE benchmark, where GPT-6 Astra Max’s performance drops by 8.3% points when compared to DeepSWE, and Fable 5.1 Max’s performance drops by 17.1% points when compared to APEX-SWE.

Learn more at https://t.co/ts1UA222FA

#ArtificialIntelligence #AIInnovation #DataQuality