← Back to the wire

AI coding agents still need human cleanup on nearly three out of four tasks in a real-world test

AchievementBenchmarkNov 8, 2026

Alibaba and university researchers introduced Co-Design Bench, a coding-agent benchmark built from real user change requests, and found that GPT-6 Astra, the best of ten models tested, fully completes only 26.4 percent of tasks, despite meeting about 70 percent of requirements on average. The researchers cite saturation of existing benchmarks, where models exceed 99 percent, and recommend reporting models and their harnesses together, since the working environment strongly influenced results.

Receipt № 27911 source · awaiting confirmation ◐

Evidence

1source· awaiting independent confirmation

No score is assigned. Sources and their independence are shown in the citation chain below.

Citation chain · 1 source

01med
the-decoder.comBloghttps://the-decoder.com/ai-coding-agents-still-need-human-cleanup-on-nearly-three-out-of-four-tasks-in-a-real-world-test/This source explains how Co-Design Bench was constructed from difficult real user requests and highlights that the working environment, not just the model, strongly determines coding-agent performance.
PRIMARY
AlibabaCompanyGPT-6 AstraModel
Canonical: https://the-decoder.com/ai-coding-agents-still-need-human-cleanup-on-nearly-three-out-of-four-tasks-in-a-real-world-test/