Anthropic details how it had to redesign its take-home test for hiring performance engineers as Claude kept defeating it, and releases the original test
What we learned from three iterations of a performance engineering take-home that Claude keeps beating.
Context & Ripple Effects
Anthropic had already reported that Opus 4.5 outscored human candidates on the earlier assessment, making the hiring exercise an unusually direct benchmark of model capability against a role-specific human task. Releasing the original version makes that progression more inspectable.
The redesign also sits alongside Anthropic's report that employees use Claude heavily for debugging and code understanding, with reported productivity gains in its internal use survey. The assessment is therefore both a hiring tool and evidence of where coding assistance is encroaching on performance-engineering work.
First-order effects
- Anthropic must use a harder or differently structured evaluation to distinguish performance-engineering applicants once Claude can reliably complete the prior task under its stated constraints.
- Candidates and outside evaluators can inspect the released original test, enabling more direct comparison of Claude's result with the task the company previously used in hiring.
Second-order effects
- Performance-engineering hiring teams may place less weight on static take-home exercises and more on evaluations that test judgment, verification, and work performed with AI tools.
- The disclosed test gives model developers and benchmark users a concrete, role-relevant artifact for testing coding agents, increasing pressure to separate genuine engineering capability from benchmark-specific performance.
Third-order effects
- If role-specific assessments keep becoming solvable by frontier models, technical recruiting will shift from measuring unaided task completion toward measuring how people direct, audit, and extend AI-generated work.
- This is a data point in a feedback loop: AI capability changes the work used to screen AI-adjacent talent, while that talent is hired to improve the systems that changed the screen.
The trend: Frontier coding models are turning formerly differentiating technical work samples into moving targets, forcing hiring and evaluation systems to co-evolve with AI-assisted engineering.