Looking at CPU/GPU Benchmark Optimizations in Galaxy S 4
Somehow both Anand and I ended up with international versions of Samsung's Galaxy S 4, equipped with the first generation Exynos 5 Octa (5410) SoC. Anand bought an international model GT-I9500 while I held out for the much cooler …
Context & Ripple Effects
AnandTech has been tracking Samsung's flagship silicon since its Galaxy S III performance preview in May 2012, so a deep dive into the international GT-I9500's Exynos 5 Octa (5410) follows an established beat. What makes this one different is the finding that the S 4's CPU and GPU behave differently under benchmark applications than in ordinary use — with The Verge reporting Samsung claiming innocence in the face of benchmark-rigging allegations, and AllThingsD, the BBC, Telegraph, ZDNet and others picking the story up the same day.
Timing sharpens the stakes. Samsung just posted a record $8.5B quarterly operating profit, up 47% year over year, but its mobile division profit slipped 3.5% sequentially and IDC estimates show smartphone share losses in Q2 2013 — so the Galaxy S 4's competitive standing against rival flagships is under more scrutiny than any Samsung phone before it.
First-order effects
- Reviewers and buyers relying on published S 4 scores are comparing numbers generated at boosted clock speeds the phone does not sustain in normal apps, while Samsung faces public allegations — which it denies — that it white-lists benchmark software.
- The same Galaxy S 4 sold with Qualcomm's Snapdragon instead of the Exynos 5410 is not implicated, splitting Samsung's own lineup into variants with divergent credibility.
Second-order effects
- Rival Android OEMs whose flagships compete on spec-sheet and benchmark superiority are pushed to audit or publicly defend their own firmware behavior, since AnandTech's methodology can be run against any device.
- Benchmark publishers face pressure to detect vendor-specific optimizations, threatening the comparability of the cross-phone score charts that anchor launch-cycle reviews.
Third-order effects
- If every vendor optimizes for whatever tool reviewers standardize on, the industry drifts toward measuring sustained real-world performance and thermals rather than short burst scores — shifting review authority to outlets capable of instrumenting devices themselves.
The trend: Smartphone SoC evaluation is moving from peak synthetic benchmark scores toward sustained, real-world performance measurement as vendors learn to game the benchmarks reviewers rely on.