Sony researchers: standardized skin tone scales used by Google, Meta, and others to test their image algorithms don't capture red and yellow hues in human skin
Paresh Dave / Wired :
Context & Ripple Effects
Google had already replaced the Fitzpatrick Scale with the 10-grade Monk scale for AI diversity work, while its camera teams were adjusting white balance and exposure for darker skin tones. Sony’s finding challenges whether the evaluation layer beneath such efforts is broad enough, not merely how individual models are tuned.
The issue reaches beyond photography: Google had also updated image search to increase racial diversity in results, making the choice of skin-tone representation relevant across image-generation, ranking, and vision systems.
First-order effects
- Teams at Google, Meta, and other developers using these standardized scales have evidence that their current tests can miss red and yellow skin hues, leaving a gap in how they assess image-algorithm performance.
- Camera and computer-vision groups may need to supplement scale-based evaluations with tests that capture the omitted hue variation, alongside prior work on white-balance and exposure adjustments for darker skin tones.
Second-order effects
- Claims of more faithful or fair image rendering become harder to compare when vendors rely on a common measurement tool with identified blind spots; product tests such as Pixel Real Tone evaluations may require broader test sets.
- Benchmark and dataset design becomes a competitive concern for image-AI suppliers, because improvements to models cannot demonstrate broad coverage if the underlying evaluation categories are incomplete.
Third-order effects
- If this critique is borne out in wider testing, the industry may move from a single standardized skin-tone scale toward multi-dimensional benchmarks that separate tone from hue and other imaging conditions.
- The broader shift is from treating fairness as a model-tuning problem to treating it as a measurement and validation problem, where the benchmark itself can constrain what developers detect and improve.
The trend: Computer-vision developers are increasingly being judged not only on model outputs but on whether their benchmarks represent the variation those systems must handle.