How Do Top Android Developers QA Test Their Apps?
A couple weeks ago I ran this post showing how one Hong Kong developer, Animoca, tests its Android games. The company, which has had more than 70 million downloads, tests every one of their apps on about 400 different devices.
Context & Ripple Effects
Android's developer experience had already been framed around testing difficulty in coverage of developers calling Android a mess and an Android test-market proposal. Animoca gives that problem a concrete operating scale: it tests every app across about 400 devices.
With Android reported to hold 49% of the mobile OS market, compatibility work is not a niche engineering task; it is part of reaching a large and varied handset base.
First-order effects
- Animoca absorbs the cost of testing each Android game on roughly 400 devices, aiming to catch device-specific failures before they affect its audience of more than 70 million downloads.
Second-order effects
- Other Android developers face a sharper trade-off between funding broad device coverage and accepting a higher risk of inconsistent app behavior across handsets.
Third-order effects
- If broad device matrices remain necessary, Android app release economics will favor publishers able to build or buy systematic QA capacity rather than relying on a small set of reference devices.
The trend: Android's expanding handset base is making compatibility testing a core competitive capability for mobile app publishers.