Twitter's photo cropping algorithm favors people with slimmer, younger-looking faces, and lighter skin; a researcher was awarded $3,500 for the discovery
Twitter had already acknowledged that its image-cropping system favored white people and women in an internal investigation. Its public bounty for cropping-bias findings turned that assessment into an external test of the model.
The award identifies additional appearance-based disparities in a feature Twitter had introduced to select the most salient part of an image for previews. It adds evidence that the issue was broader than the initially documented racial and gender skew.
First-order effects
Twitter pays the researcher $3,500 and must treat lighter skin tone, apparent age, and facial thinness as documented failure modes in its photo-preview cropping system.
People whose images receive less prominent automated crops face unequal visual treatment in Twitter’s preview surface, rather than merely a technical misclassification.
Second-order effects
Twitter’s bounty program gives outside researchers a paid route to test ranking and presentation models, increasing pressure on the company to validate its own bias reviews against independent findings.
The accumulating findings make continued automated preview cropping harder to defend as a neutral convenience feature; later coverage records Twitter ending web photo-preview cropping after ending it on mobile.
Third-order effects
If platforms respond to repeated visual-model bias reports by removing automated presentation decisions rather than refining them, product teams will increasingly favor user-controlled defaults over opaque “salience” systems.
Bug bounties can become part of algorithmic accountability, but their value depends on whether platforms provide researchers enough access to detect harms beyond the categories companies initially test.
The trend: Consumer platforms are moving from treating algorithmic bias as an internal model-quality issue toward external testing and, where failures persist, less automated user-facing presentation.
@ruchowdh That all being said, I have mixed feelings. First, algorithmic harms are not only “bugs”. Crucially, a lot of harmful tech is harmful not because of accidents, unintended mistakes, but rather by design. This comes from maximization of engagement and, in general, profit.…
The algo that was cropping out Black folks had also “learned to ignore people with white or gray hair, who wore headscarves for religious or other reasons, or who used wheelchairs...” https://www.nbcnews.com/...
Twitter users flagged problems with the photo-cropping algorithm last year, and so the company organized a bug bounty to analyze them. It's released the results, with the winner feeding the algo AI-generated faces (below) to see which it preferred https://www.theverge.com/... htt…
My latest for @WIRED: An “AI bias bounty” finds that Twitter's notorious image algorithm discriminates by age, weight, and culture, as well as gender and race. The contest shows how outside experts can hold AI accountable, by will companies let them? https://www.wired.com/...
“We create these filters because we think that's what beautiful is, and that ends up training our models and driving these unrealistic notions of what it means to be attractive,” says @ruchowdh, director of Twitter's META team. https://www.theverge.com/...
@ruchowdh externalizing the costs to others. As an example, amplifying gentrification, driving down wages, spreading clickbait and misinformation are not necessarily due to “biased” algorithms.
And now I am back to give some thoughts no one asked for! I do believe this program is a great initiative. As mentioned in the first Twitter Spaces discussion, the external bias auditing work is often done, but not rewarded, which is a similar situation to early security research
it is still wild / not at all wild actually that twitter's initial response to folks who spent time and effort and expertise pointing out the deficiencies in their cropping algorithm amounted to, “you are wrong and mean” https://www.nbcnews.com/...
Wow, this was an unexpected conclusion of the week! My submission was awarded the 1st place in the Twitter's Algorithmic Bias program. Many thanks to @ruchowdh, @TwitterEng META team, and the jury...
These are the kinds of studies that I think about when Apple tells me that they are going to recognize “sexually explicit” images in iMessage using machine learning. https://twitter.com/...