Predictim, a service using AI to generate character scores for babysitters based on years of online activity, provides questionable recommendations to parents
When Jessie Battaglia started looking for a new babysitter for her 1-year-old son, she wanted more information … Tweets: @zeynep , @justinhendrix , @katecrawford , @ai , @carnage4life , @etbrooking , @robinberjon , @chrismessina , @mathbabedotorg , @erictopol , @alexroy144 , @hunterwalk , @mat , @kierisi , @lydnicholas , @lmsacasas , @dhh , @dhh , @dhh , and @profcarroll Tweets: Zeynep Tufekci / @zeynep : Not a drill. One of the largest babysitter sites will pilot folding “AI” ratings based on social media scans “into the site's current array of sitter screenings.” China is explicitly building AI social control tools. We're drifting via business model.http://www.washingtonpost.co m/ ... http://twitter.com/... Justin Hendrix / @justinhendrix : A company called Predictim is offering parents what tech firms are selling to employers: artificial-intelligence systems that analyze a person's speech, facial expressions & social media history to determine if a person would be a suitable babysitter.http://www.washingtonpost.co m/ ... Kate Crawford / @katecrawford : Well yes, this AI sitter screening is error-prone, based on broken assumptions, and privacy invading. What's worse - it's a horrifying symptom of the growing power asymmetry between employers and job seekers. And low wage workers don't get to opt out. http://www.washingtonpost.com/ ... Anand Iyer / @ai : Wow, don't even know where to start with this. I get that parents think that this is the best thing for their children, by getting them the “perfect” babysitter. Subjecting potential sitters to this kind of invasive behavior will only deter from wanting to join the profession. http://twitter.com/... Dare Obasanjo / @carnage4life : Using AI to scan the Facebook pages of potential baby sitters to see if they are good people is winning the race for most bogus AI claim yet.As an industry we need a watchdog that requests training data & published papers from such BS claiming companieshttps://t.co/eXLIrSkKXK Emerson T. Brooking / @etbrooking : via late capitalism, the dawn of U.S. “social credit” is already well underway. at least the Chinese CCP is up front about it. http://twitter.com/... Robin Berjon / @robinberjon : This is part of what pisses me off with the current “AI ethics” fad. We don't need AI ethics for this, just plain old ethics. Or, you know, just some fucking human decency. http://twitter.com/... Chris Messina / @chrismessina : I mean, this is inevitable, even if the recommendations are “questionable”. What parent WOULDN'T consult such a service if it were widely available? http://www.washingtonpost.com/ ... Cathy O'Neil / @mathbabedotorg : I'd love to see @jovialjoy do a false negatives analysis broken down by race and gender on this shit algorithm. http://www.washingtonpost.com/ ... Eric Topol / @erictopol : While many of us are concerned of unleashing #AI tools for #healthcare, its use to screen your babysitter... @Predictim, without any published data, marketing a $25 scan of social media posts to score a candidates's offline life :-( http://www.washingtonpost.com/ ... by @drewharwell A1 WP http://twitter.com/... Alex Roy / @alexroy144 : 2016 Facebook: We can build good AI 2017 Facebook: We can fix bad AI 2018: Facebook can't fix their AI 2019 Predictim: We can build good AI http://www.washingtonpost.com/ ... @hunterwalk : Should make the parents consent to being scanned as well, and only let them hire sitters at same/lower score than their own http://twitter.com/... Mat Honan / @mat : “But the danger of hiring a problematic or violent babysitter, he added, makes the AI a necessary tool for any parent hoping to keep his or her child safe."That is some serious fucking bullshit http://www.washingtonpost.com/ ... Jesse Mostipak / @kierisi : “...as Predictim's technology influences parents' thinking, it remains entirely [...] vulnerable to quiet biases over how an appropriate babysitter should share, look and speak.” I guarantee you there is nothing quiet about the race/class/gender biases in these systems. http://twitter.com/... Lydia Nicholas / @lydnicholas : Cripes this is horrific. Look; algorithms to support decisions in care and to help find good matches could be helpful but not a black box that forces you to smile for a machine interviewer, mines your social media and produces an unexplained numerical score of character. http://twitter.com/... LM Sacasas / @lmsacasas : Thesis: all such tools are symptoms and accelerators of the breakdown of trust and judgment that emerges organically within well-functioning, human-scale communities. http://twitter.com/... @dhh : You don't have to be a part of building this dystopian hellscape. Maybe you can find a way to excuse working at Facebook, but Predicitim is so beyond the pale that not even military-grade cognitive dissonance tolerance can excuse you. This is Terrible Ethics in Software 101. @dhh : Who can we thank for funding this ethically-deprived dumpster fire? Your friends at @UCBerkeley and their @SkyDeck_Cal fund. Putting science to work in dystopian science oh-my-god-it's-not fiction @dhh : Predictim's black-box algorithm analyzes babysitters' social media accounts, reducing them to a single fit score. Beyond disgusting. Downright evil. Social media algorithms prod you to be the worst/fake you can be, hiring algorithms reject you for it. https://www.washingtonpost.com/ ... David Carroll / @profcarroll : Shot: China builds the ultimate Black Mirror surveillance state dystopia using personal data for social control. http://www.bloomberg.com/...Chaser: USA Land of the free, home of the brave. http://www.washingtonpost.com/ ...
Context & Ripple Effects
This lands a day after the Post's companion report on how Predictim pressures babysitters to grant broad access to their social media before any evaluation happens — today's piece shows what those scans actually produce: character scores the paper found questionable. The criticism echoing across the cited researchers, including Cathy O'Neil, echoes her earlier warning about algorithmic screening.
The story fits a pattern the corpus has tracked since 2016, when employers began turning to algorithms to sift job applications and poorer applicants were more likely to lose out. Predictim moves the same logic from résumés to people's private lives — and Zeynep Tufekci flagged that one of the largest babysitter sites plans to pilot folding such AI ratings into its existing sitter screenings.
First-order effects
- Parents like Jessie Battaglia receive sitter evaluations of unverifiable accuracy, while babysitters must surrender years of online history for an opaque score they cannot see or contest.
- If major childcare platforms pilot folding AI ratings into their existing screenings, as Tufekci noted, the practice scales from a startup experiment to default infrastructure for finding care work.
Second-order effects
- Rival babysitting and gig-care platforms face pressure to adopt comparable scoring to match the perceived rigor, making broad social-media consent a de facto condition of employment in care work.
- Social platforms whose histories feed these scans — Facebook among the named entities — become upstream suppliers to background-check products, exposing them to fresh scrutiny over third-party data use.
Third-order effects
- Consumer scoring extends the arc already visible in predictive algorithms setting police patrols, prison sentences, and probation rules: consequential judgments about individuals delegated to models whose error rates and bias go unaudited.
- If the pattern holds, the line between a voluntary profile and an algorithmic background check dissolves for ordinary workers, with the burden of proving character shifting onto the scored rather than the scorer.
The trend: Opaque AI scoring is migrating from employment screening into intimate personal decisions like childcare, running ahead of any verification of its accuracy or fairness.