How Rana el Kaliouby and her company Affectiva developed software to sense emotions from video
We Know How You Feel — Computers are learning to read emotion, and the business world can't wait. Three years ago, archivists at A.T. & T. stumbled upon a rare fragment of computer history … Tweets: @jasonsilva , @vaughanbell , @interfluidity and @laurenthaug Tweets: Jason Silva / @jasonsilva : “Computers are learning to read emotion” http://www.newyorker.com/... Vaughan Bell / @vaughanbell : Computers are learning to read emotion, and the business world can't wait http://www.newyorker.com/... Steve Randy Waldman / @interfluidity : We were going to help the autistic, then we realized our tech cld be used to surveil ppl and target ads http://www.newyorker.com/... ht @thezhanly Laurent Haug / @laurenthaug : Dubai projects to run all its CCTV footages through an emotion detection software to determine a happiness index http://www.newyorker.com/...
Context & Ripple Effects
This New Yorker profile captures Affectiva near its commercial inflection point: Rana el Kaliouby's team had built software that reads emotion from video, pitched first as an aid for autism and then repurposed — as one of the article's own Twitter respondents noted — for advertising research. The piece framed facial coding as a coming business layer, with 'the business world' waiting to deploy it.
The decade since has run on two tracks. Commercially, the category grew into what the Guardian's $20B emotion-detection industry report described, with Affectiva itself later closing a $26M round led by Aptiv for emotion and object detection. Scientifically, it unraveled: researchers found facial expressions match actual feelings only 20%-30% of the time, and the WSJ reported there are no universal expressions of emotion to train on. The Atlantic now finds the same tools monitoring white-collar workers' moods.
First-order effects
- Brands and market researchers gained a scalable instrument for measuring audience reactions to video content — Affectiva's immediate paying customers — replacing small-panel self-reporting with automated face analysis.
- Affectiva converted academic credibility from el Kaliouby's MIT lineage into venture backing, including the later $26M Aptiv-led round that tied its emotion AI to automotive object detection.
Second-order effects
- As deployment scaled toward a $20B market, critics forced a scientific counter-offensive — the accuracy studies and the universal-expressions debunking now function as due-diligence material buyers must weigh against vendor claims.
- Automotive became a second front: the Aptiv investment pushed emotion sensing from ad testing toward in-car driver monitoring, widening the set of contexts where faces are continuously read.
Third-order effects
- If the pattern holds, emotion AI migrates from optional consumer research to compulsory workplace infrastructure — mood tracking embedded in white-collar jobs — shifting the battleground to privacy law, bias audits, and whether regulators accept a technology whose own scientific base is contested.
- The deeper structural risk the corpus documents: systems trained on stereotypical expressions institutionalize those assumptions, so flawed affect science gets baked into hiring, management, and safety decisions faster than the underlying research can be corrected.
The trend: Emotion recognition is maturing from a novelty profiling tool into ambient workplace infrastructure even as its foundational science comes apart — adoption outpacing validation.