How Rana el Kaliouby and her company Affectiva developed software to sense emotions from video
Raffi Khatchadourian / New Yorker : Tweets: @jasonsilva , @vaughanbell , @laurenthaug and @interfluidity Tweets: Jason Silva / @jasonsilva : “Computers are learning to read emotion” http://www.newyorker.com/... Vaughan Bell / @vaughanbell : Computers are learning to read emotion, and the business world can't wait http://www.newyorker.com/... Laurent Haug / @laurenthaug : Dubai projects to run all its CCTV footages through an emotion detection software to determine a happiness index http://www.newyorker.com/... Steve Randy Waldman / @interfluidity : We were going to help the autistic, then we realized our tech cld be used to surveil ppl and target ads http://www.newyorker.com/... ht @thezhanly
Context & Ripple Effects
This 2015 New Yorker profile of Rana el Kaliouby and Affectiva's video-based emotion-sensing software is the origin story of what later coverage tracks as a full industry arc: within four years the technology had grown into a $20B emotion-detection market, with Dubai reportedly planning to run CCTV footage through such software to compute a citywide happiness index.
The profile also plants the tension every later story picks up — surveillance and targeted-advertising uses of tech pitched as helping autistic people — which matures into researchers finding facial expressions match actual emotions only 20%-30% of the time, and into emotion AI reaching white-collar workplaces by 2026.
First-order effects
- Brands become the immediate customers: Realeyes raises a $12.4M Series B to sell AT&T and Mars facial-expression gauging, proving ad-testing demand outpaces any assistive use.
- Dubai's reported plan to pipe CCTV through emotion detection makes governments, not just advertisers, early adopters — putting Affectiva-style software directly into public-space surveillance.
Second-order effects
- Commercial success forces the science into the open: experts publicly challenge the foundational claim that expressions map to feelings, threatening the validity of every contract sold on it.
Third-order effects
- If adoption continues despite the disputed science, the endpoint visible in the corpus is workplace monitoring — emotion AI tracking employees' moods — where accuracy flaws translate into biased hiring and management decisions rather than misread ads.
The trend: Emotion-sensing AI is scaling from lab demo to advertising, government, and workplace infrastructure faster than its scientific foundation has been validated, making governance the binding constraint.