Inside the DNC's plan to replace its aging data repository Vertica with the Data Warehouse, a more powerful BigQuery-based system, before the 2020 election
IN JULY OF 2017, as Raffi Krikorian settled into his new office at the Democratic National Committee, Hillary Clinton's words were still ringing in his ears. Tweets: @issielapowsky , @wired , @saikatc , @issielapowsky , @issielapowsky , @lizjaff , @jesslivmo , @issielapowsky , @wired , and @raffi Tweets: Issie Lapowsky / @issielapowsky : Exclusive: Remember how Hillary Clinton bashed the DNC's data after 2016? I got the scoop on how @TheDemocrats are replacing their crumbling data infrastructure before 2020 and talked to folks throughout the party about what needs to come next http://www.wired.com/... @wired : Exclusive: Hillary Clinton once called the Democratic National Committee's data infrastructure “mediocre to poor, nonexistent, wrong.” This is how the Democrats plan on fixing it before the 2020 presidential election. http://www.wired.com/... http://twitter.com/... Saikat Chakrabarti / @saikatc : This is cool. So much campaign infrastructure gets built as hacky stored SQL procedures duct taped together. Having a real query API opens up the ability for apps that grassroots coders make to be turned into real tech that campaigns can deploy widely. Good job @DNC. http://twitter.com/... Issie Lapowsky / @issielapowsky : But that doesn't mean all the DNC's problems are solved. 2020 contenders are still being dramatically outspent by the Trump campaign on digital advertising. “That definitely has me worried,” @raffi, the DNC's outgoing CTO told me http://www.wired.com/... Issie Lapowsky / @issielapowsky : In 2016, @TheDemocrats's data infrastructure “was a shit show” says the Clinton campaign's then-director of product. The system that housed all the party's data would crash for days at a time. We got an inside look at the DNC's plan to fix it http://www.wired.com/... Liz / @lizjaff : Something good happened @TheDemocrats fixed vertica. Any candidate now gets access to a real data warehouse. That's a big deal. It's not sexy, it's real and it was essential in the first step to build up our tech infrastructure. http://www.wired.com/... Jess Morales Rocketto / @jesslivmo : This is a really good story that smells of truthiness, rather than hyperbole, which rarely happens when it comes to @TheDemocrats tech. Welcome to the era of the Data Warehouse, y'all. http://twitter.com/... Issie Lapowsky / @issielapowsky : And questions remain about who's going to lead the party's hotly contested, but crucial data exchange now that @jomalleydillon, who was supposed to run it, left to become Beto O'Rourke's campaign manager http://www.wired.com/... @wired : Democrats are also preparing to set up a much-anticipated data exchange that will allow the party and outside political groups to share their data for the first time, without running afoul of campaign finance laws. http://www.wired.com/... @raffi : this is the story of how @TheDemocrats tech team is replacing the foundational data infrastructure of the party. this is the thing i'm the most proud of during my tenure at @TheDemocrats. http://twitter.com/...
Context & Ripple Effects
This story sits at the end of a rebuild arc that began with the FBI investigation into the Russian hack of the DNC and Hillary Clinton's post-2016 criticism of the party's data operation. Since then the committee has hired Silicon Valley engineering staff and hardened its cybersecurity, and CTO Raffi Krikorian's mandate has expanded from defense to infrastructure.
The move matters because the DNC's role between elections is to be the shared data layer for every Democratic campaign — and per the relationships here, the Vertica replacement comes bundled with two bigger plays: giving candidates real access to the new Data Warehouse, and standing up a data exchange so the party and outside groups can share data without tripping campaign finance rules.
First-order effects
- 2020 primary campaigns get programmatic access to a BigQuery-scale warehouse instead of the aging Vertica system Clinton publicly faulted, changing what voter-targeting work is feasible for down-ballot operations.
- The planned data exchange gives outside political groups a compliance-safe channel to pool data with the party, formalizing coordination that previously happened ad hoc.
Second-order effects
- Deepening the stack on Google's BigQuery ties the party's core electoral infrastructure to a single Big Tech vendor even as its own presidential candidates voice extreme worries about Big Tech — a tension already visible in the growing split between Silicon Valley and the Democratic Party.
- A warehouse is only as good as its upkeep: the one-off, startup-style funding pattern that plagues Democratic campaign tech projects is exactly the model that leaves systems like this unmaintained after election day.
Third-order effects
- If the pattern holds, national parties consolidate into permanent cloud-based data platforms that rent infrastructure and broker data-sharing between cycles — shifting power over campaign targeting toward whoever controls the exchange and its compliance rules.
The trend: Political parties are rebuilding themselves as always-on cloud data platforms, with vendor choice and data-exchange governance becoming as consequential as the software itself.