Microsoft and Databricks, an Apache Spark-based analytics platform, bring the Azure Databricks cloud service out of beta
Microsoft is bringing its Azure Databricks cloud service out of beta today to help its customers better process massive amounts of data, powered by a partnership unlike anything the tech titan has done before.
Context & Ripple Effects
Microsoft had already backed Apache Spark on its own terms — [[a:870327|wiring Spark support into HDInsight, Cortana Intelligence Suite, Power BI and R Server back in 2016]] — but Azure Databricks going general availability marks a different posture: instead of wrapping Spark in Microsoft-branded services, the company is distributing a startup's platform as a first-class Azure service, which VentureBeat notes is a partnership structure new for Microsoft.
The move lands mid-way through Databricks' funding arc — a $60M Series C in late 2016, then a $140M Series D led by Andreessen Horowitz in 2017 — giving the startup capital to scale just as the largest enterprise cloud agrees to carry it.
First-order effects
- Azure customers processing large-scale data can now run Databricks' Spark-based analytics as a generally available managed service rather than a beta, with Microsoft handling the cloud infrastructure side.
- Databricks gains Microsoft's enterprise sales channel and Azure's customer base as distribution, converting its venture-backed platform into a revenue engine without building its own cloud.
Second-order effects
- Microsoft's own Spark-touching products from the 2016 effort now sit alongside a partner offering the same workload, setting up internal competition that surfaces two years later when Microsoft launches Azure Synapse Analytics as its own unified big-data workspace.
- Rival clouds face pressure to offer comparably managed Spark services or risk losing big-data workloads to Azure, since the default path for Spark teams shifts toward whichever cloud hosts Databricks best.
Third-order effects
- The arrangement previews the dependency tension baked into hyperscaler-startup partnerships: Databricks keeps raising at rising valuations ($250M at a $2.75B valuation by early 2019) while riding a partner's infrastructure, and by 2024 it ships an AI/BI visualization tool explicitly competing with Microsoft and Salesforce — partners becoming rivals once they no longer need the channel.
- If the pattern holds, cloud providers increasingly function as distribution platforms for third-party open-source analytics engines rather than building every layer themselves, with ownership of the customer relationship becoming the contested ground.
The trend: Enterprise clouds are absorbing third-party open-source analytics platforms as managed first-class services, trading short-term workload gravity for long-term competition with the partners they legitimize.