Moving A Petabyte of Data
I made a speech last week at which I asserted it was faster to send a petabyte of data from San Francisco to Hong Kong by sailboat, than by the internet. — I got quite a few "how can that possibly be true" kinds of questions, so here's the math.
Context & Ripple Effects
A year earlier, researchers had publicized a lab transmission record of 60 DVDs per second, a reminder that headline bandwidth numbers and what enterprises actually get across a wide-area link are different things. Against that backdrop, Sun's Jonathan Schwartz published his speech assertion plus supporting math: moving a petabyte from San Francisco to Hong Kong beats the internet when the pipe between the two endpoints is thin relative to the payload.
The argument matters because it reduces bulk data movement to arithmetic — payload size divided by available throughput versus transit time — rather than intuition about how fast 'the internet' is.
First-order effects
- Anyone planning a petabyte-scale transfer faces a scheduling decision, not just a procurement one: at typical deployed wide-area speeds, weeks of transfer time compete with days of freight, so the cheapest fast option may be disks in a hull rather than leased bandwidth.
- Schwartz's published math gives CIOs and architects a reusable back-of-envelope template for comparing transport modes instead of defaulting to the network.
Second-order effects
- Network operators and equipment vendors are pressured to close the gap that makes physical shipment win — the same pressure behind the pursuit of ever-higher transmission records like the 60-DVDs-per-second result — because customers moving large datasets will otherwise route around their pipes.
- Storage vendors benefit on the other side: as media density rises, each shipped disk carries more bytes, quietly improving sneakernet's economics without any change in bandwidth.
Third-order effects
- If dataset growth continues to outrun deployed wide-area bandwidth, bulk data transport stratifies into two tiers — networks for interactive traffic and physical carriage for massive one-time moves — making freight carriers an unglamorous part of data-center architecture.
- Whether the sailboat stays ahead depends entirely on the slope of real-world link upgrades versus payload growth; the corpus's string of speed records suggests lab capability is not the binding constraint, deployment is.
The trend: As enterprise datasets grow faster than deployed wide-area bandwidth, bulk data movement increasingly splits between live networks and physical transport, with lab speed records highlighting the deployment gap.