3 Reasons Why Twitter Will NOT Index the Links You Share (Updated)
Techmeme is on fire this morning with discussion of Rafe Needleman's CNet post about Twitter's supposed plans to index the content of links shared over the microblogging service. Ex-Googler turned Twitter exec …
Context & Ripple Effects
The spark was Rafe Needleman's CNet post floating — as a rumor, not a confirmed plan — that Twitter might start indexing the content behind the links users share, which Marketing Pilgrim promptly framed as a threat to 'squash Google's spider' and Techmeme pushed through the morning's conversation. ReadWriteWeb's piece takes the contrarian position: three reasons the indexing never happens.
For the SEO crowd this is familiar ground. Twitter had already stripped ranking value from its own profile links when it added nofollow to bio URLs back in September 2008 [[a:1183559]], and Search Engine Land's March 2009 argument over whether tweets are 'incredibly valuable or utterly useless' as a link building tool ended without consensus [[a:1184990]]. This post plants itself firmly on the skeptic side of Needleman's speculation.
First-order effects
- Link builders treating tweeted URLs as ranking assets get their working answer: the content behind those links stays out of the index, so the measurable return from sharing is referral traffic, not link equity.
- Needleman's post forces Twitter to clarify its position publicly — if its ex-Googler executives were quietly weighing deeper indexing, the SEO community's immediate backlash surfaces the cost before any build-out begins.
Second-order effects
- Google keeps its default role as the web's indexer by omission: if Twitter won't expose link content, any real-time search play must run through partnerships or Twitter's own search product rather than organic crawling.
- Competing engines and aggregators watching the debate see an opening — a major platform withholding its shared-link corpus leaves real-time indexing territory up for whoever strikes terms first.
Third-order effects
- If platforms keep their streams and shared links walled off from crawlers, discovery splits structurally into two markets — platform-ranked social search versus crawler-built web search — and publishers must optimize for each separately.
The trend: Search is splintering into platform-owned real-time indexes and crawler-based web indexes, with each social network deciding how much of its stream the open web gets to rank.