YouTube Offline, Pakistan Telecom Blamed

YouTube was knocked offline for two hours Sunday when Pakistan Telecom claimed its IP addresses, sparking a debate about whether the outage was a botched effort to block Pakistanis' access to the site, or a deliberate political IP hijacking. David Ulevitch of OpenDNS said that YouTube was down "because Pakistan Telecom has decided to (accidentally probably) hijack their IP address space which means that nobody in the world can reach Youtube." Posts to the North American Network Operators Group (NANOG) confirmed that Pakistan Telecom (AS17557) made a change to routing tables that redirected an IP block for YouTube - which included all of the DNS servers that direct traffic to YouTube.

Hong Kong-based provider PCCW, which is upstream of Pakistan Telecom, at first adopted the errant routing change announced by Pakistan Telecom but addressed the error after the video portal had been offline for two hours. YouTube has reportedly added a DNS server in a separate network block to prevent a repeat of the incident.

The Pakistan government moved to block access to YouTube because of anti-Islamic clips posted on the site. A government decided to block YouTube because it contained "blasphemous content, videos and documents," a government official told AFP. "The site will remain blocked till further orders," he said. The blockage was believed to relate to cartoons published by Danish newspapers in 2005 and reprinted earlier this month.

"For about two hours, traffic to YouTube was routed according to erroneous Internet Protocols," YouTube spokesperson Ricardo Reyes said in a statement to News.com. "Many users around the world could not access our site. We have determined that the source of these events was a network in Pakistan. We are investigating and working with others in the Internet community to prevent this from happening again."

At first blush, most network professionals thought it unlikely that the move was intentional. The BBC quoted a network professional who called the routing change "probably a simple mistake by an engineer at Pakistan Telecom. There's nothing to suggest this was malicious." An attempt by a smaller network to hijack YouTube's DNS traffic could be expected to immediately degrade performance on the hijackers' network.

The erroneous IP assignments spread across the net within 1 minute, 45 seconds of its announcement by Pakistan Telecom, according to a timeline by Renesys. It took about 80 minutes for YouTube to inform its providers that the route had been hijacked, according to Renesys, which provides network monitoring and optimization services. "In this case, PCCW (3491) did not validate Pakistan Telecom's (17557) advertisement for 208.65.153.0/24," wrote Martin Brown of Renesys. "By accepting this advertisement and readvertising to its peers and providers PCCW was propagating the wrong route."

Network operators immediately began assessing ways to detects IP hijackings - whether intentional or not - and restore Internet routing tables as quickly as possible. Pakistan Telecom was able to hijack YouTube's traffic due to the nature of Internet routing, which favors the most specific assignment of an IP address. This allows a large hosting provider with many IP addresses to partition large blocks of IPs to customers. Because users of IP addresses change, network providers "announce" the changes, which are then recognized by other providers.

On Sunday Pakistan Telecom announced that it now was using IP addresses that belonged to YouTube in San Mateo, Calif. PCCW adopted the change, not realizing that the IP did not rightfully belong to Pakistan Telecom. NANOG participants said YouTube reached out directly to PCCW to address the issue. It's not clear why it took so long for PCCW to reverse the error, but weekend staffing at Asia-Pacific providers was also an issue in the 2005 domain hijacking of Panix.com, as the New York ISP had trouble reaching officials at Australian registrar Melbourne IT to convince them that the domain transfer was fraudulent.