Mobile Network Trombones are Way Out of Tune With the 5G Symphony

Mobile broadband isn't just for smartphones and laptops with LTE eSIMs. Developments like industrial IoT, the Internet of Medical Things (IoMT), connected vehicles, 5G devices like drones, smart cities, and rugged edge computing locations will all make the expansion of mobile broadband capacity more urgent. That, plus the amount of internet traffic already going over mobile networks, makes it important to have far better connections between cloud networks and mobile networks than today – especially in geographies like Africa, where mobile networks have outpaced fixed networks. Data centers need to deliver compute, storage, and other services to an ever-growing number of networked devices and users.

Putting compute closer to data sources and users with distributed networks of edge computing data centers is only part of the answer. Another big part, which will take a lot of time and investment, is improving those links between mobile and cloud networks. Verizon’s wireless network, for example, needs to get a whole lot better at communicating with Microsoft’s Azure cloud network.

Or with the network of a company like Cloudflare, which offers Content Delivery Network services and DDoS protection, but has been increasingly getting into cloud computing services for developers. Better connection between its network and mobile networks is a long-term goal for Cloudflare’s recently announced expansion of edge computing locations with edge data center specialists EdgeMicro and Vapor IO, beyond the immediate goal of putting more compute and caching capacity closer to more users.

“The idea is that mobile networks could open up parts of the radio access network that traditionally weren't available for IP peering, and providers like ourselves could interconnect with somebody like an AT&T or T-Mobile or Verizon not in tens of places but potentially hundreds or thousands,” Nitin Rao, head of global infrastructure at Cloudflare, told DCK.

Until relatively recently, all AT&T interconnections for mobile traffic happened in just two locations in the US, he said. But then, “we as an industry realized that two wasn’t enough, and for the vast majority of services that number is closer to ten, and in some cases slightly higher.” That still adds dozens or hundreds of milliseconds to each network roundtrip, because a packet must travel to the interconnection hub and then back in a long, trombone-shaped (or hairpin-shaped) loop.

“Data was an afterthought for today’s mobile networks,” Matt Trifiro, Vapor IO’s chief marketing officer, told us. “You might think you could easily peel a data packet off a wireless network, but sometimes packets don't even get an IP address until they're in another state. With one carrier [he didn’t say which one] you don’t get an IP address until it gets to Dallas.

“Even if you have an entire edge infrastructure, if it’s [traveling] over the legacy wireless network, that packet’s going to trombone all the way to Dallas, get an IP address, then get routed on the internet, then have to come all the way back, so you get no benefit to being on the edge.”

As a result, Rao said, “it feels like as an industry we're looking at that number [of interconnects] once again and saying, what would it take for us to get to a hundred? That means different architecture choices in terms of how both cloud providers like ourselves, but also mobile networks architect their network.”

That will take time. “There’s a significant amount of technical work to be done in terms of rearchitecting how these networks are designed,” he warned. “We are a little ways away from being able to run compute at cell tower aggregation points or cable-head and the like.”

Other vendors in the edge computing ecosystem are seeing the same need to better bridge cloud and mobile networks. “Enabling edge computing efficiently and cost-effectively requires new approaches to distributed cloud networking, including interconnection to public clouds and ecosystem partners located across multiple edge data centers,” Mike Capuano, chief marketing officer at Pluribus Networks, told us. (Pluribus recently announced software-defined networking product designed to run in telco central offices.)

Edge Specialists Hope They Can Help Carriers Act Faster

5G is promising to deliver a fiber-like connection wirelessly. That may already be pushing mobile networks to look for more interconnections with the cloud at the edge, Rao suggested.

“From our vantage point it does feel like this year we're making progress from a series of companies across the stack just talking about it to actually reaching into their pocketbooks and making investments of time and capital,” he said. “Mobile networks are seriously thinking about this, and it shows up on their roadmaps.”

Having met with many of the major players recently, Jason Bourg, VP of strategy at EdgeMicro, said he’d also been observing mobile operators becoming more receptive to the idea. “We just have to get everybody moving in the right direction, which is a little bit like herding cats at times,” he said.

Having Vapor IO and EdgeMicro deploying their edge data centers (both started relatively recently) could help encourage internet providers to improve these connections. “We hope it becomes a slightly easier decision for them to set up many interconnection points rather than just a few,” Rao said.

Peering and Local Breakout

EdgeMicro is trying to replicate the benefits of internet peering exchanges in tier-two markets for both terrestrial and mobile networks, Bourg told us. (Vapor is building exchanges in the cities it deploys to as well.) “We’ve always envisioned stopping the tromboning, the hair-pinning, and the backhauling of traffic by creating an exchange for private peering and public internet traffic in these markets where it doesn’t exist,” Bourg said.

The company started with terrestrial networking, partly because that’s easier than the mobile side and partly because customers couldn’t wait until mobile operators were ready to make the necessary architectural changes in their networks. EdgeMicro has developed its own mobile edge traffic exchange, capable of “local breakout” to reroute some mobile traffic at the 4G LTE network edge to its data centers, with the rest going on to the mobile network core.

4G can be adapted for local breakout like this, but it’s standard in 5G, Trifiro pointed out. “It allows you to peel off the data stream and assign an IP address at the packet gateway – which could be located just about anywhere, because it’s going to be a virtualized network function,” he explained. “So, in the same data center as the Cloudflare server, you could have a virtualized packet gateway for the wireless company you’re cross-connecting to, and that connection is almost instantaneous at that point.”

Vapor IO is establishing similar relationships with last-mile network providers, having them terminate in its network and allowing them to offload some of their backhaul from congested legacy networks, like 2G, and avoid expensive upgrades that could otherwise cost them billions.

Cloudflare is an ideal partner in these relationships, because its DDoS-protection and CDN services deliver a lot of caching capacity. “If a user or a device makes a request for some content that’s cached and sitting in one of our Kinetic Edge data centers, then we can offload that request onto our network and serve it from a Cloudflare server in that region on that network,” Trifiro said. “It never hits the carrier’s backhaul network, it never actually hits the internet, it’s served in a very low-latency way.”

The Wireline-Wireless Convergence

5G, he pointed out, is not exclusively a wireless technology. “We’re seeing convergence of wireline and wireless networks, and some of the companies that run both are converging on a 5G core. Cable companies and so on are implementing a common core.” That means full network transparency to the computing backend, regardless of whether a connection is 5G, fiber to the home, or a cable modem.

For many 5G workloads, predictable latency will be as important as low latency. “In a typical network, 30 milliseconds of latency means it averages 30 milliseconds; sometimes it might be 10 milliseconds; sometimes it might be 100 milliseconds,” Trifiro said. “And an autonomous drone that you’re controlling over 5G can’t tolerate 100 milliseconds. For a lot of low-latency workloads it's more about controlling the jitter than the actual latency.”

That requires reducing the number of network hops a packet has to travel between source and destination, which in turn requires more interconnection points with mobile networks. Rao’s hundred interconnections could be conservative. “I think every city is going to have to have ten,” Trifiro told us. “They don’t need ten today, but competitive pressure is going to drive that – and the technology [to do it] exists.”