eBay Shares Server Designs as It Rebuilds IT Infrastructure

Online marketplace eBay has learned a lot about its IT infrastructure since its founding in 1997, and now it's looking to rebuild its internal systems and share those strategies to help other businesses grow and thrive.

That's the strategy behind the company's recently announced plans to open-source the designs of the latest servers it is developing and building, as well as to share details of the many open-source software platforms it uses so other companies can work toward similar business successes.

The knowledge sharing comes as the company is "replatforming," or developing and installing new servers and other hardware to replace existing infrastructure as it takes advantage of the latest advances in IT systems architecture, according to a recent post by Mazen Rawashdeh, eBay's vice president of platform engineering. The replatforming is being done over three years.

"Open source is fueling the transformation of eBay's infrastructure with technologies like Kubernetes, Envoy, MongoDB, Docker and Apache Kafka," wrote Rawashdeh. "By sharing our innovations with the community, we are hoping to give back."

To make it all happen, eBay designed and is building the servers and then plans to make the details of the hardware and software publicly available in the fourth quarter. Plans for how this information will be shared are still being made.

"We were striving to solve our own problems to start, but want to share our knowledge through open source," Manoj Wadekar, the director of eBay's hardware engineering department, told Channel Futures. "That's the philosophy of being open. We're consuming it and now we're sharing it. We do believe this is important for us to do."

Manoj Wadekar

Jon Langshaw, eBay's chief of staff in the platform engineering department, said that companies are usually tight-lipped about such information and won't share the technical details of their success with other companies.

"From eBay's standpoint ... not only are we giving back to open source, but this is the hardware we're using so it can help other businesses," Langshaw told Channel Futures. "We're trying to solve eBay's technology problems first and foremost, so then we can help others and ultimately benefit by having more eyeballs on our open-source code and hardware."

John Langshaw

eBay is already running its massive infrastructure on open-source code, and the replatforming efforts will continue to build on that, said Langshaw. The continuing replatforming is part of a three-year process.

The replatforming is "huge and absolutely transformative for eBay," he said. "We are taking an old platform built over the last 10-15 years and we are totally changing things, from hardware to software."   

The next generation of eBay's servers are specialized to meet the complex demands of the online auction company's business. The goal of the new hardware is to enable improved customer experiences while creating greater efficiencies and reducing the company's dependence on third parties, wrote Rawashdeh. eBay has about 175 million active users and more than 1.1 billion live listings on a regular basis around the world.

The replatforming of the company's back-end infrastructure included designing its own hardware and artificial intelligence engine, decentralizing clusters of data centers, moving to an edge-computing architecture and leveraging the latest in open-source technologies, according to the company.

The company's IT systems process about 300 billion data queries each day and include more than 500 petabytes of storage each day. That equals about 1 trillion songs, or 2.5 million hours of movies.

"We are using servers and hardware that we designed, reducing our dependence on third parties," wrote Rawashdeh. "This allows us to more specifically cater to eBay's needs, offering greater predictability to our business, enabling more control and promoting flexibility."

The creation of the prototypes and deployment of the custom hardware took nine months to complete.

"With this shift, we are homogenizing our infrastructure, leading to significant development and operational efficiencies," he wrote. "With the transformation that we have made and the massive data that funnels through eBay, we used open source to build an in-house AI engine that is highly shareable across all of our teams and aims to increase productivity, collaboration and training. It is allowing our data scientists and engineers to experiment, build products and experiences for customers, and leverage AI at scale."