Western Digital's Long Trip from Open Standards to Open Source Chips

Western Digital, the company known to most as a seller of hard drives, has gone all-in on open source. While not a stranger to open source software -- and far from being the first hardware company to embrace open source -- it recently became a pioneer by diving headfirst into the uncharted waters of open source silicon. And it seems fitting for a company started in the 1970s as a chipmaker.

Like many companies that started life in the proprietary world, it took the storage vendor a few baby steps to get into open source software. In an interview with Data Center Knowledge, Western Digital's senior director of systems and software technologies Jorge Campello said the journey began with open standards.

Open Standards Pave the Way

"Open standards have been an integral part of how we do business for a long while," he said. "I think we've been very deeply engaged in those, even longer than we have in open source, because even back before open source was pervasive, even in the world of proprietary software, the way that interoperability worked was we depended on standards."

Not all standards were free and open. He pointed out large OEMs that insisted on certain specifications he referred to as "de facto standards." These were "not real standards, but more of companies monopolizing niche markets."

While this has happened in the world of storage arrays and systems, with vendors nurturing their own ecosystems and infrastructures resulting in vendor lock-in, it hasn't been an issue at the individual storage device level. "On the devices side, SCSI and ATA have been ruling interface standards for a long time, and more recently MvME with SSDs," Campello said.

From Open Standards to Linux and Open Source

The baby steps from open standards to open source software consisted of getting involved in open source projects "here and there," but the projects were relatively small, such as tools for measuring network throughput and performance. "I think we got involved more in earnest with SMR (single magnetic recording) and the necessity to make substantial changes to the software stack in order to make use of the technology," he said.

The company worked pretty much like any other hardware vendor otherwise, selling much of its hardware in combination with proprietary software solutions by partners like Microsoft. The onus would be on these software partners to enable new technology for their customers, Campello explained.

But the playing field has shifted for hardware vendors in the modern cloud-native and software-defined world. Instead of waiting for software vendors to write the code to integrate new hardware products into their infrastructure, the hardware vendors now frequently seed the pot with some homegrown code and let the open source community build on it.

"The deciding factor for the success of these open source projects is that you go forward and contribute pieces of technology to it," he said. "Instead of having to wait for someone else to do it for you, now you have the opportunity to go forward, and if you want to advance the technology in a certain area, contribute software solutions in that area. In most of these projects there's peer review, and if it's important enough it gets picked up; if it's not, it doesn't. But you have the opportunity to go out and enable it."

Western Digital leadership recognized "the reality of a world that's driven by open source software and we started to get engaged and contribute to foster technologies that we were developing."

The transition from proprietary to open source, from controlling and owning all aspects of a product to trusting an outside community to develop code that may or may not be useful, is often difficult for companies that are used to being in control from beginning to end. Western Digital isn't immune to such issues, and the discussion about whether it's worth developing software that will enable a competitor arises regularly, he said. But "you come to the realization that if you're driving the development of the technology, both in standards and software, then you're in the leading edge -- you have the time-to-market edge. And when you develop it in open source and based on standards, then you get wider adoption."

Western Digital's early experience working with the Linux kernel development team -- which Campello said was the company's first foray into open source -- has proven to be valuable experience. By working within the structure that Linux devs already had in place, he said, the company learned that in open source the old model of simply developing code and submitting it doesn't work. "You have to become an actual member of the community by contributing and sticking around, because bugs happen, other architectural changes have to be compatible with things that you put in, and you become part of the community that's maintaining that project," he said.

On the plus side, having software for your hardware being developed in the open cracks the door for outside developers to find new ways to consume the hardware. A lot of the distributed computing work by "the hyperscale movement" has been done in open source, leading to the storage devices being used "in completely different ways" than they were as part of proprietary legacy storage arrays and systems, Campello said. "The failure modes are different, the things that are important in terms of performance and tradeoffs are different, and as that grows, we have to learn about those and then adapt our products to those. It's kind of a chicken and egg. If someone starts finding a new use, we figure out that use is nice, so let's make our product better for that use. Then it evolves."

Pioneering Open Silicon

While it's become common for hardware companies to get into open source software, there hasn't been a big shift to open source hardware, outside of a handful of hyperscalers open-sourcing specs and designs for some of their data center gear through the Open Compute Project. Design is only a part of the enormous cost of producing and distributing hardware.

But in the last couple of years, some niches emerged where the open source approach to hardware does have clear advantages. This has been especially true in the computer chip arena, where the open source reduced instruction set architecture RISC-V has been proven to save manufacturers on massive licensing costs, even over "low cost" alternatives such as Arm. In addition, hardware makers can use the architecture to design and manufacture custom chips specifically for the devices in which they will be used instead of designing the devices around available chips.

RISC-V's open source approach has been so successful that in December MIPS, a once popular but now long-in-the-tooth RISC architecture with DNA going back to the 1980s, went open source, and this week IBM announced that it's open sourcing its flagship Power architecture.

Last year, Western Digital created a stir when it said it would shift to using only RISC-V chips in its products, expecting to produce two billion chips annually within a couple of years. In parallel with that announcement, the company became an investor in SiFive, the leading RISC-V company that says it can design and bring to market RISC-V chips for about 10 percent of what it would cost to do the same with a proprietary architecture.

Earlier this year, Western Digital open sourced its RISC-V-based SweRV Core and later worked with SiFive on a partnership with PlatformIO Labs to bring open source tools to the RISC-V developer community.

"We believe that it's only valuable to us if it's successful in the industry," Campello said about the RISC-V-based PlatformIO partnership. "To be successful in the industry, you need to have the whole ecosystem of software and hardware freely available. It takes a while to get to the point where it's easy to adopt, and you have to invest in all these components."

Broadening the Linux Connection

According to him, Western Digital has also expanded its work with Linux developers to include enabling RISC-V. Although many Linux distributions have been ported to run on the open source processor, the company prefers to work upstream, with kernel developers, while working directly with distro developers on things like back-porting RISC-V capability to older kernel versions the distros still support.

RISC-V's biggest advantage is the versatility made possible by its open source design, Campello said.

"That's exactly one of the main advantages of an open architecture like that with a permissive license," he said. "We can start from a core functionality that's useful for everyone and then adapt it to our needs as opposed to you get what you get and adapt yourself around it."