GPU-as-a-Service: What IT Pros Need to Know

As if the cloud computing industry weren't already inundated in acronyms, here's a new one for IT pros to learn: GPUaaS.

Actually, those of you keeping score will notice that that's an acronym inside an acronym, because it stands for GPU-as-a-service.

But we're not here to fetishize acronyms. We're here to educate you about cloud computing trends, and GPU-as-a-service is one of them. Keep reading for a look at what GPUaaS means, why it's an important concept today, and where to find GPU-as-a-service solutions.

What Is GPU-as-a-Service?

GPU-as-a-service — also known as GPUaaS or sometimes just GaaS — is a type of cloud computing offering that makes servers with GPUs, or graphical processing units, available on demand.

GPU-as-a-service falls within the category of infrastructure-as-a-service, or IaaS, because it's a way of delivering infrastructure — specifically, servers equipped with GPUs — via a network connection.

How Does GPU-as-a-Service Work?

GPU-as-a-service offerings work in the same way as any standard IaaS solution: Service providers deploy servers that include GPUs, then allow customers to connect to those servers over the internet using SSH or a remote desktop tool. Once connected, customers can deploy workloads of their choosing. Pricing is based primarily on how long the server runs.

Related:Cloud Migration vs. Cloud Transformation: What's the Difference, Exactly?

If that sounds very similar to deploying any type of server instance in the cloud — not just one that gives you access to GPUs — it's because it is. All of the major cloud providers now offer server instances — like these from AWS and these from Azure — that feature powerful GPUs as part of their standard lineup of cloud servers. In most cases, the instances are virtual machines, although you can find bare-metal cloud servers that offer GPUs, such as AWS' G4dn instances.

Why Is GPU-as-a-Service Important?

In virtually all cases, a cloud server instance equipped with a powerful GPU costs more to operate than an instance that provides similar CPU and memory resources but does not include a GPU. So, why would you choose to deploy a cloud server with a GPU?

The answer is that you'd be running a workload that can benefit from or requires a GPU, such as I model training or large-scale database analytics. For workloads like these, GPUs can deliver tremendous performance boosts because they can crunch numbers at very high speeds — especially in cases where calculations can be performed in parallel.

Thus, although GPUs were originally designed primarily for rendering video, they can prove valuable for any workload that requires parallel computation on a massive scale. And, given the momentum surrounding AI, it's likely that more and more businesses will be turning to GPU-as-a-service solutions to help train and deploy AI-dependent applications in the years to come.

You could, of course, go out and buy your own GPU-equipped servers to handle such workloads. But renting access to such servers through the cloud using a GPU-as-a-service offering may be a more economical approach, especially if you wouldn't use your own servers at full capacity on a permanent basis.

GPU-over-IP: A Variant on GPUaaS

Purchasing temporary access to cloud-based GPU servers using GPUaaS isn't the only way to access powerful GPUs without acquiring them yourself. You can also take advantage of GPU-over-IP, a variant on the GPUaaS idea.

With GPU-over-IP, you don't rent an entire server and deploy workloads on it to leverage its GPUs. Instead, you connect your GPU-dependent workloads to a remote server's GPU over the internet, using software that exposes the GPU in "raw" form via the network.

The performance levels of GPU-over-IP tend to be lower than those of GPUaaS because some resources are wasted in the process of exposing the GPU to the network. But the advantage is that you don't have to rent an entire server and deploy your workloads on it; you can take advantage of remote GPUs while leaving your workloads in place.

Where to Find GPU-as-a-Service

As noted above, you can find GPUaaS offerings from all of the major cloud providers. Some smaller ones offer GPU-enabled server instances, too. NVIDIA maintains a helpful chart listing which cloud vendors offer access to servers equipped with its GPUs.

If you want to explore the GPU-over-IP approach, check out Juice Labs, which builds software that can expose GPUs over a network. A free and open source edition of its solution is available. Juice Labs doesn't provide GPU-enabled infrastructure; you have to source that on your own. But it does make it possible to take GPU infrastructure from one source (like a partner's data center) and connect it to workloads located elsewhere.