How to Set up a CI/CD Pipeline

If you know anything about DevOps, you know that CI/CD pipelines are an essential part of any DevOps operation. But how do you actually set up a CI/CD pipeline? Which CI/CD processes do you need to implement, and how do you tie them together to build out a complete pipeline? Those are the real questions that DevOps teams need to answer to put DevOps theory into practice.

Although CI/CD pipelines can vary significantly from one organization to another, the guidelines below will help you understand how to put a pipeline together based on your needs and priorities.

What Is a CI/CD Pipeline?

A CI/CD pipeline is a set of processes that, when combined, allow teams to deliver software continuously, according to the principles behind DevOps.

Again, CI/CD pipelines can come in many shapes and sizes. Virtually all pipelines, however, include at least the following core processes:

Some pipelines are much more sophisticated than this. They could include additional processes, such as security testing that a team integrates into the CI/CD pipeline to practice DevSecOps. Some teams also incorporate techniques like release branching (which helps teams manage complex feature updates) and canary deployments (which can mitigate the risks associated with a bad production deployment) into their pipelines.

But if you’re setting up a CI/CD pipeline for the first time, it’s best to start with the basic core processes, and then add complexity from there if you need to.

Three Ways to Set up a CI/CD Pipeline

To set up any CI/CD pipeline, you need to identify the processes you will include, then implement tools that allow you to execute those processes.

There are three main ways to go about doing this.

1. DIY CI/CD

The first is what you could call a DIY approach to CI/CD. In this model, you set up each CI/CD tool separately, then integrate them together yourself.

This is the hardest way to build a CI/CD pipeline because you need to do a lot of tool deployment and integration on your own. But it offers the advantage of giving you maximum control over which tools you use and how they connect to each other.

This approach may also be attractive if you already have some CI/CD processes in place and want to build them out into a full CI/CD pipeline. For example, you might be doing continuous integration, but not continuous deployment. With a DIY approach to CI/CD, you can take the tools you already have, then fill in the process gaps to construct a complete pipeline.

Note that DIY CI/CD doesn’t necessarily imply that everything runs on-premises. There is nothing stopping you from hosting your self-deployed pipeline in a public cloud if you like, provided you are willing to configure and manage the cloud environment that hosts your tools.

2. Fully managed CI/CD

The second strategy for creating a CI/CD pipeline is to use a fully managed solution from a vendor that provides a ready-made pipeline for you. Services like GitLab, Azure DevOps fit this bill.

Most of these solutions can be deployed either on-prem on your own infrastructure, or within a managed public cloud environment. In the latter setup, you essentially get CI/CD as a service.

The advantage of fully managed CI/CD is that it’s the fastest and simplest way to roll out a CI/CD pipeline. The disadvantage is that you can only use whichever tools the vendor gives you (or supports via third-party integrations), and customizability is limited.

3. Self-managed CI/CD platform

The third approach is to use a ready-made CI/CD platform, but manage and customize it yourself. Solutions like Jenkins, which originated as a continuous integration server but is now a more-or-less complete CI/CD suite all-in-one platform, allow you to do this.

You can set up a platform like this wherever you like--in the cloud or on-prem--and customize it by integrating third-party tools into your pipeline as you see fit.

This strategy is less work than building your own pipeline from scratch using individual tools, and it provides more control over pipeline design and configuration than you could achieve using a fully managed CI/CD suite.

CI/CD Tailored to Your Needs

One of the great things about CI/CD is that you can set up your pipeline in whichever way makes most sense to you. If you want simplicity, go with a managed CI/CD solution. If you are all about customization, build it all from scratch yourself. And if you want a convenient middle ground between those two approaches, take a CI/CD platform like Jenkins and customize it to fit your needs.