Solving the Challenges of Multicloud Cost Management

This article was written by Michael Arkoosh, director of technical optimization at Vega Cloud.

If you have a multicloud architecture — as 98% of enterprises that use the cloud do — one of the most important steps you can take to manage cloud costs effectively is tracking all of your cloud spending from a central vantage point.

Unfortunately, that's often much easier said than done. The cloud providers don't make it very simple even to pull billing data in a standardized way, let alone interpret it consistently side-by-side.

Nonetheless, finding a way to centralize cloud spend management is critical if you want to optimize your ability to understand what you're spending and avoid wasting money in the cloud. And given the right strategies and tools, implementing centralized bill tracking is feasible enough.

The Importance of Multicloud Cost Management

There are two main reasons why tracking cloud spending from a centralized location is important.

One is that in some cases, businesses deploy workloads that span multiple clouds, and monitoring the costs associated with each cloud is the only way to determine the total cost of the workload. For instance, you might deploy an application in one cloud but have a "warm standby" environment for the same app provisioned in a different cloud for disaster recovery purposes. In that case, you'd need to know what each cloud provider is charging for the infrastructure the app consumes in order to track your total cost of running the app.

Related:7 Cloud Finance Metrics to Track to Better Control Cloud Costs

The other reason that makes centralized cloud spend tracking crucial is the fact that you can't effectively monitor or optimize your total cloud spending unless you can view granular spending data from across all of your clouds side-by-side. Knowing that you reduced your spending last month in one cloud isn't very helpful if you don't know how your spending changed in a different cloud — or how fluctuations in cloud spend align with different types of cloud services across your various cloud environments.

If you like analogies, think of it this way: Centralized cloud spend monitoring is like tracking what you spend in total for groceries from multiple stores over the course of a month. Collecting and interpreting this data from a central location is the only way to determine whether your overall grocery spending is trending up or down. You'd be in a much weaker position to manage your grocery costs if you only monitored how much you spent at each individual store, without having an easy way to track overall spending across all stores.

Related:Cloud Cost Calculators: Benefits and Limitations

The Challenges of Centralized Cloud Spend Management

But as I mentioned, centralized spend management in the cloud can be tough to implement.

The issue starts with the mere task of importing billing data from cloud providers. Although all major public clouds generate bills that detail what you spent each month, they expose the billing data in a different way. Amazon Web Services (AWS) generates a large CSV file, for example, while Google Cloud Platform (GCP) pushes its customers' billing data into BigQuery, a data analytics service hosted on its platform. For its part, Azure expects customers to import billing data using APIs. This means that simply getting all of your billing data into a central location requires implementing multiple data importation workflows.

Once the data is centralized, comparing it can be challenging because each cloud provider structures billing data a bit differently. For example, AWS records charges for cloud servers based on how many hours they were used, whereas Azure reports cloud server billing data based on daily usage — so you can't compare your total cost for cloud servers per hour or per day without converting the data supplied by each provider.

Consider, too, that GCP breaks cloud server spending into separate costs for compute and memory. AWS and Azure don't do this; they report billing information based only on the total resources consumed by a cloud server. Thus, if you use AWS or Azure, you need to disaggregate the data yourself if you want the level of granularity that GCP provides by default — and doing so is important if you need to make an apples-to-apples comparison of what you spend for both compute and memory across clouds.

Making Cloud Billing More Centralized and Consistent Across Clouds

Faced with these challenges, what can an organization do to manage cloud spending effectively across multiple cloud platforms?

One approach is to try to build your own cloud spend management system from scratch. If you have sufficient software development and data management resources within your organization, it's not especially challenging to build a tool that pulls in billing data from each cloud provider, then transforms and aggregates it in ways that make it possible to draw at least basic comparisons of spending data across clouds. However, the big drawback of this approach is that whenever a cloud provider changes its billing structures even slightly or introduces new billing metrics — which happens frequently and often with no warning — you have to update your spend management tool.

Fortunately, this fraught, DIY approach to managing cloud spending is not the only option. Some third-party cloud cost management tools can automatically pull and transform billing data from all major cloud providers, making it easy to compare and optimize costs without having to build and maintain your own spend management tool.

Solutions like these are on track to grow even more robust in the near- to medium-term future thanks to initiatives like FOCUS, an effort by the FinOps Foundation to create an open standard for managing cloud billing data. FOCUS promises to make it easier for organizations to collect and interpret bills from multiple clouds in a consistent way.And no matter which tools you implement for centralizing cloud spend management, making changes within your organization is important as well for achieving a consistent approach to tracking billing data. Businesses should educate FinOps stakeholders — including both engineers, who play a key role in ensuring that cloud resources are configured in ways that enable effective cost tracking, and finance teams, who take the lead in interpreting billing data — in the intricacies of multicloud spend tracking. They should also ensure that various teams adopt a shared strategy for monitoring cloud spending. And they should engage with outside groups, like the FinOps organization, to keep abreast of advancements designed to make it easier to track spending across clouds.