From Patches to Performance: What Enterprises Need to Know About Meltdown and Spectre

By the time news of the Meltdown and Spectre vulnerabilities went public earlier this week, cloud providers including Microsoft, Amazon Web Services (AWS) and Google had already put in work behind the scenes to apply patches to servers and update applications.

Since Google researchers discovered the Meltdown and Spectre vulnerabilities last year, its engineering team has been rolling out patches across its product suite. Microsoft notified customers of planned Azure maintenance a few weeks ago, instructing them to reboot VMs, while AWS deployed rolling reboots, usually an early indication of a massive patch update.

“For all intents and purposes, by the time the news broke, cloud providers were almost done deploying the patches for this. Because it’s such a big issue, because the cloud providers take ownership of security at the hypervisor level on themselves, they roll it out globally,” Misha Govshteyn, co-founder and SVP of products at managed cloud security firm Alert Logic said. About one-third of Alert Logic’s business comes from cloud providers, he added.

At this point, Google has patched its G Suite applications, and Google Cloud Platform; AWS has patched across Amazon EC2, including customer instances; and Microsoft is automatically rebooting VMs that still require the update.

So with all the work being done to patch at the hypervisor level by these cloud providers, does that mean enterprises can sit back and rely on the automated patches? Not even close.

“This is not the time to skip a patch cycle; enterprises need to be aggressively proactive about following the guidelines from the respective cloud provider,” Scott Lambert, vice president of Threat Intelligence for Alert Logic said. “Just because most of the major cloud providers have done their part, you still have that shared responsibility and you need to update all of your systems accordingly.”

“I think patching practices across enterprises are uneven to say the least, and as we saw with the Equifax breach its failure to patch against a critical vulnerability that left them exposed for a long time,” Govshteyn added.

Ryan Sommers, manager of Threat Intelligence and Incident Response at LogRhythm, said that it is also important for enterprises to patch their operating systems.

“Even though the infrastructure is patched, and that would mean that one attacked VM wouldn’t be able to read the memory of another VM, that doesn’t mean that each individual VM is patched yet," he said, which means "a process in one individual VM could read memory from another process running in that same VM when it otherwise shouldn’t be able to."

For example, AWS recommends customers patch their instance operating systems to “strengthen the protections that these operating systems provide to isolate software running within the same instance.” Google recommends the same, as does Microsoft.  

While the vulnerabilities affect individual machines as well, the impact on cloud providers is potentially devastating should the vulnerabilities be exploited. At this time there are a couple of complex proof of concepts but no exploits in the wild, Govshteyn said, calling the pure cloud risk at this point “somewhere between low to nonexistent.”

“On the cloud side there is some nuance involved in this. A lot of the reports we have seen in the media have been accurately stating that this affects cloud providers much more, and one of the reasons is that this flaw affects any cloud provider that uses virtualization – which means all of them,” he said. “You don’t need to necessarily attack one of those cloud providers in order to exploit it, you just have to buy a cloud instance and you may be able to delete data from your neighbors, other companies, which opens the doors for a whole lot of hacks.”

Govshteyn calls the Meltdown and Spectre vulnerabilities “second order” flaws – in other words, an attacker would need to find another way to enter the local system before Meltdown and Spectre flaws could be exploited.

“On one hand it is very widespread so if you’re able to get access to the local system, it will help you greatly to move to the next system and the next,” Govshteyn said. “It is a very dangerous lateral movement tool, but you first have to get access to the local system somehow. One thing that is very different from what we saw with Struts for example, which is what took Equifax down, is that I am not sure there is any way to exploit this flaw directly to initially gain access to the system, it’s a second order flaw.”

“Aside from aggressively patching the biggest thing people need to do is pay attention to all of the detection alarms for other threats” because those will be the delivery mechanism for the Meltdown and Spectre flaws, he said.

Even though sys admins should be diligent in deploying the patches, Sommers said that it is still not known how effective the patches are, which makes it even more important to watch for abnormalities.

“One thing we don’t know yet is how effective the patches are. It wouldn’t be the first time that a company has tried to patch vulnerability and someone discovers a new way to exploit a slightly different vulnerability,” Sommers said. “Admins should be diligent to watch for any kind of abnormal activity on their systems, any kind of unauthorized access, or other unusual things that might suggest an attack is in progress or has occurred.”

From the beginning, reports of cloud performance degradation have accompanied the news of the Meltdown and Spectre vulnerabilities, with some estimates being as high as 30 percent performance loss.

The experts agree that it is way too early to determine what the actual effect on performance will be.

“The estimates have been pretty dire, but ultimately it’s going to be impossible to know until people actually test this in a lab and we are probably going to see the truth emerge as people actually test their workloads,” Govshteyn said.

The performance impact will be workload dependent. For example, if a workload doesn’t need to make kernel calls, there should be no impact, Govshteyn said.

Sommers said, “Given the nature of these vulnerabilities and what might need to be done to patch them, as everyone has said all along, is it is going to be very workload dependent. If you have a workload that doesn’t use the areas of the operating system or the hardware that are affected by this as much as other types of workloads you might not be as effected.”