SolarWinds Hack Followed Years of Warnings of Weak Cybersecurity

(Bloomberg) -- Congress and federal agencies have been slow or unwilling to address warnings about cybersecurity, shelving recommendations that are considered high priority while investing in programs that have fallen short.

The massive cyber-attack by suspected Russian hackers, disclosed in December, came after years of warnings from a watchdog group and cybersecurity experts. For instance, the Cyberspace Solarium Commission, which was created by Congress to come up with strategies to thwart sizable cyber-attacks, presented a set of recommendations to Congress in March that included additional safeguards to ensure more trusted supply chains.

By then, the alleged Russian hackers may have already breached the government’s software supply chain, in a brazen attack that targeted federal agencies, technology giants including Microsoft Corp. and cybersecurity companies. U.S. officials said the attack, which was disclosed in December 2020, is ongoing; investigators haven’t yet revealed the extent of the damage.

“The fact is that we have known for a long time that the government is vulnerable,” said Cristina Chaplain, a former director at the Government Accountability Office, which has been harshly critical of U.S. government cybersecurity. “A lot of people had a sense that we were still vulnerable to something like this happening.”

The GAO has studied government cybersecurity practices and issued approximately 3,000 recommendations in the last decade that agencies could implement to make their networks more secure. Of those, nearly 20% haven’t been fully addressed, including 75 of the highest priority recommendations, according to a September report.

A more recent GAO report, in December, identified seven basic steps that agencies could take to manage risks specifically related to the digital supply chain -- such as developing a process for reviewing suppliers to avoid buying insecure software -- and found that these steps were barely practiced across 23 government agencies. In the recent attack, the suspected Russian hackers installed malicious code into software from Texas-based SolarWinds Corp., which is widely used by government agencies and private sector companies to manage computer networks.

It’s hard to know whether the Solarium’s recommendations -- some of which were approved by Congress on Jan. 1 as part of the National Defense Authorization Act -- would have thwarted such a sophisticated cyber-attack had they been put in place sooner. But Representative Mike Gallagher, a Republican from Wisconsin who co-chairs the Cyberspace Solarium Commission, said, “The federal government would have at least detected this sooner and been able to mitigate the damage much more quickly.”

Cybersecurity in the U.S. government is divided among several agencies, but protection of computer networks in civilian agencies is largely left to the Department of Homeland Security’s Cybersecurity and Infrastructure Security Agency, known as CISA, and the agencies themselves.

“Upon learning of this cyber campaign in mid-December, CISA immediately began working to understand the scope of the campaign, share information and detections and assist compromised entities with remediation,” Brandon Wales, the acting director of CISA, said in a statement. “We released an emergency cyber directive to help federal agencies identify whether their networks were exposed to this activity, and within 72 hours of release, 100% of the identified affected devices were taken offline.”

Wales added that agency heads are responsible for securing their systems, while CISA’s role is to “understand enterprise-wide cybersecurity risk and ensure that technical information, detections and remediation guidance are shared swiftly and broadly.”

The warnings about cybersecurity risks, and missed opportunities to improve defenses, date back to at least 2003.

That year, the U.S. government offered a free software update management system to civilian agencies to track software updates constantly peppering their networks -- and checking for vulnerabilities. Congress approved $11 million for the system, which was built by private contractors. But there were few takers, so the program, known as Patch Authentication and Dissemination Capability, eventually folded, according to Jim Jaeger, a former brigadier general in the U.S. Air Force who was then vice president of cybersecurity at Veridian Corp., one of two companies contracted to build PADC.

“The question is, if the PADC system still existed, how would it have evolved to keep up with today’s threat environment?” Jaeger said. “That it died is indicative of a lack of focus on a problem that security experts have warned about for 15 to 20 years. We have been concerned that the patch update process could become a vector for large-scale attacks.”

The same year, in response to a growing number of cyber-attacks, the Department of Homeland Security created the first iteration of a cybersecurity defense system known as Einstein to detect potential intrusions in government networks. Billions of dollars have been spent on Einstein, which the agency describes as the cyber equivalent to a surveillance and alarm system in a government facility.

But for years GAO has warned about problems with Einstein, foreshadowing its apparent failure to detect the SolarWinds hack. In a 2016 report, GAO found that the system was only “partially” meeting its objectives and made nine recommendations for improving Einstein. But two years later, GAO concluded that DHS had “not taken sufficient actions to ensure that it successfully mitigates cybersecurity risks on federal and private-sector computer systems and networks.” In a December 2018 report, GAO found that eight of the recommendations hadn’t been fully implemented.

A CISA official, who spoke on the condition of anonymity, said Einstein’s success depends on information sharing. Once indicators of the recent attack were shared by the private sector, Einstein was used to identify compromised government networks and notify agencies. The official added that, to the knowledge of CISA officials, no threat intrusion detection or prevention system in existence had found the attackers, who had been in systems since at least March.

In 2015, the federal government launched a “30-day cybersecurity sprint” after Chinese hackers pulled off an audacious cyber-attack, stealing detailed personal information on 22 million Americans from the U.S. Office of Personnel Management.

Tony Scott, who served as the U.S. chief information officer at the time and led the effort, said the cybersecurity flaws that were identified afterward were vast and included vulnerabilities in the digital supply chain. But Scott -- who currently leads a private sector security practice -- said that even some of the most basic protocols were missing, so his 2015 effort focused on such things as two-factor authentication, updating systems to include security patches and guarding who had privileged access to critical systems. “It was like patching a leaky roof. We plugged the holes on a temporary basis,” he said in an interview.

For Scott’s office, the sprint was intended as a first step to mitigate imminent danger, not a solution that could prevent cyber-attacks in the years to come. But, following the immediate aftermath of the Chinese hack, cybersecurity improvements lost their urgency in Congress and subsequent steps fell short, he said. For example, his office requested $3 billion in funding to replace old insecure government systems -- a “patchwork” system forming the “ultimate vulnerability” for U.S. cybersecurity -- but Congress has, to date, only appropriated a fraction of this amount, he said.

However, some improvements in cybersecurity in recent years have given members of the Solarium and GAO cause for cautious optimism. For example, CISA became its own agency within DHS in 2018 -- a reorganization that elevated the importance of cybersecurity within the U.S. government.

Another area of progress, officials and lawmakers say, is the 2021 National Defense Authorization Act, which gave CISA additional authority to test government networks for weaknesses and called for the establishment of a National Cyber Director within the executive branch to coordinate security across the government. These were key priorities for the Solarium, according to Mark Montgomery, the Solarium’s executive director, who is also a senior fellow at the Foundation for Defense of Democracies.

However, the 25 Solarium recommendations passed as part of the defense bill didn’t include its major proposals for upgrading digital supply chain security.

“The Solarium wanted to be the 9/11 Commission without the 9/11,” said Gallagher. “What SolarWinds reveals is that it’s September 10th in cyberspace, and we are vulnerable.”