Getting to the Root of Slammer

Wow! It's been a while since my commentaries generated such a wide range of heated opinions. Two weeks ago, I slammed DBAs (pun intended) for failing to apply the hotfix that would have shut down the SQL Slammer worm ("After the Slammer,"). Last week, I apologized to DBAs for oversimplifying the Slammer situation and laying all the blame on their shoulders ("SQL Server DBAs Deserve an Apology," ). I also asked you, the readers, to share what you thought Microsoft could and should do to help us maintain secure systems. This week, let's look at a sampling of feedback about my Slammer columns and your suggestions about how Microsoft can help us avoid Slammer-like problems in the future.

After reading "SQL Server DBAs Deserve an Apology," some people still believe that DBAs shoulder most of the blame for SQL Slammer. The following reader noted, "Don't bow to the pressure. Your original assessment was right on the mark. The patch for MS02-039 was one file: SSNETLIB.DLL. Applying it meant copying one file and taking a 2-minute outage, depending on such things as database size. This crying about no installer is bull. People are afraid to apply patches and/or don't want to bother testing because either they don't know what to test or their application testing is an arduous manual process. These problems, however, are no excuse for not applying security patches within 6 months of their release."

Another reader wrote, "As a former military policeman, I remember one major lesson that has carried over to the civilian world: There is no room for political correctness if you must maintain real security. Security is not a gray area. [Your system] is either secure or it isn't."

Clearly, there's plenty of blame to go around when evaluating how the Slammer worm was able to attack and spread so quickly. However, SQL Slammer was simply a manifestation of the real problem: SQL Server professionals aren't applying hotfixes and service packs that fix known problems. Although many DBAs had valid reasons for not applying the patch, we can expect other SQL Slammer-like attacks unless we understand and solve the root problem. Why aren't SQL Server professionals applying hotfixes and service packs soon after their release, and how can we resolve these underlying problems? Here are some representative arguments and suggestions from the huge volume of reader mail I received.

A common theme among the feedback I received is that Microsoft needs to add SQL Server support to Windows Update and other crucial Microsoft notification services, especially because several Microsoft applications install Microsoft SQL Server Desktop Engine (MSDE) by default. As one reader pointed out, "I never thought about SQL Server service packs because I imagined that the automatic Windows Update service would recognize the need for them. [One way to make sure fixes are applied is to] ensure that the automatic Windows Update recognizes every Microsoft product on the machine and its need for service packs."

For years, Microsoft has rolled out SQL Server service packs, explaining in the readme.txt file that you can't uninstall the service pack. I received a deluge of comments saying that this practice must change. Despite the technical difficulties that Microsoft would face in changing this behavior for SQL Server, service packs and hotfixes must come with a reliable, easy-to-use uninstall feature, or people will continue to be wary of installing the latest and greatest patch. In the blunt words of one reader, "The onus is on Microsoft to provide all updates, service packs, hotfixes, and so on with an installer. The installers must log the changes and provide an uninstall capability. Period. Nothing less than this should ever be acceptable."

Another hurdle to applying patches comes from the sometimes out-of-synch mix of SQL Server and third-party products. Many readers use Commercial Off-the-Shelf (COTS) software, and their third-party vendors have strict guidelines about what versions of software they support. One reader reported, "I generally don't apply a service pack to SQL Server until I get the go-ahead from the third-party software company. It tests its software with the service pack before giving users the OK to run the new service pack. This could take 6 months or even longer." In this Catch-22 situation, DBAs must choose between rolling out a vital SQL Server patch or voiding the service policy from a vendor or even breaking the vendor's application. Microsoft can't work with every ISV to make sure their products support every patch. But Microsoft should at least ensure that top vendors, such as major ERP companies, have tested service packs in advance and will OK their installation.

Another frequent gripe is that DBAs can't afford the downtime required to install patches and service packs. As one reader noted, "Microsoft must realize that many systems run 24 x 7. The focus should be on how fixes, upgrades, and so on can be installed with almost no downtime—and without rebooting." Microsoft plans to address this problem in the upcoming Yukon release of SQL Server, but don't expect any progress on this roadblock until then.

The Slammer worm exposed a pervasive problem in the SQL Server community, and we won't solve the multiple, underlying causes of this problem overnight. However, the status quo is unacceptable. Microsoft, third-party software vendors, businesses, and IT staff must all find ways to make sure important patches are installed in a timely manner. The alternative is the likelihood of continuing Slammer-esque attacks.