Disk Fragmentation Study Makes Annual Appearance

Sometimes it takes a while, years even, for a message to sink in. A couple of weeks ago, the market research company IDC issued a report about the high cost of disk fragmentation. The report is based on testing that the National Software Testing Laboratory (NSTL) conducted.

Disk fragmentation is one of those nasty little facts of computing that affect everything from desktop computing to larger-scale servers. Fragmentation is inevitable because of the way hard disks operate. When a user initially saves files to disk, the system saves the files in contiguous clusters. As additional files are saved, in general, they too are stored in contiguous clusters on the disk. When users delete files, the space they once consumed becomes available for new files. As this process continues, the disk becomes filled with discrete spaces once occupied by the various deleted files. At some point, no contiguous clusters large enough to save a specific file remain. Consequently, files to be saved are broken into pieces and stored in the discrete clusters that are available, which isn't necessarily a bad thing because the fragmentation process completely fills the empty disk space. Without fragmentation, disk space would be underutilized. In addition, many hard disks come from the factory with a certain percentage of bad clusters. The ability to divide a file into fragments for storage minimizes the impact of those unusable clusters.

However, disks can become highly fragmented. A study by American Business Research conducted in 2001 polled 100 large corporations and found that 56 percent of the Windows 2000 and Windows NT workstations in those companies contained files that were saved in fragments varying in number from 1050 to 8162 total fragments. The study showed that, in 25 percent of the files that were stored in fragments, the fragments totaled more than 10,000 per individual file. The figures showed an increase in fragmentation for servers: 33 percent of the servers in the study contained individual files that were stored in more than 10,000 fragments.

The performance of highly fragmented disks deteriorates significantly. Each fragment requires a separate I/O process because the read head must jump from track to track to reassemble a selected file. Most users intuitively recognize this phenomenon when programs take longer to boot and files load more slowly. But poor performance is only part of the problem. Disk fragmentation can cause system reliability problems and drain Help desk resources.

The good news is that performance increases after defragmentation operations can be impressive. In a test that used Windows XP and Microsoft Small Business Office XP, the NSTL found that Microsoft Outlook showed an increase in performance and ability to load and unload information and respond to specific actions ranging from 67.9 percent to as much as 176.1 percent after disk defragmentation. Microsoft Excel performance increased 83.7 percent. For Win2K and NT servers and workstations, defragmentation improved performance by 61.9 percent to as much as 219 percent. Such results have led IDC to conclude that effective disk defragmentation processes can help companies avoid some hardware upgrades.

The question is, how can an administrator conduct an accurate cost-benefit analysis to decide whether to invest time and resources in defragmentation? Some simple solutions to disk fragmentation are available. For example, virtually every computer with a bundled OS contains a defragmentation utility program. Unfortunately, most users don't bother to defragment their disks. Those who do often find that defragmentation can be a long, anxiety-inducing process. In any case, leaving enterprise systems' health in the hands of end users isn't good administrative practice. However, manually defragmenting disks is labor-intensive and time-consuming. Several products on the market can manage defragmentation at the network level, but a dispute has simmered between competing vendors as to the best way to compare products. As with many benchmarks, the conditions under which the testing of defragmentation products is conducted dramatically affect the results.

Even with the impressive performance improvements that disk defragmentation can provide, you need to carefully weigh the amount of time and money you want to invest in a defragmentation tool. Keep in mind that defragmentation is essentially a tuning process and not a panacea for system performance woes.