File Allocation Tables: An Old But Still Useful Technology

From the very beginning, NTFS has been my choice for file system partitioning because it’s secure, reliable, and self-healing. In short, NTFS provides a rock-solid foundation for data storage. However, NTFS can’t compete with FAT in one area: speed. FAT has been always faster than NTFS. FAT has much less overhead because it’s not secure, reliable, or self-healing.

Systems administrators who love solid, reliable performance prefer to use NTFS instead of FAT. However, FAT can be invaluable in applications in which speed is the overriding consideration, such as in IP TV applications. In these applications, TV signals from satellites are encoded to digital data. This data is typically fed to Windows servers, which distribute it to secondary servers. The secondary servers, in turn, pass the data to clients.

Video data consists of many tiny files whose size is usually only several kilobytes. These tiny files transfer from disk to disk, so they’re constantly saved and deleted. Because they’re volatile (i.e., not permanently stored), data security, reliability, and the ability to self-heal aren’t primary concerns. The main concern is performance: You must distribute the files as quickly as possible.

In an IP TV project in which I took part, hard disk performance had created a bottleneck. Although we used speedy Serial Attached SCSI (SAS) disks, the hard disk performance degraded over time and eventually crashed the application. When we observed the hard disk performance in Performance Monitor, the Avg. Disk Queue Length counter was constantly over 2, which meant the hard disk couldn’t cope with the requests.

To solve this problem, I made two suggestions:

We decided to try these changes, so we used individual disks without RAID and used FAT32 to partition them. The results turned out to be better than we had hoped. There is no hard disk bottleneck now. The Avg. Disk Queue Length counter has been constantly under 1 and often close to 0. No special action is required to control fragmentation because the files are very small and cleaned out periodically.

My suggestions were in compliance with the Keep It Simple, Stupid (KISS) principle. In general, simplicity is best.