NTFS5 vs. FAT32

Get the inside facts about Windows 2000's newest file systems

When Windows NT 4.0 was a fledgling OS, I wrote "NTFS vs. FAT," October 1996, which generated a great deal of reader feedback. In that article, I dissected NT 4.0's two primary file systems. I have a sense of déjà vu as I examine Microsoft's latest version of NT—Windows 2000. Win2K's file systems—NTFS 5.0 (NTFS5) and FAT32—are updated versions of their NT 4.0 counterparts. Here's the lowdown about these additions to the NT file-system family and tips about how and when to use each file system. Here, too, is information about how to avoid some common file-system-related pitfalls when you use Win2K.

Win2K File-System Facts
Compatibility with legacy file systems is an important aspect of OS upgrades. Win2K supports several file systems, and its support is essentially a superset of NT 4.0's support. (For more information about file systems, see "File-System Resources," page 104.) Win2K still supports FAT12, FAT16, and the CD-ROM File System (CDFS), which is the International Organization for Standardization (ISO)-9660-compliant NT file system you use to read CD-ROMs. However, Microsoft doesn't plan to add features or upgrade these systems, except for a possible future addition of support for the UNIX-centric Rock Ridge format in CDFS. NTFS, which is the premier Win2K file system, continues to exist but sports a new file-system structure and new capabilities. Microsoft also included in Win2K a Universal Disk Format (UDF) file system that the company introduced in Windows 98. Currently, Win2K's UDF version, which complies with the ISO-13346 standard, mainly supports DVD-ROM media. However, Microsoft plans to eventually replace the CD-ROM-centric CDFS specification with UDF. Win2K also supports a slightly more recent version of UDF (i.e., UDF 1.50) than Win98 supports (i.e., UDF 1.02).

The Optical Storage Technology Association (OSTA) developed the UDF standard and has released UDF 2.0. UDF 2.0 adds several new features, including support for long and Unicode filenames, ACLs, power calibration, and stream files. Although the UDF specification supports media-write operations (e.g., CD Recordable—CD-R, CD-Rewritable—CD-RW, DVD-RAM), Win2K's UDF 1.50 provides a read-only implementation. Microsoft has promised to provide an updated UDF 2.0-compliant driver for Win2K in the near future, although the company hasn't specified whether the driver will include write support. Until then, independent software vendors (ISVs) will use customized UDF drivers or proprietary packet-writing software to provide Win2K with the UDF 2.0 capabilities that UDF 1.50 lacks.

First Stop—NTFS5
Microsoft added several new features to Win2K, such as Active Directory (AD); advanced storage management features such as disk quotas, the Encrypting File System (EFS), and Hierarchical Storage Management (HSM); and the application deployment capabilities that Group Policy Objects (GPOs) and IntelliMirror provide. These features are part of Win2K's appeal, but Microsoft needed to update NTFS to NTFS5 to support them. Win2K supports only NTFS5 and automatically converts disk volumes with previous versions of NTFS to the new format during Win2K setup and as Win2K mounts the volumes. This automatic-conversion behavior has implications for multiboot systems running more than one version of NT. (For more information about automatic conversion, see the sidebar "Automatic NTFS5 Conversions: What You Need to Know.") NTFS5's new features and capabilities support Win2K's disk quotas, file encryption, reparse points, directory junctions, volume mount points, sparse files, and the change journal.

Disk quotas. The NTFS specification has contained meta data structures to support user disk quotas for some time, but Win2K with NTFS5 is the first NT version that can natively use these disk-quota structures. Disk-quota management is available in Win2K on a per-user, per-volume basis (i.e., you can set different quotas for several users on every volume). User SIDs identify file ownership and thus disk-quota usage on volumes. You store disk-quota information on the actual volumes rather than as a separate database. This feature makes NTFS more efficient and flexible when you're using applications such as clustering products and Storage Area Networks (SANs).

Administrators use disk quotas to control how much disk space users can consume on local and network-based storage volumes. Win2K shows the user the remaining disk quota on a volume as the total free space left on that volume, rather than as the volume's actual capacity. Therefore, users don't see the actual volume capacity and don't question why their disk-quota limit is set so low when free space is available on the server's hard disk. Also, applications running on the user's system don't detect the free-space information and won't create temporary or cache files whose sizes are a function of the amount of available disk space. This feature is significant because when an application creates a temporary or cache file, the application might allocate more space to the file if the application believes it has rights to more disk space than the system will let the user access.

Besides the quota support NTFS5 provides at the file-system level, Microsoft added an open-quota management API, which vendors can access. Like the disk-defragmentation API that debuted in NT 4.0, Win2K's disk-quota API lets ISVs extend Win2K's quota-management capabilities. This functionality benefits organizations that might find Win2K's built-in quota management insufficient. To enable and manage disk quotas in Win2K, you select the Quota tab from the Properties dialog box of any NTFS disk volume, as Screen 1 shows.

File encryption. Previously, you needed to use third-party utilities to encrypt stored data on NT systems. However, Win2K adds an important storage-management and security feature—EFS. New NTFS5 and Win2K features let you use a public-key security scheme to encrypt files, folders, or volumes to support EFS. When a user requests encryption, EFS uses the file encryption key (FEK) to encrypt each target file. The user's key encrypts the FEK, which creates the Data Decryption Field (DDF). You can also use a specially designated security agent's key, known as the Recovery Agent, to separately encrypt the FEK to create the Data Recovery Field (DRF). The Recovery Agent, which an IT or corporate manager typically holds, can decrypt and retrieve encrypted data just as a user can. This key lets organizations prevent users from encrypting data to the point at which the company can't retrieve the data. As with quota management, you can access Win2K's file-encryption feature, which Screen 2 shows, through the Properties dialog box of any NTFS-based file, folder, or volume.

Reparse points. NTFS5 supports an important new Win2K feature—reparse points. Win2K and Win2K programs use reparse points to trap operations on objects within an NTFS structure and run program code before returning file data to the user or calling application. Microsoft introduced this open method in Win2K to extend file-system features and support.

Directory junctions. Directory junctions are NTFS directories that Win2K associates with a special type of reparse point. These reparse points let you configure a particular NTFS directory to point to another NTFS directory—even one on a different volume, as long as that volume is on the same system. For example, you might want to map a common shared folder on the same server into several users' home directories so that users can access this directory without changing to a different drive letter. You can use a directory junction to link the common folder (e.g., common), which might exist in a different file-system namespace area, to a subdirectory under each user's home directory. Users then have a common subdirectory under their individual home directory (e.g., D:usersjimcommon, D:usersbobbycommon) that lets them access a common shared folder. Directory junctions therefore let you link logical file-system namespaces to volume roots (root directories) or subdirectories on a local system's volumes. This ability to create a unified file-system namespace that contains resources from disparate locations is similar to how Dfs works with network server file-share resources. (For more information about the differences between these two technologies, see the sidebar "Directory Junctions vs. Dfs.") Directory junctions also let you build hybrid storage volumes that use a mix of storage classes (e.g., RAID 1, RAID 5, non-fault-tolerant).

Volume mount points. Volume mount points are file-system objects that use reparse points to let you map an NTFS5 folder to an entire volume (i.e., only to an entire volume, unlike directory junctions). For example, you can take an NTFS disk volume (I'll call it the K drive, although drive letters aren't technically necessary) and mount it to a folder named data on the J drive (i.e., a separate NTFS volume). The J:data contents are now the K drive's contents. Volume mount points let users and administrators extend a volume's capacity without migrating data or repartitioning. As do directory junctions, volume mount points provide additional file-system namespace flexibility and let you build hybrid volumes containing several storage classes.

Sparse files. NTFS5 supports sparse files, which are files that typically contain large consecutive 0-bit areas. You can mark particular files as sparse files to ensure that the NTFS file system allocates space for only meaningful data within these files. NTFS stores only range information that describes where the file system will locate sparse data and doesn't waste space storing this data bit by bit. Sparse files therefore improve storage efficiency for files on NTFS5 volumes that contain sparse data and for applications that use the files.

Change journal. One problem with large file volumes is that operations that need to analyze changes to files (such as a backup program that analyzes file date stamps and timestamps to determine which files you need to back up) put an enormous load on the server's disk subsystem. Luckily, Win2K provides a new feature called the change journal that alleviates this problem. The change journal is a volume-specific log that details all file changes on that volume. To keep the change journal's size in check, Microsoft designed the log file to be circular, which means that the change journal eventually overwrites old log data with new data (each log entry is approximately 80 bytes). The change journal logs operational changes such as modifications and deletions. However, these log entries reference only general operations to the files, not the data in those files. The major benefit that the change journal provides is reducing the work that applications, such as the Indexing Service and File Replication System, that reference this type of information need to do. The change journal paves the way for ISVs to write more efficient Win2K applications and utilities, which can significantly reduce server disk I/O and thus improve overall system performance. However, Win2K turns the change journal off by default on an NTFS5 volume. As a result, the application or user must enable the feature to use it.

FAT32—Finally
Before Microsoft released Win2K, only Win98 users were able to enjoy technologies such as multiple-monitor support, DirectX 7.0, DVD, and Universal Serial Bus (USB) devices. When you used another OS, you needed to maintain dual-boot systems to boot into Win9x to play a favorite game or download pictures from a USB digital camera. Similarly, NT users weren't able to take advantage of the FAT32 file system that Win98 and Win95 OEM Service Release (OSR) 2.0 use. (The read-only FAT32 for the NT 4.0 driver and its read/write Winternals Software counterpart are two exceptions to this restriction.) However, Win2K provides full read/write support for FAT32 volumes to remove these earlier constraints. To realize the benefits of Win2K's FAT32 support, you need to understand the FAT32 file system and its features.

Microsoft introduced FAT32 in Win95 OSR 2.0 and designed the software as the successor file system to FAT (aka FAT16). FAT32 improves on its venerable predecessor in several areas. For starters, FAT32 is substantially more efficient in how it uses disk space because FAT32 supports much smaller cluster sizes than FAT supports on comparable volume sizes. A cluster, or allocation unit, is the minimum unit of disk storage you use to write file data to a volume; you determine the cluster size at the time you format a volume. All stored files on a volume, regardless of their actual size, must end on an even-cluster boundary. This constraint, in conjunction with the requirement that even the tiniest (e.g., 1KB ) files must consume at least one cluster's worth of disk space, results in a large amount of wasted space on FAT volumes.

Although FAT32 has larger cluster sizes and therefore minimizes wasted disk space better than FAT does, NTFS is better still. Table 1 presents information about FAT's cluster usage for several volume-size ranges, and Table 2 gives FAT32's cluster usage per volume-size range. Table 3 provides information about NTFS5 cluster sizes for several volume-size ranges.

Microsoft also created FAT32 improvements that relate to fault tolerance. FAT32 provides several features to ensure that crucial on-disk data structures are available. For example, FAT32 can relocate the root directory to another location on the disk if you damage the portion of the disk that housed the original location. FAT32 can also use a backup copy of its namesake allocation table if the primary copy is damaged or unreadable. Finally, FAT32 provides an expanded boot record that houses backup copies of critical-volume data structures. These levels of redundancy make FAT32 much more fault-resilient than its FAT cousin, although still not quite as robust as NTFS, which includes transactional logging features. However, for those systems in which you choose to deploy FAT32 volumes, your data is safer than it previously was with comparable FAT volumes.

Although Win2K provides general support for the FAT32 file system, this support has its limitations. For example, although Win9x OSR 2.0 can create FAT32 volumes as large as a theoretical maximum of 2TB (the practical limitation is 127.53GB), Win2K can't create FAT32 volumes greater than 32GB. Microsoft has stated that it imposed this limitation because the company wants to steer Win2K customers away from FAT32 and toward NTFS for volumes. I don't agree with Microsoft's approach to restricting FAT32's usage on large volumes, but NTFS will provide better performance and stability with volumes this large. The good news is that you can use existing Win9x volumes that exceed the 32GB limit in Win2K (i.e., Win2K can inherit volumes that are larger than 32GB, but it can't create them). Another FAT32-related limitation, and a major Microsoft oversight, is that you can't convert a FAT volume to FAT32 in Win2K. You can convert FAT and FAT32 volumes to NTFS, but Microsoft doesn't provide a FAT32 conversion tool for existing FAT volumes. Unless Microsoft decides to offer this tool in the Microsoft Windows NT Server 4.0 Resource Kit or a future service pack, you'll need to use Win9x's tools (or a third-party utility) to make this conversion.

Making the Choice
Now that you understand the NTFS5 and FAT32 features and the differences between these file systems, you must decide when and where to use them. Although the title of this article seems to pit the two file systems against each other, the truth is that they're complementary. The appropriate choice for a particular volume is fairly straightforward after you consider the intended use of that volume. First, don't ever use FAT32 on a Win2K server because FAT32 doesn't provide the security essential for server disk volumes. Second, only the NTFS5 file system can support the majority of Win2K's advanced OS features, such as AD and Remote Installation Services (RISs). In many cases, you don't need to guess which file system to use because various Win2K dialog boxes inform you that a particular feature requires an NTFS volume.

One somewhat dated argument for using FAT (or FAT32, in the case of Win2K) on server volumes relates to the boot partition. Before Win2K's release, many administrators used FAT on boot partitions because a DOS or Win9x boot disk can easily access and recover FAT volumes in the event of disaster. However, the addition of the Win2K Recovery Console (RC) invalidates this argument. The RC is a special alternative boot selection that you can install on a Win2K system. (To install this option, run winnt32/cmdcons from the Win2K CD-ROM.) You can use the RC to carry out several recovery-related operations on NTFS volumes, such as file copying and renaming. Now that Win2K includes the RC, your best choice of file systems for all Win2K server volumes is NTFS.

Although I generally recommend you use NTFS, you might want to use another file system in certain cases. For example, suppose you maintain a multibooting system that contains another OS (e.g., Win9x, Linux, OS/2, DOS) that requires access to a particular volume. On that volume, you need to use a file system that represents a common denominator between the two OSs. (For more information about useful utilities for multiboot systems, see the sidebar "Fantastic File-System Utilities.") Another valid use for FAT partitions is on system partitions that third-party boot-manager applications use. Several of these utilities require you to install them into a very small FAT volume on the first hard disk. Neither FAT32 (as a result of its 512MB-minimum-size requirement) nor NTFS (because of its high overhead on small volumes and because no third-party boot managers support installation on NTFS volumes) is appropriate, so FAT is the only viable choice.

Table 4 shows you the best file system to choose for your volumes. You can select from several excellent third-party Winternals Software utilities, including NTFSDOS, FAT32 for Windows NT 4.0, and NTFS for Win98, to help you mitigate some of these multi-OS accessibility problems.

Windows 2000's New File Systems Are Winners
Win2K's NTFS5 and FAT32 help the OS achieve a new level of performance, compatibility, and manageability. NTFS5 also provides the backbone for many of Win2K's best new features. After you understand these file systems' features, internal capabilities, and limitations, and learn which file system to use for certain scenarios, you can better plan and manage your Win2K disk volumes.