The Promise of Advanced Format Hard Drives

Back in the time of the first hard drives, back when wild mainframes still roamed the world, a standard sector size materialized into being. That size was 512 bytes. In a sector editor, that single sector looked like Figure 1. The 512-byte sector remained relatively unchanged until just a couple years ago, when some drives hit the marketplace with a new sector size of 4,096 bytes. These drives are affectionately referred to as 4K sector drives or simply 4K drives. The industry has chosen the term for this new media as advanced format. Here’s how these new drives are poised to fundamentally change the storage industry.

Figure 1: 512-byte sector 0 (zero) on physical drive 0 (zero) in Disk Probe

Why the Change?

There are a couple reasons for this increase in sector size. A minor reason would be because of drive size. Hard drives that are 2TB in size are becoming pretty common. And with a 512-byte sector size, we start to see some of the limitations of how large a single drive can be, using the old Master Boot Record (MBR) style of partitioning. But this isn’t a large concern, as the industry is moving more toward using GUID Partition Table (GPT) disks rather than MBR style disks. I authored a couple of blog entries that covered GPT versus MBR in detail: “Understanding the 2 TB Limit in Windows Storage” (blogs.technet.com/askcore/archive/2010/02/18/understanding-the-2-tb-limit-in-windows-storage.aspx) and “GPT in Windows” (blogs.technet.com/b/askcore/archive/2010/10/08/gpt-in-windows.aspx).

But, as I previously stated, drive size is a minor concern. The real concern is the juggling act between areal density, Error-Correcting Code (ECC), and the signal-to-noise ratio (SNR).

You should be able to see the juggling act now. If drive manufacturers increase areal density, SNR decreases, so more ECC is necessary to help maintain decent error rates, thus reducing the media’s space efficiency. Eventually we reach a point at which any gain made in areal density is almost completely lost to additional ECC.

Enter the 4K Drive!

The older 512-byte sector drives actually lose quite a bit of space just to storing ECC for each sector. So, the idea is that if we have larger sectors with better optimized ECC, the amount of space used in error correction would be greatly reduced, and at the same time a drive’s ability to detect errors in an individual sector would be improved. The returned space can then be used for file storage instead. By moving to drives with 4K sectors, we increase areal density. The ECC used for a single 4KB sector is significantly less than it would be for eight 512-byte sectors. And the less space used for ECC, the more space we have available for storing files!

Two Types of 4K Drives

There are two types of 4K sector drives. The first and most popular type is the 512-byte sector emulation drive (512e). Although under the hood it’s actually a 4K sector drive, it emulates a 512-byte sector drive by presenting two different sector sizes to the OS. It presents a 512-byte logical sector to use as the unit of addressing, and presents a 4K physical sector to use as the unit of atomicity.

The second type is the 4K Native drive, which is harder to find but has been seen with some USB-based hard drives over 2TB in size—probably because hardware vendors are holding off until the industry has had more time to prepare for them. No sense in providing the next new thing if only a few people can use it! The 4K Native drives don’t have this emulation, and both the logical and physical sector size is 4K.

There’s another way to check whether you have a 4K drive, but this method will discover only whether your drive is 4K Native or 512 Native. Run msinfo32 from Start/Run, and under Components select Storage, then Disks. You’ll see 4096 in the Bytes/Sector section that Figure 2 shows.

 
Figure 2: Checking whether you have a 4K drive

What About NTFS?

The new 4K sector size at the physical disk level actually lends itself well to the way NTFS works. The unit that NTFS works with is referred to as a cluster or allocation unit. The default cluster size for NTFS is already 4K. Most volumes out there are already using a cluster size that’s equal to the new hard disk sector size. The index blocks used by NTFS are 4K, as well.

Once the 4K Native drives start having more of a presence, NTFS will greatly benefit from the change. Currently, NTFS file records are only 1K in size. On 4K Native drives, these files records will also increase to 4K in size. This means that files with a greater complexity (fragmentation) will have a greatly reduced chance of hitting certain file system limitations, such as the fragmentation limit and practical file-size limitations for compressed and sparse files. You can find more information about the impact of advanced format drives on NTFS in the blog post “NTFS and 4K Disks” (blogs.msdn.com/b/ntdebugging/archive/2011/06/28/ntfs-and-4K-disks.aspx).

Supportability Issues

Although it’s truly an exciting time for those of us interested in storage, you can’t have a change of this magnitude without problems. Sector size has been the same for so long that the software industry has come to depend on it. Programmers have relied on a constant sector size for decades. From the Windows side, there are a number of gotchas we need to know about before making the jump to 4K drives. (Note that the hotfixes listed in this section apply to 512e drives only. There’s currently no down-level support for 4K Native disks on any released version of Windows.)

Windows XP/2003/2003 R2. Microsoft doesn’t support advanced format drives with deployments on Windows XP, Windows 2003, and Windows 2003 R2. So, support must come from the disk provider. The decision was made not to spend cycles on older versions that were released before the first rumblings of these new drives.

Windows Vista/Server 2008. There are a few issues that have already been identified and fixed in Windows Vista and Windows Server 2008. These fixes are included in a single rollup update, “A hotfix rollup that improves Windows Vista and Windows Server 2008 compatibility with Advanced Format disks” (support.microsoft.com/kb/2553708).

Windows 7/Server 2008 R2. Windows 7 and Server 2008 R2 also need to be patched to work best with advanced format drives. See “An update that improves the compatibility of Windows 7 and Windows Server 2008 R2 with Advanced Format Disks is available” (support.microsoft.com/kb/982018).

Hyper-V when hosting VHDs. Because of the 512-byte writes used with VHD files, reduced performance can result from hosting the VHD files on 512e drives. This is due to extra steps added when I/O is performed using a process called Read-Modify-Write. What happens is that for every 512-byte sector changed by the child system, the parent must read in the 4,096-byte sector that contains the 512-byte sector in question, make the change, then write the 4,096-byte sector back to disk. If the child needs to make 10 changes, this process will occur 10 times. In this situation, Hyper-V is completely incompatible with a 4K Native drives. See the Microsoft article “Using Hyper-V with large sector drives on Windows Server 2008 and Windows Server 2008 R2” (support.microsoft.com/kb/2515143).

Microsoft SQL Server. SQL Server will query the drive to find out sector size. If the drive is 512e, the response will be that the sectors are 512 bytes. So SQL Server will do all its I/O in 512 bytes. This results in the same Read-Modify-Write process being triggered. This is described in greater detail in the blog post “SQL Server - New Drives Use 4K Sector Size” (blogs.msdn.com/b/psssql/archive/2011/01/13/sql-server-new-drives-use-4K-sector-size.aspx).

Windows Home Server. Although the latest version of Windows Home Server supports 512e drives, the older version doesn’t. According to Microsoft, “It is important to note that some hard disk manufacturers are releasing advanced format drives in the same base models as traditional 512-byte sector drives. Because of this, it is critical that you make sure that the drive you are purchasing is not an advanced format drive if you are running Windows Home Server v1.” Unfortunately, it isn’t always easy to determine whether a drive is using advanced format until you’ve already purchased the drive. It’s often necessary to do some online research to make sure you know what you’re getting.

Microsoft Exchange Server. Exchange also has some specific rules about 4K drive use. For example, if you plan on using 4K drives, it is an “all or none” situation. Either all your databases are on 4K drives, or none of them are. Don’t mix and match. Also, the use of 4K Native drives isn’t supported. Only 512e drives are supported, and even then only with Exchange 2010 SP1 or better. You can find a full list of the rules for Exchange in the TechNet article “Understanding Storage Configuration: Exchange 2010 SP1” (technet.microsoft.com/en-us/library/ee832792.aspx#Phys).

Worried About Supportability?

Microsoft’s supportability stance is detailed in the article “Information about Microsoft support policy for large-sector drives in Windows” (support.microsoft.com/kb/2510009). The article not only outlines the supportability of 512e drives but also provides deeper information about some of the issues we’ve discovered as well as links to workarounds and fixes. As new issues are discovered, Microsoft will add them to this article.

Keep in mind that even if you’ve updated Windows to support 512e drives, you still have to be using hardware that supports their use. Otherwise you might not see the drive correctly. If the controller or enclosure you’re using with the drive fails to understand the new hardware, it will misreport the drive layout to Windows.

Repercussions

This is a huge change to the way we store data. And I don’t think we’re going to know everything it’s going to affect until the public starts using these drives. That being said, I’m fairly sure that it will affect things like data recovery, computer forensics, backup software, and disk utilities—as well as OSs. Doing a bit of research beforehand can help you determine whether the time is right to make the jump to advanced format drives.

Other resources that help explain these drives’ interaction with Microsoft products are “512-byte Emulation (512e) Disk Compatibility Update,” at msdn.microsoft.com/en-us/library/hh182553.aspx, and “Understanding the Impact of Large Sector Media for IT Pros,” at technet.microsoft.com/en-us/library/hh147334(WS.10).aspx. The latter article has some great quick-reference tables and an in-depth description of the Read-Modify-Write process that I mentioned earlier.