Using Ntdsutil to Defrag AD

If you run a Windows 2000 network, you most likely use Active Directory. AD, like most other directory services, uses a database—in AD's case, a version of the Microsoft Extensible Storage Engine (ESE)—to store information. AD is the heart and soul of Win2K, so you need to take extra care to ensure its health. Periodically using the Ntdsutil utility (ntdsutil.exe) to perform an offline defragmentation can help you achieve this goal.

Get to Know AD and Your Defragmentation Options
Before you begin working with Ntdsutil, get to know the key AD database files—edb.log, ntds.dit, res1.log, res2.log, and edb.chk—all of which reside in %systemroot%tds on a domain controller (DC) by default. During AD installation, Dcpromo lets you specify alternative locations for these log files and database files, as Figure 1, page 76, shows. The best practice for a Win2K DC is to install the OS files, transaction logs, and database files on separate spindles. The most common approach is to install the files on separate mirrored drives (i.e., RAID 1).

Any change to AD triggers a write operation, which causes ESE to write all changes to both the database file and the log file. AD periodically flushes the log file as the OS commits sets of database updates. System performance determines how fast the system writes data. At process termination, AD rolls back any open transactions. During AD installation, Win2K creates the blank log files res1.log and res2.log, each of which has an initial size of 10MB. Should the system run out of disk space to store changes, AD will rename res1.log and res2.log and begin using those files.

Two types of defragmentation—online and offline—take place. By default, online defragmentation automatically happens every 12 hours as part of AD's garbage-collection process. This process runs independently on each DC, releasing free space within the AD database file. (For more information about the process, see the Microsoft article "The Active Directory Database Garbage Collection Process" at http://support.microsoft.com/?kbid=198793.) The good news about online defragmentation is that it's automatic and the DC stays online. The bad news is that online defragmentation doesn't reduce the total size of ntds.dit; it only reclaims free space from within the file. To reduce the total size of the database on disk, you must perform an offline defragmentation, which compacts ntds.dit. The easiest way to determine when to run an offline defragmentation is to turn up garbage-collection logging to level 1. Doing so causes the directory service agent (DSA) to log an event that reports the database size and the estimated amount of internal free space.

You might never need to run an offline defragmentation of AD under typical operations. However, after a major object move or a migration to Win2K, such an exercise can prove beneficial. An example of an event that might prompt an offline defragmentation is using a third-party product that creates duplicate accounts in AD, merges those placeholder accounts with the original accounts, then deletes the placeholder accounts later during a cleanup process. This type of addition and deletion can fragment AD internally, but you can use Ntdsutil to defrag and compact the database and regain needed disk space. (Note that if under typical conditions a system volume's free disk space is so small that compacting the database will significantly increase the free space, Ntdsutil might not be able to solve the problem.)

Keep in mind that because DCs only replicate changes, performing an offline defragmentation on ntds.dit on one DC won't affect ntds.dit on other DCs. Hence, you must manually perform an offline defragmentation on each DC. Because of AD's design, ntds.dit's size will differ on each DC.

Running Ntdsutil
The Microsoft article "Performing Offline Defragmentation of the Active Directory Database," (http://support.microsoft.com/?kbid=232122) explains the offline defragmentation process. I've summarized the process, adding a few steps as a safety measure to ensure that any mistakes you make won't be catastrophic.

After completing an offline defragmentation, immediately back up the DC or at least perform a system state backup. After you've completed this backup and you're sure that AD is working properly, you can delete the temporary backup copy of ntds.dit that you created in Step 3.

Testing the Process
To determine how much disk space Ntdsutil could reclaim, I built a standalone test server running Win2K Advanced Server Service Pack 3 (SP3) in an isolated network. After running Dcpromo and installing AD, I wrote a Visual Basic (VB) application that added 25,000 users and 25,000 groups to AD. After the addition of these 50,000 objects, ntds.dit grew from 10.256MB to 221.1MB. I ran Ntdsutil, which decreased ntds.dit to 184.6MB. I then deleted the 50,000 objects.

When you delete an object in AD, the object isn't truly deleted but is simply marked for deletion at a later date. This process is called tombstoning, and the default tombstone-lifetime setting is 30 days. Objects are tombstoned rather than immediately deleted to ensure that the deletion has sufficient time to replicate throughout the forest. The garbage-collection process scans for and permanently deletes any object with an expired tombstone lifetime. To speed up the deletion process, I used ADSI Edit (a Win2K Support Tool, which you can install from the Win2K CD-ROM's supporttools directory) to set the garbage-collection attribute (garbageCollPeriod) to 1 hour, advanced the date on my test server ahead by 30 days, and rebooted. (Note that in a production environment, you should never advance the system time to force deletion of tombstoned objects. Doing so can result in problems with replication, backups, and Kerberos ticket lifetimes.)

I let the server run overnight so that AD would perform a regular online defragmentation. The next morning, I ran Ntdsutil again. The database file shrank back to the default size of 10.256MB.

Keeping AD Healthy
Given all the day-to-day tasks that occur in AD (e.g., adding users, deleting users, modifying group membership), you can see how AD can become fragmented within a short period. Proper monitoring and preventive maintenance of AD can help ensure that AD functions at an optimum level and is important to the overall health of your Win2K network. Win2K doesn't offer any built-in tools to perform these crucial tasks natively, but several third-party products, such as NetIQ's AppManager Suite or NetPro's DirectoryAnalyzer, are available. For more information about monitoring and maintaining AD and the tools you can use to do so, see "Best Systems Management Products," September 15, 2002, http://www.winnetmag.com, InstantDoc ID 26316; "Practice Proactive AD Maintenance," August 2002, InstantDoc ID 25637; "DirectoryAnalyzer 1.04," November 2000, InstantDoc ID 15725; and "Monitoring Your AD-Enabled Network," September 2000, InstantDoc ID 9645.

Corrections to this Article: