Real-World Scripting: Data Migration with Robocopy, Part 1

Sometimes writing a script entails more than writing and testing code that performs a certain task. When a script is to perform a major IT undertaking such as data migration, you need to write a script that not only performs the task but also minimizes associated risks and downtime.

For example, my company had a file server with a shared 250GB RAID 5 array that was quickly filling up with users' files, despite ongoing efforts to remove old and obsolete files. To meet the increasing memory needs, the company purchased new disks. The IT server team installed the new disks and configured them into a 500GB RAID 5 array, which the team connected to the same file server.

I had to write a script that migrated the six top-level folders while retaining the complicated NTFS file and folder permissions. I worked with the server team that maintains the server to determine the script's requirements. We determined that the script needed to

RobocopyDataMigration.bat is the script I created. You can find RobocopyDataMigration.bat in the Code Library on the Windows Scripting Solutions Web site (http://www.winscriptingsolutions.com). This month and next, I'll describe how the script works and how you can adapt it for your environment.

Copying the Data
To copy the file and folder structure, RobocopyDataMigration.bat uses Robocopy 1.96. You can find Robocopy 1.96 in the Microsoft Windows 2000 Resource Kit or the Microsoft Windows NT Server 4.0 Resource Kit, Supplement 4. The pathname for the resource kit's folder depends on which version you install. RobocopyDataMigration.bat uses the default NT pathname of C:treskit. The default pathname for the Win2K resource kit is C:program filesresource kit. If you install the Win2K resource kit, though, I recommend you install it in a folder with no spaces in the name, similar to the NT default location. Using a name with spaces in a batch command can create problems when the command executes. In the past, many administrators used the workaround of enclosing such commands in double quotes. In Win2K, this workaround produces inconsistent results. Furthermore, nesting a command enclosed in double quotes within a For command can produce inconsistent results in both Win2K and NT.

Robocopy is a powerful and useful utility because of its many options, or switches. To customize RobocopyDataMigration.bat to your environment, you need to learn about these switches. To obtain a short description of them, you can run the command

Robocopy /???

at the command line to view the online Help file. (The Help switch depends on the Robocopy version. Robocopy 1.96 uses the /??? switch, but earlier versions might use a different switch.) The 22-page document robocopy.doc explains each switch in detail. (Although robocopy.doc might show 1.95 as the version number, the installed version is actually 1.96.)

When you use a utility with this many switches, I recommend that you include the Help file's switch information in your script. To capture Robocopy's Help file output, run the command

Robocopy.bat /???> C:temprobohelp.txt

Then, open robohelp.txt, copy the applicable text, and paste that text at the beginning or end of your script. You need to use the Rem command or a double colon (::) to comment out the added text. Including switch information in a script lets you quickly and easily adapt the script for different applications. (The script RobocopyDataMigration.bat already includes the switch information.)

To copy the data and maintain the NTFS permissions, RobocopyDataMigration.bat uses the Robocopy command syntax that Listing 1 shows. In this syntax, E:test is the directory you're copying files from (i.e., the source folder) and F:test is the directory you're copying files to (i.e., the destination folder). The various switches in the command in Listing 1 serve different functions:

You need to customize the switches in RobocopyDataMigration.bat to suit your environment. For example, if you plan to copy thousands of files and hundreds of directories, you can temporarily modify the script to run the Robocopy command with the /create switch. This switch creates the destination directory structure with no files and minimizes the fragmentation of the directories on the destination disk. (For more information about the /create switch, see robocopy.doc.) If you decide you want the functionality of both the /e and /purge switches, an alternative is to use the /mir switch. Specifying the /mir switch is equivalent to specifying the /e and /purge switches.

Creating a Log
Having a script record its results in a log is always a good idea because you then have a record to review. I recommend that you create a separate log each time you run the script. If you continually append the results to one file, the file is hard to read because the script appends the new data to the bottom of the log. I also recommend that the filename include the date and time you ran the script. This practice prevents you from inadvertently overwriting existing logs with new data. To include the date and time in the filename, you need to first capture this information. RobocopyDataMigration.bat uses the resource kits' Now utility to capture the current date and time. Some characters in date and time strings might be illegal characters in a filename. For example, the colon (:), forward slash (/), and backslash () are illegal filename characters. Thus, you need to replace any illegal filename characters with legal filename characters. To do so, you can use the syntax that Listing 1 shows. In this syntax, OldChar is the character to replace, NewChar is the replacement character, OldVariable is the variable that holds the text needing replacement, and NewVariable holds the text that results after replacement. RobocopyDataMigration.bat uses this syntax to replace colons with hyphens. The script also uses this syntax to replace spaces with hyphens to improve the log filename's readability.

Next Month
Next month, I'll continue my discussion of how RobocopyDataMigration.bat works. I'll show you how to verify the copy operation results, determine how long the system will need to be offline, and permit graceful fallback. I'll also show you how to adapt the script for your environment.