Robocopy Over the Network

In "Add Robocopy to Your Toolbelt," September 2005, I introduced you to one of the Microsoft Windows Server 2003 Resource Kit's most useful and powerful utilities. This time, I want to dig a little deeper into Robocopy and show you some more of its capabilities—specifically, in regards to networking.

In September, I left off with an example wherein I wanted to copy files from a share named \markswsmyfiles to one named \centralservermarksfiles. Taking the vicissitudes of networking into account, however, I wanted Robocopy to retry its copies at least 13 times in the event of a network failure, and to wait 30 seconds between retries. That command looked like

robocopy \markswsmyfiles \centralservermarksfiles /w:30 /r:13

This command handles the sometimes-unreliable nature of networks, but I'd also like to handle another aspect of networking: bandwidth.

If the myfiles share is large, I might chew up a fair amount of my network's bandwidth. Robocopy lets me ameliorate its effect on the network in three ways—by letting me specify the time range in which it will run, the method with which it retries copies, and the length of time it waits between sending blocks of data.

When to Run
The /rh option lets me specify when a given copy operation may run. The option's parameters are /rh:hhmm-hhmm, where the hhmm variables let you specify the range of acceptable run times in military hours and minutes. For example, I could initiate a Robocopy command at any hour of the day but have the utility wait until 11pm to actually start—while also telling Robocopy to suspend its operation if it's still copying at 5:30am. To do so, I would run the command

robocopy \markswsmyfiles \centralservermarksfiles /w:30 /r:13 /rh:2300-0530

Suppose that this Robocopy job requires 10 hours. I type the Robocopy command at noon on Monday, but Robocopy waits until 11pm Monday to start the job. By Tuesday at 5:30am, the job wouldn't be complete, but it would stop copying nonetheless, waiting until 11 pm Tuesday to resume its task. (Clearly, if the myfiles share changes a lot from day to day, the job might never finish!)

Retrying from Where?
Another bandwidth-gobbler is file-copy restarts. Suppose I've asked Robocopy to copy a 2GB PST file across the network, and the copy job fails after copying 1.9GB. When Robocopy retries the copy, does it start from the file's beginning or does it pick up where the copy failed? By default, Robocopy starts from scratch, but adding the /z option tells Robocopy to restart a failed copy where it left off.

Be careful with the /z option. Robocopy can restart copies in midstream only by keeping close track of their progress, and that imposes overhead. If you're copying a lot of small files, the /z option might cost more than it's worth. But using the /z option to tell Robocopy to invest time tracking that PST file copy would pay dividends in time that's not wasted on from-scratch restarts. Unfortunately, you can't tell Robocopy to keep track of copies of files larger than a certain size, but we might see that functionality in a later version of the tool.

Waiting Between Blocks
Finally, with the /ipg option, Robocopy lets me reduce the amount of network bandwidth the tool can consume. Like all copy routines, Robocopy copies data in 64KB blocks. Typically, Robocopy just sends one block after another, with no pause between the blocks. Adding the /ipg:n option, however, tells Robocopy to wait n milliseconds between blocks. But what value do you use? Well, 64KB is about half a million bits. Therefore, over a 100Mb Ethernet connection, that 64KB file copy would take about ¹/₂₀₀ of a second—5 milliseconds.

Five milliseconds is a theoretical best-case figure. In a busy real-world network with normal software overhead, you might expect no more than 20 percent of that speed, making the actual time perhaps 25 milliseconds. Assuming, then, that Robocopy can effectively transfer only one block every 25 milliseconds, setting /ipg:25 would cut Robocopy's network usage by 50 percent. Setting /ipg:50 would cut Robocopy's network usage by 66 percent, and setting /ipg:75 would reduce network usage by 75 percent of its unencumbered speed. You get the idea.

Network Usability
Robocopy's designers understood that, in the real world, their utility would be doing most of its work across networks. The /rh, /z, and /ipg options reflect that usability and greatly improve Robocopy's usefulness.