Backing Up Virtual Server Guests

Creating backups of virtual server guests creates some challenges for virtualization in a production environment. If you’re running an enterprise-class virtualization product such as VMware’s ESX Server with the Infrastructure bundle, you have some options like VMware Consolidated Backup or Vizioncore’s ESXRanger that let you back up a virtual server guest while it’s still running. However, if you’re running one of the free virtualization products, such as Microsoft Virtual Server 2005 or VMware Server, you must shut down the virtual server guest to get a clean image backup of the virtual server guest files stored on the host. Shutting down the virtual server host can be problematic, especially if the server must run 24x7. In light of this problem, here’s a backup solution I developed to back up some of my clients' virtual server guests that are running in a production environment. It uses a combination of traditional backup methods along with a backup of the virtual server files that are stored on the host. This gives clients a lot of restore/recovery flexibility in case they have to restore individual files to the virtual server guest, perform a full backup, or move the virtual server guest to a different host in the event of a host server failure.

Backup of Virtual Server Files on the Host Server Over the weekend, I run a script that shuts down the virtual server guest(s) running on the host. I install a backup agent on the host server and use VERITAS Software's Backup Exec to perform a full backup on the host. With the virtual server guest shut down, I can get a clean backup of all the virtual server guest files. This allows me to perform an image restore to a different host server if the original host crashes. In this example, the virtual server guest is called vs1 and the virtual server configuration file is called vs1.vmx. All of the virtual server guest files reside on the d: drive in Virtual Machines Folder. The batch file that stops the virtual server guest(s) contains this line:

  call vmware-cmd "d:Virtual Machinesvs1vs1.vmx" stop trysoft  

This batch file calls the vmware-cmd batch file that shuts down the virtual server guest based on the configuration stored on the vs1.vmx file.

The batch file assumes that you’ve added the folder where the virtual server vmware-cmd.bat file resides to the path environment variable on the host server; otherwise you will have to enter the full path name to the vmware-cmd.bat file. If you have multiple virtual server guests running on the host, you could add additional lines to the batch file to shut them down as well. If you want to minimize the downtime for each virtual server guest and have extra disk space, you could create a batch file that shuts down the virtual server guest, copies all the virtual server guest files to a different folder or server, then starts up the virtual server guest. You could then move the next virtual server guest until the entire virtual server guest files are copied to a different folder or server. Then you can back up these files by using your regular backup software. After the virtual server guest files are backed up, you can restart the guest with a second batch file that contains this line:

  call vmware-cmd "D:Virtual Machinesvs1vs1.vmx" start  

This command will start the virtual server called vs1. You can schedule these batch files using the Windows Scheduler utility, which is included with Windows Server 2003. The scheduler tool, combined with a scheduled backup in Backup Exec, allows you to automate this backup process.

Full and Differential Backups using a Backup Agent On each virtual server guest, I installed a backup agent, just like on a physical server. On Monday through Wednesday, I run a differential backup on the virtual server guest using Backup Exec. On Thursday night, I use Backup Exec to run a full backup. After a week of backups, I have the following files:

1. Image files of all of your virtual server guests. These files are from the backup that runs over the weekend. This image backup of the virtual server guest files allows you to move the virtual server guests to a different host in the event of a host failure or major disaster. You can even prestage the images by restoring the images onto a different virtual server host for disaster recovery purposes.

2. Three differential backups that were run on Monday through Wednesday. These differential backups allow you to restore files that were modified since the last backup and prevent you from having to restore multiple incremental restore jobs. The differentials let you restore the server as of Wednesday night with one full restore and one differential restore.

3. One full backup of each server that was run on Thursday. A full backup is necessary using a “traditional” backup method to reset the archive bit that the differential backups use to determine which files have been modified. If you don't run a full backup, the differential backup would eventually grow quite large. Alternatively, you could use the modified date of the file, but in my experience the archive bit is more reliable.

If a host server crashes, you would run the image restore job and the latest differential backup (or full backup depending on when the server crashed) to fully recover the virtual server guest. This backup approach gives you the flexibility of being able to move the virtual server guests to a different host and retain the ability to restore a single modified file, without having to restore the entire server.