HOW TO: Backup VMware ESX Virtual Machines with Doyenz

We've had many requests lately for an overview of how to backup VMware virtual machines to the Doyenz cloud. This article is a step-by-step guide for backing up VMware ESX and ESXi servers to Doyenz, and once you have started backing up to the Doyenz cloud you have many options for disaster recovery, disaster prevention and business continuity. 

Existing technologies for backup are tried and true but require a major upfront investment in dated technologies. While these technologies work, VMware adds new features that allow for faster recovery, and is agnostic to the server it is backing up. Existing solutions for backing up physical servers can be expensive due to the cost of licenses for different agents doing backup of Microsoft Exchange or SQL data. With Doyenz, you can backup Windows, Linux and anything running as a virtual machine for no additional cost.

Since VMware controls the stack and the underlying files of what consists of a server, VMDK files in particular, we can interact with them in new ways. ESX has the ability to take a ‘snapshot’, or to change how the data is being stored. When you take a snapshot, ESX separates from making changes to the VMDK files for a server, to writing those changes to a separate file, and keeping everything consistent to the virtual machine.

1. The Doyenz Agent works on ESX and ESXi servers, running as a virtual machine on each Host server. It’s a small Linux virtual machine (1GB) that we include for free as part of the service.

2. The agent works in conjunction with the vStorage APIs to interact with the VMFS filesystem.

3. As mentioned earlier, normally a VM will have a disk file on the VMFS that is the hard drive for the guest OS. In a traditional sense, as data is read and written to the drive in the guest OS, ESX makes these reads and writes to the root VMDK file, in this example, ‘guest.vmdk’.

4. When you add a machine to the Doyenz cloud, there is an initial process to upload the current server. We do this by telling the Hypervisor to take a snapshot of the machine, in this case ‘Guest’. This will take the 'guest.vmdk' file and make it read-only, and will create another file, 'Guest-001.vmdk' to capture all the writes that happen. The server ‘Guest’ is none the wiser to the changes.

5. Since the 'Guest.vmdk' file isn’t changing anymore, we upload it along with the other files that make up the server configuration to Doyenz Shadowcloud.

6. Since the server can be as big or small as you have configured, this process may take some time. You’ll notice that the 'Guest-001.vmdk' file will grow with the activity and writes that the server 'Guest' is doing as it operates. Once the initial upload of the 'Guest.vmdk' file completes, we wait for the next scheduled backup to happen. When that happens, every day we go through a number of steps. First, we tell ESX to take another snapshot, in this case the file ‘Guest-002.vmdk’ captures all the writes, and 'Guest-001.vmdk' is made read-only.

7. Now we can upload the previous snapshot ‘Guest-001.vmdk’ to Doyenz Shadowcloud.

8. Once the 'Guest-001.vmdk' is uploaded, we delete the snapshot, which commits the data that was in 'Guest-001.vmdk' into the root file, 'guest.vmdk'.

Now the agent waits until the next scheduled backup in the Shadowcloud, and repeats the process of taking a snapshot and uploading the previous one. 

As a Doyenz partner, you are able to offer your clients a reliable VMware backup solution for their Virtual Infrastructure without a huge upfront investment. You gain a consistent interface for your employees to manage physical and virtual offsite backup and disaster recovery services.

Stay tuned for the next installment in our "How To" series on validating these backups by booting your virtual servers in the cloud.