Currently we are building a fairly rock solid high availability cluster for a client. This has the "usual" ingredients: two locations, two NetApps, two clusters of three vmware ESX servers and a bunch of virtual machines running on top of the ESX servers. Also included in the mix is a VDI (now called View) virtual desktop infrastructure for running virtual windows XP clients.
This is all managed by SRM (site recovery manager) and it is almost working. But that is another story.
What got me thinking is the following.
Last week I did a
consultancy job where they had a build a fail over cluster using
DRBD. With DRBD you have a disk device
which is transparently replicated. The device file can be
used like any other,
mount all work as
Now throw KVM into the mix...
The virtual machine images must be stored on the DRBD device. Suppose
we have two servers called master and slave. On master the
kvm processes run. In a failover situation the following needs
- If master is stil available, kill all kvm processes;
- if master is still available, set the DRBD device in secondary mode or disable it all together;
- On slave make the DRBD device primary (so that it will become
rwmode. If you don't do this you get Wrong medium type errors;
- On slave start the kvm processes again.
It would even be cooler if the virtual machines could actually be copied over while still are running, but I don't know if that would be possible.
Shared storage would be possible by letting one virtual machine export (via NFS/SaMBa/iSCSI) another DRBD device.
So my site recovery manager script (SRM script) will be something along the lines of this:
#!/bin/bash # when doing a fail over call it on the old site # (if still available): srm stop # the other side call it like: srm start case $1 in stop) /etc/init.d/kvm stop drbdadm /dev/drbd/0 secondary ;; start) drbdadm /dev/drbd/0 primary -o /etc/init.d/kvm start ;; esac exit 0
Is it really that simple?
This setup will be more complex, but it will also give you al lot of flexibility.
Fancy stuff, like restarting the servers in a particular order, can all be handled in the kvm start/stop script.
It's just a thought...
KVMwill need a writeable file just to startup, so on the slave
KVMcan not even run.
Ofcourse having the two sides both in slave mode is still a problem. A split slave situation? ;-)