Building your own private cloud with XenServer

Here is an interesting project in which we had to design and build a strong, reliable, redundant, and cost-efficient private cloud. We used XCP-NG as a XenServer-based HyperVisor, and opensource technologies for the VM(s) running inside. BareMetal servers were rented from NovoServe which were able to provide both cost-effective bare metal servers and a strong redundant network.

Building the HyperVisor

Each Hypervisor is connected to a bonded x10G network (two different switches), so we can enjoy a 20G link, and will failover to a 10G link in case one of the switches goes down.

We used local SSD SATA disks, for local storage (separate disks from the OS hypervisor), these can be also upgraded to NVMe disks for even more performance.

Hypervisor1 will be our primary server, which will take all the traffic, Hypervisor2 will be our secondary server which will be in a “standby” mode.

We used to the following methodology for a one Magento store site –

1. PfSense HA firewall to maintain and manage our public IPs & ACL

2. Linux based keepalived service for floating IPs inside our WEB / LB VMs

3. Linux based lsyncd to sync configuration files/data between servers

4. Nginx as Load balancer – config is synced 1:1 from primary to secondary, IP main IP will be transformed between primary and secondary using keepalived

5. LiteSpeed for best performance webserver – config is synced 1:1 from primary to secondary, IP main IP will be transformed between primary and secondary using keepalived

6. MySQL Primary / Secondary

Using the diagram below, you see that all the traffic is directed towards hypervisor1, which hypervisor2 is always synced and on “standby” mode. In case Hypervisor1 goes down, keepalived will move the primary IP from all VMS running inside hypervisor1 into the VMS inside hypervisor2 making him now the primary, this happens LIVE, in most cases without even losing a single ping.

Planning ahead for disaster, all the data is backed up twice, once from inside the VMs using JetBackup, and once as a full VM export using Xen-Orchestra, both BTW are backing up to Wasabi S3 offsite destination so we have full control and option to recover in case of a major disaster (full site is down).

Tags: JetBackup, litespeed, nginx, xenserver