Hetzner Failover Webservice/Script KVM/qemu/libvirt

Failover IPs via Webservice

https://robot.your-server.de/doc/webservice/de.html#failover

https://wiki.hetzner.de/index.php/Failover_Skript

Nice online Json formatter: https://jsonformatter.curiousconcept.com/

Setup

• https://robot.your-server.de/preferences
  • Webservice Settings -> enter password for webservice user
  • You get the username via email

Webservice Commands

• curl -s -u "username:password" https://robot-ws.your-server.de/failover

• [
     {
        "failover":{
           "ip":"111.111.111.111",               #failover ip
           "netmask":"255.255.255.255",
           "server_ip":"222.222.222.222",        #base server
           "server_number":123456,
           "active_server_ip":"333.333.333.333"  #currently routed to
        }
     },
     {
        "failover":{
           "ip":"444.444.444.444",
           "netmask":"255.255.255.255",
           "server_ip":"222.222.222.222",
           "server_number":123456,
           "active_server_ip":"222.222.222.222"
        }
     }
  ]

Status for failover IP 111.111.111.111

• curl -s -u "username:password" https://robot-ws.your-server.de/failover/111.111.111.111

• { 
        "failover":{
           "ip":"111.111.111.111",               #failover ip
           "netmask":"255.255.255.255",
           "server_ip":"222.222.222.222",        #base server
           "server_number":123456,
           "active_server_ip":"333.333.333.333"  #currently routed to
        }
  }

• curl -s -u "username:password" https://robot-ws.your-server.de/failover/111.111.111.111 -d active_server_ip=222.222.222.222

• { 
        "failover":{
           "ip":"111.111.111.111",               #failover ip
           "netmask":"255.255.255.255",
           "server_ip":"222.222.222.222",        #base server
           "server_number":123456,
           "active_server_ip":"222.222.222.222"  #currently routed to
        }
  }

Template for a KVM/qemu/libvirt hook script

This is based on the following network configuration for Hetzner Failover IPs:

Hypervisor:

• vi /etc/network/interfaces
  

  • auto eth0
      #hetzner default config
    
    # Bridge for KVM  - nice: VMs need no network config change and work on any hypervisor
    auto br0
    iface br0 inet static
      address 192.168.1.1
      netmask 255.255.255.255
      bridge_ports none
      bridge_stp off
      bridge_fd 0
      bridge_maxwait 1
      pre-up brctl addbr br0
      # Hetzner Failover IPs - this is now handled by hook script
      #up ip route add 111.111.111.111 via 192.168.1.1

• virsh edit myserver.example.com

  • ...
        <interface type='bridge'>
          <source bridge='br0'/>
          <model type='virtio'/>
        </interface>
    ...

Virtual machine:

• vi /etc/network/interfaces

  • iface eth0 inet static
            address 111.111.111.111 #hetzner failover ip
            netmask 255.255.255.255
            dns-nameservers 213.133.100.100 213.133.98.98
            post-up ip route add 192.168.1.1 dev eth0
            post-up ip route add default via 192.168.1.1 dev eth0

Hook script

• apt-get install jq
• vi /etc/libvirt/hooks/qemu

  • #!/bin/bash
    # kvm hetzner vm config script
    # add and remove routing
    # switch failover ip if necessary
    # 2018-01-29 klemens@ull.at
    
    # log script output
    exec 1>>/var/log/qemu-hooks.log 2>&1
    
    # Exit script immediately if a command exits with a nonzero exit value
    set -e
    
    # Activate debugging - echo on, display commands, outputs to stderr!
    #set -x
    
    echo "########################################################"
    date
    
    echo "vm: ${1}"
    echo "phase: ${2}"
    echo "begin/end: ${3}"
    
    LOCAL_IP=`ifconfig eth0 | grep -Eo 'inet (addr:)?([0-9]*\.){3}[0-9]*' | grep -Eo '([0-9]*\.){3}[0-9]*'`
    echo "local ip: $LOCAL_IP"
    
    if [ "$1" == "myserver.example.com" ]; then
            VM_IP="111.111.111.111"
            echo "myserver.example.com: $VM_IP"
    
            if [ "$2" == "start" ]; then
                    echo "adding local route"
                    ip route add $VM_IP via 192.168.1.1
    
                    ACTIVE_SERVER_IP=`curl -s -u "username:password" https://robot-ws.your-server.de/failover/$VM_IP | jq -r '.failover.active_server_ip'`
                    if [ "$ACTIVE_SERVER_IP" != "$LOCAL_IP" ]; then
                            echo "changing hetzner failover ip"
                            curl -s -u "username:password" https://robot-ws.your-server.de/failover/$VM_IP -d active_server_ip=$LOCAL_IP
                    fi
    
            elif [ $2 == "stopped" ]; then
                    echo "deleting local route"
                    ip route del $VM_IP
            fi
    fi

• chmod 700 /etc/libvirt/hooks/qemu
• ?necessary? service qemu-kvm restart
• service libvirt-bin restart
• cat /var/log/qemu-hooks.log