High Availability HA Vyatta VyOS VRRP on ProfitBricks

Introduction

Configuring a high-availability configuration is key to keeping the business running. Of course, instead of just duplicating every single thing, you should perform some risk assessments to see where the critical services are and duplicate from there. One of the guaranteed points in infrastructure is your core routers or firewalls. All traffic is flowing through them and if it goes down, you're out since everything is connected to it.

This will be a pretty straight forward walk-through as VRRP is easy to setup and configure on VyOS.

The Future Layout

The infrastructure plan is as the following:

  • 2x VyOS Router/FW
  • 2x Nginx/HAProxy Load balancers
  • 2x Web Servers
  • 2x MySQL (master-master replication)

vyos_vrrp_ha

Prerequisites

You will need to have three (3) reserved IP addresses from ProfitBricks to make this work. One IP address will be the WAN adapter of VyOS-1, the second IP address will be the WAN adapter of VyOS-2, and the third IP address will be the virtual IP address (VIP) for both WAN adapters on VyOS-1 and VyOS-2. You can reserve this block of three IP addresses using the IP Manager in the top of the DCD once you have your data-center open.

I've already reserved the following IP addresses for the WAN adapters:

  • 158.222.103.45 (VyOS-1)
  • 158.222.103.48 (VyOS-2)
  • 158.222.103.49 (VyOS WAN VIP)

Installation of VyOS

  1. Download the Vyatta VyOS ISO here: http://vyos.net/wiki/Main_Page
  2. Upload to the ProfitBricks FTP server according to your region: https://www.profitbricks.com/help/FTP_Access
  3. Add a CDROM device to the server in the DCD and choose the Vyatta VyOS ISO
  4. Mark the checkbox to indicate that the server should boot from the ISO volume.
  5. Provision Changes

The server will reboot and load up the Vyatta VyOS ISO Image. The default login username is "vyos" and the password by default is "vyos". Installation will require you access the system via the Remote Console as the system is not configured with any IP addresses or running services during this phase. The install is easy to finish and straight forward..

  • Where do you want it installed? (VDA)
  • What password do you want to set for "vyos" user?
  • Etc..

Once the image is installed, just remove the image from the CDROM device in ProfitBricks and mark the HDD storage device as the boot device. Provisioning the changes will reboot the system and boot into the OS (VyOS).

WAN Configuration with VRRP

You will need to configure the WAN network adapter as well as the services (ssh) and gateways to use (.1).

  1. Log in at remote console
  2. Configure the WAN interface
    $ configure
    # set interfaces ethernet eth0 address '158.222.103.45/24'
    # set interfaces ethernet eth0 description 'wan'
    # set interfaces ethernet eth0 hw-id '<MAC ADDRESS>'
    # set system gateway-address '158.222.103.1'
    # set system host-name 'vyos-1'
    # set system name-server '8.8.8.8'
    # set service ssh port 222
    # commit
    # save
    # exit
  3. You should be able to ping 8.8.8.8 from the server now as well as Google.com or other domains. Vice-versa, you should also be able to now ping the IP address of the server.
  4. Once all of the above is confirmed, you should be able to SSH to the router now:
    [David.Moros] ➤ ssh vyos@158.222.103.45 -p222
    Warning: Permanently added '[158.222.103.45]:222' (RSA) to the list of known hosts.
    Welcome to VyOS
    Linux vyatta 3.13.11-1-amd64-vyos #1 SMP Wed Aug 12 02:08:05 UTC 2015 x86_64
    Welcome to VyOS.
    This system is open-source software. The exact distribution terms for
    each module comprising the full system are described in the individual
    files in /usr/share/doc/*/copyright.
    Last login: Wed Apr 20 14:07:36 2016
    vyos@vyos:~$
    
  5. SSH to VyOS-1 and configure VRRP for WAN (eth0):
    $ configure
    # set interfaces ethernet eth0 vrrp vrrp-group 2 advertise-interval '1'
    # set interfaces ethernet eth0 vrrp vrrp-group 2 preempt 'true'
    # set interfaces ethernet eth0 vrrp vrrp-group 2 priority '100'
    # set interfaces ethernet eth0 vrrp vrrp-group 2 sync-group 'wansync'
    # set interfaces ethernet eth0 vrrp vrrp-group 2 virtual-address '158.222.103.49'
    # commit
    # save
    # exit
    

LAN Configuration, DHCP, and VRRP

For this example, we'll go with a 10.50.50.0/24 subnet.
  1. Configure LAN:
    $ configure
    # set interfaces ethernet eth1 address '10.50.50.1/24'
    # set interfaces ethernet eth1 description 'lan'
    # set interfaces ethernet eth1 hw-id '<MAC ADDRESS>'
    # commit
    # save
  2. Configure LAN DHCP Server:
    # set service dhcp-server disabled 'false'
    # set service dhcp-server shared-network-name LAN1 authoritative 'disable'
    # set service dhcp-server shared-network-name LAN1 subnet 10.50.50.0/24 default-router '10.50.50.3'
    # set service dhcp-server shared-network-name LAN1 subnet 10.50.50.0/24 domain-name 'vylocal'
    # set service dhcp-server shared-network-name LAN1 subnet 10.50.50.0/24 lease '86400'
    # set service dhcp-server shared-network-name LAN1 subnet 10.50.50.0/24 start 10.50.50.100 stop '10.50.50.254'
    # commit
    # save
    # exit
  3. Configure VRRP for LAN:
    $ configure
    # set interfaces ethernet eth1 vrrp vrrp-group 10 advertise-interval '1'
    # set interfaces ethernet eth1 vrrp vrrp-group 10 preempt 'true'
    # set interfaces ethernet eth1 vrrp vrrp-group 10 priority '100'
    # set interfaces ethernet eth1 vrrp vrrp-group 10 virtual-address '10.50.50.3/24'
    # set interfaces ethernet eth1 vrrp vrrp-group 10 sync-group 'lansync'
    # commit
    # save
    # exit
  4. Check VRRP Status with "show vrrp detail":
    $ show vrrp detail
    Interface: eth0
    --------------
      Group: 2
      ----------
      State:                        MASTER
      Last transition:              5s
    
      Source Address:
      Priority:                     100
      Advertisement interval:       1 sec
      Authentication type:          none
      Preempt:                      enabled
    
      Sync-group:                   wansync
    
      VIP count:                    1
        158.222.103.49/32
    
    Interface: eth1
    --------------
      Group: 10
      ----------
      State:                        MASTER
      Last transition:              0s
    
      Source Address:
      Priority:                     100
      Advertisement interval:       1 sec
      Authentication type:          none
      Preempt:                      enabled
    
      Sync-group:                   lansync
    
      VIP count:                    1
        10.50.50.3/24
    

Note: You may have noticed the DHCP server will be handing out a default-router address of '10.50.50.3'. 10.50.50.3 will be the VIP on the LAN side and will be shared between VyOS-1 and VyOS-2 so that there's redundancy for DHCP server, routing, etc.

Setup of VyOS-2 (the slave)

Per the same instructions as router one, set up the second VyOS router (VyOS-2). Be sure to make necessary changes to the IP addresses, MAC addresses (hw-id), and the priorities for the VRRP. Here's a configuration dump from VyOS-1 using "show configuration commands":

vyos@vyos-1:~$ show configuration commands

set interfaces ethernet eth0 address '158.222.103.45/24'
set interfaces ethernet eth0 description 'wan'
set interfaces ethernet eth0 hw-id '02:01:14:e0:87:52'
set interfaces ethernet eth0 vrrp vrrp-group 2 advertise-interval '1'
set interfaces ethernet eth0 vrrp vrrp-group 2 preempt 'true'
set interfaces ethernet eth0 vrrp vrrp-group 2 priority '100'
set interfaces ethernet eth0 vrrp vrrp-group 2 sync-group 'wansync'
set interfaces ethernet eth0 vrrp vrrp-group 2 virtual-address '158.222.103.49'
set interfaces ethernet eth1 address '10.50.50.1/24'
set interfaces ethernet eth1 description 'lan'
set interfaces ethernet eth1 hw-id '02:01:1f:80:6f:84'
set interfaces ethernet eth1 vrrp vrrp-group 10 advertise-interval '1'
set interfaces ethernet eth1 vrrp vrrp-group 10 preempt 'true'
set interfaces ethernet eth1 vrrp vrrp-group 10 priority '100'
set interfaces ethernet eth1 vrrp vrrp-group 10 sync-group 'lansync'
set interfaces ethernet eth1 vrrp vrrp-group 10 virtual-address '10.50.50.3/24'
set interfaces loopback 'lo'
set service dhcp-server disabled 'false'
set service dhcp-server shared-network-name lan1 authoritative 'disable'
set service dhcp-server shared-network-name lan1 subnet 10.50.50.0/24 default-router '10.50.50.3'
set service dhcp-server shared-network-name lan1 subnet 10.50.50.0/24 domain-name 'vylocal'
set service dhcp-server shared-network-name lan1 subnet 10.50.50.0/24 lease '86400'
set service dhcp-server shared-network-name lan1 subnet 10.50.50.0/24 start 10.50.50.100 stop '10.50.50.254'
set service ssh port '222'
set system config-management commit-revisions '20'
set system console device ttyS0 speed '9600'
set system gateway-address '158.222.103.1'
set system host-name 'vyos-1'
set system name-server '8.8.8.8'
set system ntp server '0.pool.ntp.org'
set system ntp server '1.pool.ntp.org'
set system ntp server '2.pool.ntp.org'
set system package repository community components 'main'
set system package repository community distribution 'helium'
set system package repository community url 'http://packages.vyos.net/vyos'
set system syslog global facility all level 'notice'
set system syslog global facility protocols level 'debug'

Here's the edited version of what we will provide to VyOS-2 once we get the WAN setup so we can just SSH to the device and paste the new configuration in:

set interfaces ethernet eth0 address '158.222.103.48/24'
set interfaces ethernet eth0 description 'wan'
set interfaces ethernet eth0 hw-id '02:01:1a:95:4f:9d'
set interfaces ethernet eth0 vrrp vrrp-group 2 advertise-interval '1'
set interfaces ethernet eth0 vrrp vrrp-group 2 preempt 'true'
set interfaces ethernet eth0 vrrp vrrp-group 2 priority '50'
set interfaces ethernet eth0 vrrp vrrp-group 2 sync-group 'wansync'
set interfaces ethernet eth0 vrrp vrrp-group 2 virtual-address '158.222.103.49'
set interfaces ethernet eth1 address '10.50.50.2/24'
set interfaces ethernet eth1 description 'lan'
set interfaces ethernet eth1 hw-id '02:01:02:67:ee:45'
set interfaces ethernet eth1 vrrp vrrp-group 10 advertise-interval '1'
set interfaces ethernet eth1 vrrp vrrp-group 10 preempt 'true'
set interfaces ethernet eth1 vrrp vrrp-group 10 priority '50'
set interfaces ethernet eth1 vrrp vrrp-group 10 sync-group 'lansync'
set interfaces ethernet eth1 vrrp vrrp-group 10 virtual-address '10.50.50.3/24'
set interfaces loopback 'lo'
set service dhcp-server disabled 'false'
set service dhcp-server shared-network-name lan1 authoritative 'disable'
set service dhcp-server shared-network-name lan1 subnet 10.50.50.0/24 default-router '10.50.50.3'
set service dhcp-server shared-network-name lan1 subnet 10.50.50.0/24 domain-name 'vylocal'
set service dhcp-server shared-network-name lan1 subnet 10.50.50.0/24 lease '86400'
set service dhcp-server shared-network-name lan1 subnet 10.50.50.0/24 start 10.50.50.100 stop '10.50.50.254'
set service ssh port '222'
set system config-management commit-revisions '20'
set system console device ttyS0 speed '9600'
set system gateway-address '158.222.103.1'
set system host-name 'vyos-2'
set system name-server '8.8.8.8'
set system ntp server '0.pool.ntp.org'
set system ntp server '1.pool.ntp.org'
set system ntp server '2.pool.ntp.org'
set system package repository community components 'main'
set system package repository community distribution 'helium'
set system package repository community url 'http://packages.vyos.net/vyos'
set system syslog global facility all level 'notice'
set system syslog global facility protocols level 'debug'

Now, on VyOS-2, you can see that this is the VRRP backup slave, opposed to the MASTER that VyOS-1 is currently configured for:

Interface: eth0
--------------
  Group: 2
  ----------
  State:                        BACKUP
  Last transition:              1m14s

  Master router:                158.222.103.45
  Master priority:              100

  Source Address:
  Priority:                     50
  Advertisement interval:       1 sec
  Authentication type:          none
  Preempt:                      enabled

  Sync-group:                   wansync

  VIP count:                    1
    158.222.103.49/32

Interface: eth1
--------------
  Group: 10
  ----------
  State:                        BACKUP
  Last transition:              1m14s

  Master router:                10.50.50.1
  Master priority:              100

  Source Address:
  Priority:                     50
  Advertisement interval:       1 sec
  Authentication type:          none
  Preempt:                      enabled

  Sync-group:                   lansync

  VIP count:                    1
    10.50.50.3/24
  • Add firewall rules to allow VRRP across LAN interfaces
    VyOS-1:
    
    set firewall name internal rule 10 action 'accept'
    set firewall name internal rule 10 description 'Allow LAN'
    set firewall name internal rule 10 protocol 'vrrp'
    set firewall name internal rule 10 source address '10.50.50.2'
    
    VyOS-2:
    
    set firewall name internal rule 10 action 'accept'
    set firewall name internal rule 10 description 'Allow LAN'
    set firewall name internal rule 10 protocol 'vrrp'
    set firewall name internal rule 10 source address '10.50.50.1'

After all of this has been completed, try rebooting router-1 while watching tcpdump on router-2. Traffic should start flowing across router-2 instead and if you check "show vrrp detail" you should see that the backup has taken over. Once router-1 comes back online, router-2 will handover the master status back to router-1.

You may also like...

2 Responses

  1. frederic says:

    Hello. If I configure the master is that there will be replication of the data automatically on the slave?

    • David says:

      No, you will have to configure the slave router independently. There is some third-party applications or software out there that can Synchronize configuration(s) amongst a number of VyOS routers (like users and other configurations) like this one: https://github.com/keshavdv/vyattta-config-sync

      I have not personally used it but you might want to give that a try. Other than that, if you exit configuration mode and run "show configuration commands" you can basically copy/paste this into a fresh VyOS system to put an entire configuration in. You should be careful though and change a few things so that they are not conflicting.

Leave a Reply

Your email address will not be published. Required fields are marked *