High Availability HA Vyatta VyOS VRRP on ProfitBricks
Introduction
Configuring a high-availability configuration is key to keeping the business running. Of course, instead of just duplicating every single thing, you should perform some risk assessments to see where the critical services are and duplicate from there. One of the guaranteed points in infrastructure is your core routers or firewalls. All traffic is flowing through them and if it goes down, you're out since everything is connected to it.
This will be a pretty straight forward walk-through as VRRP is easy to setup and configure on VyOS.
The Future Layout
The infrastructure plan is as the following:
- 2x VyOS Router/FW
- 2x Nginx/HAProxy Load balancers
- 2x Web Servers
- 2x MySQL (master-master replication)
Prerequisites
You will need to have three (3) reserved IP addresses from ProfitBricks to make this work. One IP address will be the WAN adapter of VyOS-1, the second IP address will be the WAN adapter of VyOS-2, and the third IP address will be the virtual IP address (VIP) for both WAN adapters on VyOS-1 and VyOS-2. You can reserve this block of three IP addresses using the IP Manager in the top of the DCD once you have your data-center open.
I've already reserved the following IP addresses for the WAN adapters:
- 158.222.103.45 (VyOS-1)
- 158.222.103.48 (VyOS-2)
- 158.222.103.49 (VyOS WAN VIP)
Installation of VyOS
- Download the Vyatta VyOS ISO here: http://vyos.net/wiki/Main_Page
- Upload to the ProfitBricks FTP server according to your region: https://www.profitbricks.com/help/FTP_Access
- Add a CDROM device to the server in the DCD and choose the Vyatta VyOS ISO
- Mark the checkbox to indicate that the server should boot from the ISO volume.
- Provision Changes
The server will reboot and load up the Vyatta VyOS ISO Image. The default login username is "vyos" and the password by default is "vyos". Installation will require you access the system via the Remote Console as the system is not configured with any IP addresses or running services during this phase. The install is easy to finish and straight forward..
- Where do you want it installed? (VDA)
- What password do you want to set for "vyos" user?
- Etc..
Once the image is installed, just remove the image from the CDROM device in ProfitBricks and mark the HDD storage device as the boot device. Provisioning the changes will reboot the system and boot into the OS (VyOS).
WAN Configuration with VRRP
You will need to configure the WAN network adapter as well as the services (ssh) and gateways to use (.1).
- Log in at remote console
- Configure the WAN interface
$ configure # set interfaces ethernet eth0 address '158.222.103.45/24' # set interfaces ethernet eth0 description 'wan' # set interfaces ethernet eth0 hw-id '<MAC ADDRESS>' # set system gateway-address '158.222.103.1' # set system host-name 'vyos-1' # set system name-server '8.8.8.8' # set service ssh port 222 # commit # save # exit
- You should be able to ping 8.8.8.8 from the server now as well as Google.com or other domains. Vice-versa, you should also be able to now ping the IP address of the server.
- Once all of the above is confirmed, you should be able to SSH to the router now:
[David.Moros] ➤ ssh vyos@158.222.103.45 -p222 Warning: Permanently added '[158.222.103.45]:222' (RSA) to the list of known hosts. Welcome to VyOS Linux vyatta 3.13.11-1-amd64-vyos #1 SMP Wed Aug 12 02:08:05 UTC 2015 x86_64 Welcome to VyOS. This system is open-source software. The exact distribution terms for each module comprising the full system are described in the individual files in /usr/share/doc/*/copyright. Last login: Wed Apr 20 14:07:36 2016 vyos@vyos:~$
- SSH to VyOS-1 and configure VRRP for WAN (eth0):
$ configure # set interfaces ethernet eth0 vrrp vrrp-group 2 advertise-interval '1' # set interfaces ethernet eth0 vrrp vrrp-group 2 preempt 'true' # set interfaces ethernet eth0 vrrp vrrp-group 2 priority '100' # set interfaces ethernet eth0 vrrp vrrp-group 2 sync-group 'wansync' # set interfaces ethernet eth0 vrrp vrrp-group 2 virtual-address '158.222.103.49' # commit # save # exit
LAN Configuration, DHCP, and VRRP
For this example, we'll go with a 10.50.50.0/24 subnet.- Configure LAN:
$ configure # set interfaces ethernet eth1 address '10.50.50.1/24' # set interfaces ethernet eth1 description 'lan' # set interfaces ethernet eth1 hw-id '<MAC ADDRESS>' # commit # save
- Configure LAN DHCP Server:
# set service dhcp-server disabled 'false' # set service dhcp-server shared-network-name LAN1 authoritative 'disable' # set service dhcp-server shared-network-name LAN1 subnet 10.50.50.0/24 default-router '10.50.50.3' # set service dhcp-server shared-network-name LAN1 subnet 10.50.50.0/24 domain-name 'vylocal' # set service dhcp-server shared-network-name LAN1 subnet 10.50.50.0/24 lease '86400' # set service dhcp-server shared-network-name LAN1 subnet 10.50.50.0/24 start 10.50.50.100 stop '10.50.50.254' # commit # save # exit
- Configure VRRP for LAN:
$ configure # set interfaces ethernet eth1 vrrp vrrp-group 10 advertise-interval '1' # set interfaces ethernet eth1 vrrp vrrp-group 10 preempt 'true' # set interfaces ethernet eth1 vrrp vrrp-group 10 priority '100' # set interfaces ethernet eth1 vrrp vrrp-group 10 virtual-address '10.50.50.3/24' # set interfaces ethernet eth1 vrrp vrrp-group 10 sync-group 'lansync' # commit # save # exit
- Check VRRP Status with "show vrrp detail":
$ show vrrp detail Interface: eth0 -------------- Group: 2 ---------- State: MASTER Last transition: 5s Source Address: Priority: 100 Advertisement interval: 1 sec Authentication type: none Preempt: enabled Sync-group: wansync VIP count: 1 158.222.103.49/32 Interface: eth1 -------------- Group: 10 ---------- State: MASTER Last transition: 0s Source Address: Priority: 100 Advertisement interval: 1 sec Authentication type: none Preempt: enabled Sync-group: lansync VIP count: 1 10.50.50.3/24
Note: You may have noticed the DHCP server will be handing out a default-router address of '10.50.50.3'. 10.50.50.3 will be the VIP on the LAN side and will be shared between VyOS-1 and VyOS-2 so that there's redundancy for DHCP server, routing, etc.
Setup of VyOS-2 (the slave)
Per the same instructions as router one, set up the second VyOS router (VyOS-2). Be sure to make necessary changes to the IP addresses, MAC addresses (hw-id), and the priorities for the VRRP. Here's a configuration dump from VyOS-1 using "show configuration commands":
vyos@vyos-1:~$ show configuration commands set interfaces ethernet eth0 address '158.222.103.45/24' set interfaces ethernet eth0 description 'wan' set interfaces ethernet eth0 hw-id '02:01:14:e0:87:52' set interfaces ethernet eth0 vrrp vrrp-group 2 advertise-interval '1' set interfaces ethernet eth0 vrrp vrrp-group 2 preempt 'true' set interfaces ethernet eth0 vrrp vrrp-group 2 priority '100' set interfaces ethernet eth0 vrrp vrrp-group 2 sync-group 'wansync' set interfaces ethernet eth0 vrrp vrrp-group 2 virtual-address '158.222.103.49' set interfaces ethernet eth1 address '10.50.50.1/24' set interfaces ethernet eth1 description 'lan' set interfaces ethernet eth1 hw-id '02:01:1f:80:6f:84' set interfaces ethernet eth1 vrrp vrrp-group 10 advertise-interval '1' set interfaces ethernet eth1 vrrp vrrp-group 10 preempt 'true' set interfaces ethernet eth1 vrrp vrrp-group 10 priority '100' set interfaces ethernet eth1 vrrp vrrp-group 10 sync-group 'lansync' set interfaces ethernet eth1 vrrp vrrp-group 10 virtual-address '10.50.50.3/24' set interfaces loopback 'lo' set service dhcp-server disabled 'false' set service dhcp-server shared-network-name lan1 authoritative 'disable' set service dhcp-server shared-network-name lan1 subnet 10.50.50.0/24 default-router '10.50.50.3' set service dhcp-server shared-network-name lan1 subnet 10.50.50.0/24 domain-name 'vylocal' set service dhcp-server shared-network-name lan1 subnet 10.50.50.0/24 lease '86400' set service dhcp-server shared-network-name lan1 subnet 10.50.50.0/24 start 10.50.50.100 stop '10.50.50.254' set service ssh port '222' set system config-management commit-revisions '20' set system console device ttyS0 speed '9600' set system gateway-address '158.222.103.1' set system host-name 'vyos-1' set system name-server '8.8.8.8' set system ntp server '0.pool.ntp.org' set system ntp server '1.pool.ntp.org' set system ntp server '2.pool.ntp.org' set system package repository community components 'main' set system package repository community distribution 'helium' set system package repository community url 'http://packages.vyos.net/vyos' set system syslog global facility all level 'notice' set system syslog global facility protocols level 'debug'
Here's the edited version of what we will provide to VyOS-2 once we get the WAN setup so we can just SSH to the device and paste the new configuration in:
set interfaces ethernet eth0 address '158.222.103.48/24' set interfaces ethernet eth0 description 'wan' set interfaces ethernet eth0 hw-id '02:01:1a:95:4f:9d' set interfaces ethernet eth0 vrrp vrrp-group 2 advertise-interval '1' set interfaces ethernet eth0 vrrp vrrp-group 2 preempt 'true' set interfaces ethernet eth0 vrrp vrrp-group 2 priority '50' set interfaces ethernet eth0 vrrp vrrp-group 2 sync-group 'wansync' set interfaces ethernet eth0 vrrp vrrp-group 2 virtual-address '158.222.103.49' set interfaces ethernet eth1 address '10.50.50.2/24' set interfaces ethernet eth1 description 'lan' set interfaces ethernet eth1 hw-id '02:01:02:67:ee:45' set interfaces ethernet eth1 vrrp vrrp-group 10 advertise-interval '1' set interfaces ethernet eth1 vrrp vrrp-group 10 preempt 'true' set interfaces ethernet eth1 vrrp vrrp-group 10 priority '50' set interfaces ethernet eth1 vrrp vrrp-group 10 sync-group 'lansync' set interfaces ethernet eth1 vrrp vrrp-group 10 virtual-address '10.50.50.3/24' set interfaces loopback 'lo' set service dhcp-server disabled 'false' set service dhcp-server shared-network-name lan1 authoritative 'disable' set service dhcp-server shared-network-name lan1 subnet 10.50.50.0/24 default-router '10.50.50.3' set service dhcp-server shared-network-name lan1 subnet 10.50.50.0/24 domain-name 'vylocal' set service dhcp-server shared-network-name lan1 subnet 10.50.50.0/24 lease '86400' set service dhcp-server shared-network-name lan1 subnet 10.50.50.0/24 start 10.50.50.100 stop '10.50.50.254' set service ssh port '222' set system config-management commit-revisions '20' set system console device ttyS0 speed '9600' set system gateway-address '158.222.103.1' set system host-name 'vyos-2' set system name-server '8.8.8.8' set system ntp server '0.pool.ntp.org' set system ntp server '1.pool.ntp.org' set system ntp server '2.pool.ntp.org' set system package repository community components 'main' set system package repository community distribution 'helium' set system package repository community url 'http://packages.vyos.net/vyos' set system syslog global facility all level 'notice' set system syslog global facility protocols level 'debug'
Now, on VyOS-2, you can see that this is the VRRP backup slave, opposed to the MASTER that VyOS-1 is currently configured for:
Interface: eth0 -------------- Group: 2 ---------- State: BACKUP Last transition: 1m14s Master router: 158.222.103.45 Master priority: 100 Source Address: Priority: 50 Advertisement interval: 1 sec Authentication type: none Preempt: enabled Sync-group: wansync VIP count: 1 158.222.103.49/32 Interface: eth1 -------------- Group: 10 ---------- State: BACKUP Last transition: 1m14s Master router: 10.50.50.1 Master priority: 100 Source Address: Priority: 50 Advertisement interval: 1 sec Authentication type: none Preempt: enabled Sync-group: lansync VIP count: 1 10.50.50.3/24
- Add firewall rules to allow VRRP across LAN interfaces
VyOS-1: set firewall name internal rule 10 action 'accept' set firewall name internal rule 10 description 'Allow LAN' set firewall name internal rule 10 protocol 'vrrp' set firewall name internal rule 10 source address '10.50.50.2' VyOS-2: set firewall name internal rule 10 action 'accept' set firewall name internal rule 10 description 'Allow LAN' set firewall name internal rule 10 protocol 'vrrp' set firewall name internal rule 10 source address '10.50.50.1'
After all of this has been completed, try rebooting router-1 while watching tcpdump on router-2. Traffic should start flowing across router-2 instead and if you check "show vrrp detail" you should see that the backup has taken over. Once router-1 comes back online, router-2 will handover the master status back to router-1.
Hello. If I configure the master is that there will be replication of the data automatically on the slave?
No, you will have to configure the slave router independently. There is some third-party applications or software out there that can Synchronize configuration(s) amongst a number of VyOS routers (like users and other configurations) like this one: https://github.com/keshavdv/vyattta-config-sync
I have not personally used it but you might want to give that a try. Other than that, if you exit configuration mode and run "show configuration commands" you can basically copy/paste this into a fresh VyOS system to put an entire configuration in. You should be careful though and change a few things so that they are not conflicting.