How To Sync Two Apache Web Servers

Reading Time: 8 minutes

Load balancing and replicating multiple servers has a great array of benefits, though orchestrating and keeping them in sync can be very tricky. Here, we will walk through some of the load balancing options available, as well as setting up a very basic one-way replication sync between two or more servers behind a load balancer.

What is server replication?

Load balancing is a way to increase the processing power and redundancy of your web application by spreading the traffic among multiple different servers. Traffic is orchestrated by a load balancer and the web nodes are orchestrated by another data replication mechanism. That is to say, the load balancer itself has nothing to do with data replication; it only routes traffic to the web nodes. Something else is necessary to keep the web nodes’ data and configuration in sync, which is server replication.

There are a variety of methods for syncing files between web servers. These fall into four overlapping categories, which are synchronous and asynchronous, as well as one-way and two-way. Most synchronous sync types are two way, while asynchronous sync types could be one-way or two-way.

Synchronous sync types instantly share files between servers, via a shared storage node (such as a SAN or object store), and/or by coordinating a shared local or remote file system (OCFS2). These methods are complicated to configure, since beyond mounting the file systems at the same time, all servers must also communicate about when they are ready to write files and write to all locations at the same time. Liquid Web offers Managed Replication in our Enterprise Hosting offerings, removing the planning and maintenance burden from your shoulders so you can focus on your application.

Asynchronous sync types are simpler to set up but do not share files instantaneously. After files are completely written to one location, they are pushed out to another location by a service running on that server (lsyncd) or by a regularly timed sync cron. These are generally set up in one direction, so that a master server replicates out to slave servers.

What are the advantages and drawbacks of server replication?

As I mentioned, load balancing multiple servers serving the same set of data will increase the processing power behind your website, as well as introduce some redundancy. If one of your replicated nodes fails, the other (or others) can continue to serve traffic while the failed node is repaired or replaced. Load balanced systems can also be scaled easily; nodes can be added when more traffic is expected, and taken down if they are no longer needed.
But, with more servers comes more configuration. Keeping all of the server nodes in sync with each other requires additional applications to be set up and running, as well as adding additional hardware for load balancing. It is also a good idea to dedicate the web nodes to Apache only, so database information should be offloaded to a separate server as well. Further, though the software may be freely available, multiple servers and appliances cost more money to run than a single server, so load balanced clusters are inherently more expensive.

What are the requirements for setting up replication?

To set up server replication, you need two servers and one load balancer at bare minimum. But it is recommended to have a separate database server or cluster as well, to further increase redundancy.

You should also plan for the type of replication you want to use. Some replication types require additional hardware or configuration beyond what is covered here. If you are interested in those types of replication, chat with our architects about our Managed Replication products.

For the purposes of this article, we will use Liquid Web’s Cloud Load Balancer, along with two core-managed VPS web servers, and one core-managed VPS database server, connected to each other via the Cloud Private Network with private IPs. We will call the web nodes web01 and web02, and the database node db01. The web01 node is set up with password-less SSH keys into web02. All servers have one public IP.

We will also assume that Apache is set up on each web server for name-based virtual hosting and is installed at /etc/httpd/. PHP is also installed on both machines, and their configuration files are static and matched. Finally, our database server has mariadb installed and running, and the firewall is open for external mysql connections.

Note
Why aren’t we using cPanel? Account replication in cPanel requires rather a bit more replication work than we would be able to cover in this article, since cPanel account updates and creates also need to be synced. But, our Managed Replication services are built on cPanel servers, making your hosting management easier. We will use core-managed servers here for simplicity.

Step 1: Set Up Apache

In order to sync the Apache VirtualHost files, we will set up a single folder with single configuration files for each domain. In our main Apache config file at /etc/httpd/conf/httpd.conf, we will add the following line at the very end:

IncludeOptional vhosts/*.conf

This will allow Apache to load configuration files from the /etc/httpd/vhosts/ folder, if there are any, which we will make next. From the command line, run:

mkdir /etc/httpd/vhosts

Perform these steps on both web01 and web02.

In this folder, we can set up our individual domains’ configuration files. Make sure they end in .conf so that Apache will load them. In our example, we will make two files: domain.com.conf and domain.net.conf. These both will be set up with valid VirtualHost blocks, which we won’t get into the details of here. For this exercise, the docroots of the domains are at /var/www/domain.com/ and /var/www/domain.net/. We only need to make these configuration files on web01 for now, since we will sync them later.

Step 2: Set Up Databases

If your application requires it, set up databases on your dedicated database server, and add grants so that all of the web nodes can connect. For instance, after configuring the database user with a good strong password, you might run the following grant statement if your web nodes had private IPs of 192.168.0.11 and 192.168.0.12:

mysql -e “grant all privileges on your_db.* to your_user@’192.168.0.11’; grant all privileges on your_db.* to your_user@’192.168.0.12’”

To introduce elasticity, and if you are sure that all of your private IPs will have the same prefix, you might consider running this instead:

mysql -e “grant all privileges on your_db.* to your_user@’192.168.0.%’”

This has allowed all IPs that start with 192.168.0 to access this database, if they have valid credentials. Now, if you add another web node, it will also be able to access the database without additional grants being made.

We can now connect the web node to the database using its configuration file. On WordPress, for instance, this file is wp-config.php. Enter the appropriate connection credentials, using the private IP of the database server as the DB_HOST. We use the private IP so that we don’t waste public bandwidth on MySQL communication. Setting up the host connection to a hostname or IP like this will help ensure that the wp-config.php file will work on all your web servers the same way.

Note
Not using a database server? If you don’t have a separate node for databases, and are hosting them on web01, resist using ‘localhost’ in your configuration file! Once copied to your other web nodes, the connection won’t work. Use ‘web01’ or the exact private IP for web01 instead.

Step 3: Install and Configure LSyncD

For our asynchronous one-way replication, we will use Live Sync Daemon (lsyncd). This is a freely available daemon which can watch a folder for activity, and then replicate that activity with rsync in another local or remote location. We need to add the EPEL repository to install it via yum:

yum -y install epel-release
yum -y install lsyncd

Now that lsyncd is installed, we can configure it for each folder we want to sync to other nodes. In this case, we will be syncing the vhost directory we made earlier, as well as the docroots for each domain. In the /etc/lsyncd.conf file, delete the example sync command, and set up the following block of data:

sync {
default.rsyncssh,
source = "/var/www/domain.com",
host = "192.168.0.12",
targetdir = "/var/www/domain.com",
rsync = {
binary = "/usr/bin/rsync",
archive = true,
hard_links = true,
update = true
}
}

Notice the required commas after all but the last configuration lines. This is because the sync command can also be set up on one line. Make sure you don’t add a comma after the final element of any curly brace array.

Based off of this block, we can set up similar blocks for /var/www/domain.net and any other running sites. Just add them all one after the other in the /etc/lsyncd.conf file.

Lastly, enable and start lsyncd:

systemctl enable lsyncd
systemctl start lsyncd

Note
This final command should return no output, unless there was a problem. If there was, run systemctl status lsyncd and double check your config file syntax.

You can test that it’s working by checking out the contents of the directory on web02. Everything should be there! Make an update to a file, and check the target to see how long it takes to arrive. It should take just 5-10 seconds for lsyncd to pick up the change and copy the file.

Step 4: Set Up Apache Configuration Replication

All of the above is sufficient for servers that do not regularly change the number of domains they host. But, if you configure new domains or update your Apache configuration frequently, you need to set up a script to help you sync this and restart Apache on the other servers. We can call this from lsyncd as the rsync execution binary and have it run post-sync tasks. Create a file called /root/vhostsync.sh that looks like this:

#!/bin/bash
/usr/bin/rsync "$@"
[ $? -eq 0 ] && ssh 192.168.0.12 “systemctl reload httpd”

Exit your editor, then add execute permissions to the file:

chmod 700 /root/vhostsync.sh

This wrapper will perform the rsync task for you with the arguments passed by lsyncd, and if that is successful, connect to web02 by IP and reload the apache configuration. Now, head back into /etc/lsyncd.conf and add this sync block:

sync {
default.rsyncssh,
source = "/etc/httpd/vhosts",
host = "192.168.0.12",
targetdir = "/etc/httpd/",
rsync = {
binary = "/root/vhostsync.sh"
}
}

Since we declare our script as the rsync binary, this script will execute instead of running rsync by itself. Reload lsyncd to add this sync to the running config:

systemctl restart lsyncd

Once it reloads, lsyncd should copy the folder over to web02 and reload Apache. And, since we already set up the folder to be included, web02 should now also be serving the two virtualhost blocks we made for domain.com and domain.net.

Step 5: Route Traffic

That should just about do it! The final step is to test the configuration by connecting directly to web01 and web02 using hosts file modification, and make sure the content being served from both machines is the same. Also ensure that both servers can properly reach their database on db01.

Now, create the Cloud Load Balancer to route traffic to the IPs for web01 and web02. Use your hosts file again to connect to the load balancer’s VIP to ensure you can reach the nodes.
After testing, public DNS for the hosted domains can be changed to the VIP of the load balancer, allowing traffic to be routed into your cluster. Done!

Caveats

As you may have noticed, this only sets up replication from web01 to web02, not vice versa. Therefore, if you upload content or make changes to your website, you should always do so from web01. It is possible in some Load Balancer solutions to route traffic for your website’s administrative panel, such as /wp-admin, to only one of your nodes, i.e. web01. This will allow content creators to properly upload data only to the master web node for correct replication. This is not an option on the Cloud Load Balancer, though it is available in our Shared and Dedicated Load Balancer offerings.

If you decide to add a new domain to the servers, you will need to do all the following:

    • Create the document root on web01
    • Create any databases on db01 and set up SQL grants
    • Add a new virtualhost file to the /etc/httpd/vhosts/ folder
    • Add a new sync statement to /etc/lsyncd.conf for the new docroot
    • Restart lsyncd and apache on web01

Additionally, when adding or removing a server, new blocks will need to be set up manually inside of /etc/lsyncd.conf to sync to this new host too, as well as a new vhostsync wrapper for restarting Apache on those new nodes after configuration sync.

Note
None of this is an issue with our Managed Load Balancing solutions, which will be scaled as sites or servers are connected.

Finally, though this does provide some redundancy, your web01 server is still the master node, since it is running lsyncd. If web01 goes down, your website will still work, but you ought not add to or modify your sites until it can be repaired or replaced.

Do I Still Need Backups?

Absolutely!

Server replication is not a stand-in for server backups. If a hard disk crashes or your RAM goes bad, the other web node will still be able to stand up traffic. However, if your website is compromised, or you accidentally delete files, these changes will be replicated over to the other web node, removing your “backups”. Make sure you have local and full-server backups for disaster recovery.

If you are interested in setting up your own load balanced server replication, or would like more information about our managed products to help you with synchronous or asynchronous replication, chat with a hosting advisor today!

Install Rsync and Lsync on CentOS, Fedora or Red Hat

Reading Time: 4 minutes

Have you ever needed to copy files from your local computer over to your web server? You may have previously used File Transfer Protocol (FTP) applications for this task, but FTP is prone to being insecure and can be challenging to work with over the command line. What if there was a better way? In this tutorial, we’ll be covering two popular utilities in the Linux world to securely assist in file transfers, rsync and lsyncd. We’ll show you how to install and use both in this article. Let’s dig in!

Continue reading “Install Rsync and Lsync on CentOS, Fedora or Red Hat”

Setup a Development Environment in Ubuntu

Reading Time: 4 minutes

Often we want to edit our domain’s code, but on a production website, this can be dangerous. Making changes to the production site would not only allow all of the Internet to see unfinished changes but could also cause errors to display. As a workaround, we’ll create a testing domain or “dev” domain to work out any bugs and changes to the site.

As a warning, this is advanced technical work. It’s possible to make mistakes and cause downtime on your live domain. If you are not 100% confident, it may be a good idea to hire a system admin or developer to copy the domain for you.

Continue reading “Setup a Development Environment in Ubuntu”

How to Install lsyncd on Fedora 22

Reading Time: < 1 minute

lsyncd is an rsync-based tool that monitors specified directories (including subdirectories) for updates and modifications, then syncs those changes to a specified destination.

Pre-Flight Check

  • These instructions are intended specifically for installing the lsyncd on Fedora 22.
  • I’ll be working from a Liquid Web Self Managed Fedora 22 server, and I’ll be logged in as root.

Continue reading “How to Install lsyncd on Fedora 22”

How to Install lsyncd on Fedora 21

Reading Time: < 1 minute

lsyncd is an rsync-based tool that monitors specified directories (including subdirectories) for updates and modifications, then syncs those changes to a specified destination.

Pre-Flight Check

  • These instructions are intended specifically for installing the lsyncd on Fedora 21.
  • I’ll be working from a Liquid Web Self Managed Fedora 21 server, and I’ll be logged in as root.

Continue reading “How to Install lsyncd on Fedora 21”

How to Install lsyncd on Fedora 20

Reading Time: < 1 minute

lsyncd is an rsync-based tool that monitors specified directories (including subdirectories) for updates and modifications, then syncs those changes to a specified destination.

Pre-Flight Check
  • These instructions are intended specifically for installing the lsyncd on Fedora 20.
  • I’ll be working from a Liquid Web Core Managed Fedora 20 server, and I’ll be logged in as root.

Continue reading “How to Install lsyncd on Fedora 20”

How to Install lsyncd on CentOS 7

Reading Time: < 1 minute

lsyncd is an rsync-based tool that monitors specified directories (including subdirectories) for updates and modifications, then syncs those changes to a specified destination.

Pre-Flight Check
  • These instructions are intended specifically for installing the lsyncd on CentOS 7.
  • I’ll be working from a Liquid Web Core Managed CentOS 7 server, and I’ll be logged in as root.

Continue reading “How to Install lsyncd on CentOS 7”

How to Install lsyncd on CentOS 6

Reading Time: < 1 minute

lsyncd is an rsync-based tool that monitors specified directories (including subdirectories) for updates and modifications, then syncs those changes to a specified destination.

Pre-Flight Check
  • These instructions are intended specifically for installing the lsyncd on CentOS 6.
  • I’ll be working from a Liquid Web Core Managed CentOS 6 server, and I’ll be logged in as root.

Continue reading “How to Install lsyncd on CentOS 6”

How to Install lsyncd on Ubuntu 12.04 LTS

Reading Time: < 1 minute

lsyncd is an rsync-based tool that monitors specified directories (including subdirectories) for updates and modifications, then syncs those changes to a specified destination.

Pre-Flight Check
  • These instructions are intended specifically for installing the lsyncd on Ubuntu 12.04 LTS.
  • I’ll be working from a Liquid Web Core Managed Ubuntu 12.04 LTS server, and I’ll be logged in as root.

Continue reading “How to Install lsyncd on Ubuntu 12.04 LTS”