Configuring multiple Galera cluster nodes in a single machine

What I love about Galera cluster is it's simplicity, SST/IST protocol and the native MySQL feel.

SST(state transfer protocol) is nothing but a mechanism used by Galera to transfer data between the nodes. Galera supports 3 types of SST’s

               1)      mysqldump

               2)      rsync

               3)      xtrabackup( Xtrabackup should be installed separately)

Xtrabackup is the most preferred one with less downtime since it locks only when the schema is fetched.

There are lots of superb features in Galera that we could talk about, but we shall skip that and concentrate on what the topic is about

Practically speaking I wouldn't recommend running multiple Galera nodes in a single machine. But you might run into a situation where you want to create a development environment and you have a single machine. The procedure that I am going to explain here might be handy in such situations.

Every Galera reference manual instructs us to go for a minimum of 3 nodes even if it is possible with 2 nodes. So we are also going for a 3 node setup in a single machine.

This procedure might not be useful for those who do not have any basic idea on galera cluster. I consider that you already have a basic idea and working mechanism of galera.

The first thing to understand while setting up multiple nodes in a single machine is that each galera node needs two different ports that are not used by any other process/daemon.

              1)      MySQL port ( default 3306) should be differenct

              2)      Port for State transfer SST/IST( default 4567) should be different

              3)      My.cnf(/etc/my.cnf) files should be different

              4)      Data directories should be different

              5)      Sockets must be different(/tmp/mysql.sock)

I am listing down the details of the 3 nodes in the image below

Now download binary tar ball of either Percona XtraDB Cluster or Mariadb Galera cluster (Both has galera cluster libraries included). I used Percona XtraDB Cluster 5.6(PXC) whose download link is shared below

http://www.percona.com/redir/downloads/Percona-XtraDB-Cluster-56/LATEST/binary/tarball/Percona-XtraDB-Cluster-5.6.20-rel68.0-25.7.886.Linux.x86_64.tar.gz

The first step is to install 3 separate MySQL instances with different cnf’s, ports (3310, 3311, and 3312), data directory and sockets. Once you are done with installing and configuring the 3 MySQL instances, keep them stopped.

      1)   Now follow the below steps to configure cluster in Node1.

Edit the cnf of Node1 and add the below changes

[root@cluster1 ~]# vim /etc/my_3310.cnf

datadir=/data/new/gal1
basedir=/usr/local/maria_cluster/
wsrep_provider=/usr/lib64/galera/libgalera_smm.so
query_cache_size=0
binlog_format=row
default_storage_engine=innodb
innodb_autoinc_lock_mode=2
wsrep_cluster_address=gcomm:/10.0.0.20:4568,10.0.0.20:4569
wsrep_cluster_name=cluster1
wsrep_node_name=node1
wsrep_sst_method=rsync
wsrep_node_address=10.0.0.20:4567   

Now start the cluster Node1 using the below command

[root@cluster1 ~]# mysqld_safe --defaults-file=/etc/my_3310.cnf  --wsrep-new-cluster --user=mysql &

 

The option "--wsrep-new-cluster" is a must when starting the first node. If this option is not used galera cluster will look for all the nodes mentioned in “wsrep_cluster_address”, but those nodes are down.

   

      2)  Now to add Node2 to the cluster follow the below steps

[root@cluster1 ~]# vim /etc/my_3311.cnf

datadir=/data/new/gal2
basedir=/usr/local/maria_cluster/
wsrep_provider=/usr/lib64/galera/libgalera_smm.so
query_cache_size=0
binlog_format=row
default_storage_engine=innodb
innodb_autoinc_lock_mode=2
wsrep_cluster_address=gcomm:/10.0.0.20:4567,10.0.0.20:4569
wsrep_cluster_name=cluster1
wsrep_node_name=node2
wsrep_sst_method=rsync
wsrep_node_address=10.0.0.20:4568 

 Now use the below command to start Node2 

[root@cluster1 ~]# mysqld_safe --defaults-file=/etc/my_3311.cnf  --user=mysql &

      3)  Now add Node3 to the cluster.

[root@cluster1 ~]# vim /etc/my_3312.cnf

datadir=/data/new/gal3
basedir=/usr/local/maria_cluster/
wsrep_provider=/usr/lib64/galera/libgalera_smm.so
query_cache_size=0
binlog_format=row
default_storage_engine=innodb
innodb_autoinc_lock_mode=2
wsrep_cluster_address=gcomm:/10.0.0.20:4567,10.0.0.20:4568
wsrep_cluster_name=cluster1
wsrep_node_name=node3
wsrep_sst_method=rsync
wsrep_node_address=10.0.0.20:4569 

Now use the below command to start Node3

[root@cluster1 ~]# mysqld_safe --defaults-file=/etc/my_3312.cnf --user=mysql &

 

Now to test the setup , login to each of the nodes and create some databases/tables and check if  you are able to see those databases/tables in the other nodes and you are good to go.