lsyncd
HowTo: Two way file sync between three or more servers
I was working in a project have multiple server, just assume that more than two, all those webservers are under loadbalancer and which doesn’t having centralize storage. So one thing I can do is sync files between the servers.
The first priority to avoid unwanted process and sync the file only if any updates happen, this action will effect all the servers.
I used the tool called Lsync + Unison
Lsyncd
Source : https://code.google.com/p/lsyncd/
Lsyncd watches a local directory trees event monitor interface (inotify or fsevents). It aggregates and combines events for a few seconds and then spawns one (or more) process(es) to synchronize the changes. By default this is rsync. Lsyncd is thus a light-weight live mirror solution that is comparatively easy to install not requiring new filesystems or blockdevices and does not hamper local filesystem performance.
Unison
Source : http://olex.openlogic.com/packages/unison
Unison is a file-synchronization tool for Unix and Windows. It allows two replicas of a collection of files and directories to be stored on different hosts (or different disks on the same host), modified separately, and then brought up to date by propagating the changes in each replica to the other. License GPLv3
Please follow the implementation steps below for all the servers, because each server should check for updates
Server1 : 192.168.1.51
Server2: 192.168.1.52
Server3: 192.168.1.53
Shared folder is same on all the server /home/syncfuser/fileupload
SSH port : 10022
Web root writable user : syncfuser
Installation
Add additional Centos repository here I used CentOS 6.4 64bit. you may get different version from the give link
http://wiki.centos.org/AdditionalResources/Repositories/RPMForge
[root@srv-51 ~]# rpm -Uvh http://pkgs.repoforge.org/rpmforge-release/rpmforge-release-0.5.3-1.el6.rf.x86_64.rpm Retrieving http://pkgs.repoforge.org/rpmforge-release/rpmforge-release-0.5.3-1.el6.rf.x86_64.rpm warning: /var/tmp/rpm-tmp.iORM9p: Header V3 DSA/SHA1 Signature, key ID 6b8d79e6: NOKEY Preparing... ########################################### [100%] 1:rpmforge-release ########################################### [100%]
[root@srv-51 ~]# rpm --import http://apt.sw.be/RPM-GPG-KEY.dag.txt
Installing Lsycd , unison
[root@srv-51 ~]# wget http://lsyncd.googlecode.com/files/lsyncd-2.0.6.tar.gz
[root@srv-51 ~]# tar -zxvf lsyncd-2.0.6.tar.gz
[root@srv-51 ~]# cd lsyncd-2.0.6 ; make ; make install
[root@srv-51 ~]# yum install pkgconfig lua.x86_64 lua-devel.x86_64 lua-static.x86_64 gcc unison -y
[root@srv-51 ~]# touch /var/log/lsyncd/{lsyncd,lsyncd-status}.log
[root@srv-51 ~]# vi /etc/init.d/lsyncd
#!/bin/bash
#
# chkconfig: - 85 15
# description: Lightweight inotify based sync daemon
#
# processname: lsyncd
# config: /etc/lsyncd.conf
# config: /etc/sysconfig/lsyncd
# pidfile: /var/run/lsyncd.pid
# Source function library
. /etc/init.d/functions
# Source networking configuration.
. /etc/sysconfig/network
# Check that networking is up.
[ "$NETWORKING" = "no" ] && exit 0
LSYNCD_OPTIONS="-pidfile /var/run/lsyncd.pid /etc/lsyncd.conf"
if [ -e /etc/sysconfig/lsyncd ]; then
. /etc/sysconfig/lsyncd
fi
RETVAL=0
prog="lsyncd"
thelock=/var/lock/subsys/lsyncd
start() {
[ -f /etc/lsyncd.conf ] || exit 6
echo -n $"Starting $prog: "
if [ $UID -ne 0 ]; then
RETVAL=1
failure
else
daemon /usr/local/bin/lsyncd $LSYNCD_OPTIONS
RETVAL=$?
[ $RETVAL -eq 0 ] && touch $thelock
fi;
echo
return $RETVAL
}
stop() {
echo -n $"Stopping $prog: "
if [ $UID -ne 0 ]; then
RETVAL=1
failure
else
killproc lsyncd
RETVAL=$?
[ $RETVAL -eq 0 ] && rm -f $thelock
fi;
echo
return $RETVAL
}
reload(){
echo -n $"Reloading $prog: "
killproc lsyncd -HUP
RETVAL=$?
echo
return $RETVAL
}
restart(){
stop
start
}
condrestart(){
[ -e $thelock ] && restart
return 0
}
case "$1" in
start)
start
;;
stop)
stop
;;
restart)
restart
;;
reload)
reload
;;
condrestart)
condrestart
;;
status)
status lsyncd
RETVAL=$?
;;
*)
echo $"Usage: $0 {start|stop|status|restart|condrestart|reload}"
RETVAL=1
esac
exit $RETVAL
chmod +x /etc/init.d/lsyncd
Before start configuring lsync please make sure that password less login is enabled for all the servers.
Create a new conf file /etc/lsyncd.conf
[root@srv-51 ~]# vi /etc/lsyncd.conf
settings = {
logfile = "/var/log/lsyncd/lsyncd.log",
statusFile = "/var/log/lsyncd/lsyncd-status.log",
maxDelays = 3
}
runUnison2 = {
maxProcesses = 1,
delay = 3,
onAttrib = "export HOME=/root ; /usr/bin/unison -batch -confirmbigdel=false -confirmbigdel=false /home/syncfuser/fileupload ssh://syncfuser@192.168.1.52:10022//home/syncfuser/fileupload",
onCreate = "export HOME=/root ; /usr/bin/unison -batch -confirmbigdel=false -confirmbigdel=false /home/syncfuser/fileupload ssh://syncfuser@192.168.1.52:10022//home/syncfuser/fileupload",
onDelete = "export HOME=/root ; /usr/bin/unison -batch -confirmbigdel=false -confirmbigdel=false /home/syncfuser/fileupload ssh://syncfuser@192.168.1.52:10022//home/syncfuser/fileupload",
onModify = "export HOME=/root ; /usr/bin/unison -batch -confirmbigdel=false /home/syncfuser/fileupload ssh://syncfuser@192.168.1.52:10022//home/syncfuser/fileupload",
onMove = "export HOME=/root ; /usr/bin/unison -batch -confirmbigdel=false /home/syncfuser/fileupload ssh://syncfuser@192.168.1.52:10022//home/syncfuser/fileupload",
}
runUnison3 = {
maxProcesses = 1,
delay = 3,
onAttrib = "export HOME=/root ; /usr/bin/unison -batch -confirmbigdel=false /home/syncfuser/fileupload ssh://syncfuser@192.168.1.53:10022//home/syncfuser/fileupload",
onCreate = "export HOME=/root ; /usr/bin/unison -batch -confirmbigdel=false /home/syncfuser/fileupload ssh://syncfuser@192.168.1.53:10022//home/syncfuser/fileupload",
onDelete = "export HOME=/root ; /usr/bin/unison -batch -confirmbigdel=false /home/syncfuser/fileupload ssh://syncfuser@192.168.1.53:10022//home/syncfuser/fileupload",
onModify = "export HOME=/root ; /usr/bin/unison -batch -confirmbigdel=false /home/syncfuser/fileupload ssh://syncfuser@192.168.1.53:10022//home/syncfuser/fileupload",
onMove = "export HOME=/root ; /usr/bin/unison -batch -confirmbigdel=false /home/syncfuser/fileupload ssh://syncfuser@192.168.1.53:10022//home/syncfuser/fileupload",
}
sync{ runUnison2, source="/home/syncfuser/fileupload" }
sync{ runUnison3, source="/home/syncfuser/fileupload" }
Please use -confirmbigdel=false if you have clear Idea about what you are doing, which will give the power to remove the file forcefully even if the directory is to be empty.
With out -confirmbigdel=false will stop syncing (crash lsync) for the scenario ( directory going to be empty) . It will protect the file from accidentally removal commad rm -rf * , if it is a server you don’t need to worry about it because the file removal is handled by the application.
Sample logs for file syncs .
[root@srv-51 fileupload]# cat > samplefile-sv1.txt This is the sample file from server1. ^C
tail -f /var/log/lsyncd/lsyncd.log
Fri Jun 27 18:41:44 2014 Normal: Event Delete spawns shell "export HOME=/root ; /usr/bin/unison -batch -confirmbigdel=false /home/syncfuser/fileupload ssh://syncfuser@192.168.1.53:10022//home/syncfuser/fileupload" Fri Jun 27 18:41:44 2014 Normal: Event Delete spawns shell "export HOME=/root ; /usr/bin/unison -batch -confirmbigdel=false /home/syncfuser/fileupload ssh://syncfuser@192.168.1.51:10022//home/syncfuser/fileupload" Contacting server... Contacting server... Connected [//srv-51//home/syncfuser/fileupload -> //srv-52//home/syncfuser/fileupload] Looking for changes Waiting for changes from server Reconciling changes Connected [//srv-52//home/syncfuser/fileupload -> //srv-53//home/syncfuser/fileupload] Looking for changes Waiting for changes from server Nothing to do: replicas have been changed only in identical ways since last sync. Fri Jun 27 18:41:44 2014 Normal: Retrying Delete on /home/syncfuser/fileupload//.unison.samplefile-sv1.txt.f629b5a5e1d6f2942bd1ec2ad54122b6.unison.tmp = 0 Fri Jun 27 18:41:44 2014 Normal: Event Create spawns shell "export HOME=/root ; /usr/bin/unison -batch -confirmbigdel=false /home/syncfuser/fileupload ssh://syncfuser@192.168.1.51:10022//home/syncfuser/fileupload" Reconciling changes Nothing to do: replicas have been changed only in identical ways since last sync. Fri Jun 27 18:41:44 2014 Normal: Retrying Delete on /home/syncfuser/fileupload//.unison.samplefile-sv1.txt.f629b5a5e1d6f2942bd1ec2ad54122b6.unison.tmp = 0 Fri Jun 27 18:41:44 2014 Normal: Event Create spawns shell "export HOME=/root ; /usr/bin/unison -batch -confirmbigdel=false /home/syncfuser/fileupload ssh://syncfuser@192.168.1.53:10022//home/syncfuser/fileupload" Contacting server... Contacting server... Connected [//srv-51//home/syncfuser/fileupload -> //srv-52//home/syncfuser/fileupload] Looking for changes Waiting for changes from server Reconciling changes Nothing to do: replicas have not changed since last sync. Fri Jun 27 18:41:44 2014 Normal: Retrying Create on /home/syncfuser/fileupload//samplefile-sv1.txt = 0 Connected [//srv-52//home/syncfuser/fileupload -> //srv-53//home/syncfuser/fileupload] Looking for changes Waiting for changes from server Reconciling changes Nothing to do: replicas have not changed since last sync. Fri Jun 27 18:41:44 2014 Normal: Retrying Create on /home/syncfuser/fileupload//samplefile-sv1.txt = 0 Fri Jun 27 18:41:54 2014 Normal: Event Delete spawns shell "export HOME=/root ; /usr/bin/unison -batch -confirmbigdel=false /home/syncfuser/fileupload ssh://syncfuser@192.168.1.51:10022//home/syncfuser/fileupload" Fri Jun 27 18:41:54 2014 Normal: Event Delete spawns shell "export HOME=/root ; /usr/bin/unison -batch -confirmbigdel=false /home/syncfuser/fileupload ssh://syncfuser@192.168.1.53:10022//home/syncfuser/fileupload" Contacting server... Contacting server... Connected [//srv-52//home/syncfuser/fileupload -> //srv-53//home/syncfuser/fileupload] Connected [//srv-51//home/syncfuser/fileupload -> //srv-52//home/syncfuser/fileupload] Looking for changes Waiting for changes from server Looking for changes Waiting for changes from server Reconciling changes Reconciling changes Nothing to do: replicas have been changed only in identical ways since last sync. Fri Jun 27 18:41:54 2014 Normal: Retrying Delete on /home/syncfuser/fileupload//.unison.samplefile-sv1.txt.f629b5a5e1d6f2942bd1ec2ad54122b6.unison.tmp = 0 Fri Jun 27 18:41:54 2014 Normal: Event Create spawns shell "export HOME=/root ; /usr/bin/unison -batch -confirmbigdel=false /home/syncfuser/fileupload ssh://syncfuser@192.168.1.51:10022//home/syncfuser/fileupload" Nothing to do: replicas have been changed only in identical ways since last sync. Fri Jun 27 18:41:54 2014 Normal: Retrying Delete on /home/syncfuser/fileupload//.unison.samplefile-sv1.txt.f629b5a5e1d6f2942bd1ec2ad54122b6.unison.tmp = 0 Fri Jun 27 18:41:54 2014 Normal: Event Create spawns shell "export HOME=/root ; /usr/bin/unison -batch -confirmbigdel=false /home/syncfuser/fileupload ssh://syncfuser@192.168.1.53:10022//home/syncfuser/fileupload" Contacting server... Contacting server... Connected [//srv-51//home/syncfuser/fileupload -> //srv-52//home/syncfuser/fileupload] Connected [//srv-52//home/syncfuser/fileupload -> //srv-53//home/syncfuser/fileupload] Looking for changes Waiting for changes from server Looking for changes Reconciling changes Nothing to do: replicas have not changed since last sync. Fri Jun 27 18:41:54 2014 Normal: Retrying Create on /home/syncfuser/fileupload//samplefile-sv1.txt = 0 Waiting for changes from server Reconciling changes Nothing to do: replicas have not changed since last sync. Fri Jun 27 18:41:54 2014 Normal: Retrying Create on /home/syncfuser/fileupload//samplefile-sv1.txt = 0
[root@srv-52 fileupload]# cat >> samplefile-sv1.txt
File edited from server2. ^C Log:
Fri Jun 27 18:43:54 2014 Normal: Event Modify spawns shell "export HOME=/root ; /usr/bin/unison -batch -confirmbigdel=false /home/syncfuser/fileupload ssh://syncfuser@192.168.1.53:10022//home/syncfuser/fileupload" Fri Jun 27 18:43:54 2014 Normal: Event Modify spawns shell "export HOME=/root ; /usr/bin/unison -batch -confirmbigdel=false /home/syncfuser/fileupload ssh://syncfuser@192.168.1.51:10022//home/syncfuser/fileupload" Contacting server... Contacting server... Connected [//srv-51//home/syncfuser/fileupload -> //srv-52//home/syncfuser/fileupload] Connected [//srv-52//home/syncfuser/fileupload -> //srv-53//home/syncfuser/fileupload] Looking for changes Looking for changes Waiting for changes from server Waiting for changes from server Reconciling changes changed ----> samplefile-sv1.txt local : changed file modified on 2014-06-27 at 18:43:50 size 65 rw-r--r-- srv-53 : unchanged file modified on 2014-06-27 at 18:41:51 size 38 rw-r--r-- Propagating updates UNISON 2.40.63 started propagating changes at 18:43:54.25 on 27 Jun 2014 [BGN] Updating file samplefile-sv1.txt from /home/syncfuser/fileupload to //srv-53//home/syncfuser/fileupload Reconciling changes changed ----> samplefile-sv1.txt local : changed file modified on 2014-06-27 at 18:43:50 size 65 rw-r--r-- srv-51 : unchanged file modified on 2014-06-27 at 18:41:51 size 38 rw-r--r-- Propagating updates UNISON 2.40.63 started propagating changes at 18:43:54.25 on 27 Jun 2014 [BGN] Updating file samplefile-sv1.txt from /home/syncfuser/fileupload to //srv-51//home/syncfuser/fileupload [END] Updating file samplefile-sv1.txt UNISON 2.40.63 finished propagating changes at 18:43:54.25 on 27 Jun 2014 Saving synchronizer state [END] Updating file samplefile-sv1.txt UNISON 2.40.63 finished propagating changes at 18:43:54.25 on 27 Jun 2014 Saving synchronizer state Synchronization complete at 18:43:54 (1 item transferred, 0 skipped, 0 failed) Fri Jun 27 18:43:54 2014 Normal: Retrying Modify on /home/syncfuser/fileupload//samplefile-sv1.txt = 0 Synchronization complete at 18:43:54 (1 item transferred, 0 skipped, 0 failed) Fri Jun 27 18:43:54 2014 Normal: Retrying Modify on /home/syncfuser/fileupload//samplefile-sv1.txt = 0
[root@srv-53 fileupload]# rm -rf samplefile-sv1.txt [root@srv-53 fileupload]# ll total 0 Log :
Connected [//srv-51//home/syncfuser/fileupload -> //srv-53//home/syncfuser/fileupload] Connected [//srv-52//home/syncfuser/fileupload -> //srv-53//home/syncfuser/fileupload] Looking for changes Waiting for changes from server Looking for changes Reconciling changes deleted ----> samplefile-sv1.txt local : deleted srv-51 : unchanged file modified on 2014-06-27 at 18:53:14 size 0 rw-r--r-- Propagating updates UNISON 2.40.63 started propagating changes at 18:53:31.85 on 27 Jun 2014 [BGN] Deleting samplefile-sv1.txt from //srv-51//home/syncfuser/fileupload Waiting for changes from server [END] Deleting samplefile-sv1.txt UNISON 2.40.63 finished propagating changes at 18:53:31.85 on 27 Jun 2014 Saving synchronizer state Reconciling changes deleted ----> samplefile-sv1.txt local : deleted srv-52 : unchanged file modified on 2014-06-27 at 18:53:14 size 0 rw-r--r-- Propagating updates UNISON 2.40.63 started propagating changes at 18:53:31.85 on 27 Jun 2014 [BGN] Deleting samplefile-sv1.txt from //srv-52//home/syncfuser/fileupload [END] Deleting samplefile-sv1.txt UNISON 2.40.63 finished propagating changes at 18:53:31.85 on 27 Jun 2014 Saving synchronizer state Synchronization complete at 18:53:31 (1 item transferred, 0 skipped, 0 failed) Fri Jun 27 18:53:31 2014 Normal: Retrying Delete on /home/syncfuser/fileupload//samplefile-sv1.txt = 0 Synchronization complete at 18:53:31 (1 item transferred, 0 skipped, 0 failed) Fri Jun 27 18:53:31 2014 Normal: Retrying Delete on /home/syncfuser/fileupload//samplefile-sv1.txt = 0
Now You have unique file in all the servers.
Tech's you need !!!!! 