Keep Remote Directory Up-to-date

Question

I absolutely love the Keep Remote Directory Up-to-date feature in Winscp. Unfortunately, I can't find anything as simple to use in OS X or Linux. I know the same thing can theoretically be accomplished using changedfiles or rsync, but I've always found the tutorials for both tools to be lacking and/or contradictory.

I basically just need a tool that works in OSX or Linux and keeps a remote directory in sync (mirrored) with a local directory while I make changes to the local directory.

---

Update

Looking through the solutions, I see a couple which solve the general problem of keeping a remote directory in sync with a local directory manually. I know that I can set a cron task to run rsync every minute, and this should be fairly close to real time.

This is not the exact solution I was looking for as winscp does this and more: it detects file changes in a directory (while I work on them) and then automatically pushes the changes to the remote server. I know this is not the best solution (no code repository), but it allows me to very quickly test code on a server while I develop it. Does anyone know how to combine rsync with any other commands to get this functionality?

Answer 1

I have the same issue. I loved winscp "keep remote directory up to date" command. However, in my quest to rid myself of Windows, I lost winscp. I did write a script that uses fileschanged and rsync to do something similar much closer to real time.

How to use:

• Make sure you have fileschanged installed
• Save this script in /usr/local/bin/livesync or somewhere reachable in your $PATH and make it executable
• Use Nautilus to connect to the remote host (sftp or ftp)
• Run this script by doing livesync SOURCE DEST
• The DEST directory will be in /home/[username]/.gvfs/[path to ftp scp or whatever]

A Couple downsides:

• It is slower than winscp (my guess is because it goes through Nautilus and has to detect changes through rsync as well)
• You have to manually create the destination directory if it doesn't already exist. So if you're adding a directory, it won't detect and create the directory on the DEST side.
• Probably more that I haven't noticed yet
• Also, do not attempt to synchronize a SRC directory named "rsyncThis". That will probably not be good :)

---

#!/bin/sh

upload_files()
{
    if [ "$HOMEDIR" = "." ]
    then
        HOMEDIR=`pwd`
    fi

    while read  input
    do
        SYNCFILE=${input#$HOMEDIR}
        echo -n "Sync File: $SYNCFILE..."
        rsync -Cvz --temp-dir="$REMOTEDIR" "$HOMEDIR/$SYNCFILE" "$REMOTEDIR/$SYNCFILE" > /dev/null
        echo "Done."
    done
}


help()
{
    echo "Live rsync copy from one directory to another.  This will overwrite the existing files on DEST."
    echo "Usage: $0 SOURCE DEST"    
}


case "$1" in
  rsyncThis)
    HOMEDIR=$2
    REMOTEDIR=$3
    echo "HOMEDIR=$HOMEDIR"
    echo "REMOTEDIR=$REMOTEDIR"
    upload_files
    ;;

  help)
    help
    ;;

  *)
    if [ -n "$1" ] && [ -n "$2" ]
    then
        fileschanged -r "$1" | "$0" rsyncThis "$1" "$2"
    else
        help
    fi
    ;;
esac

Answer 2

I used to have the same setup under Windows as you, that is a local filetree (versioned) and a test environment on a remote server, which I kept mirrored in realtime with WinSCP. When I switched to Mac I had to do quite some digging before I was happy, but finally ended up using:

• SmartSVN as my subversion client
• Sublime Text 2 as my editor (already used it on Windows)
• SFTP-plugin to ST2 which handles the uploading on save (sorry, can't post more than 2 links)

I can really recommend this setup, hope it helps!

Answer 3

It seems like perhaps you're solving the wrong problem. If you're trying to edit files on a remote computer then you might try using something like the ftp plugin for jedit. http://plugins.jedit.org/plugins/?FTP This ensures that you have only one version of the file so it can't ever be out of sync.

Answer 4

lsyncd seems to be the perfect solution. it combines inotify (kernel builtin function which watches for file changes in a directory trees) and rsync (cross platform file-syncing-tool).

lsyncd -rsyncssh /home remotehost.org backup-home/

Quote from github:

Lsyncd watches a local directory trees event monitor interface (inotify or fsevents). It aggregates and combines events for a few seconds and then spawns one (or more) process(es) to synchronize the changes. By default this is rsync. Lsyncd is thus a light-weight live mirror solution that is comparatively easy to install not requiring new filesystems or blockdevices and does not hamper local filesystem performance.

Answer 5

Well, I had the same kind of problem and it is possible using these together: rsync, SSH Passwordless Login, Watchdog (a Python sync utility) and Terminal Notifier (an OS X notification utility made with Ruby. Not needed, but helps to know when the sync has finished).

1. I created the key to Passwordless Login using this tutorial from Dreamhost wiki: http://cl.ly/MIw5

   1.1. When you finish, test if everything is ok… if you can't Passwordless Login, maybe you have to try afp mount. Dreamhost (where my site is) does not allow afp mount, but allows Passwordless Login. In terminal, type:

   ssh [e-mail] You should login without passwords being asked :P

2. I installed the Terminal Notifier from the Github page: http://cl.ly/MJ5x

   2.1. I used the Gem installer command. In Terminal, type:

   gem install terminal-notifier

   2.3. Test if the notification works.In Terminal, type:

   terminal-notifier -message "Starting sync"

3. Create a sh script to test the rsync + notification. Save it anywhere you like, with the name you like. In this example, I'll call it ~/Scripts/sync.sh I used the ".sh extension, but I don't know if its needed.

   #!/bin/bash terminal-notifier -message "Starting sync" rsync -azP ~/Sites/folder/ [e-mail]:site_folder/ terminal-notifier -message "Sync has finished"

   3.1. Remember to give execution permission to this sh script. In Terminal, type:

   sudo chmod 777 ~/Scripts/sync.sh 3.2. Run the script and verify if the messages are displayed correctly and the rsync actually sync your local folder with the remote folder.

4. Finally, I downloaded and installed Watchdog from the Github page: http://cl.ly/MJfb

   4.1. First, I installed the libyaml dependency using Brew (there are lot's of help how to install Brew - like an "aptitude" for OS X). In Terminal, type:

   brew install libyaml

   4.2. Then, I used the "easy_install command". Go the folder of Watchdog, and type in Terminal:

   easy_install watchdog

5. Now, everything is installed! Go the folder you want to be synced, change this code to your needs, and type in Terminal:

     watchmedo shell-command
         --patterns="*.php;*.txt;*.js;*.css" \
         --recursive \
         --command='~/Scripts/Sync.sh' \
         .
   

   It has to be EXACTLY this way, with the slashes and line breaks, so you'll have to copy these lines to a text editor, change the script, paste in terminal and press return.

   I tried without the line breaks, and it doesn't work!

   In my Mac, I always get an error, but it doesn't seem to affect anything:

   /Library/Python/2.7/site-packages/argh-0.22.0-py2.7.egg/argh/completion.py:84: UserWarning: Bash completion not available. Install argcomplete.

   Now, made some changes in a file inside the folder, and watch the magic!