Controlling rsync with Python?

Question

I've been wanting to write a python script that would run several instances of rsync in sequence for backing up data to a different computer.

At the moment I just have this text file with the commands I use and I've just been copy-pasting them into the terminal, and it seems kinda silly.

I want to be able to use python to do this for me. I know very vaguely how to use subprocess.popen, but I have no clue how to get python to interact with rsync directly, like for entering my password for me. Can python do that?

Something like:

if theProccess.proccessResponse == "Password:" :
    theProccess.respond(string)

Or is the best that I can do is just have it, or even a bash script, just run the rsyncs in sequence and have to type my password in over and over again?

Thanks in advance.

Answer 1

If you just need to enter the password, you can try populating the RSYNC_PASSWORD environmental variable or using the --password-file option.

Answer 2

I don't think it supports rsync out of the box, but paramiko might have some components you could recycle?

Answer 3

I use rsync to back up all of my clients' web sites. A script is triggered by cron and it uses Makefiles for each client because of their different needs.

Rather than do something that enters the password, use ssh-keygen to create a public/private key pair and put your public key on the remote machine. This gives you secure, no-password connections. This also means you don't have to expose the rsync port to the world. After you get past the learning curve on this (and it's not very steep) ssh is most definitely your friend.

Answer 4

If you need to programatically control a sub-process, you should look into using pexpect.

Answer 5

I feel bad, for answering this late, but I feel like everyone else's answer was wrong. They did KINDA answer your question, but not directly as they could have.

More to the point, you had asked how to grab the password interactively. To do so I would suggest the built-in getpass. In short, you are not interacting with rsync. You are grabbing the password from the user RIGHT before you execute rsync, and passing that into rsync. Another option is to allow the user to pass it in as an option, most of my command line scripts use optparse

import getpass
password = getpass.getpass('Password for %s: ' % opts.user)
try:
    #code that uses password
except Exception, e:
    # test to see if str(e) is really an invalid password error, if so tell the user and return or loop, up to you
    # else 
    raise Exception(e) # the error that was raised in the first place

To continue, I stumbled upon your question because I was looking for something similar. Just an FYI to anyone else out there, I ended up referencing the two of these stack overflow links: calling rsync from python subprocess.call and Python Subprocess.Popen from a thread

Answer 6

I've made a package called paralle_rsync that does rsync commands in parallel. You can use it within fabric to perform the operation on multiple hosts at the same time.