SSH to machine through a middle host

Question

In my work with my professor I have to ssh into our server and from there I ssh into each node to run our programs. I am trying to write a python program that will let me do everything that I need to do on the remote node from my local machine. The commands that I will be running on the nodes are:

• cp files from local machine to remote nodes
• run a program on each node
• retrieve files from the nodes to my local machine
• maybe make it possible to copy over a fortran program and compile it on the nodes and also to check the nodes to see if any programs are running.

Right now I make my input files on my local machine, scp them to the server, then I copy the files to each node and run our fluid_dynamics program on each node. I then do the reverse to get our output back to my local machine.

I was looking at paramiko but I can not figure out how I can use it to get from my local machine to the nodes because I must go through the server. local -ssh--> server -ssh--> nodes

Is there a way to do this in python or should I try something else such as: using:

os.system(ssh -t server ssh node 'command')   

or making a bash scripts on the server for each of the different commands (compile.sh, move_inputs.sh, retrieve_outputs.sh) and then just connecting to the server and running the bash scripts.

Sorry if this doesn't make sense or if it is worded badly, any help is appreciated.

Additional Info: The reason I am using python is because I want the program to be able to make the input files, send them to the nodes and retrieve the output files, and to finally generate graphs of our data. I already have some code to generate our input files and to make the graphs from the outputs.

Answer 1

You don't need Python to do this. Check the ProxyCommand configuration option for SSH. Here is a tutorial that explains the details.

Answer 2

Use plink root@10.112.10.1 -pw password ls -l

Download plink and copy it to your windows machine

Answer 3

You can do this by creating a tunnel through your server to the node:

import os, sys, shlex
import subprocess
import paramiko

cmd = "ssh -f -N -p " + str(serverport) + " -l " + serveruser + " -L " + str(tunnelport) + ":" + nodehost + ":" + str(nodeport) + " " + serverhost
args = shlex.split(cmd)
tun = subprocess.Popen(args)
stat = tun.poll()

Once the tunnel is set up you can ftp to the nodes:

transport = paramiko.Transport(("127.0.0.1", tunnelport))
transport.connect(username=nodeusername, password=nodepw)
sftp = paramiko.SFTPClient.from_transport(transport)
sftp.put(localfile, remotefile)

Or you can connect and execute a command using paramiko.SSHClient().connect("127.0.0.1", port=port, username=user, password=pw) and paramiko.SSHClient().exec_command(command).

Then the tunnel process can be killed thus:

p = subprocess.Popen(['ps', '-A'], stdout=subprocess.PIPE)
out, err = p.communicate()
for line in out.splitlines():
    if cmd in line:
        pid = int(line.split(None, 1)[0])
        os.kill(pid, signal.SIGKILL)

Answer 4

You can do it with Paramiko:

proxy_command = 'ssh -i %s %s@%s nc %s %s' % (proxy_key, proxy_user, proxy_host, host, 22)

proxy = paramiko.ProxyCommand(proxy_command)

client = paramiko.SSHClient()
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
client.connect(host, username=user, password=password, sock=proxy)

stdin, stdout, stderr = client.exec_command('echo HELLO')
print "Echo: %s" % (", ".join(stdout.readlines()))
client.close()

It works with SFTPClient too:

proxy_command = 'ssh -i %s %s@%s nc %s %s' % (proxy_key, proxy_user, proxy_host, host, 22)

proxy = paramiko.ProxyCommand(proxy_command)

transport = paramiko.Transport(proxy)
transport.connect(username=user, password=password)

sftp = paramiko.SFTPClient.from_transport(transport)

Answer 5

With a trick from my colleague, you can ssh/scp from local to nodes directly.

Edit your ~/.ssh/config:

Host *
ControlMaster auto
ControlPath ~/.ssh/master-%r@%h:%p

Host node1 node2 or node*
ProxyCommand ssh server 'nc -w 5 %h 22'

Have fun!