I'm using Paramiko to tail -f
a file on a remote server.
Previously, we were running this via ssh -t
, but that proved flaky, and the -t
caused issues with our remote scheduling system.
My question is how to kill tail when the script catches a SIGINT?
My script (based on Long-running ssh commands in python paramiko module (and how to end them))
#!/usr/bin/env python2
import paramiko
import select
client = paramiko.SSHClient()
client.load_system_host_keys()
client.connect('someserver', username='victorhooi', password='blahblah')
transport = client.get_transport()
channel = transport.open_session()
channel.exec_command("tail -f /home/victorhooi/macbeth.txt")
while True:
try:
rl, wl, xl = select.select([channel],[],[],0.0)
if len(rl) > 0:
# Must be stdout
print channel.recv(1024)
except KeyboardInterrupt:
print("Caught control-C")
client.close()
channel.close()
exit(0)
The script catches my Ctrl-C successfully, and ends. However, it leaves the tail -f
process running on the remote system,.
Neither client.close() nor channel.close() seem to terminate it.
What command can I issue in the except block to kill it?
The remote server is running Solaris 10.
There is one way to do this. It works like on the shell
The option -t is opening a pseudo pty to help ssh to track how long this process should last. the same can be done via pormiko via
prior to execute_command(...). This will not open a shell like it does with channel.invoke_shell(), it just requests such a pseudo interface to tie all processes to. The effect can also be seen if ps aux is issued on the remote machine, the process is now attached to the sshd with a ptxXY interface.
Here's a way to obtain the remote process ID:
And here's how to use it (replace
...
with the bits in the original question):I just hit this issue and wasn't in a position to issue a pkill to close the process at the end.
A better solution is to change the command you are running to:
This will let you run your tail command for as long as you need. As soon as you send a newline to the remote process the kill %1 will execute and stop the tail command you backgrounded. (for reference: %1 is a jobspec and used to describe the first process that has been backgrounded in your session, ie the tail command)
You can use get_pty as described in https://stackoverflow.com/a/38883662/565212.
E.g. scenario - When to call client/channel.close():
Step1: Execute a remote command that writes to the log file.
Step2: Spawn a thread that executes the tail command and blocks in the readline loop
Step3: In main thread, when the command returns, you know there will be no more logs, kill the tail thread.
You should use ssh keepalives... the problem you have is that the remote shell has no way of knowing (by default) that your ssh session was killed. Keepalives will enable the remote shell to detect that you killed the session
Set the keepalive value as low as you like (even 1 second)... after several seconds, the remote shell will see that the ssh login died, and it will terminate any processes that were spawned by it.
While not the most efficient method, this should work. After you CTRL+C; In the KeyboardInterrupt handler you could
exec_command("killall -u %s tail" % uname)
like so:This would kill any open processes named
tail
. That may cause issues though if you havetail
s open that you dont want to close, if thats the case you couldgrep
aps
, get the pid andkill -9
it.First, set tail to read
n
lines from end of file before following. setn
to a unique nuber liketime.time()
, since tail doesn't care if that number is larger then the number of lines in the file, the large number fromtime.time()
shouldnt cause issues and will be unique. Then grep for that unique number in theps
: