piping in shell via Python subprocess module

问题:

So I'm trying to query for the top 3 CPU "intensive" processes on a given machine, and I found this shell command to do it: ps -eo pcpu,pid,user,args | sort -k 1 -r | head -3

I want to use this data inside a Python script, so I need to be able to capture the output of the above command via the subprocess module. The following works, but just returns a huge string since I'm not restricting it to the top 3:

psResult = subprocess.check_output(['ps', '-eo', 'pcpu,user,args'])

I'm not quite sure how this subprocess.check_output works.. in a meager attempt I tried:

subprocess.check_output(['ps', '-eo', 'pcpu,user,args', '|', 'sort', '-k', '1', '-r', '|', 'head', '-3'])

Which gives me an error: ps: illegal argument: |

How do I use the pipe | symbol inside Python, or use some other way to do the sorting without having to do incredible amounts of parsing on the huge string returned by psResult = subprocess.check_output(['ps', '-eo', 'pcpu,user,args'])?

Thanks! Regards, -kstruct

回答1:

You can pass the shell=True argument to execute the plain shell command:

import subprocess
subprocess.check_output('ps -eo pcpu,pid,user,args | sort -k 1 -r | head -3',
                        shell=True)

Alternatively, use the sorting options of ps and Python's built-in string functions like this:

raw = subprocess.check_output('ps -eo pcpu,pid,user,args --sort -pcpu')
first_three_lines = list(raw.split('\n'))[:3]

回答2:

Some others have suggested using shell=True, and this answer is fine if you are passing trusted input to the shell. However, shell=True introduces some insecurities. For security, the docs recommend the following:

output=`dmesg | grep hda`
# becomes
p1 = Popen(["dmesg"], stdout=PIPE)
p2 = Popen(["grep", "hda"], stdin=p1.stdout, stdout=PIPE)
p1.stdout.close()  # Allow p1 to receive a SIGPIPE if p2 exits.
output = p2.communicate()[0]

回答3:

it should work if you use:

subprocess.check_output("ps -eo pcpu,pid,user,args | sort -k 1 -r | head -3", shell=True)

then the command is run exactly like this using /bin/sh, so the pipes will work.

回答4:

Why use external commands at all? Use psutil:

import psutil
def cpu_percentage(proc):
    try:
        return proc.get_cpu_percent()
    except psutil.AccessDenied:
        return float('-inf')

top3 = sorted(psutil.process_iter(), key=cpu_percentage, reverse=True)[:3]
for proc in top3:
    # do whatever