运行命令并获得它的stdout和stderr，几乎是实时的，就像在终端中一样

Question

我试图在Python中找到一种运行其他程序的方法：

1. 可以记录正在运行的程序的stdout和stderr 分开。在
2. 正在运行的程序的stdout和stderr可以是 以近乎实时的方式查看，如果子进程挂起，则 用户可以看到。（即，我们不会等待执行完成之前 向用户打印stdout/stderr）
3. 奖金标准： 正在运行的程序不知道它是通过python运行的，因此 不会做意想不到的事情（比如将其输出分块而不是 实时打印，或退出，因为它需要终端 查看其输出）。这个小标准意味着我们需要 我想用私人助理吧。在

这是我目前所得到的。。。 方法1：

def method1(command):
    ## subprocess.communicate() will give us the stdout and stderr sepurately, 
    ## but we will have to wait until the end of command execution to print anything.
    ## This means if the child process hangs, we will never know....
    proc=subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True, executable='/bin/bash')
    stdout, stderr = proc.communicate() # record both, but no way to print stdout/stderr in real-time
    print ' ######### REAL-TIME ######### '
    ########         Not Possible
    print ' ########## RESULTS ########## '
    print 'STDOUT:'
    print stdout
    print 'STDOUT:'
    print stderr

方法2

^{pr2}$

方法三：

def method3(command):
    ## This method is very much like method1, and would work exactly as desired
    ## if only proc.xxx.read(1) wouldn't block waiting for something. Which it does. So this is useless.
    proc=subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True, executable='/bin/bash')
    print ' ######### REAL-TIME ######### '
    out,err,outbuf,errbuf = '','','',''
    firstToSpeak = None
    while proc.poll() == None:
            stdout = proc.stdout.read(1) # blocks
            stderr = proc.stderr.read(1) # also blocks
            if firstToSpeak == None:
                if stdout != '': firstToSpeak = 'stdout'; outbuf,errbuf = stdout,stderr
                elif stderr != '': firstToSpeak = 'stderr'; outbuf,errbuf = stdout,stderr
            else:
                if (stdout != '') or (stderr != ''): outbuf += stdout; errbuf += stderr
                else:
                    out += outbuf; err += errbuf;
                    if firstToSpeak == 'stdout': sys.stdout.write(outbuf+errbuf);sys.stdout.flush()
                    else: sys.stdout.write(errbuf+outbuf);sys.stdout.flush()
                    firstToSpeak = None
    print ''
    print ' ########## RESULTS ########## '
    print 'STDOUT:'
    print out
    print 'STDERR:'
    print err

要尝试这些方法，您需要import sys,subprocess,pexpect

pexpect是纯python，可以与

sudo pip install pexpect

我认为解决方案将涉及python的pty模块——这有点像是一门黑色艺术，我找不到任何知道如何使用的人。或许如此知道：） 作为提醒，我建议你用curlwww.google.com'作为一个测试命令，因为它出于某种原因在stderr上打印出它的状态：D

---

更新-1:
好的，所以pty图书馆不适合人类消费。文档本质上是源代码。 任何提出的解决方案是阻塞的，而不是异步的，在这里是行不通的。padraiccunningham的Threads/Queue方法工作得很好，尽管添加pty支持是不可能的，而且它是“脏的”（引用Freenode的python）。 似乎唯一适合生产标准代码的解决方案是使用Twisted框架，它甚至支持pty作为布尔开关来运行进程，就像从shell调用的一样。 但是将Twisted添加到项目中需要完全重写所有代码。这真是太糟糕了：/

更新-2:

Two answers were provided, one of which addresses the first two criteria and will work well where you just need both the stdout and stderr using Threads and Queue. The other answer uses select, a non-blocking method for reading file descriptors, and pty, a method to "trick" the spawned process into believing it is running in a real terminal just as if it was run from Bash directly - but may or may not have side-effects. I wish I could accept both answers, because the "correct" method really depends on the situation and why you are subprocessing in the first place, but alas, I could only accept one.

Answer 1

虽然J.F.Sebastian的答案无疑解决了问题的核心，但我运行的是Python2.7（它不在最初的标准中），所以我就把它扔给那些只想剪切/粘贴一些代码的疲惫旅行者。 我还没有对此进行彻底的测试，但在我尝试过的所有命令中，它似乎工作得非常完美：） 您可能需要将.decode（'ascii'）更改为.decode（'utf-8'）-im仍在测试该位。在

#!/usr/bin/env python2.7
import errno
import os
import pty
import sys
from select import select
import subprocess
stdout = ''
stderr = ''
command = 'curl google.com ; sleep 5 ; echo "hey"'
masters, slaves = zip(pty.openpty(), pty.openpty())
p = subprocess.Popen(command, stdin=slaves[0], stdout=slaves[0], stderr=slaves[1], shell=True, executable='/bin/bash')
for fd in slaves: os.close(fd)

readable = { masters[0]: sys.stdout, masters[1]: sys.stderr }
try:
    print ' ######### REAL-TIME ######### '
    while readable:
        for fd in select(readable, [], [])[0]:
            try: data = os.read(fd, 1024)
            except OSError as e:
                if e.errno != errno.EIO: raise
                del readable[fd]
            finally:
                if not data: del readable[fd]
                else:
                    if fd == masters[0]: stdout += data.decode('ascii')
                    else: stderr += data.decode('ascii')
                    readable[fd].write(data)
                    readable[fd].flush()
except: pass
finally:
    p.wait()
    for fd in masters: os.close(fd)
    print ''
    print ' ########## RESULTS ########## '
    print 'STDOUT:'
    print stdout
    print 'STDERR:'
    print stderr