How to deal with multiple subprocess and read stdout while process running ?

Question

I have one script named "adb_keycode_test.sh", it will input keyevent via adb, then I want to capture logcat(while script running) from device and writing them into a file, the code are listed below:

proc_logcat = subprocess.Popen('adb logcat | grep KeyCode', shell=True, stdout=subprocess.PIPE)
proc_script = subprocess.check_output(['./adb_keycode_test.sh'])
text = proc_logcat.communicate ==> hangs here
#text = proc_logcat.stdout.read() ==> hangs too

It can run perfectly without "proc_logcat.stdout.read()", my question is how to run "adb_keycode_test.sh" and "adb logcat" simultaneously, and terminate "adb logcat" after keycode script completed ?

Answer 1

Kuanlin Chen
Short:
because stdout is for writing and stdin is for reading

Long:
Reading from stdout while multithreading will evoke a race condition. AND RACE CONDITIONS ARE EVIL!!

Answer 2

Loeschzwerg Thanks for clarifying that for Kuanlin Chen. I was just about to sleep when I sent that link and couldn't explain further at the time.

Answer 3

Loeschzwerg David Carroll
I try to use PIPE like this:

from subprocess import Popen, PIPE, STDOUT
proc_script = Popen(['./adb_keycode_test.sh'], stdout=PIPE, stdin=PIPE, stderr=STDOUT)
text = proc_script.communicate(input='1\n2\n')[0]
print(text)

I am using python 2.7.6, this solution works for me.

Answer 4

Loeschzwerg David Carroll
But I go one step further, try to get output from proc_logcat:

from subprocess import Popen, PIPE, STDOUT
cmd = "adb logcat | grep KeyCode"
proc_logcat = Popen(cmd.split(' '), stdout=PIPE, stdin=PIPE, stderr=STDOUT)
proc_script = Popen(['./adb_keycode_test.sh'], stdout=PIPE, stdin=PIPE, stderr=STDOUT)
text = proc_logcat.communicate(input='1\n2\n')[0] ===> Hangs here !
os.kill(proc_logcat.pid, signalSIGTERM)
print(text)

In my opinion, "adb logcat" will not stop until terminate it by "Ctrl+C", proc_logcat didn't return, so it will not go to the next line, is there any better way to do this ?

Answer 5

This should help:

https://stackoverflow.com/a/39477247/5087494

Answer 6

Should have more explain about "pipes" and why do I never read from stdout ?

Answer 7

Loeschzwerg Thanks!

Answer 8

@Loeschzwerg

What do you mean by "pipes" ?

Answer 9

Solved:

cmd = "adb logcat | grep KeyCode"
proc_logcat = subprocess.Popen(cmd.split(' '), stdout=subprocess.PIPE)
proc_script = subprocess.check_call(['./adb_keycode_test.sh'])
os.kill(proc_logcat.pid, signal.SIGTERM)
text = proc_logcat.stdout.read()

Answer 10

Reading from stdout is really bad coding style. NEVER do that!
Use pipes instead.