Trying to write a script that checks a directory for files which then uses the names of the files found to insert in to a subprocess command as shown below:
for filename in os.listdir('/home/dross/python/scripts/var/running/'):
print(str(filename))
cmd = 'app_query --username=dross --password=/home/dross/dross.txt "select row where label = \'Id: ' + filename + '\' SHOW status"'
print(cmd)
query = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE)
query.wait()
If I run the command manually from the command line there are 2 possible values returned "Error:No result" or "True"
When the "Error: No result" condition is true the script returns the same however when the "True" condition is present nothing is returned.
If the result of the print statement is copied and pasted in to the os command line it runs and returns "True"
What could be the deception I am seeing here ?
Is there a better approach to achieve what I am trying to do ?
You seem to be missing a call to .communicate(), to read the results of the command through the pipe.
In your original query = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE) then anything send to stderr will be displayed on the screen, which seems to be what's happening for your error message. Anything sent to stdout will be sent to the pipe, ready for reading with communicate()
Some experimenting, showing that you won't see what's written to the subprocess.PIPE channels unless you communicate with the command you've run, and that stderr will display to the terminal if it's not redirected:
>>> import subprocess
>>> query = subprocess.Popen('echo STDERR 1>&2', shell=True, stdout=subprocess.PIPE)
STDERR
>>> query.wait()
0
>>> print(query.communicate())
('', None)
>>> query = subprocess.Popen('echo STDERR 1>&2', shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
>>> query.wait()
0
>>> print(query.communicate())
('', 'STDERR\n')
>>> query = subprocess.Popen('echo STDOUT', shell=True, stdout=subprocess.PIPE)
>>> query.wait()
0
>>> print(query.communicate())
('STDOUT\n', None)
So, to use your code from the question, you want something like this:
for filename in os.listdir('/home/dross/python/scripts/var/running/'):
print(filename) # print can convert to a string, no need for str()
cmd = 'app_query --username=dross --password=/home/dross/dross.txt "select row where label = \'Id: ' + filename + '\' SHOW status"'
print(cmd)
query = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
query.wait()
output, error = query.communicate()
print("stdout: {}".format(output))
print("stderr: {}".format(error))
Related
I want to check the DNS value from my system.
If the command goes wrong, the error should be stored in a different variable.
This is what I have so far:
proc = subprocess.Popen(['echo', '"to stdout"'], stdout=subprocess.PIPE,)
stdout_value = proc.communicate()
print '\tstdout:', repr(stdout_value)
subprocess.call('echo #user', shell=True)
#subprocess.check_call('echo #HOME', shell=True)
You should try this :
It captures errorcode, stdout and stderr from a command you passed as an argument :
import shlex
from subprocess import Popen, PIPE
def get_exitcode_stdout_stderr(cmd):
"""
Execute the external command and get its exitcode, stdout and stderr.
"""
args = shlex.split(cmd)
proc = Popen(args, stdout=PIPE, stderr=PIPE)
out, err = proc.communicate()
exitcode = proc.returncode
#
return exitcode, out, err
cmd = "..." # arbitrary external command, e.g. "python mytest.py"
exitcode, out, err = get_exitcode_stdout_stderr(cmd)
For your need, I think you can use a python module to get what you want instead of using the bash cmd line. For example, to get your fully qualified domain name you can use :
socket.getfqdn()
Here is the code in question:
import subprocess
import re
import os
p = subprocess.Popen(["nc -zv 8.8.8.8 53"], stdout=subprocess.PIPE, shell = True)
out, err = p.communicate()
regex = re.search("succeeded", out)
if not regex:
print ("test")
What i want it to do is to print out test if the regex does not match the netcat command. Right now im only matching on "succeeded" but that's all i need because the netcat command prints out :
Connection to 8.8.8.8 53 port [tcp/domain] succeeded!
The code runs fine but it matches when it shouldn't ?
The output is coming out stderr not stdout:
stderr=subprocess.PIPE
You can simplify to using in and you don't need shell=True:
p = subprocess.Popen(["nc", "-zv", "8.8.8.8", "53"], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = p.communicate()
if "succeeded" not in err:
print ("test")
You can also redirect stderr to STDOUT and use check_output presuming you are using python >= 2.7:
out = subprocess.check_output(["nc", "-zv", "8.8.8.8", "53"],stderr=subprocess.STDOUT)
if "succeeded" not in out:
print ("test")
I am using subprocess module to interact with output of the linux commands. below is my code.
import subprocess
import sys
file_name = 'myfile.txt'
p = subprocess.Popen("grep \"SYSTEM CONTROLLER\" "+ file_name, stdout=subprocess.PIPE, shell=True)
(output, err) = p.communicate()
print output.strip()
p = subprocess.Popen("grep \"controller\|worker\" "+ file_name, stdout=subprocess.PIPE, shell=True)
(output, err) = p.communicate()
lines = output.rstrip().split("\n")
print lines
My program hangs while executing second subprocess i.e.
p = subprocess.Popen("grep \"controller\|worker\""+ file_name,stdout=subprocess.PIPE, shell=True)
I got to know that the reason of process hang is buffer redirected to subprocess.PIPE is getting filled, which blocks the process from writing further.
I want to know if there is any way to avoid the buffer full situation so that my program keeps on executing without any hang issue ?
The actual issue is that there is a whitespace missing between the pattern and the filename and therefore grep waits for input on the standard input (stdin).
"buffer full" (.communicate() is not susceptible) or p.stdout.read() (it fixes nothing: it loads the output into memory and unlike .communicate() it fails if more than one pipe is used) are a red herring here.
Drop shell=True and use a list argument for the command:
#!/usr/bin/env python
from subprocess import Popen, PIPE
p = Popen(["grep", r"controller\|worker", file_name], stdout=PIPE)
output = p.communicate()[0]
if p.returncode == 0:
print('found')
elif p.returncode == 1:
print('not found')
else:
print('error')
As it says at https://docs.python.org/3/library/subprocess.html#subprocess.Popen.communicate:
Note: The data read is buffered in memory, so do not use this method
if the data size is large or unlimited.
Instead, use the file objects to read the text as it is produced:
output = p.stdout.read()
As long as no other pipes (e.g. stderr) fill up while you are reading, the process shouldn't be blocked.
I know how to search if specific word exists in subprocess output. But how can I print out only one specific line, if I know this line always looks like:
This is the line: some text
"Some text" can have different values.
What I have:
variable = subprocess.call(["some", "command"], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = output.communicate()
What I want to get after executing the script:
This is the line: some text
import subprocess
p = subprocess.Popen(["some", "command"], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = p.communicate()
stdout=stdout.split("\n")
for line in stdout:
if line.startswith("This is the line:"):
print line
Have been trying to get something like this to work for a while, the below doesn't seem to be sending the correct arg to the c program arg_count, which outputs argc = 1. When I'm pretty sure I would like it to be 2. ./arg_count -arg from the shell outputs 2...
I have tried with another arg (so it would output 3 in the shell) and it still outputs 1 when calling via subprocess.
import subprocess
pipe = subprocess.Popen(["./args/Release/arg_count", "-arg"], shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = pipe.communicate()
result = out.decode()
print "Result : ",result
print "Error : ",err
Any idea where im falling over? I'm running linux btw.
From the documentation:
The shell argument (which defaults to False) specifies whether to use
the shell as the program to execute. If shell is True, it is
recommended to pass args as a string rather than as a sequence.
Thus,
pipe = subprocess.Popen("./args/Release/arg_count -arg", shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
should give you what you want.
If shell=True then your call is equivalent to:
from subprocess import Popen, PIPE
proc = Popen(['/bin/sh', '-c', "./args/Release/arg_count", "-arg"],
stdout=PIPE, stderr=PIPE)
i.e., -arg is passed to the shell itself and not your program. Drop shell=True to pass -arg to the program:
proc = Popen(["./args/Release/arg_count", "-arg"],
stdout=PIPE, stderr=PIPE)
If you don't need to capture stderr separately from stdout then you could use check_output():
from subprocess import check_output, STDOUT
output = check_output(["./args/Release/arg_count", "-arg"]) # or
output_and_errors = check_output(["./args/Release/arg_count", "-arg"],
stderr=STDOUT)