I am trying to use subprocess module with Popen to fetch log from a specified URL, However, I am not able to fetch the log and the program returns me a blank.
I have been using the below mentioned code:
import subprocess
url = r'C:\project\dummy\pro'
mycmd = ['svn', 'log', url]
log = subprocess.Popen(mycmd, shell=True,
stdin = subprocess.PIPE, stdout = subprocess.PIPE,
stderr = subprocess.PIPE)
result = log.wait()
out1, err = log.communicate()
print out1
I need the output string to use as next part of the program. Any help would be appreciated.
Try without shell=True:
log = subprocess.Popen(mycmd,
stdin = subprocess.PIPE, stdout = subprocess.PIPE,
stderr = subprocess.PIPE)
Related
I'm trying to understand the behavior of stdout_data and stderr_data of python's subprocess module (Popen.communicate()). When output of a command is slightly large, stderr is non-empty. stdout has expected output. Output is only few KB and not MBs in size in below example. Official documentation has this Note
Note: The data read is buffered in memory, so do not use this method if the data size is large or unlimited.
I'm assuming it is referring to GBs of output and unable to fit in RAM. It should not apply to my case.
minimum working example of the problem.
import subprocess
def test():
# cmd = ["curl", "www.google.com"] # This has non-empty content in stderr
cmd = ["echo", "Hello World"] # This has empty stderr
p = subprocess.Popen(cmd,
stdout = subprocess.PIPE,
stderr = subprocess.PIPE,
shell=False)
stdout, stderr = p.communicate()
if stderr:
print("Failed")
if __name__ == "__main__":
test()
When cmd = ["curl", "www.google.com"] content of stderr is amount of data downloaded (bytes received, transferred, average speed etc.)
I'm trying to understand the internals of why this is happening.
I want to check the DNS value from my system.
If the command goes wrong, the error should be stored in a different variable.
This is what I have so far:
proc = subprocess.Popen(['echo', '"to stdout"'], stdout=subprocess.PIPE,)
stdout_value = proc.communicate()
print '\tstdout:', repr(stdout_value)
subprocess.call('echo #user', shell=True)
#subprocess.check_call('echo #HOME', shell=True)
You should try this :
It captures errorcode, stdout and stderr from a command you passed as an argument :
import shlex
from subprocess import Popen, PIPE
def get_exitcode_stdout_stderr(cmd):
"""
Execute the external command and get its exitcode, stdout and stderr.
"""
args = shlex.split(cmd)
proc = Popen(args, stdout=PIPE, stderr=PIPE)
out, err = proc.communicate()
exitcode = proc.returncode
#
return exitcode, out, err
cmd = "..." # arbitrary external command, e.g. "python mytest.py"
exitcode, out, err = get_exitcode_stdout_stderr(cmd)
For your need, I think you can use a python module to get what you want instead of using the bash cmd line. For example, to get your fully qualified domain name you can use :
socket.getfqdn()
I'm having a lot of trouble passing a string to the openssl commandline tool via python's subprocess like this:
process = subprocess.Popen(
["openssl", "rsa", "-in", pathFile, "-out", "id_rsa.out"],
stdin = subprocess.PIPE,
stdout = subprocess.PIPE,
stderr = subprocess.PIPE,
shell=False
)
try:
process.communicate("some passphrase\n", timeout=2)
except:
process.kill() #openssl stays alive otherwise.
The code above times out (with and without the std redirection in the Popen). I can use openssl normally through the terminal just fine, but I really need to be able to run this as part of my python script.
Any help would be appreciated.
The section PASS PHRASE ARGUMENTS on the openssl man page explains how the passphrase input mechanism works. To make your example work, you should tell openssl to take the passphrase from stdin. Using your example as a starting point, the following works for me:
process = subprocess.Popen(
["openssl", "rsa", "-in", pathFile, "-out", "id_rsa.out", "-passin", "stdin"],
stdin = subprocess.PIPE,
stdout = subprocess.PIPE,
stderr = subprocess.PIPE,
shell=False
)
process.communicate("passphrase\n")
I try to get the output of a curl command by using the python subprocess. But the output is empty. Below is my source code:
def validateURL(url):
p = subprocess.Popen("curl",
stdin = subprocess.PIPE,
stdout = subprocess.PIPE,
stderr = subprocess.PIPE,
shell = False)
p.stdin.write("http://validator.w3.org/check?uri=" + url + "\n")
p.stdin.close()
stdout_data = p.stdout.read()
print stdout_data
result = re.findall("Error", stdout_data)
print result # empty here
if (len(result) != 0):
return 'ERR'
else:
return 'OK'
Why?
PS: I run this piece of code on my mac os and I use Python 2.7.
Drop the stderr = subprocess.PIPE,, and see the error message printed by curl. Act accordingly to fix it.
One possible reason is that the URL should be specified as a command-line argument, and not on stdin:
p = subprocess.Popen(("curl", "http://..."), stdout=subprocess.PIPE)
You were passing data to Popen after it executed the command.
Try this:
def validateURL(url):
p = subprocess.Popen(["curl", "http://validator.w3.org/check?uri=" + url + "\n"],
stdin = subprocess.PIPE,
stdout = subprocess.PIPE,
stderr = subprocess.PIPE,
shell = False)
stdout_data = p.stdout.read()
print stdout_data
result = re.findall("Error", stdout_data)
print result # empty here
if (len(result) != 0):
return 'ERR'
else:
return 'OK'
You're not specifying the URL on the command line, so curl is printing an error message and exiting. Thus, there is no output on stdout. You're trying to send the URL on standard input, but curl does not work that way.
Instead, try:
p = subprocess.Popen(["curl", "http://validator.w3.org/check?uri=" + url],
stdout=subprocess.PIPE, shell=False)
Or, you know, just use urllib2 (or requests) and do it in native Python instead of shelling out to curl and dealing with all that plumbing.
This is a follow-up from Invoke pstools in Python script
When I open a command prompt and execute
D:\pstools\psloggedon.exe -l -x \\10.10.10.10
I get
DOMAIN\user
But when I execute the script
import sys, subprocess, socket, string
import wmi, win32api, win32con
pst = subprocess.Popen(
["D:\pstools\psloggedon.exe", "-l", "-x", "\\10.10.10.10"],
stdout = subprocess.PIPE,
stderr = subprocess.PIPE
)
out, error = pst.communicate()
print out, "is output"
I get
Error opening HKEY_USERS for \10.10.10.10
is output
How do I get the subprocess to read the IP address as \10.10.10.10 instead of \10.10.10.10
By the way, I tried to add third backslash
pst = subprocess.Popen(
["D:\pstools\psloggedon.exe", "-l", "-x", "\\\10.10.10.10"],
stdout = subprocess.PIPE,
stderr = subprocess.PIPE
)
And the output is
Error opening HKEY_USERS for .0.139.40
is output
As suggested by lejlot's comment you have to use "\\" because "\" is an escape character in python.