We Keep Coding

Problem printing popen stdout from subprocess

i need put the output command to a variable.
I was trying this:
import os
import subprocess
output = subprocess.Popen(["ls", "-l"], stdout=subprocess.PIPE)
print (output.stdout)
output.terminate()
but i get
'open file '<fdopen>', mode 'rb' at 0xb76db5a0>'
what is the problem ? It's okay ?
i use python 2.6.6.

output.stdout is a file object. You can use the read method of the file object to get the content of the output:
print(output.stdout.read())
or you can use the Popen.communicate method instead:
stdout, stderr = output.communicate()
print(stdout)

python subprocess missing arguments

Have been trying to get something like this to work for a while, the below doesn't seem to be sending the correct arg to the c program arg_count, which outputs argc = 1. When I'm pretty sure I would like it to be 2. ./arg_count -arg from the shell outputs 2...
I have tried with another arg (so it would output 3 in the shell) and it still outputs 1 when calling via subprocess.
import subprocess
pipe = subprocess.Popen(["./args/Release/arg_count", "-arg"], shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = pipe.communicate()
result = out.decode()
print "Result : ",result
print "Error : ",err
Any idea where im falling over? I'm running linux btw.

From the documentation:
The shell argument (which defaults to False) specifies whether to use
the shell as the program to execute. If shell is True, it is
recommended to pass args as a string rather than as a sequence.
Thus,
pipe = subprocess.Popen("./args/Release/arg_count -arg", shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
should give you what you want.

If shell=True then your call is equivalent to:
from subprocess import Popen, PIPE
proc = Popen(['/bin/sh', '-c', "./args/Release/arg_count", "-arg"],
stdout=PIPE, stderr=PIPE)
i.e., -arg is passed to the shell itself and not your program. Drop shell=True to pass -arg to the program:
proc = Popen(["./args/Release/arg_count", "-arg"],
stdout=PIPE, stderr=PIPE)
If you don't need to capture stderr separately from stdout then you could use check_output():
from subprocess import check_output, STDOUT
output = check_output(["./args/Release/arg_count", "-arg"]) # or
output_and_errors = check_output(["./args/Release/arg_count", "-arg"],
stderr=STDOUT)

python subprocess issue with Nmap

Im trying to get a linux binary to send its standard output to a variable by using subprocess. But just keep getting tracebacks.
>>> import subprocess
>>>nmap -sn -Pn todd.ns.cloudflare.com --script dns-check-zone --script-args='dns-check-zone.domain=www.macmonster.com
Any Ideas (oh and Im using Python2.7).
Ideally I would like to avoid using Shell=true to avoid any security concerns.
Thanks,

shlex to the rescue!
The module shlex will take a string containing the whole shell command and split it up exactly how Popen and check_output expect it. Like this:
import shlex, subprocess
cmd = "/usr/bin/nmap -sn -Pn todd.ns.cloudflare.com --script dns-check-zone --script-args='dns-check-zone.domain=www.macmonster.com'"
args = shlex.split(cmd)
output = subprocess.check_output(args)
When you look at contents of args you'll see:
>>> print args
['/usr/bin/nmap', '-sn', '-Pn', 'todd.ns.cloudflare.com', '--script', 'dns-check-zone', '--script-args=dns-check-zone.domain=www.macmonster.com']
Note that shlex split up the option "--script dns-check-zone" into two tokens. On the other hand it kept "--script-args='dns-check-zone.domain=www.macmonster.com'", but removed the single-quotes.

import subprocess
output = subprocess.check_output(["/usr/bin/nmap", "-sP", "-n", "172.16.1.0/24"])
output = subprocess.check_output(["/usr/bin/nmap", "-sP", "-n", "172.16.1.0/24"], stderr=subprocess.STDOUT)

Have you tried this:
p = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
var = p.stdout.read()
print var
I would have used communicate, but it returns an odd list-type thing.

Unwanted new lines using python Popen and pandoc to parse html?

I am trying to convert several pieces of html to latex using python and pandoc and I have got stuck with a couple of problems.
To communicate my python script with pandoc I use subprocess.Popen, redirecting stdout to a file I am saving for including it in a latex template.
If I use the classic way of implementing Popen
from subprocess import Popen, PIPE, STDOUT
filedesc = open('myfile.tex','w')
args = ['pandoc', '-f', 'html', '-t', 'latex']
p = Popen(args, stdout=PIPE, stdin=PIPE, stderr=STDOUT)
outp, err = p.communicate(input=html)
filedesc.write(outp)
I get the lines with an additional new line where there shouldn't be any:
> \textbf{M. John Harrison} (Rugby, Warckwickshire, 1945) is a contemporary
>
> English writer.
This is (misteriously?) easy to solve by changing the stdout=PIPE to the file descriptor:
from subprocess import Popen, PIPE, STDOUT
filedesc = open('myfile.tex','w')
args = ['pandoc', '-f', 'html', '-t', 'latex']
p = Popen(args, stdout=filedesc, stdin=PIPE, stderr=STDOUT)
outp, err = p.communicate(input=html)
# not needed
# filedesc.write(outp)
But if I want to use a string buffer, the same problem occurs, since i cannot use it as the stdout parameter.
Any idea on how to stop Popen/pandoc from doing this?
Thanks!

Well, it seems to be a "kind of bug" in python's PIPE (???).
I am executing this code in a Windows system. This means that when a new line is entered, they are in the CR+LF (\r\n) style rather than the (cleaner) LF (\n) new line in unix-style.
At the time I introduce a large html text to be converted by pandoc, the output is returned by the pipe to the command line. Thus, every time the standard column width is reached, an ugly "new line" character is introduced. In my case, a CR+LF. This was making my output look so weird.
The dirty solution I have implemented is to add a replace('\r\n','\n') before writing the output but I am not sure if it's the most elegant one.
from subprocess import Popen, PIPE, STDOUT
html = '<p><b>Some random html code</b> longer than 80 columns ... </p>'
filedesc = open('myfile.tex','w')
args = ['pandoc', '-f', 'html', '-t', 'latex']
p = Popen(args, stdout=PIPE, stdin=PIPE, stderr=STDOUT)
outp, err = p.communicate(input=html)
filedesc.write(outp.replace('\r\n','\n'))**strong text**

Popen.communicate escapes a string I send to stdin

I am trying to spawn a process using Popen and send it a particular string to its stdin.
I have:
pipe = subprocess.Popen(cmd, shell=True, stdin=subprocess.PIPE)
pipe.communicate( my_stdin_str.encode(encoding='ascii') )
pipe.stdin.close()
However, the second line actually escapes the whitespace in my_stdin_str. For example, if I have:
my_stdin_str="This is a string"
The process will see:
This\ is\ a\ string
How can I prevent this behaviour?

I can't reproduce it on Ubuntu:
from subprocess import Popen, PIPE
shell_cmd = "perl -pE's/.\K/-/g'"
p = Popen(shell_cmd, shell=True, stdin=PIPE)
p.communicate("This $PATH is a string".encode('ascii'))
In this case shell=True is unnecessary:
from subprocess import Popen, PIPE
cmd = ["perl", "-pE" , "s/.\K/-/g"]
p = Popen(cmd, stdin=PIPE)
p.communicate("This $PATH is a string".encode('ascii'))
Both produce the same output:
T-h-i-s- -$-P-A-T-H- -i-s- -a- -s-t-r-i-n-g-

Unless you know you need it for some reason, don't run with "shell=True" in general (which, without testing, sounds like what's going on here).