I'm trying to run a .mpkg installer on UNIX and when I run this code:
p = subprocess.Popen(['/Path/to/File.mpkg'], stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE)
return p
The output is this:
<subprocess.Popen object at 0x11384xxx>
My first question is - Is there an easier way to run a mpkg installer on UNIX?
Secondly - I can't seem to figure out how to use the subprocess.Popen object to my benefit.
Sadly I’m not sure what mpkg is, but there are two options.
Either it is a self-running package, perhaps a shell script, akin to the .run format sometimes used for Unix software. In this case, your invocation of Popen is correct, as long as you have the execute permission on File.mpkg (check with ls -l /Path/to/File.mpkg). The installer should be running fine.
Or, it is intended to be processed by a system tool, like .deb packages are handled with the dpkg program. In this case, you need something like this:
p = subprocess.Popen(['/usr/bin/dpkg', '-i', '/Path/to/File.deb'], ...)
or, alternatively:
p = subprocess.Popen('dpkg -i /Path/to/File.deb', ..., shell=True)
Now, what you do with this Popen object depends on what you want to achieve. If you wish to get the output of the process, you need to call Popen.communicate, like this:
p = subprocess.Popen(
['/Path/to/File.mpkg'],
stdout=subprocess.PIPE, stderr=subprocess.PIPE,
stdin=subprocess.PIPE)
(out, err) = p.communicate()
Now out contains the standard output and err the standard error.
If you just want to invoke the command and wait until it completes, you can also use the subprocess.call shortcut.
Related
I am writing a program that needs to know whether or not it's in a virtual machine, a neat way I found was to check the system information in the cmd, however, I can't find a way to write in the cmd and read what the cmd replies.
I would like something that can write "WMIC COMPUTERSYSTEM GET MANUFACTURER" to the cmd and also check what the results are.
You want to use the subprocess module
import subprocess
proc = subprocess.Popen(
[
'WMIC',
'COMPUTERSYSTEM',
'GET',
'MANUFACTURER'
],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE
)
out, err = proc.communicate()
print(out.decode().strip())
I'm trying to use subprocess.Popen() to run a command in my script. The code is:
output = Popen(["hrun DAR_MeasLogDump " + log_file_name], stdout=subprocess.PIPE, stderr = subprocess.PIPE, executable="/bin/csh", cwd=cwdir, encoding='utf-8')
When I print the output, it's printing out the created shell output and not the actual command that's in the list. I tried getting rid of executable='/bin/csh', but then Popen wouldn't even run.
I also tried using subprocess.communicate(), but it didn't work either. I would also get the shell output and not the actual command run.
I want to completely avoid using shell=True because of security issues.
EDIT: In many different attempts, "hrun" is not being recoognized. "hrun" is a Pearl script that is being called, DAR_MeasLogDump is the action and log_file_name is the file that the script will call its action on. Is there any sort of set up or configuration that needs to be done in order for "hrun" to be recognized?
I think the problem is that Popen requires a list of every part of the command (command + options), the documentation for Popen inside subprocess has an example for that. So for that line in your script to work, you would need to write it like this:
output = Popen(["/bin/csh", "hrun", "DAR_MeasLogDump", log_file_name], stdout=subprocess.PIPE, stderr = subprocess.PIPE)
I've removed the executable argument, but I guess it could work that way as well.
Try:
output = Popen(["-c", "hrun DAR_MeasLogDump " +log_file_name], stdout=subprocess.PIPE, stderr = subprocess.PIPE, executable="/bin/csh", cwd=cwdir, encoding='utf-8')
csh is expecting -c "full command here". Without -c I think it just tries to open it as a file.
Specifying an odd shell and an explicit cwd seems completely out of place here (assuming cwdir is defined to the current directory).
If the first argument to subprocess is a list, no shell is involved.
result = subprocess.run(["hrun", "DAR_MeasLogDump", log_file_name],
stdout=subprocess.PIPE, stderr = subprocess.PIPE,
universal_newlines=True, check=True)
output = result.stdout
If you need this to be run under a legacy version of Python, maybe use check_output instead of run.
You generally want to avoid Popen unless you need to do something which the higher-level wrapper functions cannot do.
You are creating an instance of subprocess.Popen but not executing it.
You should try:
p = Popen(["hrun", "DAR_MeasLogDump ", log_file_name], stdout=subprocess.PIPE, stderr = subprocess.PIPE, cwd=cwdir, encoding='utf-8')
out, err = p.communicate() # This will get you output
Args should be passed as a sequence if you do not use shell=True, and then using executable should not be required.
Note that if you are not using advanced features from Popen, the doc recommends using subprocess.run:
from subprocess import run
p = run(["hrun", "DAR_MeasLogDump ", log_file_name], capture_output=True, cwd=cwdir, encoding='utf-8')
out, err = p.communicate() # This will get you output
This works with cat example:
import subprocess
log_file_name='-123.txt'
output = subprocess.Popen(['cat', 'DAR_MeasLogDump' + log_file_name],
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT)
stdout, stderr = output.communicate()
print (stdout)
print (stderr)
I think you need only change to your 'hrun' command
It seems the same problem that I had at the beginning of a project: you have tried with windows "environment variables". It turns out that when entering the CMD or powershell it does not recognize perl, java, etc. unless you go to the folder where the .exe .py .java, etc. is located and enter the cmd, where the java.exe, python.py, etc. is.
In my ADB project, once I added in my environment variables, I no longer needed to go to the folder where the .exe .py or adb code was located.
Now you can open a CMD and it will execute any command even from your perl , so the interpreter that uses powershell will find and recognize the command.
I am trying to pass a pretty long bash command into my Popen command. The command is this -
'/usr/local/bin/drush --alias-path=/data/scripts/drush_aliases #test pml -y | /bin/grep -i dblog | /bin/grep -i enabled'
When passing the whole command in one go, the Popen command doesn't return the correct output in Cron. In order to remedy this, I am trying to split it apart into a list (as seen in "command") as pass it in in order to get around the issue.
In my full code, I'm chaining together several different Popen objects. However, my bug can be reproduced with only the following:
command = ['/usr/local/bin/drush', '--alias-path=/data/scripts/drush_aliases', '#test', 'pml', '-y']
try:
process = subprocess.Popen(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
output = process.communicate()
What can cause this hang?
One of the most common reasons for a process to hang is if it's trying to read input from stdin.
You can work around this by explicitly passing a closed pipe (or a handle on /dev/null) on stdin:
process = subprocess.Popen(command,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
stdin=subprocess.PIPE) ## this is new
communicate() will close the pipe passed to stdin after writing any content passed to it as an argument, preventing the process from hanging.
In Python 3.2 or newer, you can also use stdin=subprocess.DEVNULL.
I am trying to find the path to the MATLAB executable using Python when it is not in PATH. I am using subprocess.Popen to execute locate and grepping the result, however this creates a Resource Unavailable error:
locate = subprocess.Popen(['locate', 'matlab'], stdout=subprocess.PIPE)
grep = subprocess.Popen(['grep', '/bin/matlab$'], stdin=locate.stdout, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
result, err = grep.communicate()
MATLAB_PATH = result.decode('UTF-8').split()
The result variable is empty and err variable is :
b'grep: (standard input): Resource temporarily unavailable\n'
I have tried your code on linux with python 3.5.2 and 3.6.1 and it does work:
locate = subprocess.Popen(['locate', 'find'], stdout=subprocess.PIPE)
grep = subprocess.Popen(['grep', '/bin/find$'], stdin=locate.stdout, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
grep.communicate()
(b'/usr/bin/find\n', b'')
For the records: locate find gives 1619 lines.
For completeness I have also tried locate fdafad (gibberish) and it also works.
It does also work when the code is in a script.
edit:
Try to use communicate to interact between to two processess:
locate = subprocess.Popen(['locate', 'find'], stdout=subprocess.PIPE)
stdout, stderr = locate.communicate()
grep = subprocess.Popen(['grep', '/bin/find$'], stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
print(grep.communicate(input=stdout))
NOTE: the second part of the answer has been written before the asker updated the question with information about the PATH
However there is a much better ways to find executables using python:
from distutils.spawn import find_executable
find_executable('find')
'/usr/bin/find'
If you insist in using shell functions, why don't use something like which.
Adding just a little bit more information on why this error occurred.
This looks to be a problem with eventlet using "green" threads and non-blocking IO and locate not generating output fast enough. That is, eventlet assumes that the stdout is to be used by python. Eventlet uses non-blocking IO to assist in cooperative threading. This means the file descriptor behind locate.stdout that you pass to grep has already been set to non-blocking. If grep tries to read from stdin when it is empty then you will get that error.
An easy solution would be to do both commands in a single shell (so Python doesn't get to mess with the pipe between the two sub-processes).
eg.
result = subprocess.check_output('locate matlab | grep /bin/matlab$', shell=True).strip()
I'm new to Python, and haven't used Linux in years, so I'm not sure where I'm getting tangled up. I'm trying to use Popen to run sql files in MySQL on Ubuntu.
Here is the relevant code:
command = ['mysql', '-uUSER', '-pPWD','-h192.168.1.132', '--database=dbName', '<', './1477597236_foo.sql' ]
print("command is: "+subprocess.list2cmdline(command))
proc = subprocess.Popen(
command, stderr=subprocess.PIPE, stdout=subprocess.PIPE, cwd='.'
)
the output from this is the same as if had run 'mysql --help'. The puzzling thing to me is that if i take the command output by subprocess.list2cmdline and run it directly, it runs perfectly. Also, if i replace '< file.sql' with '-e select * from foo', it runs. So, the '<' and file are causing my problem. I know WHAT is causing the problem, but nothing I've tried so far has fixed it.
tia, Craig
When a redirection or pipe or built-in command is present in the command line, shell=True is required. However, in simple cases like this, shell=True is overkill. There's a much cleaner way in order to avoid shell=True which gives better control on the input file.
if the input file doesn't exist, you get an exception before reaching the subprocess, which is easier to handle
the process runs without the shell: better portability & performance
the code:
command = ['mysql', '-uUSER', '-pPWD','-h192.168.1.132', '--database=dbName' ]
with open('./1477597236_foo.sql') as input_file:
proc = subprocess.Popen(
command, stdin = input_file, stderr=subprocess.PIPE, stdout=subprocess.PIPE )
output,error = proc.communicate()
(I added the next line which should be a communicate call: since both stdout & stderr are redirected, it's the only simple way to avoid deadlocks between both output streams)
So you need to add shell=True to your Popen call. < is a part of the shell and you can't use shell features without that parameter.
proc = subprocess.Popen( command, stderr=subprocess.PIPE, stdout=subprocess.PIPE, cwd='.',shell=True )