I'm using the following python script on Windows Server 2008:
import cgitb
import subprocess
cgitb.enable()
print "Content-Type: text/plain;charset=utf-8"
print
cmd = "git tag"
process = subprocess.Popen(cmd.split(), shell=True, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
git_output = process.communicate()[0]
print "git output = %s" % git_output
There are, in fact, some git tags. Running this script through the shell works perfectly. However, when run through IIS (7), the output seems to be empty.
I've tried directing the Popen output to a file instead of PIPE. Again, worked when running from the command line, didn't work when running through IIS.
Any ideas?
EDIT:
Following #Wooble's advice, I removed the [0] from the call to communicate to see git errors, and indeed found the enigmatic error "'git' is not recognized as an internal or external command, operable program or batch file." Of course git is installed on the system, and as I said the script works when run directly through the command line.
To no avail, I tried:
Setting the command to use the full path of the git executable
Adding the full path of the git executable directory to python's sys.path
Copying the actual git executable to the working directory - this removed the "git not recognized" error but still yielded an empty result!
Please help!!
I don't know why the path was partial when running from IIS (explanations welcome!), but adding this finally solved the problem:
git_path = "C:\\Program Files (x86)\\Git\\bin\\"
os.environ["PATH"} += git_path + ';'
Related
I'm using a software that handle video processing in a powershell subprocess.
With PyCharm, when I run my software (in debug mode) everything works as expected.
When I use pyinstaller and inno setup to make an executable and install it on windows I got this error when Sub process start :
[WinError 6] The handle is invalid
I suppose it’s due to an error in a subprocess like this piece of code :
try:
psa1_path = EnvValues.powershell_path().format(project=project)
# using this powershell : C:/Users/${USERNAME}\projects\demo\cmd\powershell.ps1 -m -v 'CC 2018' -wait windowstyle hidden
dc_logger.info(f'using this powershell : {psa1_path}')
if project:
dc_logger.info("PowerShell Rendering Started")
with open(EnvValues.frame_path(), 'a') as f:
p = subprocess.Popen(['C:\\WINDOWS\\system32\\WindowsPowerShell\\v1.0\\powershell.exe',
'-ExecutionPolicy',
'Unrestricted',
psa1_path],
stdout=f)
p.communicate()
dc_logger.info("PowerShell Done Rendering")
return True
else:
dc_logger.info("no project to render")
return False
Is the bug related to arguments passed to the subprocess ? Why executable version of the code is the only one not working ? Why I don't have the bug in the development version ?
Here's my pyinstaller cmds :
pyinstaller --onefile -w -F -i "C:\Users\my_project\icon.ico" Project.py
pyinstaller --onefile -w -F -i "C:\Users\my_project\icon.ico" Project.spec
Then I put this in InnoSetup and I install the output to my windows machine.
The problem is with psa1_path variable
C:/Users/${USERNAME}\projects\demo\cmd\powershell.ps1 -m -v 'CC 2018' -wait windowstyle hidden
This variable has parameters. And the subprocess.Popen use it as a string, then you have to set shell=True so Popen will use this string as a complete shell cmd.
p = subprocess.Popen(['C:\\WINDOWS\\system32\\WindowsPowerShell\\v1.0\\powershell.exe',
'-ExecutionPolicy',
'Unrestricted',
psa1_path],
stdout=f, shell=True, stdin=subprocess.DEVNULL)
Don't forget to add stdin arg because it's also throw the [WinError 6] The handle is invalid
So why the code works with debug code and not with executable :
It's mainly because PyCharm does additionnal configuration and setup behind the scenes when it comes to running a program.
Because when you’re going from IDE to runtime, you need additional hooks to get things going. pyinstaller doesn’t do the subprocess the same way as pyCharm for the shell part
Overview:
I have some C++ code (myProgram) that I want to execute from a Lambda function. I have built the code on an Amazon Linux EC2 instance, and it runs there without problem.
When I try to run it from a Python script in Lambda, I receive the following error message:
Command 'LD_LIBRARY_PATH=/var/task/lib; /var/task/myProgram -d' returned non-zero exit status 126.: CalledProcessError
I see from the Bash Reference Manual that 126 means the command is found but is not executable. It is executable before I upload it to Lambda, so I'm not sure how to resolve this.
Edit: Thanks to the comment below, I double checked the permissions on the Lambda instance. Indeed, the file is not marked as executable after it has been uploaded. When I try changing it with chmod, I get an error Read-only file system.
Details:
When I package this and upload it to my lambda function, the directory structure looks like this:
handler.py
myProgram
lib
(required libraries)
Here is the code for handler.py
import os
import subprocess
exepath = os.path.join(os.getcwd(), 'myProgram')
libdir = os.path.join(os.getcwd(), 'lib')
def handler(event, context):
command = 'LD_LIBRARY_PATH={}; {} -d'.format(libdir, exepath)
return subprocess.check_output(command, shell=True, stderr=subprocess.STDOUT)
If I change shell=True to shell=False then I receive a different error
[Errno 2] No such file or directory: 'LD_LIBRARY_PATH=/var/task/lib; /var/task/myProgram -d': FileNotFoundError
Question:
How can I make myProgram executable? I don't see any mention of this process in the Lambda blog on executables
I figured out the solution to this thanks to the tip from Yakk and the answer to Can't run binary from within python aws lambda function.
I had to copy myProgram to /tmp and then chmod to be able to execute the it.
command = 'cp ./myProgram /tmp/myProgram; chmod 755 /tmp/myProgram; LD_LIBRARY_PATH={}; /tmp/myProgram -d '.format(libdir)
return subprocess.check_output(command, shell=True, stderr=subprocess.STDOUT)
I am attempting to scrape a terminal window of the list of fonts installed on the curent hosting server. I have written the following code:
import subprocess
cmd = 'fc-list'
output = subprocess.Popen(cmd, stdout=subprocess.PIPE ).communicate()[0]
but when i call this code, an exception is raised:
[Errno 2] No such file or directory
I can open a terminal window, and this works fine. What am i doing wrong?
You need to provide the absolute path to the executable. When you open a terminal window you then have a shell running which will search in $PATH to find the program. When you run the program directly, via subprocess, you do not have a shell to search $PATH. (note: it is possible to tell subprocess that you do want a shell, but usually this leads to security vulnerabilities)
Here is what you would want to use:
import subprocess
cmd = '/usr/local/bin/fc-list'
output = subprocess.Popen(cmd, stdout=subprocess.PIPE ).communicate()[0]
In Python I'm using subprocess to call gsutil copy and move commands, but am currently unable to select multiple extensions.
The same gsutil command works at the terminal, but not in python:
cmd_gsutil = "sudo gsutil -m mv gs://xyz-ms-media-upload/*.{mp4,jpg} gs://xyz-ms-media-upload/temp/"
p = subprocess.Popen(cmd_gsutil, shell=True, stderr=subprocess.PIPE)
output, err = p.communicate()
If say there are four filetypes to move but the bucket is empty, the returning gsutil error from terminal is:
4 files/objects could not be transferred.
Whereas the error returned when run through subprocess is:
1 files/objects could not be transferred.
So clearly subprocess is mucking up the command somehow...
I could always inefficiently repeat the command for each of the filetypes, but would prefer to get to the bottom of this!
It seems, /bin/sh (the default shell) doesn't support {mp4,jpg} syntax.
Pass executable='/bin/bash', to run it as a bash command instead.
You could also run the command without the shell e.g., using glob or fnmatch modules to get the filenames to construct the gsutil command. Note: you should pass the command as a list in this case instead.
I have an R script that makes a couple of plots. I would like to be able to execute this script from python.
I first tried:
import subprocess
subprocess.call("/.../plottingfile.R", shell=True)
This gives me the following error:
/bin/sh: /.../plottingfile.R: Permission denied
126
I do not know what the number 126 means. All my files are on the Desktop and thus I do not think that any special permissions would be needed? I thought that this error may have had something to do with cwd = none but I changed this and I still had an error.
Next I tried the following:
subprocess.Popen(["R --vanilla --args </.../plottingfile.R>"], shell = True)
But this too gave me an error with:
/bin/sh: Syntax error: end of file unexpected.
Most recently I tried:
subprocess.Popen("konsole | /.../plottingfile.R", shell = True)
This opened a new konsole window but no R script was ran. Also, I received the following error:
/bin/sh: /.../plottingfile.R: Permission denied
Thanks.
First thing first, make sure that you have your platttingfile.R script at a place where you can access. Typically it is the same directory.
I read in the internet that there is a utility that comes called RScript which is used to execute R script from the command line. So in order to run the script you would use python like this:
import subprocess
retcode = subprocess.call(['/path/to/RScript','/path/to/plottingfile.R'])
This would return the retcode 0 upon successful completion. If your plottingfile.R is returning some kind of an output, it will be thrown on STDOUT. If it pulling up some GUI, then it would come up.
If you want to capture stdout and stderr, you do it like this:
import subprocess
proc = subprocess.Popen(['/path/to/RScript','/path/to/plottingfile.R'], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = proc.communicate()
Shell error 126 is an execution error.
The permission denied implies that you have a "permission issue" specifically.
Go to the file and make sure R/Python is able to access it.
I would try this out first:
$sudo chmod 777 /.../plottingfile.R
If the code runs, give it the correct but less accessible permission.
If this doesn't work, try changing R to Rscript.
have you tried chmod u+x /pathTo/Rscript.R ?
something likes this work usually for me:
subprocess.Popen("R --vanilla /PATH/plottingfile.R", shell = True)