I put one new alias python3.3 in my .bash_profile so as to launch easily the python3.3 version of pyzo.
I can use this alias in a terminal without any problem, but when I use something like subprocess.check_call(args = ["python3.3", onePyFile]), I have the following error.
Traceback (most recent call last):
...
File "/Library/Frameworks/pyzo2013b/lib/python3.3/subprocess.py", line 540, in check_call
retcode = call(*popenargs, **kwargs)
File "/Library/Frameworks/pyzo2013b/lib/python3.3/subprocess.py", line 521, in call
with Popen(*popenargs, **kwargs) as p:
File "/Library/Frameworks/pyzo2013b/lib/python3.3/subprocess.py", line 818, in __init__
restore_signals, start_new_session)
File "/Library/Frameworks/pyzo2013b/lib/python3.3/subprocess.py", line 1416, in _execute_child
raise child_exception_type(errno_num, err_msg)
FileNotFoundError: [Errno 2] No such file or directory: 'python3.3'
I guess that my alias is not seen everywhere. So how can I fix my problem ? What is the good way to build my own alias ?
If I try subprocess.check_call(args = ["python3.3", onePyFile], shell = True), I have the following error.
onePyFile.py: python3.3: command not found
Traceback (most recent call last):
File "mainFile.py", line 72, in <module>
shell = True
File "/Library/Frameworks/pyzo2013b/lib/python3.3/subprocess.py", line 545, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['python3.3', 'onePyFile.py']' returned non-zero exit status 127
If i just use subprocess.check_call(args = ["python3.3", onePyFile]) where the first line of onePyFile is #! /usr/bin/env python3.3, I have the following error.
env: python3.3: No such file or directory
Traceback (most recent call last):
...
I think that my problem is more about the symbolic link than the Python call. But I do not know what is wrong. Indeed this is the first time that I make a personal symbolic link with an alias.
try subprocess.check_call(args = ["python3.3", onePyFile] , shell=True, env={'ENV':path_of_bash_profile})
If shell is True, the specified command will be executed through the shell. This can be useful if you are using Python primarily for the enhanced control flow it offers over most system shells and still want convenient access to other shell features such as shell pipes, filename wildcards, environment variable expansion, and expansion of ~ to a user’s home directory. However, note that Python itself offers implementations of many shell-like features (in particular, glob, fnmatch, os.walk(), os.path.expandvars(), os.path.expanduser(), and shutil).
That's because subprocess doesn't load a shell by default (see the docs), so it doesn't get what's in your .bash_profile.
Use this:
subprocess.check_call(args = ["python3.3", onePyFile], shell=True)
Edit: Seems like glasslion was faster than me!
Edit 2: I did a bit more digging around, and found something strange. Because shell=True didn't seem to work as expected I went with a more direct approach, calling bash directly.
a.py:
from subprocess import check_call
check_call(['bash', '-c', '. ~/.bash_profile && ABC a bc'])
a.sh (what would be your python3 executable):
echo $*
I first tried with alias ABC='~/Documents/a.sh' in .bash_profile:
$ python ~/Documents/a.py
bash: ABC: command not found
Traceback (most recent call last):
File "Documents/a.py", line 6, in <module>
check_call(['bash', '-c', '. ' + expanduser('~/.bash_profile') + ' && ABC a bc'])
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/subprocess.py", line 511, in check_call
subprocess.CalledProcessError: Command '['bash', '-c', '. ~/.bash_profile && ABC a bc']' returned non-zero exit status 127
Then I switched from an alias to a function: ABC() { ~/Documents/a.sh $*; }
And it worked!
$ python ~/Documents/a.py
a bc
Bottom line is I got it to work, but I don't know why! Shells aren't reliable, so the best would be to skip the shell.
We can do that by using the shebang's principle (as Fred Mitchell suggested) in a clever way:
from subprocess import check_call
check_call(['/usr/bin/env', 'python3', onePyFile])
This will work if Python 3 was installed correctly, independently of it's path (what I suppose you wanted to achieve).
Why not just put a shebang to python3.3 in the first line of onePyFile and omit running python explicitly?
Related
Python IDLE shows an error when I am trying to extract files using winrar(UnRAR.exe):
"Traceback (most recent call last):
File "<pyshell#32>", line 1, in <module>
response=subprocess.check_output(['"C:\\Users\\B74Z3\\Desktop\\Test\\UnRAR.exe" e -p123 "C:\\Users\\B74Z3\\Desktop\\Test\\Test.rar"'], shell=True)
File "C:\Program Files\Python 3.5\lib\subprocess.py", line 629, in check_output
**kwargs).stdout
File "C:\Program Files\Python 3.5\lib\subprocess.py", line 711, in run
output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['"C:\\Users\\B74Z3\\Desktop\\Test\\UnRAR.exe" e -p123 "C:\\Users\\B74Z3\\Desktop\\Test\\Test.rar"']' returned non-zero exit status 1"
What is the problem with the Code:
import subprocess
response=subprocess.check_output(['"C:\\Users\\B74Z3\\Desktop\\Test\\UnRAR.exe" e -p123 "C:\\Users\\B74Z3\\Desktop\\Test\\Test.rar"'], shell=True)
I'd comment on this, but I don't have enough reputation to do so.
Try running the command without the shell interface, that is,
response=subprocess.check_output(["""C:\Users\B74Z3\Desktop\Test\UnRAR.exe""", "e", "-p123', """C:\Users\B74Z3\Desktop\Test\Test.rar"""])
I've also remove the complexity of adding additional backslashes from your command by using triple quotes. This is almost more precise in that you know exactly what command and arguments is being run.
Also on windows the shell=True is not needed unless you're running a shell built in command, https://docs.python.org/3/library/subprocess.html#popen-constructor:
On Windows with shell=True, the COMSPEC environment variable specifies the default shell. The only time you need to specify shell=True on Windows is when the command you wish to execute is built into the shell (e.g. dir or copy). You do not need shell=True to run a batch file or console-based executable.
As the title says:
>>> from subprocess import check_output
>>> check_output(['ln', '~/other_folder/src/models/sc_models.py', './src/models/sc_models.py'])
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "/usr/lib/python2.7/subprocess.py", line 573, in check_output
raise CalledProcessError(retcode, cmd, output=output)
CalledProcessError: Command '['ln', '~/other_folder/src/models/sc_models.py', './src/models/sc_models.py']' returned non-zero exit status 1
>>> exit()
$ ln ~/other_folder/src/models/sc_models.py ./src/models/sc_models.py
$
How can this be? How can it succeed from the command line, but fail from the Python subprocess call?
All tips are welcome!
You need to use os.path.expanduser:
On Unix and Windows, return the argument with an initial component of ~ or ~user replaced by that user‘s home directory.
import os
os.path.expanduser('~/other_folder/src/models/sc_models.py')
In [2]: os.path.expanduser("~")
Out[2]: '/home/padraic'
Python is looking for a directory named ~ in your cwd which obviously fails. When you run the code from bash the ~ is expanded, unless you were to use shell=True where the command would be passed to the shell and the shell would expand the tilde then you would need to use os.path.expanduser or pass the whole path i.e /home/user/other_folder...... I would stick to using shell=False withos.path.expanduser("~").
Here's my copy.py:
from subprocess import call
call("copy p2.txt p3.txt")
If in command prompt I use
copy p2.txt p3.txt it copies fine.
but when I use python copy.py it gives me:
Traceback (most recent call last):
File "copy.py", line 2, in <module>
call("copy p2.txt p3.txt")
File "C:\Python27\lib\subprocess.py", line 493, in call
return Popen(*popenargs, **kwargs).wait()
File "C:\Python27\lib\subprocess.py", line 679, in __init__
errread, errwrite)
File "C:\Python27\lib\subprocess.py", line 896, in _execute_child
startupinfo)
WindowsError: [Error 2] The system cannot find the file specified
If I replace the python call to copy with xcopy, it works fine.
Why would this be?
When subprocess.call()ing a command like in a shell, you'll need to specify shell=True as well.
from subprocess import call
call("copy p2.txt p3.txt", shell=True)
The reason you need to use shell=True in this case is that the copy command in Windows is not actually an executable but a built-in command of the shell (if memory serves right). xcopy on the other hand is a real executable (in %WINDIR%\System32, which is usually in the %PATH%), so it can be called outside of a cmd.exe shell.
In this particular instance, shutil.copy or shutil.copy2 might be viable alternatives.
Please note that using shell=True can lead to security hazards, or as the docs put it:
Warning: Using shell=True can be a security hazard. See the warning under Frequently Used Arguments for details.
Im trying to call a shell script from my python with a parameter but the parameter is not being passed
My Shellscript:
echo "Inside shell"
echo $0
echo $1
cd $1
pwd
for file in *.csv
do
split -l 50000 -d -a 4 "$file" "$file"
done
echo "Outside shell"
with shell=True
this_dir = os.path.dirname(os.path.abspath(__file__))
cmd = [os.path.join(this_dir,'split.sh'),fileslocation]
print 'cmd = ', cmd
process = subprocess.Popen(cmd,shell=True)
The parameters are not being passed correctly...
with Shell=True removed
cmd = ['/opt/sw/p3/src/PricesPaidAPI/split.sh', '../cookedData']
Traceback (most recent call last):
File "csv_rename.py", line 23, in <module>
process = subprocess.Popen(cmd)
File "/usr/lib/python2.7/subprocess.py", line 679, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1249, in _execute_child
raise child_exception
OSError: [Errno 8] Exec format error
Ok, I dunno if this should be a dupe, or not (as per comments). But it is a fact that shebang helps. And here is the confirmation.
popen_test.py:
import subprocess
subprocess.Popen(["./dummy.sh", "test"])
noshebang.sh:
echo "Echoing: $1"
This results with OSError: [Errno 8] Exec format error. It is because OS expects the file to be executable (duh). However, a #! - shebang in a 1st line is a special mark for the system, that the file should be executed in the shell environment specified by it. So it can be treated as an executable, even though it is not. So:
shebang.sh:
#!/bin/sh
echo "Echoing: $1"
Works.
It is essentially the same as python's shell=True. Just the system takes care of it. IMO it is a bit harder to inject random code with this approach. So I would suggest going for it.
So i'm trying to setup the python code checkers suggested in the emacs wiki. However, I'm unable to run those scripts in my command shell let alone emacs.
The section is found here:
http://www.emacswiki.org/emacs/PythonProgrammingInEmacs#toc7
And I tried the script located here and here
In both cases I changed the first line from #!usr/bin python with the full path of my python executable and when I run the scripts via
python pylint_etc_wrappers.py someModule.py
or
python pycheckers.py soemModule.py
both boil down to the same error, most likely because they try to open a subprocess. Here's the trace:
Traceback (most recent call last):
File "pycheckers.py", line 254, in <module>
runner.run(source_file)
File "pycheckers.py", line 91, in run
process = Popen(args, stdout=PIPE, stderr=PIPE)
File "C:\devel\Python\Python-2.7\Lib\subprocess.py", line 672, in __init__
errread, errwrite)
File "C:\devel\Python\Python-2.7\Lib\subprocess.py", line 882, in _execute_child
startupinfo)
WindowsError: [Error 2] The system cannot find the file specified
The second script suggests to change the first line to the path of the interpreter (which I did) and to change the path in the main function which looks something like :
os.environ['PATH'] = \
path.dirname(sys.executable) + ':' + os.environ['PATH']
which was a bit unclear to me. Any ideas?
I have pylint 0.25.1, installed using easy_install (Python 2.7, Win XP). Both pylint and pylint.bat were installed in Python27/Scripts (this directory is in my PATH).
I too get the "The system cannot find the file specified" error when running the pylint_etc_wrapper.py script unchanged.
Running pylint from the script does work if
command = 'pylint'
is changed to
command = 'pylint.bat'
Another way to make it work is to add shell=True to the Popen() call.
I can't really explain all this, but there is an unresolved Python bug that looks like it might be relevant: http://bugs.python.org/issue8557.