I feel a little foolish that I don't know this, but I tried to do it today and was surprised when it didn't work....
I have a directory C:\test with a demo script, lets call it demo.py
If i am in C:\test then I can just do python demo.py. Easy
I could also use a relative path, so from C:\, it's python test\demo.py
What if C:\test is on the path?
I was expecting to be able to now do python demo.py from anywhere however...
python: can't open file 'demo.py': [Errno 2] No such file or directory
I feel foolish because I thought this was straightforward, but I have searched around and have not found a solution. Am I fundamentally misunderstanding something here about how the Python interpreter finds scripts to run? I don't think this is anything to do with PYTHONPATH, as I understood that to relate to loading of modules inside scripts.
This is on Windows 7, by the way.
The PATH is only used to search for commands. A first way is that a Python script can be used directly as a command and in that case the PATH will be used: just use demo.py instead of python demo.py.
It will rely on OS specific ways. On Windows, file type (given by the extension - here .py) can be given default application to process them, while on Unix-like, the first line of a script can declare the program that will process it.
Alternatively, python allows to launch a module that will be searched in the PYTHONPATH (not PATH) by using python -m module or for Windows py -m module.
Related
I understand the difference between starting a python script like:
#!/usr/bin/env python
or
#!/usr/bin/python
From what I understood, just executes python as we would do in our shell, so it looks in $PATH. The second one is not a fixed path, which has the inconvenient that in a different system the python interpreter might be located in another path.
My question is, why do we need env? Why can we just do:
#!python
This works perfectly fine in my computer? Is there a reason to prefer calling env?
Short answer:
This depends on the shell. in bash #!python will just be ignored and you have to use #!/usr/bin/env python. zsh on the other hand seems to be able to handle #!python.
The long answer (for bash):
let's imagine your python file is named 'tst.py'
and has following contents
#!python
import sys
print(sys.executable, sys.version)
then you can always type
python tst.py
The first line is completely irrelevant and is not even looked at.
However if you do following (for example on linux)
chmod +x tst.py
./tst.py
Then the first line is looked at to determine which interpreter shall be used (bash, perl, python, something else?) and here at least for my OS (ubuntu) and my shell (bash) an absolute path is required for the executable name (e.g. /bin/bash, /bin/python, /usr/bin/env)
If I call ./tst.py on my ubuntu machine I get
bash: ./tst.py: python: bad interpreter: No such file or directory
Special case Windows, when typing tst.py or clicking on a python script.
If you're on windows, the line is looked at, but even if it is wrong a default python interpreter will be used. On Windows, this line could be used to select explicitly python2 or python3 for example.
Windows uses file type associations to determine which executable to call for which suffix. for .py files (python 3.x is installed) this is normally py.exe which is an executable located in the system path, that will just call a python interpreter. depending on installed versions, the shebang line and environment vars, that might indicate a virtualenv
Addendum:
The first line is interpreted by the shell or in the windows case by py.exe.
It seems bash requires absolute paths for command, whereas zsh accepts relative paths as well.
So for zsh only
#!python
works perfectly well whereas bash requiers absolute paths and thus the trick with the env command
#!/usr/bin/env python
For scripts that shall be executed by a cronjob it's best to hardcode the path of the python executable as PATH is rather minimalistic for cronjobs, so there it's best to have something like
#!/usr/bin/python3.5
or
#!/home/username/myvirtualenv/bin/python
Each Python script is written with a particular version of Python in mind. If the shebang is #!/usr/bin/env python, that puts the version used under the control of the individual caller, whose PATH may not provide the right version.
#!/usr/bin/python is a little better, as it puts the decision in the hands of the script itself. However, the script author doesn't necessarily know where the correction version of Python is on your system, so it still may not work.
The solution is to specify the correct location on your system when you install the module or script. The Python installer (pip, etc) will, at that time, rewrite any shebang containing the word python (#!python being the minimal such shebang) to the path specified by the installer.
Now when you run the script, it will point to the correct path that you have already provided, without being subject to runtime PATH lookup that could choose the wrong version.
Note that #!python is not meant to be used as-is, though for various reasons it might work for you. It's is a means towards making sure the script gets a fixed, absolute path to the correct interpreter. #!/usr/bin/python is also subject to install-time replacement, so may serve as a usable default.
In Windows 8, I often use the Python Windows Launcher like
py C:/long/long/long/long/long/path/to/prog.py ...
Is there any way to set some environment setting, such as PATH or PYTHONPATH etc, to prevent having to type the full path to prog.py?
From my basic knowledge/research, PATH only helps with the py part of the command line and PYTHONPATH only helps with imports within prog.py, so how do I deal with the path to prog.py itself??
Notes:
I cannot modify the code, not even the "shebang" line, since it is needed to work on other platforms.
I cannot cd to the directory containing the programs to run them, because the programs will do something based on the directory they're run in (they'll modify the files in the directory they're run in).
I know that if I associate .py extension with the Python Windows Launcher, then I can run prog.py as the first item in the command line, and thus use PATH, but currently my .py extension is associated with my favorite editor and I'd like to keep it that way if possible (so I can double-click any Python file in Windows Explorer and edit it).
However, if someone suggests a solution where I can have a different association for Windows Explorer versus the command line, then that could be a potential solution! (i.e. in Windows Explorer, .py opens with the editor, while on command line, .py runs with Python Windows Launcher)
Add your long path to PYTHONPATH, then invoke your program as such:
python -m prog
Python will search for a module called prog and then run it as the main module.
Answer to my own question: Actually, I'm so silly. I could just set a variable for each program path (there are only a few programs paths), i.e.. prog=C:/long/path/to/prog.py and then do py %prog% .... I guess I figured out an answer to my own question that was acceptable to me.
Update: I just found something even better. I can do
doskey prog=py C:/long/path/to/prog.py $*
and then simply prog ... afterward
Now I just have to do some crazy stuff to get the doskey command into a file that will be run every time I start a console, as described here: https://stackoverflow.com/a/21040825/5182136
Not a major issue but just an annoyance I've come upon while doing class work. I have my Notepad++ set up to run Python code straight from Notepad++ but I've noticed when trying to access files I have to use the full path to the file even given the source text file is in the same folder as the Python program being run.
However, when running my Python program through cmd I can just type in the specific file name sans the entire path.
Does anyone have a short answer as to why this might be or maybe how to reconfigure Notepad++?
Thanks in advance.
The problem is that your code is assuming that the current working directory is the same as the script directory. This is not true in general. Of course it is true if you're in a cmd window, and you cd to the script directory before running it.
If you don't want to rely on that (e.g., because you want to be able to run scripts from Notepad++, or directly from Explorer), what you want to do is use the script directory explicitly. For example:
import os
import sys
scriptdir = os.path.abspath(os.path.dirname(sys.argv[0]))
with open(os.path.join(scriptdir, 'myfile.txt')) as f:
# etc.
If you have a ton of files that your scripts reference in a ton of places, it might be better to explicitly set the working directory. Just add one line:
os.chdir(scriptdir)
For anything beyond quick&dirty scripts, it's usually better to build an installable package and use pkg_resources to access the data files. Read the Tutorial on Packaging and Distributing Projects for more details. But as long as you're only hacking up scripts to help you maintain your specific system, the scriptdir solution is workable.
In the properties of the shortcut that you use to start Notepad++, you can change its working directory, to whichever directory you're more accustomed to starting from in Python. You can also begin your python program with the appropriate os.chdir() command.
I need to run a python file using an older python 2.6 version installed with an external software.
To do this I have resorted to using os.system such as
os.system('""path/old_python.exe" "file.py""')
(note that the odd no. of ('") is due to the path containing a space c:/program files (x86)/.. as I am running on windows.)
This code string works well if run from a root directory. However, I would like to place this os.system call in a module within a sub-package to my root and thereafter run it from a root module. So the hierarchy would look like this:
/root
call_os_module.py
/subpack1
os_module.py
file.py
If I run this I receive the error:
path/old_python.exe: can't open file 'file.py': [Errno 2] No such file or directory
I have added the full path to subpack1 to sys.path. However, I still receive the same error and os can't find the file. How do I solve this?
Have a look at these questions:
How in Python to run external Python script with the current Python interpreter?
Relative paths in Python
in general: https://stackoverflow.com/search?q=python+relative+path
Is there a portable way to run a python script from a shell without writing its full path?
For example in Linux, I would like while in my home directory
cd ~
to be able to run a python script called run.py that is in say, ~/long/path/to/run.py, but I want to run it by simply typing
python run.py
instead of
python ~/long/path/to/run.py
I would hope for some kind of search path list that contains several directories just like the PATH variable, so that python run.py runs the first run.py it encounters in one of the directories.
I have considered turning run.py into an executable and adding its directory the system PATH variable, but could not find a portable way of making a python script executable.
EDIT
One year later after asking it, I am a bit less noob, and I see that my question was not very clear and did not make much sense, so after a question upvote I'll clarify some things.
1) Portable.
When I asked this I said portable. However what portable means is not clear in this case, and I did not give much emphasis to it.
the platforms: should work on POSIX (Linux, MacOS, etc.) and Windows
this still does not make much sense since windows uses cmd.exe, and POSIX uses sh, so each one could run the commands with a different syntax. So let's say that the most portable thing possible would be to feed the same input to both sh and cmd.exe, running the python script in both cases. In this case, you could run the same command from an ANSI C system function, which uses sh on POSIX and cmd on Windows. ANSI C being one of the few things that is common to Windows and POSIX, the question makes some sense in that case.
2) Executable
Next, the phrase turning run.py into an executable, is not very clear. By that I was talking about the Linux strategy of chmod +x run.py, add a shebang #!/usr/bin/env python, and adding its directory the system add ~/long/path/to/ the PATH enviroment variable. But then this won't work for windows because windows does not support an executable file metadata property like Linux and because /usr/bin/env does not necessarily exist in Windows.
3) Extension
Finally, in my head I was hoping for a solution that does not specify what kind of file run is, so that if someday we decide to make it, say, a perl file, no interfaces would change.
Therefore, writing run.py would be bad because it would specify the filetype; it would be better to be able to write just run
If the directory containing run.py is on the module search path (for example, PYTHONPATH environment variable), you should be able to run it like this:
python -m run
Here is the documentation on the -m command line option:
-m module-name
Searches sys.path for the named module and runs the corresponding .py file as a script.
You can make a python script executable by adding
#!/usr/bin/env python
to the beginning of the file, and making it executable with chmod +x.
Answer after the clarification edit
I prefer the following approach to the one suggested by #F.J. because it does not require users to specify the file type. Please note that this was not specified in the original question, so his answer to the original question was correct.
Lets call the file pytest.py to avoid conflicts with a possible existing run program.
On POSIX (MacOs, linux) do what #Petr said, which is based on what #alberge said:
chmod +x
add shebang #!/usr/bin/env python
create a directory and add it to path. Usual locations on Linux are: ~/bin/ for a single user, /usr/local/bin/ for all users
symlink (cp -s) the file under your PATH with basename pytest instead of pytest.py
On windows:
create a dir and add it to PATH. AFAIK, there is no conventional place for that, so why not C:\bin\ and ~\bin\?
add .PY to the PATHEXT environment variable so that Windows will recognize files with python extension as runnable files without requiring you to type the extension
associate python files with the python.exe interpreter (Windows Explorer > right click > check "Always use the selected program"). There is an option on the python installer that does this for you.
symlink pytest with extension into the dir under PATH (using Link Shell Extension from Windows Explorer or mklink name dest from cmd )
Now system( "pytest" ); should work in both systems ( sh under Linux, cmd under Windows )
Make python file executable (as "alberge" stated above)
Create some directory and put this directory into your PATH variable
In this directory, create links to your python scripts