I use python3 with emacs (editor and shell) under Linux OS. Why the cwd is not in the sys.path ?? How can we put it, for all sessions !!
I Thank you.
You do not want to add cwd() to the sys.path. Always adding cwd() would be a terrible idea as you can no longer control what files are available for import.
Python adds the directory of the script being executed instead.
E.g. when you run:
python.exe path/to/script.py
then path/to is automatically added to the sys.path.
Only if you run a script from the current directory is '' added to the start of the path, meaning the current working directory is searched for imports. E.g. when you run python.exe localfile.py then Python does add the current working directory, in the assumption you wont't change the current working directory while importing.
See Interface options in the Command line and environment documentation:
If the script name refers directly to a Python file, the directory containing that file is added to the start of sys.path, and the file is executed as the __main__ module.
and the sys.path documentation:
As initialized upon program startup, the first item of this list, path[0], is the directory containing the script that was used to invoke the Python interpreter. If the script directory is not available (e.g. if the interpreter is invoked interactively or if the script is read from standard input), path[0] is the empty string, which directs Python to search modules in the current directory first. Notice that the script directory is inserted before the entries inserted as a result of PYTHONPATH.
You can always add the current working directory to sys.path explicitly:
import sys
if sys.path[0] != '':
sys.path.insert(0, '')
Be careful, any python file or package in that working directory with a name matching a module you are already using in your code will mask that module, easily leading to breakage.
sys.path isn't the System Path, it's the path that python checks when looking for modules (when you execute import statements, for example).
To change it permanently, change the environment variable PYTHONPATH.
You can change your PYTHONPATH environmental variable.
This prepends locations to the default module search path.
The docs say that, when running a script from the command line,
the first item in sys.path will be the path to the script.
The docs say that sys.path is then
“initialized from the environment variable PYTHONPATH, plus an installation-dependent default.”
For example, if PYTHONPATH is empty (PowerShell: $env:PYTHONPATH = ""),
only the installation-dependent default is added.
However, if PYTHONPATH starts with an empty string,
then the current working directory (that the script is run from) is added to PYTHONPATH.
(Even just an initial separator will add this—for example a semicolon in PowerShell: $env:PYTHONPATH = ";".)
This is not always the desired behavior,
so you may wish to hesitate before making permanent changes in your PYTHONPATH.
Related
I am having a problem running my script in a cmd prompt despite it working in PyCharm. I have a folder structure as such:
MyCode # PyCharm project folder
/UsefulFunctions
/Messaging
/Texter.py
/DiscordBot
/DiscordBot.py
Within DiscordBot.py I have an import
from UsefulFunctions.Messaging import Texter
This works when I run it from PyCharm without a problem. However when I try to run from a command prompt located at the DiscordBot level it errors with:
ImportError: No module named 'UsefulFunctions'
So naturally I thought it meant that the UsefulFunctions folder was not on my path. Therefore, I went into my environment variables and added it to my PATH variable (as well as the MyCode folder for good measure). Still it encountered this error. I browsed some posts on here regarding imports (mainly Importing files from different folder) and they recommend doing something like:
import sys
sys.path.insert(0, '/path/to/application/app/folder')
import file
Or adding __init__.py files to each folder in order to get them to register as packages. I went ahead and added __init__ files to each folder and subfolder I was trying to import from, but still could not run from the command prompt...I ommitted the sys.path.insert() solution because I see no benefit from this after already explicitly adding it to my PATH variable. Another solution was to add "." before the import because supposedly otherwise it is only searching python's PATH. I attempted this as:
from .UsefulFunctions.Messaging import Texter
ImportError: attempted relative import with no known parent package
And this error shows on PyCharm now as well... I don't get why my initial script would work without a hitch on PyCharm, but the same program cannot seem to find my import when run from a prompt. Can somebody please explain the difference between PyCharm running the program and my prompt? Why will this not work despite having __init__.py files and having added MyCode and UsefulFunctions to my PATH variable on Windows?
From [Python.Docs]: Command line and environment - PYTHONPATH:
Augment the default search path for module files. The format is the same as the shell’s PATH: one or more directory pathnames separated by os.pathsep (e.g. colons on Unix or semicolons on Windows). Non-existent directories are silently ignored.
You can also find more details on [SO]: Strange error while using Pycharm to debug PyQt gui (#CristiFati's answer).
So, in order for Python to be able to load a module (package) without specifying its path, the path must be present in %PYTHONPATH% environment variable.
You mentioned %PATH% several times in the question but it's %PYTHONPATH% (MyCode must be added to it).
PyCharm does that because of (any of) the 2 checkboxes in the image below (dialog can be triggered from the menu: Run -> Edit Configurations...):
If you want to get things working from CmdLine, yo have to do the same thing there as well:
[cfati#CFATI-5510-0:e:\Work\Dev\StackOverflow\q054955891\DiscordBot]> sopr.bat
### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ###
[prompt]> set py
Environment variable py not defined
[prompt]> "e:\Work\Dev\VEnvs\py_064_03.06.08_test0\Scripts\python.exe" DiscordBot.py
Traceback (most recent call last):
File "DiscordBot.py", line 1, in <module>
from UsefulFunctions.Messaging import Texter
ModuleNotFoundError: No module named 'UsefulFunctions'
[prompt]> set PYTHONPATH=e:\Work\Dev\StackOverflow\q054955891
[prompt]> set py
PYTHONPATH=e:\Work\Dev\StackOverflow\q054955891
[prompt]> "e:\Work\Dev\VEnvs\py_064_03.06.08_test0\Scripts\python.exe" DiscordBot.py
e:\Work\Dev\StackOverflow\q054955891\UsefulFunctions\Messaging\Texter.py imported
Conversely, in PyCharm (with the content roots related checkbox from above, checked), more content roots can be added like in the image below (menu: File -> Settings..., select Project Structure then Add Content Root):
This is useful when some required modules are located deeper in the project tree (and some dirs aren't valid Python package names).
So, when dealing with this type of situation, checking [Python.Docs]: sys.path, [Python.Docs]: os.getcwd() and module path, can save lots of wasted time and headaches:
import os
import sys
print(sys.path)
print(os.getcwd())
import some_module
print(some_module)
As a side note, I personally hate names starting with My (e.g. MyCode). Such a name tells me that the purpose of whatever entity "wears" it, was not clear to the person who wrote the code. Try finding a more useful name (e.g. TestBotProject, or smth similar) :).
[SO]: PyCharm doesn't recognize installed module (#CristiFati's answer) might also contain some useful info.
Python uses the system variable PYTHONPATH, among other things, to decide what to import.
From the docs:
When a module named spam is imported, the interpreter first searches
for a built-in module with that name. If not found, it then searches
for a file named spam.py in a list of directories given by the
variable sys.path. sys.path is initialized from these locations:
The directory containing the input script (or the current directory
when no file is specified).
PYTHONPATH (a list of directory names,
with the same syntax as the shell variable PATH).
The installation-dependent default.
The reason PyCharm magically imports the module when you run the script is because of the Project Structure -> Content Root value. It points to your project directory, by default.
Check your Interpreter. It is different than your command prompt Interpreter, located in Appdata, whereas the interpreter for PyCharm is in the Workspace folder.
Set your Python path in System variables,So that you can run python -help from any where in directory
then
navigate to project folder
c:\nnnn..\mmm..\MyCode
run python c:\nnnn..\mmm..\MyCode\DiscordBot
\DiscordBot.py
or
C:\Python27\python.exe "C:\Users\Username\MyCode\DiscordBot
\DiscordBot.py" or
C:\Python27\python.exe C:\Users\Username\MyCode\DiscordBot
\DiscordBot.py
Try quotes if path has space
I used
>>> import sys
>>> print(sys.path)
and I get this:
['', 'C:\\Users\\HowLo\\AppData\\Local\\Programs\\Python\\Python36-32\\python36.zip',
'C:\\Users\\HowLo\\AppData\\Local\\Programs\\Python\\Python36-32\\DLLs',
'C:\\Users\\HowLo\\AppData\\Local\\Programs\\Python\\Python36-32\\lib',
'C:\\Users\\HowLo\\AppData\\Local\\Programs\\Python\\Python36-32',
'C:\\Users\\HowLo\\AppData\\Local\\Programs\\Python\\Python36-32\\lib\\site-packages']
I am confused why this is in there:
r'C:\Users\HowLo\AppData\Local\Programs\Python\Python36-32\python36.zip'
when I try to pull it up in the File Explorer, nothing is there.
sys.path stores a list of strings, each one (as you can tell) is a path to a location on your computer.
Python looks in these places to find modules your program can use (when you do import sys python is getting the sys module from one of the locations in sys.path)
Paths to .zip files are just as valid as paths to folders, python will try to unzip any archived files.
Now that you know what sys.path is, we can look at your "problem."
You've said that C:\Users\HowLo\AppData\Local\Programs\Python\Python36-32\python36.zip doesn't exist.
All this means is that python doesn't load any modules from there.
It really has no (meaningful) implications whatsoever.
The sys path can be appended with whatever path you choose. It is after all simply a list.
It is used to search for modules which are not in the current folder. More on that in the docs.
A list of strings that specifies the search path for modules. Initialized from the environment variable PYTHONPATH, plus an installation-dependent default.
As initialized upon program startup, the first item of this list, path[0], is the directory containing the script that was used to invoke the Python interpreter. If the script directory is not available (e.g. if the interpreter is invoked interactively or if the script is read from standard input), path[0] is the empty string, which directs Python to search modules in the current directory first. Notice that the script directory is inserted before the entries inserted as a result of PYTHONPATH.
A program is free to modify this list for its own purposes. Only strings and bytes should be added to sys.path; all other data types are ignored during import.
I am trying to make a module discoverable on a system where I don't have write access to the global site-packages directory, and without changing the environment (PYTHONPATH). I have tried to place a .pth file in the same directory as a script I'm executing, but it seems to be ignored. E.g., I created a file extras.pth with the following content:
N:\PythonExtras\lib\site-packages
But the following script, placed and run in the same directory, prints False.
import sys
print r"N:\PythonExtras\lib\site-packages" in sys.paths
The only directory in sys.path to which I have write access is the directory containing the script. Is there another (currently non-existent) directory where I could place extras.pth and have it be seen? Is there a better way to go about this?
I'm using python 2.7 on Windows. All .pth questions I could find here use the system module directories.
Edit: I've tracked down the Windows per-user installation directory, at %APPDATA%\Python\Python27\site-packages. I can place a module there and it will be imported, but if I put a .pth file there, it has no effect. Is this really not supposed to work, or am I doing something wrong?
As described in the documentation, PTH files are only processed if they are in the site-packages directory. (More precisely, they are processed if they are in a "site directory", but "site directory" itself is a setting global to the Python installation and does not depend on the current directory or the directory where the script resides.)
If the directory containing your script is on sys.path, you could create a sitecustomize.py in that directory. This will be loaded when Python starts up. Inside sitecustomize.py, you can do:
import site
site.addsitedir('/some/dir/you/want/on/the/path')
This will not only add that directory, but will add it as a "site directory", causing PTH files there to be processed. This is handy if you want to create your own personal site-packages-like-directory.
If you only need to add one or two directories to the path, you could do so more simply. Just create a tiny Python library that manipulates sys.path, and then import that library from your script. Something like:
# makepath.py
import sys
sys.path.append('/whatever/dir/you/want')
# script.py
import makepath
Edit: Again, according to the documentation, there is the possibility of a site-specific directory in %APPDATA%\Python\PythonXY\site-packages (on Windows). You could try that, if in fact you have write access to that (and not just to your script directory).
You can make a .pth (path) file in a directory already in sys.path so it can be included/
Put that line in .bashrc (.profile on Mac), the working directory is on the path. Leave it out, it isn't.
export PYTHONPATH=$PYTHONPATH
I've noticed that shells with the line have a PYTHONPATH= line in their env output, those without don't, so perhaps that matters. Python also adds the current working directory if PYTHONPATH has some valid directory but begins with a ':' character. (Simply setting PYTHONPATH to some valid directory breaks the working directory inclusion.)
Why is this so?
What is best practice for ensuring the current working directory is on sys.path?
If you want to ensure it, then explicitly add . to the pythonpath:
export PYTHONPATH=.:$OTHER_DIRS:$PYTHONPATH
If you also want to have the current working directory in your sys.path with a PYTHONPATH set to something else, then you could try to set it up in your sitecustomize.py or usercustomize.py by inserting something like:
sys.path.append(os.getcwd())
Additional note:
Since the introduction of per user site-packages it's best to put your user packages there (you can get the folder name with site.USER_SITE). pip also makes it easy to install packages thera using the --user option, so I've found that at least for me ther rarely is the necessity to mess around with PYTHONPATH.
As per the docs,(this should clear your doubts)
The PYTHONPATH IS AN ENV. VARIABLE THAT augments the default search
path for module files. The format is the same as the shell’s PATH:
one or more directory pathnames separated by os.pathsep (e.g. colons
on Unix or semicolons on Windows). Non-existent directories are
silently ignored.
In addition to normal directories, individual PYTHONPATH entries may
refer to zipfiles containing pure Python modules (in either source or
compiled form). Extension modules cannot be imported from zipfiles.
The default search path is installation dependent, but generally
begins with prefix/lib/pythonversion (see PYTHONHOME above). It is
always appended to PYTHONPATH.
An additional directory will be inserted in the search path in front
of PYTHONPATH as described above under Interface options. The search
path can be manipulated from within a Python program as the variable
sys.path.
On WinXP sp2 I'd like to have a directory of modules that other python scripts will be using called "SharedPython" in the same directory as my python scripts. so, basically:
/pythonScripts
/pythonScripts/SharedPython
as well as other python scripts on the same level as the SharedPython directory.
when I run
print sys.path
I get the following output:
C:\WINDOWS\system32\python25.zip
C:\Python25\DLLs
C:\Python25\lib
C:\Python25\lib\plat-win
C:\Python25\lib\lib-tk
C:\Python25
C:\Python25\lib\site-packages
I don't know what environment variable controls this and, in fact, I don't see one that contains all these directories.
So,
a.)how do I determine which environment variable contains this list of dirs?
and
b.)can I just add the aforementioned SharedPython dir to that list?
I've tried setting PYTHONPATH to the following: %PYTHONPATH%C:\PythonScripts\SharedPython
You need the PYTHONPATH env var. The dirs listed in it are prepended to sys.path.
The correct way to setup PYTHONPATH in your case is:
set PYTHONPATH=%PYTHONPATH%;C:\PythonScripts\SharedPython
Note the semicolon between the second % and the C:\
Those paths are added by the site module; do not change this module, but rather create a batch file that adds your paths in %PYTHONPATH% and then runs the script.