I am trying to make a module discoverable on a system where I don't have write access to the global site-packages directory, and without changing the environment (PYTHONPATH). I have tried to place a .pth file in the same directory as a script I'm executing, but it seems to be ignored. E.g., I created a file extras.pth with the following content:
N:\PythonExtras\lib\site-packages
But the following script, placed and run in the same directory, prints False.
import sys
print r"N:\PythonExtras\lib\site-packages" in sys.paths
The only directory in sys.path to which I have write access is the directory containing the script. Is there another (currently non-existent) directory where I could place extras.pth and have it be seen? Is there a better way to go about this?
I'm using python 2.7 on Windows. All .pth questions I could find here use the system module directories.
Edit: I've tracked down the Windows per-user installation directory, at %APPDATA%\Python\Python27\site-packages. I can place a module there and it will be imported, but if I put a .pth file there, it has no effect. Is this really not supposed to work, or am I doing something wrong?
As described in the documentation, PTH files are only processed if they are in the site-packages directory. (More precisely, they are processed if they are in a "site directory", but "site directory" itself is a setting global to the Python installation and does not depend on the current directory or the directory where the script resides.)
If the directory containing your script is on sys.path, you could create a sitecustomize.py in that directory. This will be loaded when Python starts up. Inside sitecustomize.py, you can do:
import site
site.addsitedir('/some/dir/you/want/on/the/path')
This will not only add that directory, but will add it as a "site directory", causing PTH files there to be processed. This is handy if you want to create your own personal site-packages-like-directory.
If you only need to add one or two directories to the path, you could do so more simply. Just create a tiny Python library that manipulates sys.path, and then import that library from your script. Something like:
# makepath.py
import sys
sys.path.append('/whatever/dir/you/want')
# script.py
import makepath
Edit: Again, according to the documentation, there is the possibility of a site-specific directory in %APPDATA%\Python\PythonXY\site-packages (on Windows). You could try that, if in fact you have write access to that (and not just to your script directory).
You can make a .pth (path) file in a directory already in sys.path so it can be included/
Related
I am having a problem running my script in a cmd prompt despite it working in PyCharm. I have a folder structure as such:
MyCode # PyCharm project folder
/UsefulFunctions
/Messaging
/Texter.py
/DiscordBot
/DiscordBot.py
Within DiscordBot.py I have an import
from UsefulFunctions.Messaging import Texter
This works when I run it from PyCharm without a problem. However when I try to run from a command prompt located at the DiscordBot level it errors with:
ImportError: No module named 'UsefulFunctions'
So naturally I thought it meant that the UsefulFunctions folder was not on my path. Therefore, I went into my environment variables and added it to my PATH variable (as well as the MyCode folder for good measure). Still it encountered this error. I browsed some posts on here regarding imports (mainly Importing files from different folder) and they recommend doing something like:
import sys
sys.path.insert(0, '/path/to/application/app/folder')
import file
Or adding __init__.py files to each folder in order to get them to register as packages. I went ahead and added __init__ files to each folder and subfolder I was trying to import from, but still could not run from the command prompt...I ommitted the sys.path.insert() solution because I see no benefit from this after already explicitly adding it to my PATH variable. Another solution was to add "." before the import because supposedly otherwise it is only searching python's PATH. I attempted this as:
from .UsefulFunctions.Messaging import Texter
ImportError: attempted relative import with no known parent package
And this error shows on PyCharm now as well... I don't get why my initial script would work without a hitch on PyCharm, but the same program cannot seem to find my import when run from a prompt. Can somebody please explain the difference between PyCharm running the program and my prompt? Why will this not work despite having __init__.py files and having added MyCode and UsefulFunctions to my PATH variable on Windows?
From [Python.Docs]: Command line and environment - PYTHONPATH:
Augment the default search path for module files. The format is the same as the shell’s PATH: one or more directory pathnames separated by os.pathsep (e.g. colons on Unix or semicolons on Windows). Non-existent directories are silently ignored.
You can also find more details on [SO]: Strange error while using Pycharm to debug PyQt gui (#CristiFati's answer).
So, in order for Python to be able to load a module (package) without specifying its path, the path must be present in %PYTHONPATH% environment variable.
You mentioned %PATH% several times in the question but it's %PYTHONPATH% (MyCode must be added to it).
PyCharm does that because of (any of) the 2 checkboxes in the image below (dialog can be triggered from the menu: Run -> Edit Configurations...):
If you want to get things working from CmdLine, yo have to do the same thing there as well:
[cfati#CFATI-5510-0:e:\Work\Dev\StackOverflow\q054955891\DiscordBot]> sopr.bat
### Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ###
[prompt]> set py
Environment variable py not defined
[prompt]> "e:\Work\Dev\VEnvs\py_064_03.06.08_test0\Scripts\python.exe" DiscordBot.py
Traceback (most recent call last):
File "DiscordBot.py", line 1, in <module>
from UsefulFunctions.Messaging import Texter
ModuleNotFoundError: No module named 'UsefulFunctions'
[prompt]> set PYTHONPATH=e:\Work\Dev\StackOverflow\q054955891
[prompt]> set py
PYTHONPATH=e:\Work\Dev\StackOverflow\q054955891
[prompt]> "e:\Work\Dev\VEnvs\py_064_03.06.08_test0\Scripts\python.exe" DiscordBot.py
e:\Work\Dev\StackOverflow\q054955891\UsefulFunctions\Messaging\Texter.py imported
Conversely, in PyCharm (with the content roots related checkbox from above, checked), more content roots can be added like in the image below (menu: File -> Settings..., select Project Structure then Add Content Root):
This is useful when some required modules are located deeper in the project tree (and some dirs aren't valid Python package names).
So, when dealing with this type of situation, checking [Python.Docs]: sys.path, [Python.Docs]: os.getcwd() and module path, can save lots of wasted time and headaches:
import os
import sys
print(sys.path)
print(os.getcwd())
import some_module
print(some_module)
As a side note, I personally hate names starting with My (e.g. MyCode). Such a name tells me that the purpose of whatever entity "wears" it, was not clear to the person who wrote the code. Try finding a more useful name (e.g. TestBotProject, or smth similar) :).
[SO]: PyCharm doesn't recognize installed module (#CristiFati's answer) might also contain some useful info.
Python uses the system variable PYTHONPATH, among other things, to decide what to import.
From the docs:
When a module named spam is imported, the interpreter first searches
for a built-in module with that name. If not found, it then searches
for a file named spam.py in a list of directories given by the
variable sys.path. sys.path is initialized from these locations:
The directory containing the input script (or the current directory
when no file is specified).
PYTHONPATH (a list of directory names,
with the same syntax as the shell variable PATH).
The installation-dependent default.
The reason PyCharm magically imports the module when you run the script is because of the Project Structure -> Content Root value. It points to your project directory, by default.
Check your Interpreter. It is different than your command prompt Interpreter, located in Appdata, whereas the interpreter for PyCharm is in the Workspace folder.
Set your Python path in System variables,So that you can run python -help from any where in directory
then
navigate to project folder
c:\nnnn..\mmm..\MyCode
run python c:\nnnn..\mmm..\MyCode\DiscordBot
\DiscordBot.py
or
C:\Python27\python.exe "C:\Users\Username\MyCode\DiscordBot
\DiscordBot.py" or
C:\Python27\python.exe C:\Users\Username\MyCode\DiscordBot
\DiscordBot.py
Try quotes if path has space
I have an input error in pycharm when debugging and running.
My project structure is rooted properly, etc./HW3/. so that HW3 is the root directory.
I have a subfolder in HW3, util, and a file, util/util.py. I have another file in util called run_tests.py.
In run_tests.py, I have the following import structure,
from util.util import my_functions, etc.
This yields an input error, from util.util import load_dataset,proportionate_sample
ImportError: No module named 'util.util'; 'util' is not a package
However, in the exact same project, in another directory (same level as util) called data, I have a file data/data_prep.py, which also imports functions from util/util.py using a similar import statement...and it runs without any problems.
Obviously, I am doing this in the course of doing a homework, so please understand: this is ancillary to the scope of the homework.
The problem goes away when I move the file to another directory. So I guess this question is How do I import a python file located in the same directory in a pycharm project? Because pycharm raises an error if I just do import util and prompts me to use the full name from the root.
Recommended Way:
Make sure to set the working folder as Sources.
You can do it in Pycharm -> Preferences -> Project: XYZ -> Project Structure
Select your working folder and mark it as Sources. Then Pycharm recognize the working folder as a Source folder for the project and you will be able to simply add other files within that folder by using
import filename.py
or
from filename.py import mudule1
=================
Not recommended way:
In Pycharmyou can simply add . before the .py file which you are going to import it from the same folder. In your case it will be
from .util import my_functions
Resource
There is a good reference also for more information with example how to implement Package Relative Imports. I would highly recommend to check this page.
Package Relative Imports
If you don't have an __init__.py create one and add this line
from util.util import my_function
then you can easily import the module in your scripts
the __init__.py tells python that it should treat that folder as a python package, it can also be used to import/load modules too.
in most cases the __init__.py is empty.
Quoting the docs:
The __init__.py files are required to make Python treat the
directories as containing packages; this is done to prevent
directories with a common name, such as string, from unintentionally
hiding valid modules that occur later on the module search path. In
the simplest case, __init__.py can just be an empty file, but it can
also execute initialization code for the package or set the __all__
variable, described later.
Right-click on the folder which you want to be marked as the source > Mark Directory as > Source root.
In my case, it worked only when I omit the extension. Example:
import filename
Note: May be a bit unrelated.
I was facing the same issue but I was unable to import a module in the same directory (rather than subdirectory as asked by OP) when running a jupyter notebook (here the directory didn't have __init__.py). Strangely, I had setup python path and interpreter location and everything. None of the other answers helped but changing the directory in python did.
import os
os.chdir(/path/to/your/directory/)
I'm using PyCharm 2017.3 on Ubuntu 16.04
I had the same issue with pycharm, but the actual mistake was that the file I was trying to import didn't have a .py extension, even though I was able to run it as a standalone script. Look in the explorer window and make sure it has a .py extension. If not, right click on the file in the explorer window, pick refactor, and then rename it with a .py extension.
In Pycharm go to "Run - Configuration" and uncheck
'Add Content root to Pythonpath' and
'Add source roots to Pythonpath',
then use
from filename import functionname
For me the issue was, the source directory was marked correctly, but my file to import was named starting with numeric value. Resolved by renaming it.
import 01_MyModuleToImport
to
import MyModuleToImport
I am using a python package called kRPC that requires a basic boilerplate of setup code to use in any given instance, so here's my question:
Once I create a generic 'kRPCboilerplate.py', where can I place it inside my Python27 directory so that I can simply type,
import kRPCboilerplate
at the beginning of all my files?
I want to install my custom Python file to my Python directory so that I don't have to copy and paste the file into every new project folder I make.
I understand that,
import boilerplate
will import 'boilerplate.py', but only if 'boilerplate.py' is set in the root directory relative to the Python file that imports it.
The program I am creating will not be distributed, so there is no need to make a module installer, which is above the scope of my abilities. I simply want to copy and paste 'kRPCboilerplate.py' to the proper directory so that I can use Import without ever having to specify a path or copy and paste the imported file into the relative directory.
Your module root directory is 'Python27\Lib' where Python27 is your main python folder which includes the python executable file. You can drag and drop the .py files into there and import it without any complications!
Bit late to reply, but the safest is to set a special environmental variable called PYTHONPATH which will add search location for Python to search for libraries:
eg in Linux terminal:
export PYTHONPATH=$PYTHONPATH:/path/to/file
note it is only the path to the file, not the filename.
If you want a more permanent solution you can add
export PYTHONPATH=$PYTHONPATH:/path/to/file
to your ~/.bashrc or ~/.profile file
In windows the environmental variables are set in the config panel.
Not sure about OSx
I've written my own mail.py module in spider (anaconda). I want to import this py file in other python (spider) files just by 'import mail'
I searched on the internet and couldn't find a clearly solution.
To import any python script, it should exist in the PYTHONPATH. You can check this with the following code:
import sys
print sys.path
To import your Python script:
Put both the scripts (main and the imported python script) in the
same directory.
Add the location of the file to be imported to the
sys.path.
For example, if the script is located as '/location/to/file/script.py':
import sys
sys.path.append('/location/to/file/')
import script
I had the same problem, my files were in same folder, yet it was throwing an error while importing the "to_be_imported_file.py".
I had to run the "to_be_imported_file.py" seperately before importing it to another file.
I hope it works for you too.
Searched for an answer for this question to. To use a .py file as a import module from your main folder, you need to place both files in one folder or append a path to the location. If you storage both files in one folder, then check the working directory in the upper right corner of the spyder interface. Because of the wrong working directory you will see a ModuleNotFoundError.
There are many options, e.g.
Place the mail.py file alongside the other python files (this works because the current working dir is on the PYTHONPATH.
Save a copy of mail.py in the anaconda python environment's "/lib/site-packages" folder so it will be available for any python script using that environment.
I did a slightly different solution approach that is less sophisticated. When I start my anaconda terminal it is at a C prompt. I just did a cd d:\mypython\lib in the beginning window before starting python. once I did that I could simply just import my own classes that I put in that library with "import MyClass as my" then I was off and running. It is interesting, I did 2 days of internet searching in my part time and could not find the answer either, until I asked a friend.
cd d:\mypython\lib
python
>>> import MyClass as my
>>> my1=my.MyClass()
>>> my1.doSomething()
worked for me on my anaconda / windows 10 environment python 3.6.6.
when calling any function from another file, it should be noted to not import any library inside the function
I believe the easiest solution is to place the directory containing your Python files into the Anaconda site-packages folder on your machine. I wrote an article outlining the whole process but, in short, you'll need to create a folder containing your python script and an __init__.py file. Then place that folder inside the site-packages folder in the Anaconda directory.
On Windows the site-packages directory is typically located at:
C:\Users\[your_username]\Anaconda3\Lib\site-packages\
On Mac the site-packages directory is typically located located at:
Users/[your_username]/opt/anaconda3/lib/[python3.8]/site-packages/
Notice, on Mac the Python version matters. You'll need to look for the directory that corresponds to the (base) Python version used by Anaconda. Also, anything I've placed inside of square brackets in the file paths above need to be changed according to your particular machine and Python version.
The file structure should look something like this:
~/
|__site-packages/
|__your_folder/
script.py
__init__.py
After you have the folder containing your script.py and __init__.py file moved into the site-packages subdirectory of Anaconda, you'll be able to import it from any script you run on your machine.
Everywhere I see Python code importing modules using import sys or import mymodule
How does the interpreter find the correct file if no directory or path is provided?
http://docs.python.org/3/tutorial/modules.html#the-module-search-path
6.1.2. The Module Search Path
When a module named spam is imported, the interpreter first searches for a built-in module with that name. If not found, it then searches for a file named spam.py in a list of directories given by the variable sys.path. sys.path is initialized from these locations:
The directory containing the input script (or the current directory when no file is specified).
PYTHONPATH (a list of directory names, with the same syntax as the shell variable PATH).
The installation-dependent default.
Note: On file systems which support symlinks, the directory containing the input script is calculated after the symlink is followed. In other words the directory containing the symlink is not added to the module search path.
After initialization, Python programs can modify sys.path. The directory containing the script being run is placed at the beginning of the search path, ahead of the standard library path. This means that scripts in that directory will be loaded instead of modules of the same name in the library directory. This is an error unless the replacement is intended. See section Standard Modules for more information.
For information on the "installation-specific default", see documentation on the site module.
Also, you can see what the current path is by using the sys module
import sys
print(sys.path)
It uses the PYTHONPATH, set as an environment variable, to find packages (folders containing __init__.py files) and modules (or, if already loaded once, retrieves the module object from sys.modules).
Python has a path variable just like the one you have inside your terminal. Python looks for modules in folders inside that path, or in the folder where your program is located.