I'm trying to run a python file, but I keep getting a ImportError.
My set up is I have a project with the following path:
/Users/John/Documents/pythonprojects/projectX
within 'projectX' I have a folder called 'components' which contains two python files titled 'py_file' and 'init'.
/Users/John/Documents/pythonprojects/projectX/components
At the top of 'py_file', I'm importing a namedTuple that is defined in 'init'
`from components import some_tuple`
When I run python py_file.py run I get ImportError: No module named components
However, if I add the lines below, I can get the file to run:
import sys
sys.path.append("..")
Any idea whats going on with this?
You should probably make sure your PYTHONPATH environment variable is set to /Users/John/Documents/pythonprojects/projectX.
Effectively, that is what you do with that line sys.path.append(".."), since the relative .. directory from py_file.py is /Users/John/Documents/pythonprojects/projectX.
from components import some_tuple
Python is looking for components in the directory you started it from. But, you started Python in your components directory, so that's not going to work. Appending ".." to your path appears to fix the problem because now Python will look in the parent directory, which does contain components.
However, the better approach is to leave sys.path alone. Figure out which .py file in your program is going to be the entry point, and make all your imports work when it executes from there. Then, always execute your program from there. If you want to quickly test some small module off in a package somewhere, write some tests!
If you'd like the flexibility of beginning execution from anywhere, you can look at relative imports. Personally I think these are better left until after absolute imports make sense.
Related
This question already has answers here:
How can I import a module dynamically given the full path?
(35 answers)
Closed 1 year ago.
When I try to do using sys in following way, it gives error: name 'path' is not defined.
Here test.py is file in folder directory. Also folder directory has __init__.py in it.
import sys
sys.path.append('c/Users/Downloads/folder')
from test import *
print(path)
Content of test.py
path="sample"
I also tried writing from test import path but it did not work. So is there any other way than sys. I am not able to find why it's not working with it, since most answers recommend this.
EDIT:
Python is used a lot in industry so I am hopeful there must be some way for that (without using or using sys). I can have multiple files with same name in my system so help provided in answer and comments are not permanent fix.
This is an interesting problem so I spend a little time investigating. After some digging, it turns out the problem is there is already another script named "test.py" in one of the sys.path directories and Python is importing that script instead of the one you want.
When you import a module, Python looks for it in each of the search paths in the order they are listed in sys.path and imports the first match. Using sys.path.append('path/to/folder') fails because it adds a search path to the end of the list, and Python already finds a match for test.py in one of the directories listed before it. This is also the reason sys.path.insert(0,'path/to/folder') works - because it inserts the specified path at the front of the list so Python will find it first.
FIX
Change the name of the file 'test.py' to be something unique that won't match any other file names. This is the probably the best way.
You could use sys.path.insert(0,'path/to/folder') which places the specified path first in the list of search paths.
Another way is to go through each of the directories in the sys.path list and remove or rename the other "test.py" file(s). Not recommended
is there any other way than sys
If both the script you are running and the script you want to import are in the same folder, you don't need sys module.
If the files are in different folders and you absolutely won't use sys, then you can set PYTHONPATH in terminal before running the script. Note that you must run the python script from inside this same shell for it to work. Also, this will not work if the folder your script is in also contains a file with the same name as the one you want to import from another folder
Linux Bash:
export PYTHONPATH="path/to/folder/"
Windows Powershell:
set PYTHONPATH="path/to/folder/"
I'm currently trying to import one of my modules from a different folder.
Like this...
from Assets.resources.libs.pout import Printer, ForeColor, BackColor
This import method works completely fine in PyCharm, however, when i try to launch the file in cmd or IDLE, i get this error.
ModuleNotFoundError: No module named 'Assets'
This is my file structure from main.py to pout.py:
- Assets
- main.py
- resources
- libs
- pout.py
Any clue about how i could fix this ?
Any help is appreciated !
Edit: The original answer was based on the assumption that the script you're running is within the folder structure given, which a re-read tells me may not be true. The general solution is to do
sys.path.append('path_to_Assets')
but read below for more detail.
Original answer
The paths that modules can be loaded from will differ between the two methods of running the script.
If you add
import sys
print(sys.path)
to the top of your script before the imports you should be able to see the difference.
When you run it yourself the first entry will be the location of the script itself, followed by various system/environment paths. When you run it in PyCharm you will see the same first entry, followed by an entry for the top level of the project. This is how it finds the modules when run from PyCharm. This behaviour is controlled by the "Add content roots to PYTHONPATH" option in the run configuration.
Adding this path programmatically in a way that will work in all situations isn't trivial because, unlike PyCharm, your script doesn't have a concept of where the top level should be. BUT if you know you'll be running the script from the top level, i.e. your working directory will be the folder containing Assets and you're running something like python Assets/main.py then you can do
sys.path.append(os.path.abspath('.'))
and that will add the correct folder to the path.
Appending sys path didn't work for me on windows, hence here is the solution that worked for me:
Add an empty __init__.py file to each directory
i.e. in Assets, resources, libs.
Then try importing with only the base package names.
Worked for me!
In Python, I use some of my functions so often that I made a file where they are written, no matter the project I am working on. How do I tell Python to add this file to path every time my code is executed so that i can use my functions, just like adding a specific path to Pathtool in Matlab? I know in Python, I could do something like
sys.path.insert(1, 'D:/Python/FunctionDirectory') and then from file_with_functions import my_function but I would have to write this code to all of my projects before start coding to use my functions. I would like to always have the path to my functions added so that I can always use them like in Matlab without worrying about adding them in every single .py file.
you can use PYTHONPATH env var to specify folder where to look for modules
https://www.tutorialspoint.com/What-is-PYTHONPATH-environment-variable-in-Python
To provide complete steps for anyone with my problem for later times, I found my file with functions can be thought of as a module from which functions can be imported just like from file_with_functions import my_function.
But, I needed to add the folder with my module to python path. Messing with environment variables in Windows didn't work for me. Fortunately here, I read about a different solution:
I simply went to my site-packages folder (to find it, import sys then print(sys.path) and look for a name containing the string 'site-packages'). In this folder, I created a new text file and simply pasted the path with my module there:
like this, closed the text file and changed the extension from .txt to .pth (name of file did not matter as long as it was a .pth, Python found it).
I have the following folder structure:
app
__init__.py
utils
__init__.py
transform.py
products
__init__.py
fish.py
In fish.py I'm importing transform as following: import utils.transform.
When I'm running fish.py from Pycharm, it works perfectly fine. However when I am running fish.py from the Terminal, I am getting error ModuleNotFoundError: No module named 'utils'.
Command I use in Terminal: from app folder python products/fish.py.
I've already looked into the solutions suggested here: Importing files from different folder, adding a path to the application folder into the sys.path helps. However I am wondering if there is any other way of making it work without adding two lines of code into the fish.py. It's because I have many scripts in the /products directory, and do not want to add 2 lines of code into each of them.
I looked into some open source projects, and I saw many examples of importing modules from a parallel folder without adding anything into sys.path, e.g. here:
https://github.com/jakubroztocil/httpie/blob/master/httpie/plugins/builtin.py#L5
How to make it work for my project in the same way?
You probably want to run python -m products.fish. The difference between that and python products/fish.py is that the former is roughly equivalent to doing import products.fish in the shell (but with __name__ set to __main__), while the latter does not have awareness of its place in a package hierarchy.
This expands on #Mad Physicist's answer.
First, assuming app is itself a package (since you added __init__.py to it) and utils and products are its subpackages, you should change the import to import app.utils.transform, and run Python from the root directory (the parent of app). The rest of this answer assumes you've done this. (If it wasn't your intention making app the root package, tell me in a comment.)
The problem is that you're running app.products.fish as if it were a script, i.e. by giving the full path of the file to the python command:
python app/products/fish.py
This makes Python think this fish.py file is a standalone script that isn't part of any package. As defined in the docs (see here, under <script>), this means that Python will search for modules in the same directory as the script, i.e. app/products/:
If the script name refers directly to a Python file, the directory
containing that file is added to the start of sys.path, and the file
is executed as the __main__ module.
But of course, the app folder is not in app/products/, so it will throw an error if you try to import app or any subpackage (e.g. app.utils).
The correct way to start a script that is part of a package is to use the -m (module) switch (reference), which takes a module path as an argument and executes that module as a script (but keeping the current working directory as a module search path):
If this option is given, [...] the current directory
will be added to the start of sys.path.
So you should use the following to start your program:
python -m app.products.fish
Now when app.products.fish tries to import the app.utils.transform module, it will search for app in your current working directory (which contains the app/... tree) and succeed.
As a personal recommendation: don't put runnable scripts inside packages. Use packages only to store all the logic and functionality (functions, classes, constants, etc.) and write a separate script to run your application as you wish, putting it outside the package. This will save you from this kind of problems (including the double import trap), and has also the advantage that you can write several run configurations for the same package by just making a separate startup script for each.
While patching a program I wrote I noticed that I had no safe way to actually get the directory of 'libs', or even 'include' from within the Python installation itself. Every single time I tried to find any python used directory, I just assumed they would always be in the same place, and thus hard-coded them in. Now you might wonder why I am having trouble with this, I mean it's just a simple join call after a sys check, right?
import os, sys
python_base_directory = sys.exec_prefix
libs_directory = os.path.join(python_base_directory, 'libs')
This code works most of the time, but will fail when you are using a virtual environment, or in my case, using travis_ci.
I want to be able to get the path to the 'libs' directory without hard coding anything.