I know that there are other questions like this, none of them worked for me. I've been creating a simple python app and decided to organize it (instead of having the app and test.py in the same directory. I tried to set it up like this:
C:\Dev\project\module\test
- project
- __init__.py
- module
- __init__.py
- module.py
- test
- __init__.py
- test_module.py
Now I've tried everything i can think of to import module.py to test_module.py
import module
import project.module
import module.module
import project.module.module
from project import module
from project.module import module
None of these work. It fails with:
ImportError: No module named 'whatever i put in above'
It's driving me nuts, shouldn't this be simple? What am i missing? I added a test that shows my PYTHONPATH using import sys print sys.path. The first item in the list is C:\Dev\project\module\test
EDIT:
I tried adding init to the top level as well and that didn't help. I know I could force edit the sys.path as many of the answers suggest.
What is the right way to do it? As in, what is the standard or sensible way to build a project to avoid this issue?
If you only need to reach the parent directory, you can use:
import sys
sys.path.insert(0, '..')
import module
This should work on both Windows and Linux.
Messing with the python path programmatically isn't usually considered great style, but it has its uses. I find it's a nice, quick way to do what you're doing (writing tests below/inside a module). Of course the easy way is to simply put your tests outside of the module. Barring that, however, just do this:
import sys
sys.path.insert(0, r'C:\Dev\project') # Add the directory containing your module.
import module
This way, you ensure that the first directory on the path is the directory that contains the module you want to import. You could also avoid hard-coding the module path (e.g. if you want to be able to move your project someplace else) by using a path that's relative to your testing script:
import os
import sys
sys.path.insert(0, os.path.abspath(os.path.join(__file__, '..', '..', '..')))
import module
This will work no matter where you run the script from, and cross-platform.
At I beginning of most of my python scripts, I always put the following:
# Path hack.
# http://stackoverflow.com/questions/6323860/sibling-package-imports
import os, sys
sys.path.insert(0, os.path.abspath('.'))
I think packaging is a bit screwed in python, yet this is a standard way for me to make it work :)
When executing my script, I always use the root directory as the working dir:
cd C:\Dev\project
python3 .\module\test\test_module.py
When importing submodules, I also always start the path from the root:
import module.module
or
from module.module import some_identifier
Related
Not a python developer and I am clearly missing something fundamental here. Using Python 3.10.7 and i am getting an error:
from ..class_one import ClassOne
ImportError: attempted relative import beyond top-level package
when attempting to execute python run_me.py script in the example below
I have the following structure with following import statements
\Project
\data_processor
data_process.py
from ..class_one import ClassOne <--getting an error here
run_me.py
from data_processor.data_process import DataProcess
class_one.py
interesting that when i type a line from ..class_one import ClassOne in data_process.py my IDE thinks its completely legit and intellisence works, suggesting that i import ClassOne.
Most of the solutions imply Python earlier than v3.3 (changed the way packages are handled), which isn't the case here.
Python does not allow relative imports beyond the top-level package. This is why you are seeing the ImportError: attempted relative import beyond top-level package error...
Try to use absolute import, like this:
from Project.class_one import ClassOne
You can also move the class_one module to a package within the current project, you can create a new package and move the class_one module to it. Then, you can use a relative import to import the ClassOne module:
\Project
\data_processor
data_process.py
from my_package.class_one import ClassOne
run_me.py
from data_processor.data_process import DataProcess
\my_package
class_one.py
Finally, it's not recommended (complexity, cause confusion and difficulty to read the code) but I think you can try to use a 'sys.path' to add the parent directory to the Python path. I think it will allow you to use a relative import:
import sys
sys.path.insert(0, '..')
from class_one import ClassOne
Why you get that error?
Remember relative imports are resolved using __package__ variable(I'm talking about ..class_one). Add a simple print statement in your data_process.py to see the value of this variable:
print(__package__)
You're in Project folder and you run python run_me.py, so this variable must be: 'data_processor'. So you can't go beyond "one" level up. (It follows the dots as we see next).
When you can go one level up?
In run_me.py change your import statement from:
from data_processor.data_process import DataProcess
to
from Project.data_processor.data_process import DataProcess
Before running your script, we need to tell Python where to find Project folder. It doesn't know right now! We are in Project directory and there is no Project directory in where we are. So just add it to the sys.path in run_me.py:
import sys
sys.path.insert(0, PATH TO PARENT OF Project DIRECTORY)
Now run your command -> python run_me.py. Every thing works fine.
Did you notice the value of __package__? It's now 'Project.data_processor'. We went one level down, so we can go one level up .( As I said, it follows the dots.)
That was the reason. But what you can do now?
I think the most simple solution is to just stick with absolute imports. If you know in which directory you are and check the records in sys.path(Add to the paths if necessary), you won't get into problems.
You can manually hack __package__ which I do not recommend. To do so, add:
__package__ = "Project.data_processor"
at the top in your data_process.py file. Then add this to the run_me.py
import sys
sys.path.insert(0, PATH TO PARENT OF Project DIRECTORY)
This way you don't need to change your original import statement. That from data_processor.data_process import DataProcess just works.
For this option, you don't need to hack __package__ nor adding any path to the sys.path. Just change your import statement from:
from data_processor.data_process import DataProcess
to
from Project.data_processor.data_process import DataProcess
But for executing your command, you should go to the parent directory and run your script like:
python -m Project.run_me
I'm trying to import from a parent folder. The folder structure is (ignore the names)
experiments
__init__.py
poppy.py
ostepop
__ init__.py
importio.py
nopop
__ init__.py
loggio.py
I tried adding __init__.pyto every folder, but it didn't have any affect
import experiments.ostepop.loggiogives error message:
ModuleNotFoundError: No module named 'experiments'
and from ..experiments import poppy gives
ImportError: attempted relative import with no known parent package
Any tips on how to import poppy.py or loggio.py from importio.py?
import os
import sys
# Append parent directory to import path
sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
After this you can import your modules as if they were at, or under, the same directory as your source file. So from importio.py you can
import poppy # noqa
import nopop.logio # noqa
The # noqa is there to suppress "PEP 8: E402 module level import not at top of file" warning.
The __init__.py files are unnecessary for this solution, but you may want to keep them (except the top level one), if you plan to convert your subdirectories to modules one day.
If you want to understand how this solution work, please run
print(__file__)
print(os.path.abspath(__file__))
print(os.path.dirname(os.path.abspath(__file__)))
print(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
It will print:
The current file name
The current file name, with absolute path
The current directory, with absolute path
And the parent directory, with absolute path
And sys.path.insert(0, ...) inserts this path to the beginning of the search path.
They are plenty of posts already, but it's still a tricky stuff to me as well.
So, though I'm still not sure what's the best practice, anyway here is how I handled this problem.
In the module where you want to do the imports, add
import sys
import os
sys.path.append(os.path.join(sys.path[0], '..'))
sys.path[0] is the absolute path to the module without the file name. It works both when the module is and is not the top-level script. If you don't know what is top-level script and how is it related to import mechanism, read this.
os.path.join is used to probably handle system difference. Anyway, os.path.join(sys.path[0], '..') will point to the parent path.
sys.path.append add that parent path to the searching space.
So, in your case, add previous code to importio.py, then you can do
from nopop import loggio
import poppy
in importio.py as well.
__init__.py is not used at all in this solution. You can delete them.
But I've seen many repos using __init__.py. So the best practice might involve using them.
If I understand you correctly, you want to import poppy.py from importio.py and debug with file model. Under this model, Python interpreter will not think of it as a package, so this code (import experiments.ostepop.loggio and from ..experiments import poppy) is invalid.
To solve this problem, you can debug this script with package model, which means run python -m experiments in terminal. It may be run correctly.
I am writing a Python (Python3) application (not a package) and have some doubts about the correct directory structure. At the moment I have this:
myapp/
__init__.py
launch.py
core/
__init__.py
some_core_class.py
other_core_class.py
gui/
__init__.py
some_gui_class.py
other_gui_class.py
I want the application to be started with python launch.py from any place in my directory structure - of course with prepending the correct path to launch.py, e.g. python myapps/myapp/launch.py.
Inside my modules I use absolute imports, e.g. in some_core_class.py I write from myapp.core.other_core_class import OtherCoreClass. I use the same way in launch.py, e.g. from myapp.core.some_core_class import SomeCoreClass.
But then launching it for example directly from dir myapp by writing python launch.py results in ImportError: No module named 'myapp'. I found I could make it work by changing my import in launch.py to from core.some_core_class import SomeCoreClass but this does not seem to me as a correct absolute import and is inconsistent with imports in other files.
What is the best way to solve my issue? I would like to avoid adding myapp to PATH environment variable which would require manual edit by the user or an installer. How should I change my code or my imports to make the application launchable from anywhere? Is that even possible?
I have just found, I can solve it by prepending
import os
import sys
app_parent_dir = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
sys.path.append(app_parent_dir)
at the very beginning of my launch.py. But this is so ugly and feels like a hack. There must be a better way. Python should not be ugly. Awaiting for better answers...
Recently started a new Python project.
I am resolving a import module error where I am trying to import modules from the same directory.
I was following the solutions here but my situation is slightly different and as a result my script cannot run.
My project directory is as follows:
dir-parent
->dir-child-1
->dir-child-2
->dir-child-3
->__init__.py (to let python now that I can import modules from here)
->module1
->module2
->module3
->module4
->main.py
In my main.py script I am importing these module in the same directory as follows:
from dir-parent.module1 import class1
When I run the script using this method it throws a import error saying that there is no module named dir-parent.module1 (which is wrong because it exists).
I then change the import statement to:
from module1 import class1
and this seemed to resolve the error, however, the code I am working on has been in use for over 2.5 years and it has always imported modules via this method, plus in the code it refers to the dir-parent directory.
I was just wondering if there is something I am missing or need to do to resolve this without changing these import statements and legacy code?
EDIT: I am using PyCharm and am running off PyCharm
If you want to keep the code unchanged, I think you will have to add dir-parent to PYTHONPATH. For exemple, add the following on top of your main.py :
import os, sys
parent_dir = os.path.abspath(os.path.dirname(__file__)) # get parent_dir path
sys.path.append(parent_dir)
Python's import and pathing are a pain. This is what I do for modules that have a main. I don't know if pythonic at all.
# Add the parent directory to the path
CURRENTDIR = os.path.dirname(os.path.dirname(os.path.realpath(__file__)))
if CURRENTDIR not in sys.path:
sys.path.append(CURRENTDIR)
I am having trouble importing python packages only when running python from cmdline/console. However, when using pydev, everything seems to work fine.
I have the following filesystem...
---MarketData
---Parser
---Parser.py
---__init__.py
---IO
---__init__.py
---MarketSocket.py
Currently, Parser and IO are defined as python packages (they have init.py files, although there is no code in the Parser.init.py file.
I am trying to run the following line of code in MarketSocket.py
from Parser import Parser
Which should import the module 'Parser' within the package 'Parser' however, I get the following error.
ImportError: No Module Named Parser
Any help would be appreciated! This should work according to similar issues on stackOverflow, but for some odd reason it isn't.
MarketSocket.py is in the directory IO. Therefore it is not possible to find the package Parser.
The best way to resolve this, are relative imports: from ..Parser import Parser But they might not work, if you start the script like: python MarketSocket.py. To use this, you would also have to add an __init__.py to your MarketData directory.
If it doesn't work extend the sys.path like this:
import sys
sys.path.append('../')
With this addition, Python searches also the paths you want.
If I were you I would also think about restructuring your project. In my opinion executables should be (most of the time) at the top of your working tree, which is also like Python works.
the MarketSocket.py is one level below Parser and thus can't see it
do this:
import os
import sys
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
sys.path.append(os.path.dirname(__file__))
Putting an (empty) __init__.py in the MarketData directory will make the whole thing a package (and avoids the ugly path hacks). That should just work then if you call the module from the top level of the package.
You encountered an issue with relative import. Only in the parent directory you may have the access to any child package/module. So in MarketSocket.py, you need
from ..Parser import Parser
Then when you run it with -m option, the trick is you have to run it in the top level directory. So in this case
1) you would go to the parent directory of MarketData
2) in that parent directory, run python -m MarketData.IO.marketSocket