I am trying to make my own package so that I can use the files in a different folder. This package contains a few different modules and then the main module that imports all the others inside it. For example:
Folder
|- main.py
|- other.py
|- something.py
|- __init__.py
Inside the main.py I have the imports:
import other
import something
and it works just fine when running the file itself; however, I added the __init__.py file and tried to import it into a different folder. The package is recognized, but the main.py gives me the following error:
Exception has occurred: ModuleNotFoundError No module named
'univariate'
File "C:...\stats.py", line 8, in
import univariate
File "F:...\testing.py", line 7, in
from stats import stats
For clarification, the actual main file is called stats.py. This is my first experience trying to make a package so I might be missing something. Thank you.
You need to change your imports into relative imports
import .other
import .something
or to change it to absolute imports rooted to your project folder
import x.y.other
import x.y.something
you can read here about the imports
When you have a module that you're trying to import you don't need the ".py" part.
Having a folder with a init.py file (even a blank one) means that a project that contains that folder can import from it.
/myproject
| - /mymodule
| - |- stats.py
| - |- other.py
| - |- something.py
| - |- __init__.py
| - main.py
then in main.py all you need to do is import mymodule or from mymodule import stats
I always hate to FTFM someone, but here's a link to how to build packages from the official documentation. But, where this really starts to shine is when you need to package your module so that someone else can run it Digital Ocean has a pretty good tutorial here.
Related
So, I encountered a ModuleNotFoundError when trying to import a module in a subpackage that imports another subpackage under its directory (so it's a subsubpackage to the main directory). I have put empty __init__.py files under both the subdirectory and subsubdirectory. The code was run in Python 3.9.7.
Here's what the structure looks like:
|- main.py
|- subpackage/
|- __init__.py
|- submod.py
|- subsubpackage/
|- __init__.py
|_ subsubmod.py
The code
In main.py, I have:
from subpackage import submod
def main():
x = submod.test_func(3)
print(x)
if __name__ == 'main':
main()
and in submod.py, I want to import subsubmod.py under subsubpackage/, so I have:
from subsubpackage import subsubmod
def test_func(a):
return subsubmod.addone(a)
and finally, in subsubmod.py:
def addone(x):
return x+1
The error message:
Now if I run main.py, I got
Traceback (most recent call last):
File "/Users/anonymous/test/main.py", line 1, in
<module>
from subpackage import submod
File "/Users/anonymous/test/subpackage/submod.py",
line 1, in <module>
from subsubpackage import subsubmod
ModuleNotFoundError: No module named 'subsubpackage'
My question and confusion
I'm not sure what I have done wrong. I realized that submod.py can be run separately, so it seems that the issue occurs when the import goes down more than one subdirectory? I wonder if there's a way around this issue, or should I just use a different structure to organize my scripts.
Putting a dot before the package name worked for me.
from .subsubpackage import subsubmod
Looks like when you are in a package if you don't use relative import it will look for your packages somewhere else.
You can find more information here:
Python documents about import system
StackOverflow question about relative imports
This question keeps showing up on Stack Overflow and I can understand why. So today I took the time to create a nested project with 1 top module and 2 nested sub modules both with classes in them. And another class was also created in the top module folder. This is also considered a module by python. But it's in the same folder as the top module.
The skeleton nested module project is now live on Github at:
HdlHelpers Nested Modules Skeleton Project
I created a python package which I use for brain preprocessing - named bpt. I would also like to use the same package as a sub-module in another project named brain_seg.
As a results, I am having problems with import statements which either results in errors when running the package stand-alone or nested.
The structure of the bpt package is as follows:
bpt
|--__init__.py
|--module1.py
|--module2.py
|--run.py
Import statements in the file run.py look something like:
import os
import module1
import module2
.
.
.
The structure of the brain_seg package is as follows:
brain_seg
|--bpt
| |--__init__.py
| |--module1.py
| |--module2.py
| |--run.py
|
|--package2
| |--__init__.py
| |--module1a.py
|
|--__init__.py
When running the run.py script which is part of the stand alone bpt package, everything runs as expected.
If I try to run the same script as part of the brain_seg package, the following error is dispatched:
ModuleNotFoundError: No module named 'module1'
I tried to use relative imports so the nested project will work as expected, resulting in the following run.py file:
import os
from . import module1
from . import module2
.
.
.
Following the change, execution of the brain_seg project worked as expected but when I tried to call the run.py script from the stand alone bpt package, it resulted in the following error being dispatched:
ImportError: attempted relative import with no known parent package
How should I handle the imports such that both options will be supported?
Imports in bpt.run must be absolute or relative:
from bpt.module1 import ... # absolute
from .module1 import ... # relative
To run bpt.run from your repository root with your pythonpath getting correctly set, use -m:
python bpt/run.py # bad
python -m bpt.run # good
If brain_seg is not truly meant to be a package itself, but just a project consisting of multiple packages and modules, get rid of brain_seg/__init__.py.
If you do mean to use it as a package, then move it below the repo root, i.e.
- README.md (etc.)
- .gitignore (etc.)
- brain_seg
- __init__.py
- bps
- __init__.py
- main.py
and use e.g. python -m brain_seg.bps.run
from the repository root.
If you also intend to use bps as a standalone thing, I wouldn't recommend keeping it in the brain_seg hierarchy at all, but to make it pip installable (see https://packaging.python.org/tutorials/packaging-projects/) and then pip install -e ../somewhere/bps to have pip set up an editable link between your projects.
Project structure
I have the following folder structure
|
|- src
| |- mypackage
| | |- __init__.py
| | |- mymodule.py
| |- utils.egg
|- main.py
in mymodule.py file I can import the egg adding it to the sys.path as
import sys
sys.path.append('src/utils.egg')
import utils
When calling main.py everything works fine (python -m main).
Problem
The problem comes from pylint. First, it shows the following message in mymodule.py file
Unable to import 'utils' pylint(import-error)
if I ask for suggestions (CRTL + Space) when importing I got
utils.build
.dist
.utils
.setup
# |- suggestions
And from utils.utils I can acces the actual classes / functions in utils module. Of course if I import utils.utils, when executing the main script, an importing error pops up.
How can I configure my vscode setting in order fix pylint?
should I install the egg instead of copy it to the working folder?
Is my project's folder-structure ok, or it goes against recommended practices?
Extra info
In case you wonder the EGG-INFO/SOURCE.txt file looks like
setup.py
utils/__init__.py
utils/functions.py
utils.egg-info/PKG-INFO
utils.egg-info/SOURCES.txt
utils.egg-info/dependency_links.txt
utils.egg-info/top_level.txt
utils/internals/__init__.py
utils/internals/somemodule.py
utils/internals/someothermodule.py
Also, there aren't build nor dist folder in the egg.
This is an issue with Pylint itself and not the Python extension, so it will come down to however you need to configure Pylint.
As for whether you should copy an egg around or install it, you should be installing it into your virtual environment, or at least copying over the appropriate .pth file to make the egg directory work appropriately.
I'm working on a Python application consisting of a core and multiple independent modules using the core. I'm having difficulty setting up relative imports of packages.
app
|- __init__.py
|- core
|- __init__.py
|- corefile.py
|- module1
|- __init__.py
|- main.py
The __init__.py files are empty. I'm running Python 2.7.1.
main.py
from .core import *
Running python main.py results in ValueError: Attempted relative import in non-package.
Similar questions: Ultimate answer to relative python imports, How to do relative imports in Python?, Relative imports in Python
Thanks for the help.
In short, you can only use relative imports from packages that are, themselves, imported.
For example, if you had:
$ cat run.py
from app.module1 import main
main.main()
$ python run.py
Then you could use a relative import in app/module1/main.py (although it would need to be from ..core import foo, because core/ is one level above main.py).
import sys
abs_filepath = '/home/n/Documents/IMPORTANT/deep_learning/drori_2018/ final_proj/Ryans_branch/StackGAN/'
# insert your absolute filepath above as abs_filepath = '/path/to/targ/dir'
sys.path.append(abs_filepath)
Please correct it if there are problems with doing the import this way
Other Answers:
Also please see here for a thorough answer about what's going on.
stuff/
__init__.py
mylib.py
Foo/
__init__.py
main.py
foo/
__init__.py
script.py
script.py wants to import mylib.py
This is just an example, but really I just want to do a relative import of a module in a parent directory. I've tried various things and get this error...
Attempted relative import beyond toplevel package
I read somewhere that the script from where the program starts shouldn't in the package, and I tried modifying the structure for that like so...
stuff/
mylib.py
foo.py // equivalent of main.py in above
foo/
__init__.py
script.py
but got same error.
How can I accomplish this? Is this even an adequate approach?
Edit: In Python 2
After fiddling with it a bit more, I realized how to set it up, and for the sake of specificity I won't use foo bar names. My project directory is set up as...
tools/
core/
object_editor/
# files that need to use ntlib.py
editor.py # see example at bottom
__init__.py
state_editor/
# files that need to use ntlib.py
__init__.py
ntlib.py
__init__.py # core is the top level package
LICENSE
state_editor.py # equivalent to main.py for the state editor
object_editor.py # equivalent to main.py for the object editor
A line in object_editor.py looks like...
from core.object_editor import editor
A line in editor.py looks like...
from .. import ntlib
or alternatively
from core import ntlib
The key is that in the example I gave in the question, the "main" script was being run from within the package. Once I moved it out, created a specific package (core), and moved the library I wanted the editors to share (ntlib) into that package, everything was hunky-dory.
though as long "stuff" is not in your python PATH you got no choice than adding the path.
If you know the level of your script.py from stuff you can do for example:
import sys
import os
sys.path.append(os.path.join(os.path.dirname(__file__), '..', '..'))
I'm running Python 3.4.2 on Windows 7 and tore my hair out over this.
When running either of these:
python -m unittest
python -m unittest discover
...I would get the 'Attempted relative import beyond toplevel package' error.
For me, the solution was dropping the ".." in my [test_stock.py].
The line was:
from ..stock import Stock
Changed it to:
from stock import Stock
.. and it works.
Folder structure:
C:\
|
+-- stock_alerter
|
+-- __init__.py
+-- stock.py
|
\-- tests
|
+-- __init__.py
\-- test_stock.py
From the PEP it appears that you cannot use a relative import to import a file that is not packaged.
So you would need to add a __init__.py to stuff and change your imports to something like from .mylib import *
However, the PEP seems to make no allowance to keep mylib packaged up in a module. So you might be required to change how you call your library functions.
Another alternative is to move mylib into a subpackage and import it as from .libpackage import mylib
If you're on Linux or perhaps a similar *nix, you can hack this with symlinks.
stuff/
mylib.py
foo.py // equivalent of main.py in above
foo/
script.py
mylib.py -> ../mylib.py
foo2/
script2.py
mylib.py -> ../mylib.py
This is likely not a good pattern to follow.
In my case I opted for it because I had multiple executables dependent on the same library that needed to be put into separate directories.
Implementation of new executable tests shouldn't require the test writer to have a deep understanding of python imports.
tests/
common/
commonlib.py
test1/
executable1.py
executable2.py
commonlib.py -> ../common/commonlib.py
test2/
executable1.py
executable2.py
commonlib.py -> ../common/commonlib.py