I know there are several similar questions, but I'm struggling to understand the error I'm getting and browsing the docs and similar questions hasn't helped yet. If anything, the similar questions make me feel like what I'm doing is right.
I have the following files:
src/main.py
from pack import pack
if __name__ == '__main__':
pack.exec("Hello Universe!")
src/pack/pack.py
import util
def exec(text):
util.write(text)
if __name__ == '__main__':
exec("Hello World!")
src/pack/util.py
def write(text):
print(text)
*src/pack/_init_.py*
EMPTY FILE
When I run python pack.py from the src/pack directory, it works (prints "Hello World!"). However when I run python main.py from the src directory I get the following exception:
Traceback (most recent call last):
File ".../src/main.py", line 1, in <module>
from pack import pack
File ".../src/pack/pack.py", line 1, in <module>
import util
ImportError: No module named util
If I change the import line in pack.py to from . import util as suggested, effectively the opposite occours. main.py runs successfully, however now pack.py fails, raising:
Traceback (most recent call last):
File ".../src/pack/pack.py", line 1, in <module>
from . import util
ValueError: Attempted relative import in non-package
I would have thought that imports are relative to the current location, and as such you ought to be able to construct a chain of imports like this. It seems very odd to me that module is supposed to import a sibling file differently depending on where the program starts.
Can someone explain why this error occurs in one way but not the other, and if there is some way to allow this file structure to run whether I want to run from main.py or pack.py?
You will have trouble making the import work in both cases. This is because in one case you are running pack.py as the main file and in another you run it as part of a package.
When you run it as a standalone script python pack.py, the "pack" directory gets added to the PYTHONPATH, meaning that you can import any module in it. Hence, import util will work.
When you run python main.py you add the src directory to your PYTHONPATH. This means that any module or package in src, for example the pack directory, now becomes importable. Hence from pack import pack. However, to access util.py you now need to do from pack import util. You can also do from . import util from within pack.py, as you noticed.
But you can't really do both at the same time. Either src/ is the main directory or src/pack is.
The obvious, but wrong solution, is to let the main.py add the src/pack directory to the PYTHONPATH. That will work, but it's not a good idea. The correct way to do this is to make up your mind. Is src/pack a module that should be imported via import pack or is it just a folder with a bunch of Python scripts? Decide! :-)
I think in this case its' obvious that src/pack should be a module. So then treat it like a module, and make sure it's available like a module. Then you can from pack import util even when running pack.py as a main script.
How do you do that? Well, basically you either install the pack module in your site-packages, or you add the src directory to the PYTHONPATH. That last one is what you want during development. You can either do it manually with export PYTHONPATH=<path> or you can let your testrunners do it for you. You don't have a testrunner? Well you should, but that's another question. :)
For installing it permanently once you don't do development anymore, take a look at Distribute. It includes a testrunner. ;)
Your link is to python 2.7 documentation, but it looks like you are using Python 3.x.
See here: http://docs.python.org/py3k/ for the correct docs.
Python 3 removes implicit relative import. That is, you cannot import packages within the same module in the same way anymore.
You need to use from . import util, this is an explicitly relative import and is allowed. import X no longer checks the current directory. Instead it only check the entries in sys.path. This includes the directory of the script was started and python's standard library.
You are missing __init__.py in your pack directory.
Create an empty file called __init__.py in your pack directory.
Nothing in the correct documentation supports your theory that this form of importing should work.
Related
I have a file in my management/commands folder and I am trying to import my models into the report.py folder. However, I get the following error.
Traceback (most recent call last):
File "c:\Users\Timmeh\source\Python\Django Projects\env\topxgym\members\management\commands\report.py", line 2, in <module>
from members.models import ActiveMember
ModuleNotFoundError: No module named 'members'
I have a active.py file in that same folder with the same import and it works. However, the active.py is run from python manage.py active. The report.py I am just compiling/running it (not running it from python manage.py). That is the only difference.
the file structure.
Please don't forget to add __init__.py for every folder.
More here:
https://docs.python.org/3/reference/import.html#regular-packages
Try to made relative import.
from ..models import ActiveMember
if it works, this answer can help:
Absolute imports in python not working, relative imports work
Try to check your project settings.
Have you add yors members app in settings.INSTALLED_APPS list?
Here is a dirty hack which should solve the issue.
and I'll also point out an alternate one if that suits your use case.
First Option
import sys
sys.path.append(r'c:\Users\Timmeh\source\Python\Django Projects\env\topxgym')
# now your import should work
from members.models import ActiveMember
This is dirty because it manipulates sys path which is not a good practice. I'm assuming this script is just a standalone script you wish to run, hence it should not be an issue overall.
Second Option
IFF this is a possibility, you could just move your report.py and keep it inside topxgym folder (at the same level as requirements.txt)
In that case too, the same import statement would work.
side note
If you're looking to access the ORM from a standalone script, it looks like you might be missing to add the django configuration in the script as recommended by the documentation.
After your import issues are fixed by either of the above two methods, if it works, great. If it doesn't it would mean that you have to add the django.setup() config and specify a settings file.
I am trying to build a Python package. For simplification, I will only mention the parts which are relevant to the problem:
The package (directory) is called moranpycess and inside it there are three relevant files:
The initfile for the whole package __init__.py:
from .Individual import Individual
from .MoranProcess import MoranProcess
A module called Individual which contains a class Individual.
A module called MoranProcess which contains a class MoranProcess. At the top it imports the previous module with: import Individual.
I install the package with python -m pip install .
Then I run a test to see if the package can be imported properly: python -c 'import moranpycess'.
I get:
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/home/runner/work/angry-moran-simulator/angry-moran-simulator/moranpycess/__init__.py", line 18, in <module>
from .MoranProcess import MoranProcess
File "/home/runner/work/angry-moran-simulator/angry-moran-simulator/moranpycess/MoranProcess.py", line 22, in <module>
import Individual
ModuleNotFoundError: No module named 'Individual'
Error: Process completed with exit code 1.
This is strange to me since it seems that the Python interpreter can find the package, the package imports the respective classes but the interpreter also tries to execute the top-level module imports and (I don't know why) it does not find the module...
Am I doing something wrong here?
EDIT:
directory structure:
└── moranpycess
├── Individual.py
├── MoranProcess.py
└── __init__.py
Update
I am considering the solution proposed here:
https://stackoverflow.com/a/49375740/2340598
However I don't know if this is the "right" way to organize a package...
Relative imports are a bit harder to get right than absolute imports. They are also more brittle (e.g., if you move a file doing the import, it breaks), so unless you have a good reason to use relative imports I'd suggest to just use absolute ones.
Following that, your public API as it is defined in your topmost __init__.py would look like this:
from moranpycess.MoranProcess import MoranProcess
__all__ = ["MoranProcess"] # import hint that some tools use
While non-public objects can still be imported internally (e.g. in MoranProcess.py) in the following way:
from moranpycess.Individual import Individual
The good thing about absolute imports is that no matter where in your package (or outside of it) you are, they always look the same.
If done like this, users that have installed your package should be able to use your objects like this:
from moranpycess import MoranProcess # directly gets the object, not module
You should try to avoid messing with sys.path in order to get your imports to work, mainly because it isn't necessary. If all you're doing is writing regular python code that is meant to be used in a regular way, the builtin import mechanics should serve your use cases just fine.
If I were to import a package and noticed that sys.path changed, I'd suspect shenanigans and start looking for an alternative.
UPDATE (TO SHOW FINAL CODE)
Since this seemed hard to explain, I shared the project. For those that come to this question, you can see the full project here:
https://github.com/jeffnyman/pacumen
Calling out the files that were problematic for me:
graphical_pacman
https://github.com/jeffnyman/pacumen/blob/master/displays/graphical_pacman.py
layout
https://github.com/jeffnyman/pacumen/blob/master/mechanics/layout.py
With the __init__.py and setup.py files in place, I'm now able to run commands like these:
python displays/graphical_pacman.py
python mechanics/layout.py
All imports now resolve correctly when those commands are executed. You can see all the import statements I use in each file and the placement of the various __init__.py files.
ORIGINAL QUESTION
I cannot get what seems to be a simple thing to work: importing modules between directories. This is all in Python 3 so I don't want to have __init__.py files all over the place if I can help it, which many of the answers here suggest is the "right" way.
I have a structure like this:
project
displays
graphical_pacman.py
mechanics
layout.py
The layout.py file has a top level function called get_layout() that I want to call from graphical_pacman.py.
Going to the minimum code necessary, in graphical_pacman.py I have:
import layout
if __name__ == '__main__':
board = layout.get_layout("test_maze.lay")
Shows up fine in IDE, even autocompletes it for me. Running graphical_pacman.py gets me this:
File "displays/graphical_pacman.py", line 3, in <module>
import layout
ModuleNotFoundError: No module named 'layout'
Then I tried this:
from mechanics.layout import get_layout
if __name__ == '__main__':
board = mechanics.layout.get_layout("test_maze.lay")
Can't do that either:
File "displays/graphical_pacman.py", line 3, in <module>
from mechanics.layout import get_layout
ModuleNotFoundError: No module named 'mechanics'
I tried this:
from mechanics import layout
if __name__ == '__main__':
board = layout.get_layout("test_maze.lay")
I tried this:
from layout import get_layout
if __name__ == '__main__':
board = get_layout("test_maze.lay")
Doesn't work. Got this:
File "displays/graphical_pacman.py", line 3, in <module>
from layout import get_layout
ModuleNotFoundError: No module named 'layout'
I tried using relative imports (with the . in front of things) but that also doesn't work. I've also just tried using the * for my import (essentially importing everything). Also doesn't work. When I say "doesn't work" I get some variation of the above errors.
I've tried all of this running the command python graphical_pacman.py from within the displays directory and at the root project directory. The same errors occur each time.
I also tried using a sys.path, such as this sys.path.insert(0, '../mechanics'). I also tried variations on sys.path.append('../') based on other answers I've seen here. Again, all I get are variations on the above errors.
What am I missing?
2nd Update:
I have sent a pull request, check it out.
Here's what I did: I created a package out of the project root directory and installed it using pip. Details follow:
Files Added:
pacumen
__init__.py
setup.py
displays
__init__.py
library
__init__.py
mechanics
__init__.py
Contents of pacumen/__init__.py:
from . import displays
from . import library
from . import mechanics
Contents of pacumen/setup.py:
import setuptools
setuptools.setup(
name='pacumen',
version='0.01dev',
packages=['displays',
'library',
'mechanics',
],
author='jefferyman',
author_email='something#something.com',
description='Pacman. Duh.',
long_description=open('README.md').read(),
)
Contents of pacumen/displays/__init__.py:
from . import graphical_helpers
from . import graphical_pacman
from . import graphical_support
Contents of pacumen/library/__init__.py:
from . import structures
from . import utilities
Contents of pacumen/mechanics/__init__.py:
from . import grid
from . import layout
Changes in files:
pacumen/mechanics/layout.py:
from mechanics.grid import Grid
Make sure your virtual environment is active. (Instructions for that further down).
Finally, navigate to the project root directory and install your project root as a package:
pip install --editable . # or pip install -e . (Note the period at the end)
Now as long as you activate the virtual environment, you should not have any import problems. Do make sure you use the import statements of the style:
from mechanics.grid import Grid
Virtual Environment creation and activation:
For those reading, now is a good time to make a virtual environment if one isn't already made. If made make sure to activate it. If not, navigate to the root of the project dir (pacumen) and run
$ python -m venv venvdir
Then, once it is created, run:
<project-root>$ .venvdir\Scripts\activate # for windows.
OR
<project-root>$ source venvdir/bin/activate # for linux bash
Update: First things to check are that there are no circular imports, which means you are importing something in graphical_pacman from mechanics and something else in mechanics from graphical_pacman. Also make sure your module names don't conflict with built-in python module's names. If it's all good on these fronts,
Have you tried this?
from project.mechanics import layout
If that didn't work place __init__.py files in project, displays, mechanics.
In project/displays/__init__.py add from . import graphical_pacman.
In project/mechanics/__init__.py add from. import layout.
If neither of these worked, create a package out of this, navigate to the package root and install it in you environment with pip install -e .
I have something that works so I have an answer but the full answer is that what I'm doing can't be done in Python, as far as I can tell. Part of the problem here is that as I learned more, I realized my question was not worded well enough in terms of how I wanted things to execute.
Here's how things can work. If in graphical_pacman.py, I do this:
if __name__ == '__main__':
sys.path.append('../')
from mechanics import layout
board = layout.get_layout("test_maze.lay")
This finally works!
BUT ...
I have to run python graphical_pacman.py from within the displays directory rather than at the project root. (Meaning, I can't do this command from the project root: python displays\graphical_pacman.py.)
BUT ...
Execution still runs into an error:
Traceback (most recent call last):
File "graphical_pacman.py", line 18, in <module>
from mechanics import layout
File "../mechanics/layout.py", line 4, in <module>
from grid import Grid
ModuleNotFoundError: No module named 'grid'
And that's coming from this line in layout.py:
from grid import Grid
If I change that line to a relative import:
from .grid import Grid
NOW my python graphical_pacman.py command works when run from the displays directory.
BUT ...
Now I can't just run python layout.py, which also needs to be possible. That's apparently because the relative import is now in there, and so I get this error:
Traceback (most recent call last):
File "layout.py", line 4, in <module>
from .grid import Grid
ModuleNotFoundError: No module named '__main__.grid'; '__main__' is not a package
The problem I'm finding is I need to sometimes run files individually. You'll see mechanics\layout.py has an if __name__ == '__main__': section. So does displays\graphical_support.py. Those are necessary so people can see how things work by running those files directly. But eventually those will be used/imported by a file that's in the root of the directory that is orchestrating everything. And sometimes files (like layout) will depend on others (like grid).
So it seems the answer is that Python can't do what I'm trying to do, which is basically just seek flexibility.
I suppose I could handle all of this by just having all of the files in one directory, which seems to be the best answer here, and seems to be how other projects have done it. That seems like a bad approach but I'm weighing that against fighting with an import mechanism particularly since I've now discovered that the mechanism works in different ways depending on execution.
I'm still learning to code better in Python. Therefore I am trying to use some constructions in my programming. Things like name_conventions and structured packaging & modules.
That said I come across an issue which I cannot resolve, it seems.
The structure I chose is as follows:
Core
__init__.py
controller.py
Dashboard
Log
Plugins
Stats
__init__.py
controller.py
Plugin2
Plugin3
Etc..
main.py
In the Core controller.py i am trying to import the Stats(object) class.
I have tried several different ways. In my IDE (Pycharm) it works like a charm ;) When i try to start it via Command Line i get ImportErrors.
I have tried different approaches.
from Plugins.Stats.controller import Stats
from Plugins.Stats import controller
from Plugins.Stats import controller.Stats
from Plugins import Stats.controller
I also tried to put the Stats class into the Plugins.Stats.__init__.py and call it:
from Plugins.Stats import Stats
and I get this:
Traceback (most recent call last):
File "controller.py", line 4, in <module>
from Plugins.Stats.controller import Stats
ImportError: No module named 'Plugins'
The funny thing is... When i try to do the same from main.py.
It does work. So I am really confused. Am I doing something wrong?
Also are there special design patterns to handle imports?
You should add __init__.py file also in the Plugins directory so it will look like
Plugins
__init__.py
Stats
__init__.py
controller.py
You can read more about Python modules here - check out 6.4. Packages
you also can make your module you want to import visible for everyone by adding it to the pythonpath
sys.path.append("/path/to/main_folder")
before this modules are on another namespaces
You will find more info here
First put From plugin import stats, then from stats import controller and import _init_
Incase of running from the terminal it will only try to import from the current folder. You have to manually add the path of the main folder to this file using the following code at the beging of the file
import sys
sys.path.append("../")
If you want to back 2 folders add ../../ instead of ../
You should not start a submodule of a package directly as a script. This will mess up your hierarchy most of the time. Instead call it as a module:
python -m Core.controller
You can verify the following by adding this to the top of any of your scripts:
import sys
for p in sys.path:
print(repr(p))
The reason the imports work when invoked as:
python main.py
is python will add the directory of the script to the pythonpath (In this case the empty string '')
However, when running
python Core/controller.py
it will add Core to sys.path, rendering the rest of your project unimportable
You can instead use
python -m Core.controller
which'll invoke using runpy and again put the empty string on the python path.
You can read more about invocation options of the python interpreter here: https://docs.python.org/2/using/cmdline.html You can read about runpy here: https://docs.python.org/2/library/runpy.html
I am having trouble importing python packages only when running python from cmdline/console. However, when using pydev, everything seems to work fine.
I have the following filesystem...
---MarketData
---Parser
---Parser.py
---__init__.py
---IO
---__init__.py
---MarketSocket.py
Currently, Parser and IO are defined as python packages (they have init.py files, although there is no code in the Parser.init.py file.
I am trying to run the following line of code in MarketSocket.py
from Parser import Parser
Which should import the module 'Parser' within the package 'Parser' however, I get the following error.
ImportError: No Module Named Parser
Any help would be appreciated! This should work according to similar issues on stackOverflow, but for some odd reason it isn't.
MarketSocket.py is in the directory IO. Therefore it is not possible to find the package Parser.
The best way to resolve this, are relative imports: from ..Parser import Parser But they might not work, if you start the script like: python MarketSocket.py. To use this, you would also have to add an __init__.py to your MarketData directory.
If it doesn't work extend the sys.path like this:
import sys
sys.path.append('../')
With this addition, Python searches also the paths you want.
If I were you I would also think about restructuring your project. In my opinion executables should be (most of the time) at the top of your working tree, which is also like Python works.
the MarketSocket.py is one level below Parser and thus can't see it
do this:
import os
import sys
sys.path.insert(0, os.path.join(os.path.dirname(__file__), ".."))
sys.path.append(os.path.dirname(__file__))
Putting an (empty) __init__.py in the MarketData directory will make the whole thing a package (and avoids the ugly path hacks). That should just work then if you call the module from the top level of the package.
You encountered an issue with relative import. Only in the parent directory you may have the access to any child package/module. So in MarketSocket.py, you need
from ..Parser import Parser
Then when you run it with -m option, the trick is you have to run it in the top level directory. So in this case
1) you would go to the parent directory of MarketData
2) in that parent directory, run python -m MarketData.IO.marketSocket