I have a file in my management/commands folder and I am trying to import my models into the report.py folder. However, I get the following error.
Traceback (most recent call last):
File "c:\Users\Timmeh\source\Python\Django Projects\env\topxgym\members\management\commands\report.py", line 2, in <module>
from members.models import ActiveMember
ModuleNotFoundError: No module named 'members'
I have a active.py file in that same folder with the same import and it works. However, the active.py is run from python manage.py active. The report.py I am just compiling/running it (not running it from python manage.py). That is the only difference.
the file structure.
Please don't forget to add __init__.py for every folder.
More here:
https://docs.python.org/3/reference/import.html#regular-packages
Try to made relative import.
from ..models import ActiveMember
if it works, this answer can help:
Absolute imports in python not working, relative imports work
Try to check your project settings.
Have you add yors members app in settings.INSTALLED_APPS list?
Here is a dirty hack which should solve the issue.
and I'll also point out an alternate one if that suits your use case.
First Option
import sys
sys.path.append(r'c:\Users\Timmeh\source\Python\Django Projects\env\topxgym')
# now your import should work
from members.models import ActiveMember
This is dirty because it manipulates sys path which is not a good practice. I'm assuming this script is just a standalone script you wish to run, hence it should not be an issue overall.
Second Option
IFF this is a possibility, you could just move your report.py and keep it inside topxgym folder (at the same level as requirements.txt)
In that case too, the same import statement would work.
side note
If you're looking to access the ORM from a standalone script, it looks like you might be missing to add the django configuration in the script as recommended by the documentation.
After your import issues are fixed by either of the above two methods, if it works, great. If it doesn't it would mean that you have to add the django.setup() config and specify a settings file.
Related
UPDATE (TO SHOW FINAL CODE)
Since this seemed hard to explain, I shared the project. For those that come to this question, you can see the full project here:
https://github.com/jeffnyman/pacumen
Calling out the files that were problematic for me:
graphical_pacman
https://github.com/jeffnyman/pacumen/blob/master/displays/graphical_pacman.py
layout
https://github.com/jeffnyman/pacumen/blob/master/mechanics/layout.py
With the __init__.py and setup.py files in place, I'm now able to run commands like these:
python displays/graphical_pacman.py
python mechanics/layout.py
All imports now resolve correctly when those commands are executed. You can see all the import statements I use in each file and the placement of the various __init__.py files.
ORIGINAL QUESTION
I cannot get what seems to be a simple thing to work: importing modules between directories. This is all in Python 3 so I don't want to have __init__.py files all over the place if I can help it, which many of the answers here suggest is the "right" way.
I have a structure like this:
project
displays
graphical_pacman.py
mechanics
layout.py
The layout.py file has a top level function called get_layout() that I want to call from graphical_pacman.py.
Going to the minimum code necessary, in graphical_pacman.py I have:
import layout
if __name__ == '__main__':
board = layout.get_layout("test_maze.lay")
Shows up fine in IDE, even autocompletes it for me. Running graphical_pacman.py gets me this:
File "displays/graphical_pacman.py", line 3, in <module>
import layout
ModuleNotFoundError: No module named 'layout'
Then I tried this:
from mechanics.layout import get_layout
if __name__ == '__main__':
board = mechanics.layout.get_layout("test_maze.lay")
Can't do that either:
File "displays/graphical_pacman.py", line 3, in <module>
from mechanics.layout import get_layout
ModuleNotFoundError: No module named 'mechanics'
I tried this:
from mechanics import layout
if __name__ == '__main__':
board = layout.get_layout("test_maze.lay")
I tried this:
from layout import get_layout
if __name__ == '__main__':
board = get_layout("test_maze.lay")
Doesn't work. Got this:
File "displays/graphical_pacman.py", line 3, in <module>
from layout import get_layout
ModuleNotFoundError: No module named 'layout'
I tried using relative imports (with the . in front of things) but that also doesn't work. I've also just tried using the * for my import (essentially importing everything). Also doesn't work. When I say "doesn't work" I get some variation of the above errors.
I've tried all of this running the command python graphical_pacman.py from within the displays directory and at the root project directory. The same errors occur each time.
I also tried using a sys.path, such as this sys.path.insert(0, '../mechanics'). I also tried variations on sys.path.append('../') based on other answers I've seen here. Again, all I get are variations on the above errors.
What am I missing?
2nd Update:
I have sent a pull request, check it out.
Here's what I did: I created a package out of the project root directory and installed it using pip. Details follow:
Files Added:
pacumen
__init__.py
setup.py
displays
__init__.py
library
__init__.py
mechanics
__init__.py
Contents of pacumen/__init__.py:
from . import displays
from . import library
from . import mechanics
Contents of pacumen/setup.py:
import setuptools
setuptools.setup(
name='pacumen',
version='0.01dev',
packages=['displays',
'library',
'mechanics',
],
author='jefferyman',
author_email='something#something.com',
description='Pacman. Duh.',
long_description=open('README.md').read(),
)
Contents of pacumen/displays/__init__.py:
from . import graphical_helpers
from . import graphical_pacman
from . import graphical_support
Contents of pacumen/library/__init__.py:
from . import structures
from . import utilities
Contents of pacumen/mechanics/__init__.py:
from . import grid
from . import layout
Changes in files:
pacumen/mechanics/layout.py:
from mechanics.grid import Grid
Make sure your virtual environment is active. (Instructions for that further down).
Finally, navigate to the project root directory and install your project root as a package:
pip install --editable . # or pip install -e . (Note the period at the end)
Now as long as you activate the virtual environment, you should not have any import problems. Do make sure you use the import statements of the style:
from mechanics.grid import Grid
Virtual Environment creation and activation:
For those reading, now is a good time to make a virtual environment if one isn't already made. If made make sure to activate it. If not, navigate to the root of the project dir (pacumen) and run
$ python -m venv venvdir
Then, once it is created, run:
<project-root>$ .venvdir\Scripts\activate # for windows.
OR
<project-root>$ source venvdir/bin/activate # for linux bash
Update: First things to check are that there are no circular imports, which means you are importing something in graphical_pacman from mechanics and something else in mechanics from graphical_pacman. Also make sure your module names don't conflict with built-in python module's names. If it's all good on these fronts,
Have you tried this?
from project.mechanics import layout
If that didn't work place __init__.py files in project, displays, mechanics.
In project/displays/__init__.py add from . import graphical_pacman.
In project/mechanics/__init__.py add from. import layout.
If neither of these worked, create a package out of this, navigate to the package root and install it in you environment with pip install -e .
I have something that works so I have an answer but the full answer is that what I'm doing can't be done in Python, as far as I can tell. Part of the problem here is that as I learned more, I realized my question was not worded well enough in terms of how I wanted things to execute.
Here's how things can work. If in graphical_pacman.py, I do this:
if __name__ == '__main__':
sys.path.append('../')
from mechanics import layout
board = layout.get_layout("test_maze.lay")
This finally works!
BUT ...
I have to run python graphical_pacman.py from within the displays directory rather than at the project root. (Meaning, I can't do this command from the project root: python displays\graphical_pacman.py.)
BUT ...
Execution still runs into an error:
Traceback (most recent call last):
File "graphical_pacman.py", line 18, in <module>
from mechanics import layout
File "../mechanics/layout.py", line 4, in <module>
from grid import Grid
ModuleNotFoundError: No module named 'grid'
And that's coming from this line in layout.py:
from grid import Grid
If I change that line to a relative import:
from .grid import Grid
NOW my python graphical_pacman.py command works when run from the displays directory.
BUT ...
Now I can't just run python layout.py, which also needs to be possible. That's apparently because the relative import is now in there, and so I get this error:
Traceback (most recent call last):
File "layout.py", line 4, in <module>
from .grid import Grid
ModuleNotFoundError: No module named '__main__.grid'; '__main__' is not a package
The problem I'm finding is I need to sometimes run files individually. You'll see mechanics\layout.py has an if __name__ == '__main__': section. So does displays\graphical_support.py. Those are necessary so people can see how things work by running those files directly. But eventually those will be used/imported by a file that's in the root of the directory that is orchestrating everything. And sometimes files (like layout) will depend on others (like grid).
So it seems the answer is that Python can't do what I'm trying to do, which is basically just seek flexibility.
I suppose I could handle all of this by just having all of the files in one directory, which seems to be the best answer here, and seems to be how other projects have done it. That seems like a bad approach but I'm weighing that against fighting with an import mechanism particularly since I've now discovered that the mechanism works in different ways depending on execution.
I am getting this error
Traceback (most recent call last):
File "Exporter.py", line 3, in <module>
import sys,getopt,got,datetime,codecs
File "C:\Users\Rohil\Desktop\GetOldTweets-python-master\got\__init__.py", line 1, in <module>
import models
ModuleNotFoundError: No module named 'models'
my directory tree is:
C:\Users\Rohil\Desktop\GetOldTweets-python-master\got
this contains 2 folders: manager and models and 1 __init__.py file with the code :
import models
import manager
i am executing a file with the path: C:\Users\Rohil\Desktop\GetOldTweets-python-master\Exporter.py
I can't figure out what the issue is. can anyone assist me?
Set the environment variable PYTHONPATH=C:\Users\Rohil\Desktop\GetOldTweets-python-master\got (how exactly, depends on your operating system)
If you have created a directory and sub-directory then follow the below steps and please keep in mind that a directory must have an __init__.py file for python to recognize it as a package.
First run this to see all paths being searched by python:
import sys
sys.path
You must be able to see your current working directory in that list.
Now import the sub-directory and the respective module that you want to use via the import command: import subdir.subdir.modulename as abc You should now be able to use the methods in that module.
As you can see in this screenshot above I have one parent directory and two sub-directories. Under the second sub-directory I have a module named CommonFunction. On the console to the right you can see my working directory after execution of sys.path.
Does the models folder has an __init__.py file inside it ? Only then, it will be recognized as a module by python and import models would make sense.
So,
Create an empty __init__.py file in the models subfolder and then the code should work without any issues.
You should also look at this answer.
if create d or using or custom python package
check our dir. correct.
** For python 3.7 user **
from. import module_name
If you are using python3 but are installing the API through 'pip install 'such and such'. It's not gonna work on python3 console. Therefore, you'd better use 'sudo python3 -m pip install 'such and such''. Then it's gonna work!
(at least for ubuntu stuff)
:)
I'm still learning to code better in Python. Therefore I am trying to use some constructions in my programming. Things like name_conventions and structured packaging & modules.
That said I come across an issue which I cannot resolve, it seems.
The structure I chose is as follows:
Core
__init__.py
controller.py
Dashboard
Log
Plugins
Stats
__init__.py
controller.py
Plugin2
Plugin3
Etc..
main.py
In the Core controller.py i am trying to import the Stats(object) class.
I have tried several different ways. In my IDE (Pycharm) it works like a charm ;) When i try to start it via Command Line i get ImportErrors.
I have tried different approaches.
from Plugins.Stats.controller import Stats
from Plugins.Stats import controller
from Plugins.Stats import controller.Stats
from Plugins import Stats.controller
I also tried to put the Stats class into the Plugins.Stats.__init__.py and call it:
from Plugins.Stats import Stats
and I get this:
Traceback (most recent call last):
File "controller.py", line 4, in <module>
from Plugins.Stats.controller import Stats
ImportError: No module named 'Plugins'
The funny thing is... When i try to do the same from main.py.
It does work. So I am really confused. Am I doing something wrong?
Also are there special design patterns to handle imports?
You should add __init__.py file also in the Plugins directory so it will look like
Plugins
__init__.py
Stats
__init__.py
controller.py
You can read more about Python modules here - check out 6.4. Packages
you also can make your module you want to import visible for everyone by adding it to the pythonpath
sys.path.append("/path/to/main_folder")
before this modules are on another namespaces
You will find more info here
First put From plugin import stats, then from stats import controller and import _init_
Incase of running from the terminal it will only try to import from the current folder. You have to manually add the path of the main folder to this file using the following code at the beging of the file
import sys
sys.path.append("../")
If you want to back 2 folders add ../../ instead of ../
You should not start a submodule of a package directly as a script. This will mess up your hierarchy most of the time. Instead call it as a module:
python -m Core.controller
You can verify the following by adding this to the top of any of your scripts:
import sys
for p in sys.path:
print(repr(p))
The reason the imports work when invoked as:
python main.py
is python will add the directory of the script to the pythonpath (In this case the empty string '')
However, when running
python Core/controller.py
it will add Core to sys.path, rendering the rest of your project unimportable
You can instead use
python -m Core.controller
which'll invoke using runpy and again put the empty string on the python path.
You can read more about invocation options of the python interpreter here: https://docs.python.org/2/using/cmdline.html You can read about runpy here: https://docs.python.org/2/library/runpy.html
Suppose I have the following directory structure:
workspace/
__init__.py
ys_manage/
__init__.py
manage.py
ys_utils/
__init__.py
project_dicts.py
Now, suppose I need access to project_dicts.py in manage.py. Also, my $PATH includes /home/rico/workspace/ys_manage.
I need to be able to run manage.py from any directory on my machine and still be able to access project_dicts.py.
My $PYTHONPATH only has /home/rico/workspace.
If I include the following in manage.py I can run the file from ~/workspace/ys_manage but not anywhere else.
import sys
sys.path.append('..')
from ys_utils import project_dicts
It appears that the '..' gives a relative path to where the directory where the file is run, not where the file is located. Is this correct?
I wanted to try and use ys_manage/__init__.py to import project_dicts.py so that it would be available in manage.py universally. Is this a good idea?
I've never used __init__.py for anything other than a "package creator". That is, I've never used it for initialization purposes. Perhaps I'm doing it wrong.
Contents of ys_manage/__init__.py:
import sys
sys.path.append('..')
from ys_utils import project_dicts
Should I include something in manage.py to look for this import?
When I try and run manage.py I get the following error:
NameError: global name 'project_dicts' is not defined
As a secondary question, do I need to have workspace/__init__.py? I'd really rather not have it because ys_manage and ys_utils (and about a dozen other packages) are all under revision control and used by several other developers...workspace is not.
Generally, I've found trying to use relative paths for imports dangerous and very error prone. I'd suggest just putting workspace on your PYTHONPATH (or adding it programatically in __init__.py) and importing everything relative to that static location. It will make your code more easily readable too, as you'll be able to track down where imports are coming from much more quickly and clearly.
Try this instead of sys.path.append('..'):
sys.path.append(os.path.join(os.path.dirname(__file__), '..'))
(you'll need to import both sys and os).
I know there are several similar questions, but I'm struggling to understand the error I'm getting and browsing the docs and similar questions hasn't helped yet. If anything, the similar questions make me feel like what I'm doing is right.
I have the following files:
src/main.py
from pack import pack
if __name__ == '__main__':
pack.exec("Hello Universe!")
src/pack/pack.py
import util
def exec(text):
util.write(text)
if __name__ == '__main__':
exec("Hello World!")
src/pack/util.py
def write(text):
print(text)
*src/pack/_init_.py*
EMPTY FILE
When I run python pack.py from the src/pack directory, it works (prints "Hello World!"). However when I run python main.py from the src directory I get the following exception:
Traceback (most recent call last):
File ".../src/main.py", line 1, in <module>
from pack import pack
File ".../src/pack/pack.py", line 1, in <module>
import util
ImportError: No module named util
If I change the import line in pack.py to from . import util as suggested, effectively the opposite occours. main.py runs successfully, however now pack.py fails, raising:
Traceback (most recent call last):
File ".../src/pack/pack.py", line 1, in <module>
from . import util
ValueError: Attempted relative import in non-package
I would have thought that imports are relative to the current location, and as such you ought to be able to construct a chain of imports like this. It seems very odd to me that module is supposed to import a sibling file differently depending on where the program starts.
Can someone explain why this error occurs in one way but not the other, and if there is some way to allow this file structure to run whether I want to run from main.py or pack.py?
You will have trouble making the import work in both cases. This is because in one case you are running pack.py as the main file and in another you run it as part of a package.
When you run it as a standalone script python pack.py, the "pack" directory gets added to the PYTHONPATH, meaning that you can import any module in it. Hence, import util will work.
When you run python main.py you add the src directory to your PYTHONPATH. This means that any module or package in src, for example the pack directory, now becomes importable. Hence from pack import pack. However, to access util.py you now need to do from pack import util. You can also do from . import util from within pack.py, as you noticed.
But you can't really do both at the same time. Either src/ is the main directory or src/pack is.
The obvious, but wrong solution, is to let the main.py add the src/pack directory to the PYTHONPATH. That will work, but it's not a good idea. The correct way to do this is to make up your mind. Is src/pack a module that should be imported via import pack or is it just a folder with a bunch of Python scripts? Decide! :-)
I think in this case its' obvious that src/pack should be a module. So then treat it like a module, and make sure it's available like a module. Then you can from pack import util even when running pack.py as a main script.
How do you do that? Well, basically you either install the pack module in your site-packages, or you add the src directory to the PYTHONPATH. That last one is what you want during development. You can either do it manually with export PYTHONPATH=<path> or you can let your testrunners do it for you. You don't have a testrunner? Well you should, but that's another question. :)
For installing it permanently once you don't do development anymore, take a look at Distribute. It includes a testrunner. ;)
Your link is to python 2.7 documentation, but it looks like you are using Python 3.x.
See here: http://docs.python.org/py3k/ for the correct docs.
Python 3 removes implicit relative import. That is, you cannot import packages within the same module in the same way anymore.
You need to use from . import util, this is an explicitly relative import and is allowed. import X no longer checks the current directory. Instead it only check the entries in sys.path. This includes the directory of the script was started and python's standard library.
You are missing __init__.py in your pack directory.
Create an empty file called __init__.py in your pack directory.
Nothing in the correct documentation supports your theory that this form of importing should work.