Suppose that you create a Python package called app, and it contains a module called foo, which has a function called say_hello that prints out Hello! to the console.
Suppose that you also create a second Python package called boss_app that has a module main. The app and boss_app are in the same directory, and the directory is on the Python path. Also, the __init__ files are all blank. The structure is:
app
__init__.py
foo.py
boss_app
__init__.py
main.py
I want to import app into boss_app.main so that I can call app.foo.say_hello.
I use this command:
import app
app.foo.say_hello()
and I expect to see in the console:
>>>> Hello!
Instead, the behavior I get is that app is imported but it does not have access to foo.
The solution I came up with was to modify app.__init__ so that it contained the following command:
from .foo import *
Now I get the expected behavior.
Is it always necessary to make a custom __init__ for a package if that package is going to be imported from an outside package?
Saying
import app
only runs app/__init__.py and makes everything it initializes available to be used as app.SOMENAME. If you want app.foo module to be available, you need to say import app.foo. This will load the module. A common example of this distinction is probably import os vs import os.path. Just because you say import os, you won't have the contents of os.path package available.
You don't have to break modules into packages, but it helps to avoid module-name collision. It may also help to keep concepts clear in the mind of a user of these modules.
Related
I have a question. I have a directory setup like this:
folder/
main.py
/stuff/
__init__.py
function.py
/items/
__init__.py
class.py
My question is how would I import the class.py into the function.py? This setup is very specific and is unable to be changed. What would I need to put in order for this to work?
Your current directory structure seems ideal, so long as the application is started via main.py.
Python will always automatically add the parent directory of the main script to the start of sys.path (i.e. folder in your example). This means that the import machinery will give that directory priority when searching for modules and packages that are not part of the standard libarary.
Given this, you can import the classes.py module into function.py, like so:
from items import classes
(Note that I have renamed the module, because class is a python keyword).
If you later added another module to stuff, and wanted to import it into functions.py, you would do:
from stuff import another
and if a sub-package was added to items, and you wanted to import a module from that, you would do:
from items.subpackage import module
Imports specified in this top-down way can be used from any module within the application, because they are always relative to the parent directory of the main script, which has priority.
I have a python project structured like this:
repo_dir/
----project_package/
--------__init__.py
--------process.py
--------config.py
----tests/
--------test_process.py
__init__.py is empty
config.py looks like this:
name = 'brian'
USAGE
I use the library by running python process.py from the project/project/ directory, or by specifying the python file path absolutely. I'm running Python 2.7 on Amazon EC2 Linux.
When process.py looks like below, everything works fine and process.py prints brian.
import config
print config.name
When process.py looks like below, I get the error ImportError: No module named project.config.
import project.config
print config.name
When process.py looks like below, I get the error ImportError: No module named project. This makes sense as the same behavior from the previous example should be expected.
from project import config
print config.name
If I add these lines to process.py to include the library root in sys.path, all configurations above, work fine.
import os
import sys
sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
MY CONFUSION
Many resources suggest setting up python libraries to import modules using project.module_name, but it doesn't seem like sys.path appending is standard, and seems weird that I need it. I can see that the sys.path append added my library root as a path in sys, but I thought that's what the __init__.py in my library root was supposed to do. What gives? What am I missing? I know Python importing creates lots of headaches so I've tried to simplify this as much as possible to wrap my head around it. I'm going crazy and it's Friday before a holiday. I'm bummed. Please help!!
QUESTIONS
How should I set up my libraries? How should I import packages? Where should I have __init__.py files? Do I need to append my library root to sys.path in every project? Why is this so confusing?
Your project setup is alright. I renamed the directories just for clarity
in this example, but the structure is the same as yours:
repo_dir/
project_package/
__init__.py
process.py
config.py
# Declare your project in a setup.py file, so that
# it will be installable, both by users and by you.
setup.py
When you have a module that wants to import from another module in
the same project, the best approach is to use relative imports. For example:
# In process.py
from .config import name
...
While working on the code on your dev box, do your work in a Python virtualenv,
and pip install your project in "editable" mode.
# From the root of your repo:
pip install -e .
With that approach, you'll never need to muck around with sys.path -- which
is almost always the wrong approach.
I think the problem is how you're running your script. If you want the script to be living in a package (the inner project folder), you should run it with python -m project.process, rather than by filename. Then you can make absolute or explicit relative imports to get config from process.
An absolute import would be from project import config or import project.config.
An explicit relative import would be from . import config.
Python 2 also allows implicit relative imports, but they're a really bad misfeature that you should never use. With implicit relative imports, internal package modules can shadow top-level modules. For instance, a project/json.py file would hide the standard library's json module from all the other modules in the package. You can tell Python you want to forbid implicit relative imports by putting from __future__ import absolute_import at the top of the file. It's the standard behavior in Python 3.
I'm still learning to code better in Python. Therefore I am trying to use some constructions in my programming. Things like name_conventions and structured packaging & modules.
That said I come across an issue which I cannot resolve, it seems.
The structure I chose is as follows:
Core
__init__.py
controller.py
Dashboard
Log
Plugins
Stats
__init__.py
controller.py
Plugin2
Plugin3
Etc..
main.py
In the Core controller.py i am trying to import the Stats(object) class.
I have tried several different ways. In my IDE (Pycharm) it works like a charm ;) When i try to start it via Command Line i get ImportErrors.
I have tried different approaches.
from Plugins.Stats.controller import Stats
from Plugins.Stats import controller
from Plugins.Stats import controller.Stats
from Plugins import Stats.controller
I also tried to put the Stats class into the Plugins.Stats.__init__.py and call it:
from Plugins.Stats import Stats
and I get this:
Traceback (most recent call last):
File "controller.py", line 4, in <module>
from Plugins.Stats.controller import Stats
ImportError: No module named 'Plugins'
The funny thing is... When i try to do the same from main.py.
It does work. So I am really confused. Am I doing something wrong?
Also are there special design patterns to handle imports?
You should add __init__.py file also in the Plugins directory so it will look like
Plugins
__init__.py
Stats
__init__.py
controller.py
You can read more about Python modules here - check out 6.4. Packages
you also can make your module you want to import visible for everyone by adding it to the pythonpath
sys.path.append("/path/to/main_folder")
before this modules are on another namespaces
You will find more info here
First put From plugin import stats, then from stats import controller and import _init_
Incase of running from the terminal it will only try to import from the current folder. You have to manually add the path of the main folder to this file using the following code at the beging of the file
import sys
sys.path.append("../")
If you want to back 2 folders add ../../ instead of ../
You should not start a submodule of a package directly as a script. This will mess up your hierarchy most of the time. Instead call it as a module:
python -m Core.controller
You can verify the following by adding this to the top of any of your scripts:
import sys
for p in sys.path:
print(repr(p))
The reason the imports work when invoked as:
python main.py
is python will add the directory of the script to the pythonpath (In this case the empty string '')
However, when running
python Core/controller.py
it will add Core to sys.path, rendering the rest of your project unimportable
You can instead use
python -m Core.controller
which'll invoke using runpy and again put the empty string on the python path.
You can read more about invocation options of the python interpreter here: https://docs.python.org/2/using/cmdline.html You can read about runpy here: https://docs.python.org/2/library/runpy.html
I have a directory structure that looks like this:
foo/
__init__.py
db.py
database.db
main.py
Now, in db.py, I read the contents of the database:
open('database.db').read()
This works as long as I am running db.py directly. However, when I try to import the foo package in main, and then run db.py, it fails with a file not found error.
How can I avoid this? (Preferably without making any changes to the modules in the foo package)
(Note: I am aware of a solution like this Relative paths in Python but that would entail a lot of modifications to my foo package)
EDIT 1: Is it not possible to add something to __init__.py so that python can find my files?
EDIT 2: As another small example, consider what happens when I start to have nested packages:
foo/
baz/
__init__.py
mod.py
modules.json
__init__.py
db.py
database.py
main.py
Now, if main.py uses the foo package, and db.py requires the baz package, then I need to start making more complicated changes in the modules of the baz as well as the foo package, so that the file reads succeed.
This kind of a situation is happening when I need to use a git module that has several nested submodules. Obviously, it's not very convenient to make so many changes.
One solution can be to mask the open() function with a function of your own, in db.py , and in that custom open() function, you can open the file after appending the directory of __file__ . Example -
def open(file,mode='r',buffering=-1):
import __builtin__ #use import builtins for Python 3.x
import os.path
filePath = os.path.join(os.path.dirname(__file__),file)
return __builtin__.open(filePath,mode,buffering)
__builtin__.open() (or for Python 3.x - builtins.open() ) points to the built-in open() function.
I have used buffering =-1 as it seems to be the default according to documentation , you can leave out that argument altogether, if its not needed.
If possible, you should not mask (maybe use a different name like - open_file() ) , but seems like you do not want to do that.
Though I still think masking should be done only when there is no other solution. Also, please note you should not be import db.py as from db import * , that may cause similar masking of open() in the other .py file where its imported (Ideally, it is not recommended to do from <module> import * in any situtation) .
Use of this module will to resolve this import error issue, the error that you mentioned is issue about path. so the error was python interpreter coulnt find the path where your modules has specified. if you use this module it would work fine.
sys.path
A list of strings that specifies the search path for modules. Initialized from the environment variable PYTHONPATH, plus an installation-dependent default
import sys
sys.path.append('..\Desktop\New folder')# folder destination
from foo import db
This is an extract from my fabric script
def create_php_site(name):
"""
Creates a new php installation
"""
from resource.php.git import gitignore
from resource.php.nginx import nginx
when I run it, I get error "ImportError: No module named php.git"
but if I move "from resource.php.git import gitignore " outside of the function, it works. Any ideas?
Each layer of resource.php.git has __init__.py
There is git.py in resource/php/:
gitignore = """.DS_Store
._.DS_Store
"""
You are running into a namespace conflict.
You cannot name your directory resource because there's a standard python module named resource - http://docs.python.org/2/library/resource.html
And so when you declare from resource... all hell breaks loose. Python cannot find php.git because indeed, in the standard resource python module, there's no such functionalities.
I created a similar example but instead renamed resource directory to myresource. Performing the imports then work perfectly. Like this:-
>>> from myresource.php.git import gitignore
>>> gitignore
'my ignore strings'
IMPORTANT
Please also make sure that you have an __init__.py file in your myresource directory.