Using the structure from the Python docs:
sound/
__init__.py
effects/
__init__.py
echo.py
surround.py
reverse.py
Say I want to import sound.effects and get a list of available effects. I could do this by declaring a module-level variable in sound.effects and then appending to it when each .py file is imported. So sound/effects/__init__.py might look like this:
effectList = []
import echo
import surround # Could write code to import *.py instead
...
From my main code I can now access sound.effects.effectList to get a list of effects, but how do I access effectList from within echo.py to do the actual append? I'm stuck trying to get access to the variable:
# None of these work :-(
# from . import self
# from .. import effects
# import sound.effects
sound.effect.effectList.append({'name': 'echo'})
What people commonly do in this situation is create a common.py file in the module.
sound/
__init__.py
effect/
__init__.py
common.py
echo.py
surround.py
reverse.py
Then you move the code from __init__.py to common.py:
effectList = []
import echo
import surround # Could write code to import *.py instead
...
Inside __init__.py you have this:
from common import *
So now in echo.py you'd have this:
import common
common.effectList.append({'name': 'echo'})
Anything importing sound would use it like this
import sound.effect
for effect_name,effect in sound.effect.effectlist.items():
#....
I've only just started using this myself, but I believe it's common practice in the python community.
I think you should leave the "making available" to the __init__.py inside the effects package rather than have all the modules auto populate the effectList. A couple of reasons I can think of.
You can't import any of the effects except via the package if you did manage to get this work somehow (they'd except an effectList in the importing module).
You have to manually do the append in every effect you write. It would be better if you just implemented an import *.py like thing in your __init__.py that loaded everything up in the current directory and made it available.
Something like this in your __init__.py.
import os, glob
effectslist = []
for i in glob.glob("*.py"):
if i == "__init__.py":
next
print "Attempting to import %s"%i
try:
mod = __import__(os.path.splitext(i)[0])
effectslist.append(mod)
except ImportError,m:
print "Error while importing %s - %s"%(i,m)
Related
Imagine I have a module with two files, like this:
mymodule
|-- __init__.py
`-- submodule.py
mymodule/__init__.py contains:
SOME_CONSTANT_ONE = 1
SOME_CONSTANT_TWO = 2
SOME_CONSTANT_THREE = 3
...
SOME_CONSTANT_ONE_HUNDRED = 100
def initialize():
pass # do some stuff
def support_function():
pass # something that lots of other functions might need
I already know that I can use a relative import to bring in specific objects from the __init__.py file, like this:
submodule.py:
from . import initialize, support_function
def do_work():
initialize() # initialize the module
print(support_function()) # do something with the support function
But now what I want to know is if I can import all of the constants from the __init__.py file, but simultaneously have them appear in a namespace.
What won't work (what I've tried/considered):
import mymodule as outer_module works, since the import system already has knowledge of where the module is. However, if I ever need to change the name of the outer module, that code will break.
Doing import . as outer_module doesn't work.
Doing from . import * does work but puts all of the objects in __init__.py in the current namespace rather than in the sub-namespace.
Doing from . import SOME_CONSTANT_ONE as outer_constant_1, SOME_CONSTANT_TWO as outer_constant_2, SOME_CONSTANT_THREE as outer_constant_3, ... is ugly and won't bring in any new constants should they be defined later on in __init__.py.
What I really want is something like this:
submodule.py:
SOME_CONSTANT_ONE = "one!" # We don't want to clobber this.
import . as outer_module # this does not work, but it illustrates what is desired.
def do_work():
print(SOME_CONSTANT_ONE) # should print "one!"
print(outer_module.SOME_CONSTANT_ONE) # should print "1"
I know that I could move all of the constants to a constants.py file and then I should be able to import it with from . import constants (as something) but I'm working on existing code and making that change would require a lot of refactoring. While that's not a bad idea, I'm wondering, given that Python does have a way to import individual objects, and also to import the whole module by name to an explicit name, if I can maybe do something with importlib to accomplish importing everything from __init__.py into a namespace?
The loader sets __package__ which you can use:
import sys
SOME_CONSTANT_ONE = "one!" # We don't want to clobber this.
outer_module = sys.modules[__package__]
def do_work():
print(SOME_CONSTANT_ONE) # should print "one!"
print(outer_module.SOME_CONSTANT_ONE) # should print "1"
This is precisely the attribute from which relative imports are based.
See PEP 366 for details.
However, I really think the backwards-compatible refactoring which the other answer suggests is probably the better approach here.
I could move all of the constants to a constants.py file and then I should be able to import it with from . import constants (as something) but I'm working on existing code and making that change would require a lot of refactoring
You can still refactor the constants into the new constants.py module. To support existing code relying on __init__.py you can import constants into the __init__.py
# constants.py
SOME_CONSTANT_ONE = 1
SOME_CONSTANT_TWO = 2
SOME_CONSTANT_THREE = 3
... # etc
# __init__.py
from .constants import *
# submodule.py
SOME_CONSTANT_ONE = 'dont clobber me!'
from . import constants as something
print(something.SOME_CONSTANT_ONE) # Yay namespaces
# existing_code.py
from . import SOME_CONSTANT_ONE
# still works!
# no refactor required!
Typically speaking, the __init__.py file is usually left completely empty and nothing is ever directly defined there. If there are contents in __init__.py they are typically imported from within the package. https://stackoverflow.com/a/4116384/5747944
I simply want to take all my .py files from a single folder (I don't care about the sub-folders for now) and put them into a single module.
The use case I'm having here is that I'm writing some pretty standard object-oriented code and I'm using a single file for every class, and I don't want to have to write from myClass import myClass for every class into my __init__.py. I can't use Python3, so I'm still working with impand reloadand such.
At the moment I'm using
# this is __init__.py
import pkgutil
for loader, name, is_pkg in pkgutil.walk_packages(__path__):
if not is_pkg:
__import__(__name__ + "." + name)
and it doesn't seem to work, it includes the packages but it includes them as modules, so that I have to write MyClass.MyClass for a class that is defined in a file with it's own name. That's silly and I don't like it.
I've been searching forever and I'm just getting more confused how complicated this seemingly standard use case seems to be. Do python devs just write everything into a single file? Or do they always have tons of imports?
Is this something that should be approached in an entirely different way?
What you really want to do
To do the job you need to bind your class names to namespace of your __init__.py script.
After this step you will be able to just from YourPackageName import * and just use your classes directly. Like this:
import YourPackageName
c = YourPackageName.MyClass()
or
from YourPackageName import *
c = MyClass()
Ways to achieve this
You have multiple ways to import modules dynamically: __import__(), __all__.
But.
The only way to bind names into namespace of current module is to use from myClass import myClass statement. Static statement.
In other words, content of each of your __init__.py scripts should be looking like that:
#!/usr/bin/env python
# coding=utf-8
from .MySubPackage import *
from .MyAnotherSubPackage import *
from .my_pretty_class import myPrettyClass
from .my_another_class import myAnotherClass
...
And you should know that even for a dynamic __all__:
It is up to the package author to keep this list up-to-date when a new version of the package is released.
(https://docs.python.org/2/tutorial/modules.html#importing-from-a-package)
So a clear answers to your questions:
Do python devs just write everything into a single file?
No, they don't.
Or do they always have tons of imports?
Almost. But definitely not tons. You need to import each of your modules just once (into an appropriate __init__.py scripts). And then just import whole package or sub-package at once.
Example
Let's assume that there is next package structure:
MyPackage
|---MySubPackage
| |---__init__.py
| |---pretty_class_1.py
| |---pretty_class_2.py
|---__init__.py
|---sleepy_class_1.py
|---sleepy_class_2.py
Content of the MyPackage/MySubPackage/__init__.py:
#!/usr/bin/env python
# coding=utf-8
from .pretty_class_1 import PrettyClass1
from .pretty_class_2 import PrettyClass2
Content of the MyPackage/__init__.py:
#!/usr/bin/env python
# coding=utf-8
from .MySubPackage import *
from .sleepy_class_1 import SleepyClass1
from .sleepy_class_2 import SleepyClass2
As result, now we are able to write next code in our application:
import MyPackage
p = MyPackage.PrettyClass1()
s = MyPackage.SleepyClass2()
or
from MyPackage import *
p = PrettyClass1()
s = SleepyClass2()
I am having a lot of trouble understanding the python module import system.
I am trying to create a simple folder structure as follows.
SomeModule
__init__.py
AnotherModule
AnotherModule.py
__init__.py
Utils
Utils.py
__init__.py
To use SomeModule i can do:
SomeModule.Foo()
Now inside AnotherModule.py I would like to import my Utils directory.
How come I have to do
import SomeModule.AnotherModule.Utils.Foo
why cannot I just do
import Utils.Foo
To shorten up the actual function name that you'll have to call in your code, you can always do:
from SomeModule.AnotherModule.Utils import *
While this still won't allow you to get away with a shorter import statement at the top of your script, you'll be able to access all of the functions within .Utils just by calling their function name (i.e. foo(x) instead of SomeModule.AnotherModule.Utils.foo(x).
Part of the reason for the lengthy import statement goes to the comment from #wim . Have a look by typing import this in a python interpreter.
put
import sys
import SomeModule.AnotherModule
sys.modules['AnotherModule'] = SomeModule.AnotherModule
in SomeModules __init__.py
i've run through many posts about this, but still doesn't seem to work. The deal is pretty cut. I've the got the following hierarchy.
main.py
DirA/
__init__.py
hello.py
DirB/
__init__.py
foo.py
bla.py
lol.py
The__init__.py at DirA is empty. The respective one at DirB just contains the foo module.
__all__.py = ["foo"]
The main.py has the following code
import DirA
import DirB
hey() #Def written at hello.py
foolish1() #Def written at foo.py
foolish2() #Def written at foo.py
Long story short, I got NameError: name 'foo' is not defined. Any ideas? Thanks in advance.
You only get what you import. Therefore, in you main, you only get DirA and DirB. You would use them in one of those ways:
import DirA
DirA.something_in_init_py()
# Importing hello:
import DirA.hello
DirA.hello.something_in_hello_py()
# Using a named import:
from DirA.hello import something_in_hello_py
something_in_hello_py()
And in DirB, just make the __init__.py empty as well. The only use of __all__ is for when you want to import *, which you don't want because, as they say, explicit is better than implicit.
But in case you are curious, it would work this way:
from DirB import *
something_in_dirb()
By default the import * will import everything it can find that does not start with an underscore. Specifying a __all__ restricts what it imported to the names defined in __all__. See this question for more details.
Edit: about init.
The __init__.py is not really connected to the importing stuff. It is just a special file with the following properties:
Its existence means the directory is a python package, with several modules in it. If it does not exist, python will refuse to import anything from the directory.
It will always be loaded before loading anything else in the directory.
Its content will be available as the package itself.
Just try it put this in DirA/__init__.py:
foo = 42
Now, in your main:
from DirA import foo
print(foo) # 42
It can be useful, because you can import some of your submodules in the __init__.py to hide the inner structure of your package. Suppose you build an application with classes Author, Book and Review. To make it easier to read, you give each class its own file in a package. Now in your main, you have to import the full path:
from myapp.author import Author
from myapp.book import Book
from myapp.review import Review
Clearly not optimal. Now suppose you put those exact lines above in your __init__.py, you may simplify you main like this:
from myapp import Author, Book, Review
Python will load the __init__.py, which will in turn load all submodules and import the classes, making them available on the package. Now your main does not need to know where the classes are actually implemented.
Have you tried something like this:
One way
from DirA import hello
Another way
from DirA.hello import hey
If those don't work then append a new system path
You need to import the function itself:
How to call a function from another file in Python?
In your case:
from DirA import foolish1, foolish2
In my project I want to change the main package name.
I've a dir structure like this:
hallo/sub
hallo/foo
hallo/bar
And I want to change the main name for example to 'goodbye':
goodbye/sub
goodbye/foo
goodbye/bar
But as result the new name is always rejected! for example if I import
import goodbye.sub.utils as utils
It return the error
ImportError: No module named sub.utils
And clearly the old name don't works.
The file __init__.py is written in all subdirectories!
I've tried to remove all *.pyc files and cache directory, I've tried to re-clone the project in another directory, but nothing, the new name is always rejected!
I'm using python2 under *nix and I'm never moved under windows.
Some idea?
Edit:
The old name works perfectly:
import hallo.sub.utils as utils
Has always worked without any errors, the problem is the name change.
For imports from the outside, check the contents of __init__.py for variables that can throw things off -- like __all__.
Also, the typical idiom is:
from hallo.sub import utils #tyically this
import hallo.sub.utils as utils #instead of this
I can't imagine why that would make a difference, but python occasionally has silly bugs.
For imports from within the package, you can instead use relative imports. Within your hallo package you can change this:
import hallo.sub as sub
import hallo.sub.utils as utils
to this
from . import sub
from .sub import utils
Then it doesn't matter what the outer package is called.