I use __init__.py to run checks when I do from myprojects.something import blabla.
Today I started using pyzmq and I wanted to see what's going on behind the scenes. So I browsed the code in github and I find (for me) some strange usage of __init__.py there that I cannot explain myself.
For example zmq/core/__init__.py. What's the point of adding in zmq.core.__all__ the __all__'s value of zmq.core.constants, zmq.core.error, zmq.core.message, etc.?
In zmq/__init__.py I see at the end
__all__ = ['get_includes'] + core.__all__
where get_includes is a function which basically returns a list with the directory of the module and the utils directory in the parent directory.
What's the point of that? What has __init.py__ achieved by doing that?
The __all__ is for when someone does from module import * as documented here.
The only solution is for the package author to provide an explicit
index of the package. The import statement uses the following
convention: if a package’s __init__.py code defines a list named
__all__, it is taken to be the list of module names that should be imported when from package import * is encountered. It is up to the
package author to keep this list up-to-date when a new version of the
package is released. Package authors may also decide not to support
it, if they don’t see a use for importing * from their package. For
example, the file sounds/effects/__init__.py could contain the
following code:
__all__ = ["echo", "surround", "reverse"]
This would mean that from sound.effects import * would import the
three named submodules of the sound package.
One use for __all__ is a tool for package builders to allow them to structure their package in a way that works for them while making it convenient for users. Specifically in the case of pyzmq, it lets you write code such as:
import zmq
print zmq.zmq_version()
Rather than having to use the full dotted module name:
print zmq.core.version.zmq_version()
The package designers of pyzmq are using __all__ to promote namespace elements from nested modules up to the top level of their namespace so the user isn't bothered by the structure of their package.
Related
I've tried to create a new module in Python. It's git link : https://github.com/Sanmitha-Sadhishkumar/strman
After uploading and installing that using pip, I found that I could access that module as
import strman.strman as s
s.func_name
What are the changes to be made to access that as
import strman
strman.func_name
In your __init__.py file, you want to use a relative import.
from .strman import *
You have a strman package (the outer directory) and within it a strman module (the strman.py file). That's a perfectly common pattern. But without the relative import your __init__.py wasn't importing from deep enough in the hierarchy.
More generally, whenever you import from a sibling module within a project, you almost always should use a relative import, because it's explicit and avoids various complications, such as the example in your case.
You have to move the files into the main directory. Your directory will look like this
\myProject
- init.py
- strman.py
- main.py # <-- this would be the file your programming is
Read more about modules in the docs
It seems that I am still missing some basics of python. I was trying to understand submodules importing, which I feel I have not understood yet. But I have also stumbled upon something new.
I am having following two packages in two different PyDev projects:
package1
|
+--mod1.py
|
+--mod2.py
package2
|
+--__init__.py
|
+--modx.py
|
+--mody.py
In mod1, I can do import mod2. But in __init__ and modx, I cannot do import mody (Eclipse says "unresolved imports"). In __init__, I can do import .mody or from .mody import vary. In modx, I cannot do import .mody. (In fact I never saw use of . in import statement as prefix to the module. Earlier I only came across import mod and from mod import var, but never saw import .mod and from .mod import var.) Why this might be happening? I must be unaware some context which is causing this behaviour. But then I dont know what is it?
PS: I am using Python 3.4
There is a subtle different between how Python is treating both of those packages.
package1 is treated as a namespace package in that it does not contain an __init__.py file.
package2 is treated as a regular package in that it does contain an __init__.py file.
So I'll give a quick breakdown of why each step is happening:
In mod1, I can do import mod2.
This is happening due to how namespace packages are handled using absolute imports. You're most likely executing python mod1.py from the directory in which the file is stored, right (in my attempt to re-create your folder structure and test it myself locally, I did the same)? So package1 becomes your current working directory with your mod2 file being at the root of that directory.
With namespace packages, Python will default look to sys.path in an attempt to find the imports you have requested. Since your current working directory is automatically added to and included in sys.path, Python can successfully find your import mod2 request without any difficulty.
Thanks to ShadowRanger for correcting my initial response to this where I had misunderstood exactly how Python is including the current working directory in its search.
In init, I can do import .mody or from .mody import vary.
This is because Python is treating this as a regular package. The name of your regular package in this case is package2. When you use the . notation, you are asking Python to start searching for the import from the current package (which in this case is your parent package2). So you have to use import .mody to find the mody package within the current package.
If you used .. then it would import from the parent of the current package and so on.
The dot notation is useful as you are explicitly declaring that you wish to search from the current package only - so if there was another package2 package on your PYTHONPATH, Python would know which one to choose.
But in init and modx, I cannot do import mody (Eclipse says "unresolved imports").
With __init__.py this is because you have not used the dot notation and have not told Python that you wish to search for these modules in the current package. So it's looking to the Python standard library and to your PYTHONPATH for these packages and not finding them (hence your error in Eclipse). By using the dot notation, you are stating that you wish to include the current package in the search and, thus, Python will then be able to locate those files.
Using the dot notation like this, to import via from . import mody, is to use a relative import.
With modx you also have to use a relative import (see next section).
In modx, I cannot do import .mody. Why this might be happening?
This is because you're not using a relative / absolute import. You'll be using a relative import in this case. A relative import is the from . import mody syntax you've seen already. Using a relative or absolute import behaviour is default in Python.
It is now the default behaviour as, with the old Python import behaviour, suppose Python's own standard library had a package called mody. When you'd use import mody it would previously have imported mody from your package and not the standard library. This wasn't always desirable. What if you specifically wanted the standard library version?
So now your imports must be made using from . import mody or from .mody import vary syntax so as the import is very clear. If you use import and not the from... syntax, Python will assume it's a standard library or PYTHONPATH import.
By the way, sources for a lot of the above information came from the following sites:
https://docs.python.org/3/reference/import.html
https://docs.python.org/2.5/whatsnew/pep-328.html
Python modules are optional "additions" to Python that can be imported using the import command like so:
import package1
package1.mod1 # Can be accessed using this
To import individual parts of the package, use from like so:
from package1 import mod1
mod1 # Can be accessed using this
If you want to import every part of a module and use it without package., use:
from package1 i
I have this package structure:
widget/
__init__.py
core.py
extension.py
__init__.py is empty.
core.py contains this:
import widget.extension as extension
It works, but it feels like I'm side-stepping the package and just importing it from the global path (i.e. climbing out of it only to look back into it). If I just import extension in core.py it doesn't work. Does this matter? Am I wrong in the first place? Should I instead be pulling both of these modules into __init__.py?
I'm presuming you are using Python 3; in Python 2, import extension would work as Python 2 will first look for a local, relative import before looking for a global, absolute reference.
You have two more options:
from widget import extension
and
from . import extension
The latter imports relative to the current package, which allows you to rename your widget package without having to update all your internal imports. What you use is a matter of style and taste.
This question already has an answer here:
A Python module and package loading confusion
(1 answer)
Closed 9 years ago.
I am porting some matlab code to python. I need to work with packages and modules in this case. The relevant package directory structure looks like this:
toppackage
__init__.py
subpackage
__init__.py
module.py
...
In the script I use the package, I can work like this:
from toppackage.subpackage.module import SomeClass
s = SomeClass()
But I would prefer working like this:
import toppackage %somewhere at the beginning of file
s = toppackage.subpackage.module.SomeClass()
I see this is done in numpy. But I could not find it in documentation. How can I do that?
Thanks in advance.
You need to import contained packages in the __init__.py files.
You can import the packages inside the toppackage/__init__.py for example:
import toppackage.subpackage.module
or you can import just each directly contained package, so in toppackage/__init__.py:
from . import subpackage
and in toppackage/subpackage/__init__.py:
from . import module
Just importing the top-level package does not automatically make the contained packages available. You need to explicitly import the full path once, somewhere, before that works.
The numpy package imports the nested packages in the top-level __init__.py.
How this stuff works depends critically on what is in __init__.py. Whatever gets imported in that script becomes part of the package namespace.
For example, if your toppackage/__init__.py is empty, to gain access to subpackage, you'd need to do:
import toppackage
try:
p = toppackage.subpackage
except AttributeError:
print "you would see this"
import toppackage.subpackage
p = toppackage.subpackage #no error now
however, if toppackage/__init__.py included the line:
#toppackage/__init__.py
import subpackage
Then the above script would raise no Exceptions.
As noted in the comments, you can also use relative imports since you are in a package:
from . import subpackage
This avoids "namespace" conflicts -- e.g. if you have a subpackage named os it will get your subpackage rather than the python-level package.
I am just starting with python and have troubles understanding the searching path for intra-package module loads. I have a structure like this:
top/ Top-level package
__init__.py Initialize the top package
src/ Subpackage for source files
__init__.py
pkg1/ Source subpackage 1
__init__.py
mod1_1.py
mod1_2.py
...
pkg2/ Source subpackage 2
__init__.py
mod2_1.py
mod2_2.py
...
...
test/ Subpackage for unit testing
__init__.py
pkg1Test/ Tests for subpackage1
__init__.py
testSuite1_1.py
testSuite1_2.py
...
pkg2Test/ Tests for subpackage2
__init__.py
testSuite2_1.py
testSuite2_2.py
...
...
In testSuite1_1 I need to import module mod1_1.py (and so on). What import statement should I use?
Python's official tutorial (at docs.python.org, sec 6.4.2) says:
"If the imported module is not found in the current package (the package of which the current module is a submodule), the import statement looks for a top-level module with the given name."
I took this to mean that I could use (from within testSuite1_1.py):
from src.pkg1 import mod1_1
or
import src.pkg1.mod1_1
neither works. I read several answers to similar questions here, but could not find a solution.
Edit: I changed the module names to follow Python's naming conventions. But I still cannot get this simple example to work.
The module name doesn't include the .py extension. Also, in your example, the top-level module is actually named top. And finally, hyphens aren't legal for names in python, I'd suggest replacing them with underscores. Then try:
from top.src.pkg1 import mod1_1
Problem solved with the help of http://legacy.python.org/doc/essays/packages.html (referred to in a similar question). The key point (perhpas obvious to more experienced python developers) is this:
"In order for a Python program to use a package, the package must be findable by the import statement. In other words, the package must be a subdirectory of a directory that is on sys.path. [...] the easiest way to ensure that a package was on sys.path was to either install it in the standard library or to have users extend sys.path by setting their $PYTHONPATH shell environment variable"
Adding the path to "top" to PYTHONPATH solved the problem.To make the solution portable (this is a personal project, but I need to share it across several machines), I guess having a minimal initialization code in top/setup.py should work.