I'm just starting to get to the point in my python projects that I need to start using multiple packages and I'm a little confused on exactly how everything is supposed to work together. What exactly should go into the __init__.py of the package? Some projects I see just have blank inits and all of their code are in modules in that package. Other projects implement what seems to be the majority of the package's classes and functions inside the init.
Is there a document or style guide or something that describes what the python authors had in mind for the use of packages and the __init__ file and such?
Edit:
I know the point of having the __init__.py file in the simplest sense that it makes a folder a package. But why would I put a function there instead of a module in that same folder(package)?
__init__.py can be empty, but what it really does is make sure Python treats your directories correctly, provide any initialization you might need for when your package is imported (configuring the environment or something along those lines), or defining __all__ so that Python knows what to do when someone uses from package import *.
Most everything you need to know is described in the docs on Packages. Dive Into Python also has a piece on packaging.
You already know, I guess that __init__.py files are required to make Python treat the directories as containing packages.
In the above model __init__.py can remain empty.
You can can also execute initialization code for the package.
You can also set the __all__ variable.
[Edit: learnings]
When you do "from package import item", or "from package import *", then the variable __all__ can be used to import selected packages.
See : http://docs.python.org/tutorial/modules.html
Related
I feel a bit confused with how python modules work when I started looking at PyMySQL repository, see here: https://github.com/PyMySQL/PyMySQL?files=1
1) Why is there no pymysql.py file because it is imported like: import pymysql? Isnt it required to have such a file?
2) I cannot find the method connect, used like: pymysql.connect(...), anywhere. Is it possible to rename exported methods somehow?
There's a directory pymysql there. A directory can also be imported as a module*, with the advantage that it can contain submodules. Classically, there's a __init__.py file in the directory that controls what's in the top-level pymysql.* namespace.
So, the connect method you're missing will either be defined directly in pymysql/__init__.py, or defined in one of its siblings in that directory, and then imported from there by pymysql/__init__.py.
*Strictly speaking, a directory that you import like a module is really called a "package". I like to avoid that term—it's potentially confusing because the term is overloaded: what you install with pip is also called a "package" in sense 2, and that might actually contain multiple "packages" in sense 1.
See What is __init__.py for? and the official docs
I have a Python module called util. I would like to import a script in this package _util.py from another script in scripts folder.
Even if the util package has an empty __init__.py file it does not appear as a Python package but a normal directory, without the small dot on folder image.
How can I import this module?
A preliminary answer to your question is that modules or methods beginning with underscores are meant to be used internally.
_my_method() should only be referenced from within the module holding it
_my_module() should only be referenced from within the package holding it
That being said, this convention is meant to be a hint to other developers, not a strict prohibition. Perhaps the first step you can take to solve the import issue is to rename _util.py to util.py and proceed from there.
I'm fairly new to python and especially it's import mechanism. I'm not entirely sure i'm using the terminology correctly so i should apologize for that up front.
firstly, this seems to be a problem i'm having with a 3rd party import so i can't really change the structure of their release.
In the release, all of the packages are in site-packages/[ROOTFOL]/[PACKAGE]
the [ROOTFOL] does not have a __init__.py file, only the package folders have this file.
this folder is placed into site-packages and the site-packages is present in my PYTHONPATH
in the examples they provide, they use it like this:
import ROOTFOL.PACKAGE.WhateverObject as obj
I'm trying to avoid adding every single package to the PYTHONPATH as there are a bunch of them. Everything seems to work fine, however it really inhibits my ability to work with the auto-complete functionality and that is the frustrating part.
Something else i find strange, is that when the packages are installed, there is a EGG-INFO folder placed along side the package. In this there are several .txt files and one of which is namespace_packages.txt which has only the ROOTFOL. Is there some way i should be setting this to PyDev?
So, what you're seeing here is their distribution model. Usually a module will have one root import that everything stems from, but that's not necessarily the case. They're providing a package with (what I assume) is many modules that don't interact with each other; or they can all stand alone.
instead of importing each package individually, you could use the 'from' keyword:
from ROOTFOL.PACKAGE import *
which will grab everything inside that sub-module. You could e-mail the developer and ask why they deployed it this way...or you could add your own __init__.py to the root folder and,
from ROOTFOL import *
which will walk the tree. Good luck!
Here is my structure,
main.py
folder1\
button.py
folder2\
picturebutton.py
folder3\
listbox.py
folder4\
customlistbox.py
folder5\
hyperlistbox.py
Now,
I have a module called, "widget.py" and I would like to make it accessible to all the modules here so that each module will be able to say import widget or something of the sort. After googling, it appears that I have to make a package to do this.
I could not function with the examples online as I have no idea how they work, and I am hoping that one of you may be able to help me with my case.
Edit:
All the folders, (except for the root one) have an __init__.py file.
Being able to import some other module does not need for that to be a package, it needs for the widget module to be put on your PYTHONPATH. You'd do that typically by installing it (writing a setup.py file, see the standard library's distutils module).
If you did want a package though, every folder that needs to be a package needs to have an __init__.py file in it (empty is fine).
Proper way is to create a setup.py file for your package but since it may take time . Below is shortcut .
If you want to use your module it frequently like in script . Easy way is to export "PYTHONPATH" in bashrc/zshrc file and give path to the directory containing your code .
For example:
export PYTHONPATH=$PYTHONPATH:$HOME/path/to/package
Do check on terminal using
echo "$PYTHONPATH"
Happy Coding
In some __init__.py files of modules I saw such single line:
__import__('pkg_resources').declare_namespace(__name__)
What does it do and why people use it? Suppose it's related to dynamic importing and creating namespace at runtime.
It boils down to two things:
__import__ is a Python function that will import a package using a string as the name of the package. It returns a new object that represents the imported package. So foo = __import__('bar') will import a package named bar and store a reference to its objects in a local object variable foo.
From setup utils pkg_resources' documentation, declare_namespace() "Declare[s] that the dotted package name name is a "namespace package" whose contained packages and modules may be spread across multiple distributions."
So __import__('pkg_resources').declare_namespace(__name__) will import the 'pkg_resources' package into a temporary and call the declare_namespace function stored in that temporary (the __import__ function is likely used rather than the import statement so that there is no extra symbol left over named pkg_resources). If this code were in my_namespace/__init__.py, then __name__ is my_namespace and this module will be included in the my_namespace namespace package.
See the setup tools documentation for more details
See this question for discussion on the older mechanism for achieving the same effect.
See PEP 420 for the standardized mechanism that provides similar functionality beginning with Python 3.3.
This is a way to declare the so called "namespace packages" in Python.
What are these and what is the problem:
Imagine you distribute a software product which has a lot of functionality, and not all people want all of it, so you split it into pieces and ship as optional plugins.
You want people to be able to do
import your_project.plugins.plugin1
import your_project.plugins.plugin2
...
Which is fine if your directory structure is exactly as above, namely
your_project/
__init__.py
plugins/
__init__.py
plugin1.py
plugin2.py
But what if you ship those two plugins as separate python packages so they are located in two different directories? Then you might want to put __import__('pkg_resources').declare_namespace(__name__) in each package's __init__.py so that Python knows those packages are part of a bigger "namespace package", in our case it's your_project.plugins.
Please refer to the documentation for more info.