Creating a python library: how to write __init__.py - python

I have written a small Python library, currently hosted at BitBucket. As you can see, the library is called pygpstools, and it's made from 5 files:
gpstime.py → A class
satellite.py → A class
geodesy.py → A module with some geodesy methods
almanacs.py → A module with some almanac methods
constants.py → Some constants
I want to use it as written at the README. For example:
from pygpstools import GPSTime
GPSTime(wn=1751, tow=314880)
or:
import pygpstools
pygpstools.GPSTime(wn=1751, tow=314880)
But after installing my library with command python setup.py install I'm getting ImportError when trying to access GPSTime class like this.
I guess the problem is at the __init__.py file. When I asked about this at the python IRC channel, I was told that leaving it empty does the trick. But I have researched and it looks like that only tells Python that it is a module, but it is not enough to allow this kind of importing I'm looking for, as at any other library out there.
So I have tried (not currently updated at bitbucket) to use this as __init__.py:
__title__ = 'pygpstools'
__version__ = '0.1.1'
__author__ = 'Roman Rodriguez'
__license__ = 'MIT'
__copyright__ = 'Copyright 2013 Roman Rodriguez'
import almanacs
import constants
import geodesy
import gpstime
import satellite
but still doesn't work: ImportError for GPSTime.
What am I missing?

GPSTime, for example, is in the module gpstime, so its actual (relative) name is gpstime.GPSTime. So when you import gpstime in your __init__ you are actually making available the name gpstime which holds a reference to your type as gpstime.GPSTime.
So you would have to use from pygpstools import gpstime and then gpstime.GPSTime as the type name.
Obviously this is not what you want, so instead, you want to “collect” all your types in the __init__ module. You can do that by just making them available directly:
from almanacs import *
from constants import *
from geodesy import *
from gpstime import GPSTime
from satellite import *
I have used * now to import anything because I didn’t take a closer look at what actual types there are in your files. You should specify it though. It is also recommended, to define an __all__ list in your __init__ so that you can control which names are imported when writing from pygpstools import *.

Related

How to list all functions in a python module when imported with *

I am importing a module in python as this:
from myutils.user_data import *
How do I find out what are the list of methods that I have imported?
I know one workaround from here:
How to list all functions in a Python module?
being:
from inspect import getmembers, isfunction
import myutils.user_data as module_name
functions_list = getmembers(module_name, isfunction)
print(functions_list)
but this would oblige me to use the nomenclature:
module_name.mehtodA() whereas I would like to be able to use the methods as such methodA()
of course I can do:
from myutils.user_data import *
import myutils.user_data as module_name
but this is actually importing two times.
Any idea?
EDIT: Why do I need this? I am creating documentation for a module in a JupyterHub environment (in premises). I create this documentation using notebooks, i.e. anyone interested in finding out the use of a particular .py file (including utility methods) can open the notebook and play around AND the jupyter notebook can be rendered as a web site with voila. In that case I would like to print all the methods included in the particular .py file.
This is also a question that just made me curious. Someone commented you would never import with * a module. Well, why not if you know what you are importing being a few very small methods.
Generally you are rarely recommended to use the from ... import * style, because it could override local symbols or symbols imported first by other modules.
That beeing said, you could do
symbols_before = dir()
from myutils.user_data import *
symbols_after = dir()
imported_symbols = [s for s in symbols_after if not s in symbols_before]
which stores the freshly imported symbols in the imported_symbols list.
Or you could use the fact, that the module is still loaded into sys.modules and do
import sys
from inspect import getmembers, isfunction
from myutils.user_data import *
functions_list = getmembers(sys.modules['myutils.user_data'], isfunction)
print(functions_list)

Automatically import files in python

I am develepong a blender addon which has several submodules
- A/__init__.py
|
|- B/__init__.py
In A/init.py I can do import A.B to import the content of submodule B, but I would like to be able to import automatically this files. Is there any way to achieve that?
The idea is that my addon is used to implement several glTF extensions. Each extension is in its own submodule and the glTF exporter expects me to return some classes, one for each extension.
Instead of importing manually each extension and adding the class to the list of extensions, I want that to happen automatically
instead of
from A.B import B_extension
glTF2ExportUserExtensions = [A.B.B_extension]
I want something like
# A.submodules returns all the submodules and extension() returns the extension class. Im assuming each submodule have an extension() function
glTF2ExportUserExtensions = [A.submodules.extension()]
I need to return a list of classes
You can import things from an essential file that you have the imports on.
Example of ./import_file.py
import example
import ...
You can use "*" to import everything from the import file, or you can import something specific.
Example of ./main_file.py
from import_file import *
from import_file import example
Ok, I guess I understand what you want to do.
So to import several submodules under the same "submodule-name" submodules, import them in a separate file submodules.py at the same level as A/__init__.py:
from .B import B_extension as extension
from .C import C_extension as someothername
And in A/__init__.py add:
from . import submodules
Now if you import A with
import A
you can access the submodules with A.submodules.extension, A.submodules.someothername, etc...
If you want to access the submodules functions/classes/etc. directly from submodules, such as A.submodules.extension(), your submodules.py files has to look like:
from .B.B_extension import extension, anotherBmethod, SomeBClass
from .C.C_extension import Cextension, anotherCmethod, SomeCClass
If you want to have a fully automatic import, use pkgutil (credit goes to this answer), even though I strongly object importing all submodules automatically. Explicit is better than implicit. You never know what happens when you change a single line in a submodule, without testing all imports when doing it implicitly... Add this to A/__init__.py:
import pkgutil
__all__ = []
for loader, module_name, is_pkg in pkgutil.walk_packages(__path__):
__all__.append(module_name)
_module = loader.find_module(module_name).load_module(module_name)
globals()[module_name] = _module

Why does python import module imports when importing *

Let's say I have a file where I'm importing some packages:
# myfile.py
import os
import re
import pathlib
def func(x, y):
print(x, y)
If I go into another file and enter
from myfile import *
Not only does it import func, but it also imports os, re, and pathlib,
but I DO NOT want those modules to be imported when I do import *.
Why is it importing the other packages I'm importing and how do you avoid this?
The reason
Because import imports every name in the namespace. If something has a name inside the module, then it's valid to be exported.
How to avoid
First of all, you should almost never be using import *. It's almost always clearer code to either import the specific methods/variables you're trying to use (from module import func), or to import the whole module and access methods/variables via dot notation (import module; ...; module.func()).
That said, if you must use import * from module, there are a few ways to prevent certain names from being exported from module:
Names starting with _ will not be imported by import * from .... They can still be imported directly (i.e. from module import _name), but not automatically. This means you can rename your imports so that they don't get exported, e.g. import os as _os. However, this also means that your entire code in that module has to refer to the _os instead of os, so you may have to modify lots of code.
If a module contains the name __all__: List[str], then import * will export only the names contained in that list. In your example, add the line __all__ = ['func'] to your myfile.py, and then import * will only import func. See also this answer.
from myfile import func
Here is the fix :)
When you import *, you import everything from. Which includes what yu imported in the file your source.
It has actually been discussed on Medium, but for simplification, I will answer it myself.
from <module/package> import * is a way to import all the names we can get in that specific module/package. Usually, everyone doesn't actually use import * for this reason, and rather sticked with import <module>.
Python's import essentially just runs the file you point it to import (it's not quite that but close enough). So if you import a module it will also import all the things the module imports. If you want to import only specific functions within the module, try:
from myfile import func
...which would import only myfile.func() instead of the other things as well.

How to import everything inside a folder into a single module in Python2

I simply want to take all my .py files from a single folder (I don't care about the sub-folders for now) and put them into a single module.
The use case I'm having here is that I'm writing some pretty standard object-oriented code and I'm using a single file for every class, and I don't want to have to write from myClass import myClass for every class into my __init__.py. I can't use Python3, so I'm still working with impand reloadand such.
At the moment I'm using
# this is __init__.py
import pkgutil
for loader, name, is_pkg in pkgutil.walk_packages(__path__):
if not is_pkg:
__import__(__name__ + "." + name)
and it doesn't seem to work, it includes the packages but it includes them as modules, so that I have to write MyClass.MyClass for a class that is defined in a file with it's own name. That's silly and I don't like it.
I've been searching forever and I'm just getting more confused how complicated this seemingly standard use case seems to be. Do python devs just write everything into a single file? Or do they always have tons of imports?
Is this something that should be approached in an entirely different way?
What you really want to do
To do the job you need to bind your class names to namespace of your __init__.py script.
After this step you will be able to just from YourPackageName import * and just use your classes directly. Like this:
import YourPackageName
c = YourPackageName.MyClass()
or
from YourPackageName import *
c = MyClass()
Ways to achieve this
You have multiple ways to import modules dynamically: __import__(), __all__.
But.
The only way to bind names into namespace of current module is to use from myClass import myClass statement. Static statement.
In other words, content of each of your __init__.py scripts should be looking like that:
#!/usr/bin/env python
# coding=utf-8
from .MySubPackage import *
from .MyAnotherSubPackage import *
from .my_pretty_class import myPrettyClass
from .my_another_class import myAnotherClass
...
And you should know that even for a dynamic __all__:
It is up to the package author to keep this list up-to-date when a new version of the package is released.
(https://docs.python.org/2/tutorial/modules.html#importing-from-a-package)
So a clear answers to your questions:
Do python devs just write everything into a single file?
No, they don't.
Or do they always have tons of imports?
Almost. But definitely not tons. You need to import each of your modules just once (into an appropriate __init__.py scripts). And then just import whole package or sub-package at once.
Example
Let's assume that there is next package structure:
MyPackage
|---MySubPackage
| |---__init__.py
| |---pretty_class_1.py
| |---pretty_class_2.py
|---__init__.py
|---sleepy_class_1.py
|---sleepy_class_2.py
Content of the MyPackage/MySubPackage/__init__.py:
#!/usr/bin/env python
# coding=utf-8
from .pretty_class_1 import PrettyClass1
from .pretty_class_2 import PrettyClass2
Content of the MyPackage/__init__.py:
#!/usr/bin/env python
# coding=utf-8
from .MySubPackage import *
from .sleepy_class_1 import SleepyClass1
from .sleepy_class_2 import SleepyClass2
As result, now we are able to write next code in our application:
import MyPackage
p = MyPackage.PrettyClass1()
s = MyPackage.SleepyClass2()
or
from MyPackage import *
p = PrettyClass1()
s = SleepyClass2()

Keep imported modules from showing up in code completion?

How can I keep imported modules from being accessible (i.e., clogging my code-completion options)?
For example:
# testmodule.py
import os
def o_stuff():
return
When I import testmodule, I don't want os to show up every time I type testmodule.; I only want the functions/classes declared within testmodule – in this case, just o_stuff.
Is there something similar to the asterisk (i.e., from testmodule import *) that will do this?
You can define a special variable __all__ containing a list of the names to be imported by from module import * - for example:
# testmodule.py
import os
__all__ = ['o_stuff', 'more_stuff']
def o_stuff():
pass
def more_stuff():
pass
IDEs with code-completion will typically also respect __all__ (though I'm unfamiliar with Visual Studio, so I don't know whether IntelliSense does so).
An alternative, included here for completeness although I would strongly recommend against it (on the grounds that it'll annoy anyone reading your code to distraction) is to import modules as an underscore-prefixed alias:
# ugly_as_sin.py
import os as _os
def o_stuff():
return _os.name
Again, both from module import * and, typically, code-completion will ignore underscore-prefixed names.
You can try to use the __all__ in your module to see if it helps.
import os
__all__ = ['o_stuff']
def o_stuff():
return
Not familiar with Intellisense but it sounds like it could also use a bit of fine-tuning.

Categories