I have a Python package in which I would like to expose some classes. I will also need to iterate over these classes at some point so I would like to also expose __all__ as a list of classes, not strings. After trying the numerous methods to do this, I settled on the following in the __init__.py:
from .a import A
from .b import B
from .c import C
ALL = [A, B, C]
__all__ = [x.__name__ for x in ALL]
This works, as I can use from package import * to grab all the classes as well as being able to import package / package.ALL to be able to iterate over them.
Is there a cleaner (or more correct) way to do this? My main gripe with it is that If I want to add a new class, I'd have to import the classes first that add it to a list. It would be ideal if I could just add the new class in one place.
Related
Google style guide requires to import modules. Say I have a non trivial class and several non-trivial derived classes. It seems that I have to put them all in the one file otherwise will have to use different prefixes when using them.
Personally I prefer one class per file. Is there a common idiom to avoid unnecessary large files?
E.g. can I rename several imports into one as long as there is no name conflicts?
import data_main as data
import data_derived_a as data
import data_derived_b as data
With modules, no, there's really no way of doing this that I know of. If you're happy to put your class in separate files within a package, though, you can import the individual classes in __init__.py, and your client modules can import them all at once, like so.
# mypackage/a.py
class A:
pass
# mypackage/b.py
class B:
pass
# mypackage/__init__.py
from .a import A
from .b import B
# mymodule.py
from mypackage import A, B
a = A()
b = B()
# Or mymodule.py
import mypackage as p
a = p.A()
b = p.B()
I have a python directory like below.. in test.py i have a,b,c methods.
While import test i dont want user to import c method. One way is making as c method as private. Is there anyway to achieve that in __init__.py or using __import__
test
__init__.py
test.py
I have see few solution in stackoverflow am not getting a way to achieve that.
Thanks.
If you're looking for absolute private methods, then python is the wrong language for you - go to Java or C/++/# or some other language that supports the distinction between public and private. In python, if it is known something exists, then it is generally possible to access that thing no matter how hidden it is.
If you're simply trying to limit the convenient options of the user when they import your module, then you can simply selectively include or exclude methods in __init__.py. Say that you have
test.py
def a():
pass
def b():
pass
def c():
pass
and you wanted a and b to be accessible to the user but c not to be, then you could do
__init__.py
from .test import a, b
and export the folder as a module. Now, when a user does
import test
they get access to only the stuff that's in the namespace by the end of __init__.py (that is, they can get test.a and test.b, but test.c doesn't exist). Since you never included c in __init__.py, it doesn't appear there.
Note that c would still be accessible by doing
from test.test import c
which accesses the sourcefile directly.
Alternatively, you can designate on a per-file basis which names should be immediately accessible by using a built-in variable __all__. The following will have the same effect as the above code:
test.py
...
__all__ = ['a', 'b'] # note that 'c' is excluded
__init__.py
from test.py import * # imports a and b, but not c
I want to define a bunch of attributes for use in a module that should also be accessible from other modules, because they're part of my interface contract.
I've put them in a data class in my module like this, but I want to avoid qualifying them every time, similar to how you use import * from a module:
#dataclass
class Schema:
key1='key1'
key2='key2'
and in the same module:
<mymodule.py>
print(my_dict[Schema.key1])
I would prefer to be able to do this:
print(my_dict[key1])
I was hoping for an equivalent syntax to:
from Schema import *
This would allow me to do this from other modules too:
<another_module.py>
from mymodule.Schema import *
but that doesn't work.
Is there a way to do this?
Short glossary
module - a python file that can be imported
package - a collection of modules in a directory that can also be imported, is technically also a module
name - shorthand for a named value (often just "variable" in other languages), they can be imported from modules
Using import statements allows you to import either packages, modules, or names:
import xml # package
from xml import etree # also a package
from xml.etree import ElementTree # module
from xml.etree.ElementTree import TreeBuilder # name
# --- here is where it ends ---
from xml.etree.ElementTree.TreeBuilder import element_factory # does not work
The dots in such an import chain can only be made after module objects, which packages and modules are, and names are not. So, while it looks like we are just accessing attributes of objects, we are actually relying on a mechanism that normal objects just don't support, so we can't import from within them.
In your particular case, a reasonable solution would be to simply turn the object that you wanted to hold the schema into a top-level module in your project:
schema.py
key1 = 'key1'
key2 = 'key2'
...
Which will give you the option to import them in the way that you initially proposed. Doing something like this to make common constants easily accessible in your project is not unusual, and the django framework for example uses a settings.py in the same manner.
One thing you should keep in mind is that names in python modules are effectively singletons, so their values can't be changed at runtime[1].
[1] They can, but it's so hacky that it should pretty much always be treated as not possible.
Is it possible to import everything except one module from a package?
I need a lot of modules from a particular library that I use in my class, but it looks like it used the same module name for one of the modules that I need.
I need to use set operation and intersection, but when I import that library from my class, it gives me an error because of that.
I didn't want to import it separately or put the name in front of every methods since I'm using it a lot.
Is there a way for the python to import everything except for a particular method like set?
Or maybe import the set part again later?
No, there is no terminology to from ... import * except blah, bleh, bluh. You can either write your own import function to support it, or do something like:
from xyz import *
del set
which will stop shadowing the built-in set so you can use it again. Then if you need the xyz.set function you can do:
from xyz import set as xyzset
Note: from ... import * is not usually good practice, and you should make sure the modules you are using this way support it -- if they don't explicity say they were designed to be used this way, then you shouldn't (unless you enjoy debugging weird problems later ;).
I suppose what you want would be:
from thing import a, b, c, d, e, f
which will import a, b, c, d, e, f from thing.
AFAIK, there is no way to do an from thing import all but a, b, c
thats why
import thing
thing.a
exists in the first place.
I'm not sure I entirely understand what's going on (giving actual module names might help). But it's generally considered bad practice to do from ... import * at all, because then it's not obvious where particular things are coming from. Instead, do from ... import thingA, thingB, thingC.
You could also do import ... as shortname, and then refer to the methods as shortname.whatever (where shortname could obviously be something very short).
I want the interface of some module to contain a certain number of functions and classes (and nothing else). I could implement all of those in a single file, and would easily get the interface I want. But since there is a lot of code, I'd prefer to split the whole thing up into several files, say
mypackage/
__init__.py
a.py
b.py
c.py
d.py
To get the desired interface anyway, I define an __init__.py file for the package that imports all public symbols from a, b, c and d:
from a import func_a1, func_a2, ClassA1, ClassA2
from b import func_b1, func_b2, ClassB1, ClassB2
from c import func_c1, func_c2, ClassC1, ClassC2
from d import func_d1, func_d2, ClassD1, ClassD2
If I import the package using
import mypackage
the package namespace also contains the symbols a, b, c and d. These names are implementation details and not part of my interface. I don't want them to appear as "public" symbols. What is the best way of getting rid of them?
The options I considered are
Use a single module instead of a package. The interface will look fine, but the implementation will get less clear than it is now.
Add the line
del a, b, c, d
to the end of __init__.py. Works ok, but seems like a hack. (For example, you can't import __init__ any more, which works without this line.)
Rename a, b, c and d to _a, _b, _c and _d. Now they are included in mypackage's namespace as "private" symbols, which I'm fine with, but it feels a bit strange that all my filenames start with an underscore (in reality, there are of course more than four submodules).
Any better suggestions? Or thoughts on which option to prefer?
Or am I just to anal and shouldn't care about the whole thing?
If some of the files in a package are indeed implementation details, go ahead and stick an underscore in front of them -- that's what we use them for.
For example, if you look in ctypes you'll see
__init__.py
==================================================
"""create and manipulate C data types in Python"""
import os as _os, sys as _sys
__version__ = "1.1.0"
from _ctypes import Union, Structure, Array
from _ctypes import _Pointer
from _ctypes import CFuncPtr as _CFuncPtr
...
As you can see, even os and sys became implementation details in that file.
If you really want to remove the names from the namespace then you can just use the del statement on them and they'll disappear like the wind.
Here's a solution inspired by Javascript single-function modules:
def __init__module():
from os import path
def _module_export_1():
return path.abspath('../foo')
def _module_export_2():
return path.relpath('foo/bar', 'foo')
g = globals()
g['module_export_1'] = _module_export_1
g['module_export_2'] = _module_export_2
__init__module()
Although the module needs to import 'path' from os, 'path' doesn't pollute the module namespace. The only cruft in the module namespace is __init_module(), which is clearly marked private by the double-underscore prefix.
Another option would be to import needed modules at the top of each function, rather than the top of your module. After the first time a module is imported, subsequent imports are just a lookup in the sys.modules dictionary.
But I agree with the other commenters here -- the Python convention is not to worry about module namespace pollution, and just make it obvious to your module's users which parts of the namespace are your public API and which are internals.
From http://docs.python.org/tutorial/modules.html:
The import statement uses the
following convention: if a package’s
__init__.py code defines a list named __all__, it is taken to be the list of module names that should be imported
when from package import * is
encountered.
In your mypackage/__init__.py, try adding this:
# add this line, replace "..." with the rest of the definitions
# you want made public
__all__ = ['func_a1', 'func_a2', 'ClassA1', 'ClassA2', ...]
from a import func_a1, func_a2, ClassA1, ClassA2
from b import func_b1, func_b2, ClassB1, ClassB2
from c import func_c1, func_c2, ClassC1, ClassC2
from d import func_d1, func_d2, ClassD1, ClassD2