Avoiding huge files when following python google-style-guide - python

Google style guide requires to import modules. Say I have a non trivial class and several non-trivial derived classes. It seems that I have to put them all in the one file otherwise will have to use different prefixes when using them.
Personally I prefer one class per file. Is there a common idiom to avoid unnecessary large files?
E.g. can I rename several imports into one as long as there is no name conflicts?
import data_main as data
import data_derived_a as data
import data_derived_b as data

With modules, no, there's really no way of doing this that I know of. If you're happy to put your class in separate files within a package, though, you can import the individual classes in __init__.py, and your client modules can import them all at once, like so.
# mypackage/a.py
class A:
pass
# mypackage/b.py
class B:
pass
# mypackage/__init__.py
from .a import A
from .b import B
# mymodule.py
from mypackage import A, B
a = A()
b = B()
# Or mymodule.py
import mypackage as p
a = p.A()
b = p.B()

Related

is there anyway to restrict a method in module from import in python?

I have a python directory like below.. in test.py i have a,b,c methods.
While import test i dont want user to import c method. One way is making as c method as private. Is there anyway to achieve that in __init__.py or using __import__
test
__init__.py
test.py
I have see few solution in stackoverflow am not getting a way to achieve that.
Thanks.
If you're looking for absolute private methods, then python is the wrong language for you - go to Java or C/++/# or some other language that supports the distinction between public and private. In python, if it is known something exists, then it is generally possible to access that thing no matter how hidden it is.
If you're simply trying to limit the convenient options of the user when they import your module, then you can simply selectively include or exclude methods in __init__.py. Say that you have
test.py
def a():
pass
def b():
pass
def c():
pass
and you wanted a and b to be accessible to the user but c not to be, then you could do
__init__.py
from .test import a, b
and export the folder as a module. Now, when a user does
import test
they get access to only the stuff that's in the namespace by the end of __init__.py (that is, they can get test.a and test.b, but test.c doesn't exist). Since you never included c in __init__.py, it doesn't appear there.
Note that c would still be accessible by doing
from test.test import c
which accesses the sourcefile directly.
Alternatively, you can designate on a per-file basis which names should be immediately accessible by using a built-in variable __all__. The following will have the same effect as the above code:
test.py
...
__all__ = ['a', 'b'] # note that 'c' is excluded
__init__.py
from test.py import * # imports a and b, but not c

Cleanest way to expose __all__ as a list

I have a Python package in which I would like to expose some classes. I will also need to iterate over these classes at some point so I would like to also expose __all__ as a list of classes, not strings. After trying the numerous methods to do this, I settled on the following in the __init__.py:
from .a import A
from .b import B
from .c import C
ALL = [A, B, C]
__all__ = [x.__name__ for x in ALL]
This works, as I can use from package import * to grab all the classes as well as being able to import package / package.ALL to be able to iterate over them.
Is there a cleaner (or more correct) way to do this? My main gripe with it is that If I want to add a new class, I'd have to import the classes first that add it to a list. It would be ideal if I could just add the new class in one place.

How to call a class outside of the file

I am an APCS student so my primary language is Java but am currently writing a program that requires python which I know somewhat. I assume there is a way to call classes / methods outside the file in which they were created like in Java because they are both OO, but was wondering how I would go about doing this. Thanks.
Depending on how you go about packaging your code (IE: you may end up using a python package or simply a module) you will likely use the import statement in a very similar way as you do java. Some Examples:
import my_module
my_object = my_module.MyClass()
or
from my_module import MyClass
my_object = MyClass()
You use the import statement, where you either import the whole file or just the required class.
For example, my directory has file1.py and file2.py.
file2.py has a class named "A", which has the method print_hi().
So, in file1.py I do this:
from file2 import A
A.print_hi()
And I get the expected output.
Or, I could do this:
import file2
file2.A.print_hi()
which gives me the same output.But, you can't just import a method from a class which is present in another file --- you either import the file or the required class.
Thus, doing things like:
from file2.A import print_hi
or
from file2 import A.print_hi
won't work.

Conveniently import several classes from modules in a Python package

I'm currently creating a framework for a syntax tree. I have a folder/package syntax which contains one syntax tree element which is a class in each file, the structure looks then like:
syntax/List.py -> contains class List
syntax/Block.py -> contains class Block
syntax/Statement.py -> contains class Statement
Now I'd like to import the folder into my source file in a way that I can access the classes like
block = syntax.Block()
Is that possible at all? So far, I always ended up that I need syntax.Block.Block() which is not very nice...
Project structure
syntax
├── Block.py
├── __init__.py
The class
# syntax/Block.py (this file really ought to be lowercase, leave PascalCase to class names)
class Block(object):
pass
Imports in __init__
# syntax/__init__.py
from .Block import Block # Relative import with .
Using the package
In [5]: import syntax
In [6]: b = syntax.Block()
In [7]: b
Out[7]: <syntax.Block.Block at 0x108dcb990>
Alternative if you are open to some re-organization
Unlike languages that require us to put a single class into a file with same name (class Block inside file Block.py), Python is pretty flexible about this.
You can actually put many classes in syntax.py and import syntax alone, then access syntax.Block (no imports in __init__.py would be required for this)
# syntax.py
class Block(object):
pass
class List(object):
pass
Then can use as follows
import syntax
b = syntax.Block()
l = syntax.List()
The most readable way to deal with that is to do your imports into the __init__.py file
Example :
# syntax/__init__.py
from syntax.List import List
from syntax.Block import Block
from syntax.Statement import Statement
# in your main file
from syntax import List
import syntax
b = syntax.Block()
Note: you can also use the relative import syntax in your __init__.py file (for example from .List import List)
When you do an import, you define a variable. This variable is containing your imported variable, constant, function, class or whatever you want.
If you do from .List import List into your __init__.py file, you are defining the variable List into your __init__.py file refering to the class syntax.List.List.
So when you are trying you write from syntax import List you are trying to import the variable List from your __init__.py file, which point to the syntax.List.List.
from syntax import List
from syntax.List import List as OriginList
List is OriginList # True

How to keep submodule names out of the name space of a Python package?

I want the interface of some module to contain a certain number of functions and classes (and nothing else). I could implement all of those in a single file, and would easily get the interface I want. But since there is a lot of code, I'd prefer to split the whole thing up into several files, say
mypackage/
__init__.py
a.py
b.py
c.py
d.py
To get the desired interface anyway, I define an __init__.py file for the package that imports all public symbols from a, b, c and d:
from a import func_a1, func_a2, ClassA1, ClassA2
from b import func_b1, func_b2, ClassB1, ClassB2
from c import func_c1, func_c2, ClassC1, ClassC2
from d import func_d1, func_d2, ClassD1, ClassD2
If I import the package using
import mypackage
the package namespace also contains the symbols a, b, c and d. These names are implementation details and not part of my interface. I don't want them to appear as "public" symbols. What is the best way of getting rid of them?
The options I considered are
Use a single module instead of a package. The interface will look fine, but the implementation will get less clear than it is now.
Add the line
del a, b, c, d
to the end of __init__.py. Works ok, but seems like a hack. (For example, you can't import __init__ any more, which works without this line.)
Rename a, b, c and d to _a, _b, _c and _d. Now they are included in mypackage's namespace as "private" symbols, which I'm fine with, but it feels a bit strange that all my filenames start with an underscore (in reality, there are of course more than four submodules).
Any better suggestions? Or thoughts on which option to prefer?
Or am I just to anal and shouldn't care about the whole thing?
If some of the files in a package are indeed implementation details, go ahead and stick an underscore in front of them -- that's what we use them for.
For example, if you look in ctypes you'll see
__init__.py
==================================================
"""create and manipulate C data types in Python"""
import os as _os, sys as _sys
__version__ = "1.1.0"
from _ctypes import Union, Structure, Array
from _ctypes import _Pointer
from _ctypes import CFuncPtr as _CFuncPtr
...
As you can see, even os and sys became implementation details in that file.
If you really want to remove the names from the namespace then you can just use the del statement on them and they'll disappear like the wind.
Here's a solution inspired by Javascript single-function modules:
def __init__module():
from os import path
def _module_export_1():
return path.abspath('../foo')
def _module_export_2():
return path.relpath('foo/bar', 'foo')
g = globals()
g['module_export_1'] = _module_export_1
g['module_export_2'] = _module_export_2
__init__module()
Although the module needs to import 'path' from os, 'path' doesn't pollute the module namespace. The only cruft in the module namespace is __init_module(), which is clearly marked private by the double-underscore prefix.
Another option would be to import needed modules at the top of each function, rather than the top of your module. After the first time a module is imported, subsequent imports are just a lookup in the sys.modules dictionary.
But I agree with the other commenters here -- the Python convention is not to worry about module namespace pollution, and just make it obvious to your module's users which parts of the namespace are your public API and which are internals.
From http://docs.python.org/tutorial/modules.html:
The import statement uses the
following convention: if a package’s
__init__.py code defines a list named __all__, it is taken to be the list of module names that should be imported
when from package import * is
encountered.
In your mypackage/__init__.py, try adding this:
# add this line, replace "..." with the rest of the definitions
# you want made public
__all__ = ['func_a1', 'func_a2', 'ClassA1', 'ClassA2', ...]
from a import func_a1, func_a2, ClassA1, ClassA2
from b import func_b1, func_b2, ClassB1, ClassB2
from c import func_c1, func_c2, ClassC1, ClassC2
from d import func_d1, func_d2, ClassD1, ClassD2

Categories