I'm very new to Python (I'm coming from a JAVA background) and I'm wondering if anyone could help me with some of the Python standards. Is it a normal or "proper" practice to put multiple class in a module? I have been working with Django and started with the tutorials and they place their database model classes in the same module. Is this something that is normally done or should I stick with 1 class per module? Is there a reason I would do one over the other?
Hope I'm being clear and not to generic. Thanks to everyone in advance!
Here is a useful rule of thumb from what I have seen of typical Java projects:
The bottom-most package in Java should be a file in Python
What does that mean?
If your Java project was organized:
toplevel/
subproject/
Foo.java
Bar.java
subproject2/
Baz.java
Qux.java
Then your Python project should look like:
toplevel/
subproject.py <-- put class Foo, Bar here
subproject2.py <-- put class Baz, Qux here
Things to notice re: organization:
Do not use inner classes. Just put
classes in the same module
By convention, things that start with _ are "private"
It's OK to have "public variables"
Think it this way.
In java what you write is a Class where in the case of Python, you write a module instead of a class. So a module can contain several classes.
Whenever you want to use a particular class, import the respective module first and then call the class to make objects.
Here's an example.
Classes.py (This is a module named 'Classes')
class MyClass(object):
def greet(self):
print("Hello World")
class MyNextClass(object):
def greetAgain(self):
print("Hello again")
Now I can import this module from anywhere I wish
import Classes
if __name__ == '__main__':
a=Classes.MyClass()
a.greet()
b=Classes.MyNextClass()
b.greetAgain()
When in doubt, just look at Python's standard libraries :)
For example, the standard calendar module contains 31 classes. So yes, it is ok.
It is absolutely proper to do so. A module groups related functionality. If that functionality is implemented in several classes (e.g., Tree, Node, Leaf) then it is appropriate to place them together.
A module is more closely associated with a Java package than a Java class. You can also implement a module as a folder, named for the module, with an __init__.py file inside (so Python can identify the module as such; the __init__.py may also optionally include initialization code and lists of classes, functions, and sub-packages to export.)
It is certainly a normal thing to do in Python. When and why you choose one over the other is partly a matter of taste, and partly convention.
If you're still getting to know Python, and therefore its conventions, reading the style guide is well worth your time.
Related
I want to know why we use from module import module?
For example from BeautifulSoup import BeautifulSoup instead of only import BeautifulSoup.
Are both of those different?
And if yes, then how?
Yes, they are different.
There are several ways to import a module
import module
The module is imported, but all its functions, classes and variables remain within the name space of 'module . To reference them, they must be prepended with the module name:
module.somefunction() module.someclass() module.somevariable = 1
import module as md
The module is imported, but given a new name space 'md'. To reference the functions, classes and variables, they must be prepended with the new namespace:
md.somefunction() module.someclass() module.somevariable = 1
This keeps the namespaces separated, but provides a shorthand notation which makes the code more readable.
from module import *
All functions, classes and variabes from the module are imported into the current namespace as if they were defined in the current module. They can be called with their own name:
somefunction() someclass() somevariable = 1
The disadvantage is that there might be overlapping names!
from module import something
from module import something, somethingelse
This imports only 'something' (and 'somethingelse') a function, a class, a variable into the current namespace.
This is not only more efficient, but it also reduces the risk of overlapping names.
A module name and a class inside the module may have the same name. Don't let that confuse you:
import BeautifulSoup reference: BeautifulSoup.BeautifulSoup()
from BeautifulSoup import BeautifulSoup reference: BeautifulSoup()
import BeautifulSoup as bs reference: bs.BeautifulSoup()
I think it comes down to what the individual library (Python package) owner wants to do and however they think it will be easiest for people to use their product. As part of the Python community, they may take into account the current conventions. Those may be done formally as a PEP like PEP 8, informally through common practice, or as third-party formalized linters.
It can be confusing that there are some distinctions, like package vs module, that have the potential to help with understanding, but do not really mean anything to a Python application itself. For example, with your example, it is not from module import module; it is more like from package import class for BeautifulSoup. But it could be argued that a package is an external module. I have heard some call any file a module and any directory a package.
More background, if you are new to python:
In python it is all about namespacing. Namespacing has the potential to allow for very clean and flexible code organization compared to earlier languages. However it also allows that any package, module, class, function, method, variable, or whatever name can be hard for someone writing a client application to know what is what. There is even a convenience class in the standard library for faking it: types.Namespace. Then there are metaclasses and dynamic programming as well. Anyway, yes there are common conventions followed to reduce confusion, but they have to be learned separately from the language itself (like learning customs of a country, not only its language, to be able to understand common phrases).
What PEP 20 has to say about namespaces:
import this
Back to PEP 8, generally, it is accepted that classes are word-uppercase (class SomeClass:) without underscores, and functions, variables and methods are all lowercase with words separated by underscores (some_variable). Not always the case, but those are probably the most widely accepted styles. Somewhere I see things going against what is "pythonic" and/or commonplace is when a bindings library is a thin wrapper around a library from another language (e.g. C++) or when the Python code is tightly coupled with code in another language, so the styles smear together for easier transitions. Nothing says anyone has to follow specific style.
Some people prefer terseness, so they may shorten and combine words for variable names (e.g. foo bar as fb). Others may like to be explicit (e.g. foo bar as foo_bar). And still others may prefer to be generic and use comments to describe their variables (why? it may be convenient for large complex equations):
# x is for foo bar.
# y is for spam and z is for ham.
assert x + y != z
Some people really like formatters like Black that follow very strict rules, but others like flexibility or have reason to go against convention. Even in the standard libraries that come with Python, there are inconsistency from legacy code that the maintainers have left alone or allowed before the community settled on common practices that the code goes against.
This is probably a vestige from my days as a c# programmer, and I probably just need to learn the "pythonic" way of doing things. I'm relatively new to python, but have been coding in c# for years.
I have a python module within my project named 'applib' where I store a number of classes. Each class is contained within its own file of the same name (e.g. Foo.py has a class named Foo, Bar.py has a class named Bar, etc).
When I want to use a class, I have to say
import applib.Foo
followed by something like
my_foo=applib.Foo.Foo()
This results in having to specify "Foo" twice in my declaration (once for the module name, and then again for the class). In c# I could control this with a namespace directive at the top of the class file, but in python I don't know of a way to control this except for putting all my classes in a single file. I know I can also import the class by using
from applib.Foo import Foo
but that's still a duplication of namespace elements, which I want to avoid completely.
Can anyone provide me with some recommendations on a more pythonic methodology for namespace structure within an app?
I'm trying to import a function from a python module. That function is declared on the module I'm calling import from, but nevertheless I'm using that function on the other file.
Like so:
context.py
from elements import *
class Context:
def __init__(self):
pass
#staticmethod
def load():
print "load code here"
elements.py
from context import *
class Item:
def __init__(self):
Context.load() # NameError: global name 'load' is not defined
As someone who comes from Java, seems like applying the same nested class accessing logic doesn't work in Python. I'm wondering what could be the best practice here, since it doesn't seem right to put the import statement below the Context class. I searched about this but the material wasn't clear about this practice.
Also, at context.py I'm using instances of classes defined at elements, and vice versa. So my question is really what would be the best importing practice here.
Another related question: is it good practice to encapsulate functions and variables inside Classes in Python or should I use global functions/variables instead?
Ah, in Python this is considered a circular import error -- and can be incredibly frustrating. elements is importing from context and vice-versa. This may be possible in Java with magic compiler tricks but since Python is (mostly) interpreted, this isn't possible*.
Another unstated difference between Java and Python is that a Python class is closer to a hashmap with a special API than a proper Java class. As such, it is perfectly acceptable and preferable to put classes that have a tight interdependence (such as the ones you wrote) in the same Python module. This will remove the circular import error.
In general, you want to organize library modules by dependency level -- meaning, the leaves of your lib folder do not import from anything else in your project, and as you progress closer to the root, more imports are drawn upon. To the best of your ability you want your import structure to be a tree, not a spiderweb (if that makes any sense). Without a compiler, it's the only way I've found in a large (multi-million line) Python project to maintain sanity.
The above comments are generally considered best practice, this next suggestion is highly opinionated:
I would recommend structuring executable modules around I/O boundaries. It becomes very tempting to build tightly interconnected fabrics of Python objects with complicated inheritance structures passed by reference. While on a small and medium scale this offers development advantages, on a larger scale you lose the ability to easily integrate concurrency since you've taken away the ability for the code to be transfer-layer agnostic.
Edit: Okay, it IS possible by playing around with import statement ordering, using the __import__ method, etc., to hack the import framework and accomplish this. However, you should NOT do this if you intend to have a large project -- it is very brittle and difficult to explain to a team. It seems like you're more interested in best practices, which is how I directed my answer. Sorry if that was unclear.
In context.py file you should add def before __init__, also class methods do not take self:
class Context:
def __init__(self):
pass
#staticmethod
def load():
print "load code here"
then in another file:
from context import Context
class Item:
def __init__(self):
Context.load()
I am developing a rather complex application for my company following the object-oriented model in python3. The application contains several packages and sub-packages, each - of course - containing an __init__.py module.
I mostly used those __init__.py modules to declare generic classes for the package inside them, which are intended to be used as abstract templates for their respective package only.
My question is now: Is this a "nice" / "correct" / "pythonic" way to use the __init__.py module(s)? Or shall I rather declare my generic classes somewhere else?
To give an example, let's assume a package mypkg:
mypkg.__init__.py:
class Foo(object):
__some_attr = None
def __init__(self, some_attr):
self.__some_attr = some_attr
#property
def some_attr(self):
return self.__some_attr
#some_attr.setter
def some_attr(self, val):
self.__some_attr = val
mypkg.myfoo.py:
from . import Foo
class MyFoo(Foo):
def __init__(self):
super().__init__("this is my implemented value")
def printme(self):
print(self.some_attr)
It depends by what is the API you want to provide. For example the collections module in the standard library defines all the classes in the __init__.py1.
And the standard library should be pretty "pythonic", whatever that means.
However it provides mostly a "module-like" interface. It's quite rare to see:
import collections.abc
If you already have subpackages you are probably better introducing a new subpackage.
If, currently, the use of the package doesn't actually depend on subpackages you might consider putting the code in the __init__.py. Or put the code in some private module and simply import the names inside __init__.py (this is what I'd prefer)
If you are only concerned with where it's better to put the Abstract Base Classes, as shown above (collections.abc contains the abstract base classes of the collections package), and as you can see from the standard library's abc module, it's common to define an abc.py submodule that contains them.
You may consider exposing them directly from the __init__.py doing:
from .abc import *
from . import abc
# ...
__all__ = list_of_names_to_export + abc.__all__
inside your __init__.py.
1 The actual implementation used is in C however: _collectionsmodule.c
You can always put everything into one source file. The reason for splitting the more complex code into separate modules or packages is to separate the things that are mutually related from things that are unrelated. The separated things should be as independent as possible on the rest. And it applies to any level: data structures, functions, classes, modules, packages, applications.
Inside the module or inside the package should apply the same rules. I agree with Bakuriu that __init__.py should be closely related to the package infrastructure, but not neccessarily to the functionality of the module. My personal opinion is that the __init__.py should be as simple as possible. The reason firstly is that everything should be as simple as possible but not simpler. Secondly, people reading the code of your package tend to think that way. They may not expect the unexpected functionality in __init__.py. It would probably be better to create generic.py inside the package for the purpose. It is easier to document the purpose of the module (say via its top docstring).
The better the separation is from the beginning, the better can the independent features be combined in future. You get better flexibility -- both for the usage of module inside the package and also for future modifications.
It is indeed possible to use __init__.py for a specific module initialization, but I have never seen someone using it to define new functions. The only "correct" use I know is what is discussed in this subject ....
Anyway, as you might have a complex application, you can use a temporary file where you define all your needed functions instead of defining them directly in __init__.py module. This will allow you more readability it is easier to change afterwards.
Don't ever use __init__.py for anything except to define __all__. You will save so many lives if you will avoid it.
Reason: It is common for developers to look at packages and modules. But there is a problem you can stuck with sometimes. If you have package, you assume that there is a modules and code inside of it. But you will rarely count __init__.py as one, because, let's face it, most times it is just a requirement to make modules from directory importable.
package
__init__.py
mod1.py
humans.py
cats.py
dogs.py
cars.py
commons.py
Where should be located class Family? It is common class and it depends on others, because we can create family of humans, dogs and cats, even cars!
By my logic, and logic of my friends, it should be places in separated file, but I will try to find it in commons, next in humans and then... I will be embarrassed, because I don't really know where it is!
Stupid example, huh? But it gives a point.
I'm learning Python and I have been playing around with packages. I wanted to know the best way to define classes in packages. It seems that the only way to define classes in a package is to define them in the __init__.py of that package. Coming from Java, I'd kind of like to define individual files for my classes. Is this a recommended practice?
I'd like to have my directory look somewhat like this:
recursor/
__init__.py
RecursionException.py
RecursionResult.py
Recursor.py
So I could refer to my classes as recursor.Recursor, recursor.RecursionException, and recursor.RecursionResult. Is this doable or recommended in Python?
Go ahead and define your classes in separate modules. Then make __init__.py do something like this:
from RecursionException import RecursionException
from RecursionResult import RecursionResult
from Recursor import Recursor
That will import each class into the package's root namespace, so calling code can refer to recursor.Recursor instead of recursor.Recursor.Recursor.
I feel the need to echo some of the other comments here, though: Python is not Java. Rather than creating a new module for every class under the sun, I suggest grouping closely related classes into a single module. It's easier to understand your code that way, and calling code won't need a bazillion imports.
This is perfectly doable. Just create a new class module for each of those classes, and create exactly the structure you posted.
You can also make a Recursion.py module or something similar, and include all 3 classes in that file.
(I'm also new to Python from Java, and I haven't yet put anything in my __init__.py files...)
In Python you're not restricted to defining 1 class per file and few do that. You can if you want to though - it's totally up to you. A Package in Python is just a directory with an
__init__.py
file. You don't have to put anything in that file you can to control what gets imported etc.