So I have a class in a module that has some static methods. A couple of these static methods just do crc checks and stuff, and they're not really useful outside of the class (I would just make them private static methods in java or C++). I'm wondering if I should instead make them global class functions (outside of the class).
Is there any benefit for doing it either way? The class is being imported by from module import class so I'm not worried about having those modules pulled in as well. But should I just make them class methods so that from module import * is safer or something?
Prefixing the function names with a single underscore is a convention to say that they are private, and it will also prevent them from being imported with a from module import *.
Another technique is to specify an __all__ list in the module - this can just be done in the module itself (you don't need an __init__.py file)
__all__ = ['my_class_name']
This is more of a whitelist approach, so you can have full control over what gets imported without using leading underscores.
So unless your methods logically belong in the class, and from your description they don't, I would leave them as module level functions and use one of these two approaches to make them private.
Make them module-level functions, and prefix them with a single underscore so that consumers understand that they are for private use.
If they are not useful outside of the class, what is the motivation to make them module methods? Keeping them as static method makes the name space cleaner.
The only advantage to move it outside maybe so that people can reference them without using qualified them the class name. Say you have a log method that got reference in a ton of places, this may make sense as a stylistic choice.
Related
This is probably a vestige from my days as a c# programmer, and I probably just need to learn the "pythonic" way of doing things. I'm relatively new to python, but have been coding in c# for years.
I have a python module within my project named 'applib' where I store a number of classes. Each class is contained within its own file of the same name (e.g. Foo.py has a class named Foo, Bar.py has a class named Bar, etc).
When I want to use a class, I have to say
import applib.Foo
followed by something like
my_foo=applib.Foo.Foo()
This results in having to specify "Foo" twice in my declaration (once for the module name, and then again for the class). In c# I could control this with a namespace directive at the top of the class file, but in python I don't know of a way to control this except for putting all my classes in a single file. I know I can also import the class by using
from applib.Foo import Foo
but that's still a duplication of namespace elements, which I want to avoid completely.
Can anyone provide me with some recommendations on a more pythonic methodology for namespace structure within an app?
I am developing a rather complex application for my company following the object-oriented model in python3. The application contains several packages and sub-packages, each - of course - containing an __init__.py module.
I mostly used those __init__.py modules to declare generic classes for the package inside them, which are intended to be used as abstract templates for their respective package only.
My question is now: Is this a "nice" / "correct" / "pythonic" way to use the __init__.py module(s)? Or shall I rather declare my generic classes somewhere else?
To give an example, let's assume a package mypkg:
mypkg.__init__.py:
class Foo(object):
__some_attr = None
def __init__(self, some_attr):
self.__some_attr = some_attr
#property
def some_attr(self):
return self.__some_attr
#some_attr.setter
def some_attr(self, val):
self.__some_attr = val
mypkg.myfoo.py:
from . import Foo
class MyFoo(Foo):
def __init__(self):
super().__init__("this is my implemented value")
def printme(self):
print(self.some_attr)
It depends by what is the API you want to provide. For example the collections module in the standard library defines all the classes in the __init__.py1.
And the standard library should be pretty "pythonic", whatever that means.
However it provides mostly a "module-like" interface. It's quite rare to see:
import collections.abc
If you already have subpackages you are probably better introducing a new subpackage.
If, currently, the use of the package doesn't actually depend on subpackages you might consider putting the code in the __init__.py. Or put the code in some private module and simply import the names inside __init__.py (this is what I'd prefer)
If you are only concerned with where it's better to put the Abstract Base Classes, as shown above (collections.abc contains the abstract base classes of the collections package), and as you can see from the standard library's abc module, it's common to define an abc.py submodule that contains them.
You may consider exposing them directly from the __init__.py doing:
from .abc import *
from . import abc
# ...
__all__ = list_of_names_to_export + abc.__all__
inside your __init__.py.
1 The actual implementation used is in C however: _collectionsmodule.c
You can always put everything into one source file. The reason for splitting the more complex code into separate modules or packages is to separate the things that are mutually related from things that are unrelated. The separated things should be as independent as possible on the rest. And it applies to any level: data structures, functions, classes, modules, packages, applications.
Inside the module or inside the package should apply the same rules. I agree with Bakuriu that __init__.py should be closely related to the package infrastructure, but not neccessarily to the functionality of the module. My personal opinion is that the __init__.py should be as simple as possible. The reason firstly is that everything should be as simple as possible but not simpler. Secondly, people reading the code of your package tend to think that way. They may not expect the unexpected functionality in __init__.py. It would probably be better to create generic.py inside the package for the purpose. It is easier to document the purpose of the module (say via its top docstring).
The better the separation is from the beginning, the better can the independent features be combined in future. You get better flexibility -- both for the usage of module inside the package and also for future modifications.
It is indeed possible to use __init__.py for a specific module initialization, but I have never seen someone using it to define new functions. The only "correct" use I know is what is discussed in this subject ....
Anyway, as you might have a complex application, you can use a temporary file where you define all your needed functions instead of defining them directly in __init__.py module. This will allow you more readability it is easier to change afterwards.
Don't ever use __init__.py for anything except to define __all__. You will save so many lives if you will avoid it.
Reason: It is common for developers to look at packages and modules. But there is a problem you can stuck with sometimes. If you have package, you assume that there is a modules and code inside of it. But you will rarely count __init__.py as one, because, let's face it, most times it is just a requirement to make modules from directory importable.
package
__init__.py
mod1.py
humans.py
cats.py
dogs.py
cars.py
commons.py
Where should be located class Family? It is common class and it depends on others, because we can create family of humans, dogs and cats, even cars!
By my logic, and logic of my friends, it should be places in separated file, but I will try to find it in commons, next in humans and then... I will be embarrassed, because I don't really know where it is!
Stupid example, huh? But it gives a point.
I recently refactored my code to put input validation methods that are shared among several classes in their own module, validate.py. Some of these validation methods check if their input is an instance of a class, e.g. MyClass. Therefore validate.py must import MyClass so it's method is_MyClass can check if isinstance(input, MyClass). But, I want to use some validation methods from validate.py in MyClass to sanitize input to MyClass.my_method, so MyClass must import validate.py.
Something tells me I just casually refactored my way into an anti-pattern. If what I'm trying to do implies circular dependencies, then I must be Doing It Wrong™.
But, code reuse is a good idea. So what's the best practice for sharing validation methods in this way?
I think the parts of the validation code that are specific to one of the classes should probably be put into the class itself - maybe as a classmethod? That way the 'generic' validation code can just call obj.validate() at the appropriate time. You then don't need to import the classes from the generic validation code.
While Tom Dalton's answer is probably correct as far as the best design goes, it may be worth noting that import cycles often work just fine in Python.
The limitation though is that you need to use import my_module syntax and avoid top-level (global) code that uses the imported modules. Declaring functions (or classes with methods) that use the imported module is fine.
You usually run into trouble if you're using from my_module import obj or something similar, since this will only work if obj has already been defined in the other module. If that other module is in the process of importing your module, the class definition or global variable assignment may not have have happened yet.
So for your specific case, an alternative solution may be to have your validate module use import my_class, then is_MyClass can do isinstance(input, my_class.MyClass).
I have a library that interfaces with an external tool and exposes some basic keywords to use from robotframework; This library is implemented as a python package, and I would like to implement extended functionality that implements complex logic, and exposes more keywords, within modules of this package. The package is given test case scope, but I'm not entirely sure how this works. If I suggest a few ways I have thought of, could someone with a bit more knowledge let me know where I'm on the right track, and where I'm barking up the wrong tree...
Use an instance variable - if the scope is such that the python interpreter will see the package as imported by the current test case (i.e this is treated as a separate package in different test cases rather than a separate instance of the same package), then on initialisation I could set a global variable INSTANCE to self and then from another module within the package, import INSTANCE and use it.
Use an instance dictionary - if the scope is such that all imports see the package as the same, I could use robot.running.context to set a dictionary key such that there is an item in the instance dictionary for each context where the package has been imported - this would then mean that I could use the same context variable as a lookup key in the modules that are based on this. (The disadvantage of this one is that it will prevent garbage collection until the package itself is out of scope, and relies on it being in scope persistently.)
A context variable that I am as of yet unaware of that will give me the instance that is in scope. The docs are fairly difficult to search, so it's fully possible that there is something that I'm missing that will make this trivial. Also just as good would be something that allowed me to call the keywords that are in scope.
Some excellent possibility I haven't considered....
So can anyone help?
Credit for this goes to Kevin O. from the robotframework user group, but essentially the magic lives in robot.libraries.BuiltIn.BuiltIn().get_library_instance(library_name) which can be used like this:
from robot.libraries.BuiltIn import BuiltIn
class SeleniumTestLibrary(object):
def element_should_be_really_visible(self):
s2l = BuiltIn().get_library_instance('Selenium2Library')
element = s2l._element_find(locator, True, False)
It sounds like you are talking about monkeypatching the imported code, so that other modules which import that package will also see your runtime modifications. (Correct me if I'm wrong; there are a couple of bits in your question that I'm not quite following)
For simple package imports, this should work:
import my_package
def method_override():
return "Foo"
my_package.some_method = method_override
my_package, in this case, refers to the imported module, and is not just a local name, so other modules will see the overridden method.
This won't work in cases where other code has already done
from my_package import some_method
Since in that case, some_method is a local name in the place it is imported. If you replace the method elsewhere, that change won't be seen.
If this is happening, then you either need to change the source to import the entire module, or patch a little bit deeper, by replacing method internals:
import my_package
def method_override():
return "Foo"
my_package.some_method.func_code = method_override.func_code
At that point, it doesn't matter how the method was imported in any other module; the code object associated with the method has been replaced, and your new code will run rather than the original.
The only thing to worry about in that case is that the module is imported from the same path in every case. The Python interpreter will try to reuse existing modules, rather than re-import and re-initialize them, whenever they are imported from the same path.
However, if your python path is set up to contain two directories, say: '/foo' and '/foo/bar', then these two imports
from foo.bar import baz
and
from bar import baz
would end up loading the module twice, and defining two versions of any objects (methods, classes, etc) in the module. If that happens, then patching one will not affect the other.
If you need to guard against that case, then you may have to traverse sys.modules, looking for the imported package, and patching each version that you find. This, of course, will only work if all of the other imports have already happened, you can't do that pre-emptively (without writing an import hook, but that's another level deeper again :) )
Are you sure you can't just fork the original package and extend it directly? That would be much easier :)
I'm very new to Python (I'm coming from a JAVA background) and I'm wondering if anyone could help me with some of the Python standards. Is it a normal or "proper" practice to put multiple class in a module? I have been working with Django and started with the tutorials and they place their database model classes in the same module. Is this something that is normally done or should I stick with 1 class per module? Is there a reason I would do one over the other?
Hope I'm being clear and not to generic. Thanks to everyone in advance!
Here is a useful rule of thumb from what I have seen of typical Java projects:
The bottom-most package in Java should be a file in Python
What does that mean?
If your Java project was organized:
toplevel/
subproject/
Foo.java
Bar.java
subproject2/
Baz.java
Qux.java
Then your Python project should look like:
toplevel/
subproject.py <-- put class Foo, Bar here
subproject2.py <-- put class Baz, Qux here
Things to notice re: organization:
Do not use inner classes. Just put
classes in the same module
By convention, things that start with _ are "private"
It's OK to have "public variables"
Think it this way.
In java what you write is a Class where in the case of Python, you write a module instead of a class. So a module can contain several classes.
Whenever you want to use a particular class, import the respective module first and then call the class to make objects.
Here's an example.
Classes.py (This is a module named 'Classes')
class MyClass(object):
def greet(self):
print("Hello World")
class MyNextClass(object):
def greetAgain(self):
print("Hello again")
Now I can import this module from anywhere I wish
import Classes
if __name__ == '__main__':
a=Classes.MyClass()
a.greet()
b=Classes.MyNextClass()
b.greetAgain()
When in doubt, just look at Python's standard libraries :)
For example, the standard calendar module contains 31 classes. So yes, it is ok.
It is absolutely proper to do so. A module groups related functionality. If that functionality is implemented in several classes (e.g., Tree, Node, Leaf) then it is appropriate to place them together.
A module is more closely associated with a Java package than a Java class. You can also implement a module as a folder, named for the module, with an __init__.py file inside (so Python can identify the module as such; the __init__.py may also optionally include initialization code and lists of classes, functions, and sub-packages to export.)
It is certainly a normal thing to do in Python. When and why you choose one over the other is partly a matter of taste, and partly convention.
If you're still getting to know Python, and therefore its conventions, reading the style guide is well worth your time.