In Python, how do I indicate I'm overriding a method? - python

In Java, for example, the #Override annotation not only provides compile-time checking of an override but makes for excellent self-documenting code.
I'm just looking for documentation (although if it's an indicator to some checker like pylint, that's a bonus). I can add a comment or docstring somewhere, but what is the idiomatic way to indicate an override in Python?

Based on this and fwc:s answer I created a pip installable package https://github.com/mkorpela/overrides
From time to time I end up here looking at this question.
Mainly this happens after (again) seeing the same bug in our code base: Someone has forgotten some "interface" implementing class while renaming a method in the "interface"..
Well Python ain't Java but Python has power -- and explicit is better than implicit -- and there are real concrete cases in the real world where this thing would have helped me.
So here is a sketch of overrides decorator. This will check that the class given as a parameter has the same method (or something) name as the method being decorated.
If you can think of a better solution please post it here!
def overrides(interface_class):
def overrider(method):
assert(method.__name__ in dir(interface_class))
return method
return overrider
It works as follows:
class MySuperInterface(object):
def my_method(self):
print 'hello world!'
class ConcreteImplementer(MySuperInterface):
#overrides(MySuperInterface)
def my_method(self):
print 'hello kitty!'
and if you do a faulty version it will raise an assertion error during class loading:
class ConcreteFaultyImplementer(MySuperInterface):
#overrides(MySuperInterface)
def your_method(self):
print 'bye bye!'
>> AssertionError!!!!!!!

Here's an implementation that doesn't require specification of the interface_class name.
import inspect
import re
def overrides(method):
# actually can't do this because a method is really just a function while inside a class def'n
#assert(inspect.ismethod(method))
stack = inspect.stack()
base_classes = re.search(r'class.+\((.+)\)\s*\:', stack[2][4][0]).group(1)
# handle multiple inheritance
base_classes = [s.strip() for s in base_classes.split(',')]
if not base_classes:
raise ValueError('overrides decorator: unable to determine base class')
# stack[0]=overrides, stack[1]=inside class def'n, stack[2]=outside class def'n
derived_class_locals = stack[2][0].f_locals
# replace each class name in base_classes with the actual class type
for i, base_class in enumerate(base_classes):
if '.' not in base_class:
base_classes[i] = derived_class_locals[base_class]
else:
components = base_class.split('.')
# obj is either a module or a class
obj = derived_class_locals[components[0]]
for c in components[1:]:
assert(inspect.ismodule(obj) or inspect.isclass(obj))
obj = getattr(obj, c)
base_classes[i] = obj
assert( any( hasattr(cls, method.__name__) for cls in base_classes ) )
return method

If you want this for documentation purposes only, you can define your own override decorator:
def override(f):
return f
class MyClass (BaseClass):
#override
def method(self):
pass
This is really nothing but eye-candy, unless you create override(f) in such a way that is actually checks for an override.
But then, this is Python, why write it like it was Java?

Improvising on #mkorpela great answer, here is a version with
more precise checks, naming, and raised Error objects
def overrides(interface_class):
"""
Function override annotation.
Corollary to #abc.abstractmethod where the override is not of an
abstractmethod.
Modified from answer https://stackoverflow.com/a/8313042/471376
"""
def confirm_override(method):
if method.__name__ not in dir(interface_class):
raise NotImplementedError('function "%s" is an #override but that'
' function is not implemented in base'
' class %s'
% (method.__name__,
interface_class)
)
def func():
pass
attr = getattr(interface_class, method.__name__)
if type(attr) is not type(func):
raise NotImplementedError('function "%s" is an #override'
' but that is implemented as type %s'
' in base class %s, expected implemented'
' type %s'
% (method.__name__,
type(attr),
interface_class,
type(func))
)
return method
return confirm_override
Here is what it looks like in practice:
NotImplementedError "not implemented in base class"
class A(object):
# ERROR: `a` is not a implemented!
pass
class B(A):
#overrides(A)
def a(self):
pass
results in more descriptive NotImplementedError error
function "a" is an #override but that function is not implemented in base class <class '__main__.A'>
full stack
Traceback (most recent call last):
…
File "C:/Users/user1/project.py", line 135, in <module>
class B(A):
File "C:/Users/user1/project.py", line 136, in B
#overrides(A)
File "C:/Users/user1/project.py", line 110, in confirm_override
interface_class)
NotImplementedError: function "a" is an #override but that function is not implemented in base class <class '__main__.A'>
NotImplementedError "expected implemented type"
class A(object):
# ERROR: `a` is not a function!
a = ''
class B(A):
#overrides(A)
def a(self):
pass
results in more descriptive NotImplementedError error
function "a" is an #override but that is implemented as type <class 'str'> in base class <class '__main__.A'>, expected implemented type <class 'function'>
full stack
Traceback (most recent call last):
…
File "C:/Users/user1/project.py", line 135, in <module>
class B(A):
File "C:/Users/user1/project.py", line 136, in B
#overrides(A)
File "C:/Users/user1/project.py", line 125, in confirm_override
type(func))
NotImplementedError: function "a" is an #override but that is implemented as type <class 'str'> in base class <class '__main__.A'>, expected implemented type <class 'function'>
The great thing about #mkorpela answer is the check happens during some initialization phase. The check does not need to be "run". Referring to the prior examples, class B is never initialized (B()) yet the NotImplementedError will still raise. This means overrides errors are caught sooner.

Python ain't Java. There's of course no such thing really as compile-time checking.
I think a comment in the docstring is plenty. This allows any user of your method to type help(obj.method) and see that the method is an override.
You can also explicitly extend an interface with class Foo(Interface), which will allow users to type help(Interface.method) to get an idea about the functionality your method is intended to provide.

Like others have said unlike Java there is not #Overide tag however above you can create your own using decorators however I would suggest using the getattrib() global method instead of using the internal dict so you get something like the following:
def Override(superClass):
def method(func)
getattr(superClass,method.__name__)
return method
If you wanted to you could catch getattr() in your own try catch raise your own error but I think getattr method is better in this case.
Also this catches all items bound to a class including class methods and vairables

Based on #mkorpela's great answer, I've written a similar package (ipromise pypi github) that does many more checks:
Suppose A inherits from B and C, B inherits from C.
Module ipromise checks that:
If A.f overrides B.f, B.f must exist, and A must inherit from B. (This is the check from the overrides package).
You don't have the pattern A.f declares that it overrides B.f, which then declares that it overrides C.f. A should say that it overrides from C.f since B might decide to stop overriding this method, and that should not result in downstream updates.
You don't have the pattern A.f declares that it overrides C.f, but B.f does not declare its override.
You don't have the pattern A.f declares that it overrides C.f, but B.f declares that it overrides from some D.f.
It also has various features for marking and checking implementing an abstract method.

You can use protocols from PEP 544. With this method, the interface-implementation relation is declared only at the use site.
Assuming you already have an implementation (let's call it MyFoobar), you define an interface (a Protocol), which has the signatures of all the methods and fields of your implementation, let's call that IFoobar.
Then, at the use site, you declare the implementation instance binding to have the interface type e.g. myFoobar: IFoobar = MyFoobar(). Now, if you use a field/method that is missing in the interface, Mypy will complain at the use site (even if it would work at runtime!). If you failed to implement a method from the interface in the implementation, Mypy will also complain. Mypy won't complain if you implement something that doesn't exist in the interface. But that case is rare, since the interface definition is compact and easy to review. You wouldn't be able to actually use that code, since Mypy would complain.
Now, this won't cover cases where you have implementations both in the superclass and the implementing class, like some uses of ABC. But override is used in Java even with no implementation in the interface. This solution covers that case.
from typing import Protocol
class A(Protocol):
def b(self):
...
def d(self): # we forgot to implement this in C
...
class C:
def b(self):
return 0
bob: A = C()
Type checking results in:
test.py:13: error: Incompatible types in assignment (expression has type "C", variable has type "A")
test.py:13: note: 'C' is missing following 'A' protocol member:
test.py:13: note: d
Found 1 error in 1 file (checked 1 source file)

as python 3.6 and above, the functionality provided by #override can be easily implemented using the descriptor protocol of python, namingly the set_name dunder method:
class override:
def __init__(self, func):
self._func = func
update_wrapper(self, func)
def __get__(self, obj, obj_type):
if obj is None:
return self
return self._func
def __set_name__(self, obj_type, name):
self.validate_override(obj_type, name)
def validate_override(self, obj_type, name):
for parent in obj_type.__bases__:
func = parent.__dict__.get(name, None)
if callable(func):
return
else:
raise NotImplementedError(f"{obj_type.__name__} does not override {name}")
Note that here set_name is called once the wrapped class is defined, and we can get the parent class of the wrapped class by calling its dunder method bases.
for each for its parent class, we would like to check if the wrapped function is implemented in the class by
check that the function name is in the class dict
it is a callable
Using i would be as simple as:
class AbstractShoppingCartService:
def add_item(self, request: AddItemRequest) -> Cart:
...
class ShoppingCartService(AbstractShoppingCartService):
#override
def add_item(self, request: AddItemRequest) -> Cart:
...

Not only did the decorator I made check if the name of the overriding attribute in is any superclass of the class the attribute is in without having to specify a superclass, this decorator also check to ensure the overriding attribute must be the same type as the overridden attribute. Class Methods are treated like methods and Static Methods are treated like functions. This decorator works for callables, class methods, static methods, and properties.
For source code see: https://github.com/fireuser909/override
This decorator only works for classes that are instances of override.OverridesMeta but if your class is an instance of a custom metaclass use the create_custom_overrides_meta function to create a metaclass that is compatible with the override decorator. For tests, run the override.__init__ module.

In Python 2.6+ and Python 3.2+ you can do it (Actually simulate it, Python doesn't support function overloading and child class automatically overrides parent's method). We can use Decorators for this. But first, note that Python's #decorators and Java's #Annotations are totally different things. The prior one is a wrapper with concrete code while later one is a flag to compiler.
For this, first do pip install multipledispatch
from multipledispatch import dispatch as Override
# using alias 'Override' just to give you some feel :)
class A:
def foo(self):
print('foo in A')
# More methods here
class B(A):
#Override()
def foo(self):
print('foo in B')
#Override(int)
def foo(self,a):
print('foo in B; arg =',a)
#Override(str,float)
def foo(self,a,b):
print('foo in B; arg =',(a,b))
a=A()
b=B()
a.foo()
b.foo()
b.foo(4)
b.foo('Wheee',3.14)
output:
foo in A
foo in B
foo in B; arg = 4
foo in B; arg = ('Wheee', 3.14)
Note that you must have to use decorator here with parenthesis
One thing to remember is that since Python doesn't have function overloading directly, so even if Class B don't inherit from Class A but needs all those foos than also you need to use #Override (though using alias 'Overload' will look better in that case)

Here is a different solution without annotation.
It has a slightly other goal in mind. While the other proposed solutions check if the given method actually overrides a parent, this one checks, if all parent methods were overridden.
You don't have to raise an AssertionError, but can print a warning or disable it in production by checking for the env in __init__ and return before checking.
class Parent:
def a():
pass
def b():
pass
class Child(Overrides, Parent):
def a()
# raises an error, as b() is not overridden
class Overrides:
def __init__(self):
# collect all defined methods of all base-classes
bases = [b for b in self.__class__.__bases__ if b != Overrides]
required_methods = set()
for base in bases:
required_methods = required_methods.union(set([f for f in dir(base) if not f.startswith('_')]))
# check for each method in each base class (in required_methods)
# if the class, that inherits `Overrides` implements them all
missing = []
# me is the fully qualified name of the CLASS, which inherits
# `Overrides`
me = self.__class__.__qualname__
for required_method in required_methods:
# The method can be either defined in the parent or the child
# class. To check it, we get a reference to the method via
# getattr
try:
found = getattr(self, required_method)
except AttributeError:
# this should not happen, as getattr returns the method in
# the parent class if it is not defined in the cild class.
# It has to be in a parent class, as the required_methods
# is a union of all base-class methods.
missing.append(required_method)
continue
# here is, where the magic happens.
# found is a reference to a method, and found.__qualname__ is
# the full-name of the METHOD. Remember, that me is the full
# name of the class.
# We want to check, where the method is defined. If it is
# defined in an parent class, we did no override it, thus it
# is missing.
# If we did not override, the __qualname__ is Parent.method
# If we did override it, the __qualname__ is Child.method
# With this fact, we can determine if the class, which uses
# `Override` did implement it.
if not found.__qualname__.startswith(me + '.'):
missing.append(required_method)
# Maybe a warning would be enough here
if missing != []:
raise AssertionError(f'{me} did not override these methods: {missing}')

Hear is simplest and working under Jython with Java classes:
class MyClass(SomeJavaClass):
def __init__(self):
setattr(self, "name_of_method_to_override", __method_override__)
def __method_override__(self, some_args):
some_thing_to_do()

Related

When and how to check Python subclasses meet specification

I have a base class that looks something like this:
class myBaseClass:
def __init__(self):
self.name = None # All subclasses must define this
def foo(self): # All subclasses must define this
raise NotImplementedError
def bar(self): # Optional -- not all subclasses will define this
raise NotImplementedError
My API specification stipulates that anyone creating a subclass of myBaseClass must provide a meaningful value for .name, and for the function .foo(). However, .bar() is optional and calling code should be able to handle the case where that results in a NotImplementedError.
When and how should I check that subclasses contributed by third parties meet these requirements?
The options seem to be:
Build subclasses exclusively via metaclasses. However, this approach will be unfamiliar and potentially confusing to most of the contributors to my project, who tend not to be expert developers.
Add an __init_subclass__ method to the base class and use this to infer whether the subclass has overridden everything it is supposed to override. Seems to work, but feels a bit 'kludgy'.
Write build-time tests to instantiate each subclass, call each 'required' method, and verify that they do not raise a NotImplementedError. Seems like an excessive computational effort to answer such a simple question (calling .foo() may be expensive).
Ignore the issue. Deal with it if and when it causes something else to break.
I'm sure I'm not the only person who needs to deal with this issue - is there a 'correct' approach here?
Here's how I would structure it.
First off, what you're looking for here is an abstract base class. Using the built-in modules you can easily define it as such and have methods be forced to have an implementation, otherwise the class will raise an exception when instantiated.
If the name attribute needs to be set always, then you should make it part of the constructor arguments.
Because bar is not always required I wouldn't define it as a method in the base class you have. Instead I would make a child class that is also abstract and define it there as required. When checking to see if the method is available you can use isinstance.
This is what my final code would look like:
from abc import ABC, abstractmethod
class FooBaseClass(ABC):
def __init__(self, name):
self.name = name
#abstractmethod
def foo(self):
"""Some useful docs for foo"""
class FooBarBaseClass(FooBaseClass, ABC):
#abstractmethod
def bar(self):
"""Some useful docs for bar"""
When creating instances you can pick the base class you want and will be forced to define the methods.
class FooClass(FooBaseClass):
def __init__(self):
super().__init__("foo")
def foo(self):
print("Calling foo from FooClass")
class FooBarClass(FooBarBaseClass):
def __init__(self):
super().__init__("foobar")
def foo(self):
print("Calling foo from FooBarClass")
def bar(self):
print("Calling bar from FooBarClass")
Example checking if bar is callable:
def do_operation(obj: FooBaseClass):
obj.foo()
if isinstance(obj, FooBarBaseClass):
obj.bar()
Example:
do_operation(FooClass())
do_operation(FooBarClass())
Calling foo from FooClass
Calling foo from FooBarClass
Calling bar from FooBarClass
An example of invalid code
class InvalidClass(FooBaseClass):
def __init__(self):
super().__init__("foo")
InvalidClass()
Traceback (most recent call last):
File "C:\workspace\so\test.py", line 52, in <module>
InvalidClass()
TypeError: Can't instantiate abstract class InvalidClass with abstract method foo

When should I prefer #classmethod over a #staticmethod and vice versa? [duplicate]

What is the difference between a method decorated with #staticmethod and one decorated with #classmethod?
Maybe a bit of example code will help: Notice the difference in the call signatures of foo, class_foo and static_foo:
class A(object):
def foo(self, x):
print(f"executing foo({self}, {x})")
#classmethod
def class_foo(cls, x):
print(f"executing class_foo({cls}, {x})")
#staticmethod
def static_foo(x):
print(f"executing static_foo({x})")
a = A()
Below is the usual way an object instance calls a method. The object instance, a, is implicitly passed as the first argument.
a.foo(1)
# executing foo(<__main__.A object at 0xb7dbef0c>, 1)
With classmethods, the class of the object instance is implicitly passed as the first argument instead of self.
a.class_foo(1)
# executing class_foo(<class '__main__.A'>, 1)
You can also call class_foo using the class. In fact, if you define something to be
a classmethod, it is probably because you intend to call it from the class rather than from a class instance. A.foo(1) would have raised a TypeError, but A.class_foo(1) works just fine:
A.class_foo(1)
# executing class_foo(<class '__main__.A'>, 1)
One use people have found for class methods is to create inheritable alternative constructors.
With staticmethods, neither self (the object instance) nor cls (the class) is implicitly passed as the first argument. They behave like plain functions except that you can call them from an instance or the class:
a.static_foo(1)
# executing static_foo(1)
A.static_foo('hi')
# executing static_foo(hi)
Staticmethods are used to group functions which have some logical connection with a class to the class.
foo is just a function, but when you call a.foo you don't just get the function,
you get a "partially applied" version of the function with the object instance a bound as the first argument to the function. foo expects 2 arguments, while a.foo only expects 1 argument.
a is bound to foo. That is what is meant by the term "bound" below:
print(a.foo)
# <bound method A.foo of <__main__.A object at 0xb7d52f0c>>
With a.class_foo, a is not bound to class_foo, rather the class A is bound to class_foo.
print(a.class_foo)
# <bound method type.class_foo of <class '__main__.A'>>
Here, with a staticmethod, even though it is a method, a.static_foo just returns
a good 'ole function with no arguments bound. static_foo expects 1 argument, and
a.static_foo expects 1 argument too.
print(a.static_foo)
# <function static_foo at 0xb7d479cc>
And of course the same thing happens when you call static_foo with the class A instead.
print(A.static_foo)
# <function static_foo at 0xb7d479cc>
A staticmethod is a method that knows nothing about the class or instance it was called on. It just gets the arguments that were passed, no implicit first argument. It is basically useless in Python -- you can just use a module function instead of a staticmethod.
A classmethod, on the other hand, is a method that gets passed the class it was called on, or the class of the instance it was called on, as first argument. This is useful when you want the method to be a factory for the class: since it gets the actual class it was called on as first argument, you can always instantiate the right class, even when subclasses are involved. Observe for instance how dict.fromkeys(), a classmethod, returns an instance of the subclass when called on a subclass:
>>> class DictSubclass(dict):
... def __repr__(self):
... return "DictSubclass"
...
>>> dict.fromkeys("abc")
{'a': None, 'c': None, 'b': None}
>>> DictSubclass.fromkeys("abc")
DictSubclass
>>>
Basically #classmethod makes a method whose first argument is the class it's called from (rather than the class instance), #staticmethod does not have any implicit arguments.
To decide whether to use #staticmethod or #classmethod you have to look inside your method. If your method accesses other variables/methods in your class then use #classmethod. On the other hand, if your method does not touches any other parts of the class then use #staticmethod.
class Apple:
_counter = 0
#staticmethod
def about_apple():
print('Apple is good for you.')
# note you can still access other member of the class
# but you have to use the class instance
# which is not very nice, because you have repeat yourself
#
# For example:
# #staticmethod
# print('Number of apples have been juiced: %s' % Apple._counter)
#
# #classmethod
# print('Number of apples have been juiced: %s' % cls._counter)
#
# #classmethod is especially useful when you move your function to another class,
# you don't have to rename the referenced class
#classmethod
def make_apple_juice(cls, number_of_apples):
print('Making juice:')
for i in range(number_of_apples):
cls._juice_this(i)
#classmethod
def _juice_this(cls, apple):
print('Juicing apple %d...' % apple)
cls._counter += 1
Official python docs:
#classmethod
A class method receives the class as
implicit first argument, just like an
instance method receives the instance.
To declare a class method, use this
idiom:
class C:
#classmethod
def f(cls, arg1, arg2, ...): ...
The #classmethod form is a function
decorator – see the description of
function definitions in Function
definitions for details.
It can be called either on the class
(such as C.f()) or on an instance
(such as C().f()). The instance is
ignored except for its class. If a
class method is called for a derived
class, the derived class object is
passed as the implied first argument.
Class methods are different than C++
or Java static methods. If you want
those, see staticmethod() in this
section.
#staticmethod
A static method does not receive an
implicit first argument. To declare a
static method, use this idiom:
class C:
#staticmethod
def f(arg1, arg2, ...): ...
The #staticmethod form is a function
decorator – see the description of
function definitions in Function
definitions for details.
It can be called either on the class
(such as C.f()) or on an instance
(such as C().f()). The instance is
ignored except for its class.
Static methods in Python are similar
to those found in Java or C++. For a
more advanced concept, see
classmethod() in this section.
Here is a short article on this question
#staticmethod function is nothing more than a function defined inside a class. It is callable without instantiating the class first. It’s definition is immutable via inheritance.
#classmethod function also callable without instantiating the class, but its definition follows Sub class, not Parent class, via inheritance. That’s because the first argument for #classmethod function must always be cls (class).
What is the difference between #staticmethod and #classmethod in Python?
You may have seen Python code like this pseudocode, which demonstrates the signatures of the various method types and provides a docstring to explain each:
class Foo(object):
def a_normal_instance_method(self, arg_1, kwarg_2=None):
'''
Return a value that is a function of the instance with its
attributes, and other arguments such as arg_1 and kwarg2
'''
#staticmethod
def a_static_method(arg_0):
'''
Return a value that is a function of arg_0. It does not know the
instance or class it is called from.
'''
#classmethod
def a_class_method(cls, arg1):
'''
Return a value that is a function of the class and other arguments.
respects subclassing, it is called with the class it is called from.
'''
The Normal Instance Method
First I'll explain a_normal_instance_method. This is precisely called an "instance method". When an instance method is used, it is used as a partial function (as opposed to a total function, defined for all values when viewed in source code) that is, when used, the first of the arguments is predefined as the instance of the object, with all of its given attributes. It has the instance of the object bound to it, and it must be called from an instance of the object. Typically, it will access various attributes of the instance.
For example, this is an instance of a string:
', '
if we use the instance method, join on this string, to join another iterable,
it quite obviously is a function of the instance, in addition to being a function of the iterable list, ['a', 'b', 'c']:
>>> ', '.join(['a', 'b', 'c'])
'a, b, c'
Bound methods
Instance methods can be bound via a dotted lookup for use later.
For example, this binds the str.join method to the ':' instance:
>>> join_with_colons = ':'.join
And later we can use this as a function that already has the first argument bound to it. In this way, it works like a partial function on the instance:
>>> join_with_colons('abcde')
'a:b:c:d:e'
>>> join_with_colons(['FF', 'FF', 'FF', 'FF', 'FF', 'FF'])
'FF:FF:FF:FF:FF:FF'
Static Method
The static method does not take the instance as an argument.
It is very similar to a module level function.
However, a module level function must live in the module and be specially imported to other places where it is used.
If it is attached to the object, however, it will follow the object conveniently through importing and inheritance as well.
An example of a static method is str.maketrans, moved from the string module in Python 3. It makes a translation table suitable for consumption by str.translate. It does seem rather silly when used from an instance of a string, as demonstrated below, but importing the function from the string module is rather clumsy, and it's nice to be able to call it from the class, as in str.maketrans
# demonstrate same function whether called from instance or not:
>>> ', '.maketrans('ABC', 'abc')
{65: 97, 66: 98, 67: 99}
>>> str.maketrans('ABC', 'abc')
{65: 97, 66: 98, 67: 99}
In python 2, you have to import this function from the increasingly less useful string module:
>>> import string
>>> 'ABCDEFG'.translate(string.maketrans('ABC', 'abc'))
'abcDEFG'
Class Method
A class method is a similar to an instance method in that it takes an implicit first argument, but instead of taking the instance, it takes the class. Frequently these are used as alternative constructors for better semantic usage and it will support inheritance.
The most canonical example of a builtin classmethod is dict.fromkeys. It is used as an alternative constructor of dict, (well suited for when you know what your keys are and want a default value for them.)
>>> dict.fromkeys(['a', 'b', 'c'])
{'c': None, 'b': None, 'a': None}
When we subclass dict, we can use the same constructor, which creates an instance of the subclass.
>>> class MyDict(dict): 'A dict subclass, use to demo classmethods'
>>> md = MyDict.fromkeys(['a', 'b', 'c'])
>>> md
{'a': None, 'c': None, 'b': None}
>>> type(md)
<class '__main__.MyDict'>
See the pandas source code for other similar examples of alternative constructors, and see also the official Python documentation on classmethod and staticmethod.
I started learning programming language with C++ and then Java and then Python and so this question bothered me a lot as well, until I understood the simple usage of each.
Class Method: Python unlike Java and C++ doesn't have constructor overloading. And so to achieve this you could use classmethod. Following example will explain this
Let's consider we have a Person class which takes two arguments first_name and last_name and creates the instance of Person.
class Person(object):
def __init__(self, first_name, last_name):
self.first_name = first_name
self.last_name = last_name
Now, if the requirement comes where you need to create a class using a single name only, just a first_name, you can't do something like this in Python.
This will give you an error when you will try to create an object (instance).
class Person(object):
def __init__(self, first_name, last_name):
self.first_name = first_name
self.last_name = last_name
def __init__(self, first_name):
self.first_name = first_name
However, you could achieve the same thing using #classmethod as mentioned below
class Person(object):
def __init__(self, first_name, last_name):
self.first_name = first_name
self.last_name = last_name
#classmethod
def get_person(cls, first_name):
return cls(first_name, "")
Static Method: This is rather simple, it's not bound to instance or class and you can simply call that using class name.
So let's say in above example you need a validation that first_name should not exceed 20 characters, you can simply do this.
#staticmethod
def validate_name(name):
return len(name) <= 20
and you could simply call using class name
Person.validate_name("Gaurang Shah")
Only the first argument differs:
normal method: the current object is automatically passed as an (additional) first argument
classmethod: the class of the current object is automatically passed as an (additional) fist argument
staticmethod: no extra arguments are automatically passed. What you passed to the function is what you get.
In more detail...
normal method
The "standard" method, as in every object oriented language. When an object's method is called, it is automatically given an extra argument self as its first argument. That is, method
def f(self, x, y)
must be called with 2 arguments. self is automatically passed, and it is the object itself. Similar to the this that magically appears in eg. java/c++, only in python it is shown explicitly.
actually, the first argument does not have to be called self, but it's the standard convention, so keep it
class method
When the method is decorated
#classmethod
def f(cls, x, y)
the automatically provided argument is not self, but the class of self.
static method
When the method is decorated
#staticmethod
def f(x, y)
the method is not given any automatic argument at all. It is only given the parameters that it is called with.
usages
classmethod is mostly used for alternative constructors.
staticmethod does not use the state of the object, or even the structure of the class itself. It could be a function external to a class. It only put inside the class for grouping functions with similar functionality (for example, like Java's Math class static methods)
class Point
def __init__(self, x, y):
self.x = x
self.y = y
#classmethod
def frompolar(cls, radius, angle):
"""The `cls` argument is the `Point` class itself"""
return cls(radius * cos(angle), radius * sin(angle))
#staticmethod
def angle(x, y):
"""this could be outside the class, but we put it here
just because we think it is logically related to the class."""
return atan(y, x)
p1 = Point(3, 2)
p2 = Point.frompolar(3, pi/4)
angle = Point.angle(3, 2)
I think a better question is "When would you use #classmethod vs #staticmethod?"
#classmethod allows you easy access to private members that are associated to the class definition. this is a great way to do singletons, or factory classes that control the number of instances of the created objects exist.
#staticmethod provides marginal performance gains, but I have yet to see a productive use of a static method within a class that couldn't be achieved as a standalone function outside the class.
Static Methods:
Simple functions with no self argument.
Work on class attributes; not on instance attributes.
Can be called through both class and instance.
The built-in function staticmethod()is used to create them.
Benefits of Static Methods:
It localizes the function name in the classscope
It moves the function code closer to where it is used
More convenient to import versus module-level functions since each method does not have to be specially imported
#staticmethod
def some_static_method(*args, **kwds):
pass
Class Methods:
Functions that have first argument as classname.
Can be called through both class and instance.
These are created with classmethod in-built function.
#classmethod
def some_class_method(cls, *args, **kwds):
pass
#decorators were added in python 2.4 If you're using python < 2.4 you can use the classmethod() and staticmethod() function.
For example, if you want to create a factory method (A function returning an instance of a different implementation of a class depending on what argument it gets) you can do something like:
class Cluster(object):
def _is_cluster_for(cls, name):
"""
see if this class is the cluster with this name
this is a classmethod
"""
return cls.__name__ == name
_is_cluster_for = classmethod(_is_cluster_for)
#static method
def getCluster(name):
"""
static factory method, should be in Cluster class
returns a cluster object for the given name
"""
for cls in Cluster.__subclasses__():
if cls._is_cluster_for(name):
return cls()
getCluster = staticmethod(getCluster)
Also observe that this is a good example for using a classmethod and a static method,
The static method clearly belongs to the class, since it uses the class Cluster internally.
The classmethod only needs information about the class, and no instance of the object.
Another benefit of making the _is_cluster_for method a classmethod is so a subclass can decide to change it's implementation, maybe because it is pretty generic and can handle more than one type of cluster, so just checking the name of the class would not be enough.
Let me tell the similarity between a method decorated with #classmethod vs #staticmethod first.
Similarity: Both of them can be called on the Class itself, rather than just the instance of the class. So, both of them in a sense are Class's methods.
Difference: A classmethod will receive the class itself as the first argument, while a staticmethod does not.
So a static method is, in a sense, not bound to the Class itself and is just hanging in there just because it may have a related functionality.
>>> class Klaus:
#classmethod
def classmthd(*args):
return args
#staticmethod
def staticmthd(*args):
return args
# 1. Call classmethod without any arg
>>> Klaus.classmthd()
(__main__.Klaus,) # the class gets passed as the first argument
# 2. Call classmethod with 1 arg
>>> Klaus.classmthd('chumma')
(__main__.Klaus, 'chumma')
# 3. Call staticmethod without any arg
>>> Klaus.staticmthd()
()
# 4. Call staticmethod with 1 arg
>>> Klaus.staticmthd('chumma')
('chumma',)
#staticmethod just disables the default function as method descriptor. classmethod wraps your function in a container callable that passes a reference to the owning class as first argument:
>>> class C(object):
... pass
...
>>> def f():
... pass
...
>>> staticmethod(f).__get__(None, C)
<function f at 0x5c1cf0>
>>> classmethod(f).__get__(None, C)
<bound method type.f of <class '__main__.C'>>
As a matter of fact, classmethod has a runtime overhead but makes it possible to access the owning class. Alternatively I recommend using a metaclass and putting the class methods on that metaclass:
>>> class CMeta(type):
... def foo(cls):
... print cls
...
>>> class C(object):
... __metaclass__ = CMeta
...
>>> C.foo()
<class '__main__.C'>
Another consideration with respect to staticmethod vs classmethod comes up with inheritance. Say you have the following class:
class Foo(object):
#staticmethod
def bar():
return "In Foo"
And you then want to override bar() in a child class:
class Foo2(Foo):
#staticmethod
def bar():
return "In Foo2"
This works, but note that now the bar() implementation in the child class (Foo2) can no longer take advantage of anything specific to that class. For example, say Foo2 had a method called magic() that you want to use in the Foo2 implementation of bar():
class Foo2(Foo):
#staticmethod
def bar():
return "In Foo2"
#staticmethod
def magic():
return "Something useful you'd like to use in bar, but now can't"
The workaround here would be to call Foo2.magic() in bar(), but then you're repeating yourself (if the name of Foo2 changes, you'll have to remember to update that bar() method).
To me, this is a slight violation of the open/closed principle, since a decision made in Foo is impacting your ability to refactor common code in a derived class (ie it's less open to extension). If bar() were a classmethod we'd be fine:
class Foo(object):
#classmethod
def bar(cls):
return "In Foo"
class Foo2(Foo):
#classmethod
def bar(cls):
return "In Foo2 " + cls.magic()
#classmethod
def magic(cls):
return "MAGIC"
print Foo2().bar()
Gives: In Foo2 MAGIC
Also: historical note: Guido Van Rossum (Python's creator) once referred to staticmethod's as "an accident": https://mail.python.org/pipermail/python-ideas/2012-May/014969.html
we all know how limited static methods are. (They're basically an accident -- back in the Python 2.2 days when I was inventing new-style classes and descriptors, I meant to implement class methods but at first I didn't understand them and accidentally implemented static methods first. Then it was too late to remove them and only provide class methods.
Also: https://mail.python.org/pipermail/python-ideas/2016-July/041189.html
Honestly, staticmethod was something of a mistake -- I was trying to do something like Java class methods but once it was released I found what was really needed was classmethod. But it was too late to get rid of staticmethod.
The definitive guide on how to use static, class or abstract methods in Python is one good link for this topic, and summary it as following.
#staticmethod function is nothing more than a function defined inside a class. It is callable without instantiating the class first. It’s definition is immutable via inheritance.
Python does not have to instantiate a bound-method for object.
It eases the readability of the code, and it does not depend on the state of object itself;
#classmethod function also callable without instantiating the class, but its definition follows Sub class, not Parent class, via inheritance, can be overridden by subclass. That’s because the first argument for #classmethod function must always be cls (class).
Factory methods, that are used to create an instance for a class using for example some sort of pre-processing.
Static methods calling static methods: if you split a static methods in several static methods, you shouldn't hard-code the class name but use class methods
I will try to explain the basic difference using an example.
class A(object):
x = 0
def say_hi(self):
pass
#staticmethod
def say_hi_static():
pass
#classmethod
def say_hi_class(cls):
pass
def run_self(self):
self.x += 1
print self.x # outputs 1
self.say_hi()
self.say_hi_static()
self.say_hi_class()
#staticmethod
def run_static():
print A.x # outputs 0
# A.say_hi() # wrong
A.say_hi_static()
A.say_hi_class()
#classmethod
def run_class(cls):
print cls.x # outputs 0
# cls.say_hi() # wrong
cls.say_hi_static()
cls.say_hi_class()
1 - we can directly call static and classmethods without initializing
# A.run_self() # wrong
A.run_static()
A.run_class()
2- Static method cannot call self method but can call other static and classmethod
3- Static method belong to class and will not use object at all.
4- Class method are not bound to an object but to a class.
The difference occurs when there is inheritance.
Suppose that there are two classes-- Parent and Child. If one wants to use #staticmethod, print_name method should be written twice because the name of the class should be written in the print line.
class Parent:
_class_name = "Parent"
#staticmethod
def print_name():
print(Parent._class_name)
class Child(Parent):
_class_name = "Child"
#staticmethod
def print_name():
print(Child._class_name)
Parent.print_name()
Child.print_name()
However, for #classmethod, it is not required to write print_name method twice.
class Parent:
_class_name = "Parent"
#classmethod
def print_name(cls):
print(cls._class_name)
class Child(Parent):
_class_name = "Child"
Parent.print_name()
Child.print_name()
Python comes with several built-in decorators. The big three are:
#classmethod
#staticmethod
#property
First let's note that any function of a class can be called with instance of this class (after we initialized this class).
#classmethod is the way to call function not only as an instance of a class but also directly by the class itself as its first argument.
#staticmethod is a way of putting a function into a class (because it logically belongs there), while indicating that it does not require access to the class (so we don't need to use self in function definition).
Let's consider the following class:
class DecoratorTest(object):
def __init__(self):
pass
def doubler(self, x):
return x*2
#classmethod
def class_doubler(cls, x): # we need to use 'cls' instead of 'self'; 'cls' reference to the class instead of an instance of the class
return x*2
#staticmethod
def static_doubler(x): # no need adding 'self' here; static_doubler() could be just a function not inside the class
return x*2
Let's see how it works:
decor = DecoratorTest()
print(decor.doubler(5))
# 10
print(decor.class_doubler(5)) # a call with an instance of a class
# 10
print(DecoratorTest.class_doubler(5)) # a direct call by the class itself
# 10
# staticmethod could be called in the same way as classmethod.
print(decor.static_doubler(5)) # as an instance of the class
# 10
print(DecoratorTest.static_doubler(5)) # or as a direct call
# 10
Here you can see some use cases for those methods.
Bonus: you can read about #property decorator here
Instance Method:
+ Can modify object instance state
+ Can modify class state
Class Method:
- Can't modify object instance state
+ Can modify class state
Static Method:
- Can't modify object instance state
- Can't modify class state
class MyClass:
'''
Instance method has a mandatory first attribute self which represent the instance itself.
Instance method must be called by a instantiated instance.
'''
def method(self):
return 'instance method called', self
'''
Class method has a mandatory first attribute cls which represent the class itself.
Class method can be called by an instance or by the class directly.
Its most common using scenario is to define a factory method.
'''
#classmethod
def class_method(cls):
return 'class method called', cls
'''
Static method doesn’t have any attributes of instances or the class.
It also can be called by an instance or by the class directly.
Its most common using scenario is to define some helper or utility functions which are closely relative to the class.
'''
#staticmethod
def static_method():
return 'static method called'
obj = MyClass()
print(obj.method())
print(obj.class_method()) # MyClass.class_method()
print(obj.static_method()) # MyClass.static_method()
output:
('instance method called', <__main__.MyClass object at 0x100fb3940>)
('class method called', <class '__main__.MyClass'>)
static method called
The instance method we actually had access to the object instance , right so this was an instance off a my class object whereas with the class method we have access to the class itself. But not to any of the objects, because the class method doesn't really care about an object existing. However you can both call a class method and static method on an object instance. This is going to work it doesn't really make a difference, so again when you call static method here it's going to work and it's going to know which method you want to call.
The Static methods are used to do some utility tasks, and class methods are used for factory methods. The factory methods can return class objects for different use cases.
And finally, a short example for better understanding:
class Student:
def __init__(self, first_name, last_name):
self.first_name = first_name
self.last_name = last_name
#classmethod
def get_from_string(cls, name_string: str):
first_name, last_name = name_string.split()
if Student.validate_name(first_name) and Student.validate_name(last_name):
return cls(first_name, last_name)
else:
print('Invalid Names')
#staticmethod
def validate_name(name):
return len(name) <= 10
stackoverflow_student = Student.get_from_string('Name Surname')
print(stackoverflow_student.first_name) # Name
print(stackoverflow_student.last_name) # Surname
#classmethod : can be used to create a shared global access to all the instances created of that class..... like updating a record by multiple users....
I particulary found it use ful when creating singletons as well..:)
#static method: has nothing to do with the class or instance being associated with ...but for readability can use static method
My contribution demonstrates the difference amongst #classmethod, #staticmethod, and instance methods, including how an instance can indirectly call a #staticmethod. But instead of indirectly calling a #staticmethod from an instance, making it private may be more "pythonic." Getting something from a private method isn't demonstrated here but it's basically the same concept.
#!python3
from os import system
system('cls')
# % % % % % % % % % % % % % % % % % % % %
class DemoClass(object):
# instance methods need a class instance and
# can access the instance through 'self'
def instance_method_1(self):
return 'called from inside the instance_method_1()'
def instance_method_2(self):
# an instance outside the class indirectly calls the static_method
return self.static_method() + ' via instance_method_2()'
# class methods don't need a class instance, they can't access the
# instance (self) but they have access to the class itself via 'cls'
#classmethod
def class_method(cls):
return 'called from inside the class_method()'
# static methods don't have access to 'cls' or 'self', they work like
# regular functions but belong to the class' namespace
#staticmethod
def static_method():
return 'called from inside the static_method()'
# % % % % % % % % % % % % % % % % % % % %
# works even if the class hasn't been instantiated
print(DemoClass.class_method() + '\n')
''' called from inside the class_method() '''
# works even if the class hasn't been instantiated
print(DemoClass.static_method() + '\n')
''' called from inside the static_method() '''
# % % % % % % % % % % % % % % % % % % % %
# >>>>> all methods types can be called on a class instance <<<<<
# instantiate the class
democlassObj = DemoClass()
# call instance_method_1()
print(democlassObj.instance_method_1() + '\n')
''' called from inside the instance_method_1() '''
# # indirectly call static_method through instance_method_2(), there's really no use
# for this since a #staticmethod can be called whether the class has been
# instantiated or not
print(democlassObj.instance_method_2() + '\n')
''' called from inside the static_method() via instance_method_2() '''
# call class_method()
print(democlassObj.class_method() + '\n')
''' called from inside the class_method() '''
# call static_method()
print(democlassObj.static_method())
''' called from inside the static_method() '''
"""
# whether the class is instantiated or not, this doesn't work
print(DemoClass.instance_method_1() + '\n')
'''
TypeError: TypeError: unbound method instancemethod() must be called with
DemoClass instance as first argument (got nothing instead)
'''
"""
A class method receives the class as implicit first argument, just like an instance method receives the instance. It is a method which is bound to the class and not the object of the class.It has access to the state of the class as it takes a class parameter that points to the class and not the object instance. It can modify a class state that would apply across all the instances of the class. For example it can modify a class variable that will be applicable to all the instances.
On the other hand, a static method does not receive an implicit first argument, compared to class methods or instance methods. And can’t access or modify class state. It only belongs to the class because from design point of view that is the correct way. But in terms of functionality is not bound, at runtime, to the class.
as a guideline, use static methods as utilities, use class methods for example as factory . Or maybe to define a singleton. And use instance methods to model the state and behavior of instances.
Hope I was clear !
You might want to consider the difference between:
class A:
def foo(): # no self parameter, no decorator
pass
and
class B:
#staticmethod
def foo(): # no self parameter
pass
This has changed between python2 and python3:
python2:
>>> A.foo()
TypeError
>>> A().foo()
TypeError
>>> B.foo()
>>> B().foo()
python3:
>>> A.foo()
>>> A().foo()
TypeError
>>> B.foo()
>>> B().foo()
So using #staticmethod for methods only called directly from the class has become optional in python3. If you want to call them from both class and instance, you still need to use the #staticmethod decorator.
The other cases have been well covered by unutbus answer.
Class methods, as the name suggests, are used to make changes to classes and not the objects. To make changes to classes, they will modify the class attributes(not object attributes), since that is how you update classes.
This is the reason that class methods take the class(conventionally denoted by 'cls') as the first argument.
class A(object):
m=54
#classmethod
def class_method(cls):
print "m is %d" % cls.m
Static methods on the other hand, are used to perform functionalities that are not bound to the class i.e. they will not read or write class variables. Hence, static methods do not take classes as arguments. They are used so that classes can perform functionalities that are not directly related to the purpose of the class.
class X(object):
m=54 #will not be referenced
#staticmethod
def static_method():
print "Referencing/calling a variable or function outside this class. E.g. Some global variable/function."
I think giving a purely Python version of staticmethod and classmethod would help to understand the difference between them at language level (Refers to Descriptor Howto Guide).
Both of them are non-data descriptors (It would be easier to understand them if you are familiar with descriptors first).
class StaticMethod(object):
"Emulate PyStaticMethod_Type() in Objects/funcobject.c"
def __init__(self, f):
self.f = f
def __get__(self, obj, objtype=None):
return self.f
class ClassMethod(object):
"Emulate PyClassMethod_Type() in Objects/funcobject.c"
def __init__(self, f):
self.f = f
def __get__(self, obj, cls=None):
def inner(*args, **kwargs):
if cls is None:
cls = type(obj)
return self.f(cls, *args, **kwargs)
return inner
Analyze #staticmethod literally providing different insights.
A normal method of a class is an implicit dynamic method which takes the instance as first argument.
In contrast, a staticmethod does not take the instance as first argument, so is called 'static'.
A staticmethod is indeed such a normal function the same as those outside a class definition.
It is luckily grouped into the class just in order to stand closer where it is applied, or you might scroll around to find it.
One pretty important practical difference occurs when subclassing. If you don't mind, I'll hijack #unutbu's example:
class A:
def foo(self, x):
print("executing foo(%s, %s)" % (self, x))
#classmethod
def class_foo(cls, x):
print("executing class_foo(%s, %s)" % (cls, x))
#staticmethod
def static_foo(x):
print("executing static_foo(%s)" % x)
class B(A):
pass
In class_foo, the method knows which class it is called on:
A.class_foo(1)
# => executing class_foo(<class '__main__.A'>, 1)
B.class_foo(1)
# => executing class_foo(<class '__main__.B'>, 1)
In static_foo, there is no way to determine whether it is called on A or B:
A.static_foo(1)
# => executing static_foo(1)
B.static_foo(1)
# => executing static_foo(1)
Note that this doesn't mean you can't use other methods in a staticmethod, you just have to reference the class directly, which means subclasses' staticmethods will still reference the parent class:
class A:
#classmethod
def class_qux(cls, x):
print(f"executing class_qux({cls}, {x})")
#classmethod
def class_bar(cls, x):
cls.class_qux(x)
#staticmethod
def static_bar(x):
A.class_qux(x)
class B(A):
pass
A.class_bar(1)
# => executing class_qux(<class '__main__.A'>, 1)
B.class_bar(1)
# => executing class_qux(<class '__main__.B'>, 1)
A.static_bar(1)
# => executing class_qux(<class '__main__.A'>, 1)
B.static_bar(1)
# => executing class_qux(<class '__main__.A'>, 1)
tldr;
A staticmethod is essentially a function bound to a class (and consequently its instances)
A classmethod is essentially an inheritable staticmethod.
For details, see the excellent answers by others.
First let's start with an example code that we'll use to understand both concepts:
class Employee:
NO_OF_EMPLOYEES = 0
def __init__(self, first_name, last_name, salary):
self.first_name = first_name
self.last_name = last_name
self.salary = salary
self.increment_employees()
def give_raise(self, amount):
self.salary += amount
#classmethod
def employee_from_full_name(cls, full_name, salary):
split_name = full_name.split(' ')
first_name = split_name[0]
last_name = split_name[1]
return cls(first_name, last_name, salary)
#classmethod
def increment_employees(cls):
cls.NO_OF_EMPLOYEES += 1
#staticmethod
def get_employee_legal_obligations_txt():
legal_obligations = """
1. An employee must complete 8 hours per working day
2. ...
"""
return legal_obligations
Class method
A class method accepts the class itself as an implicit argument and -optionally- any other arguments specified in the definition. It’s important to understand that a class method, does not have access to object instances (like instance methods do). Therefore, class methods cannot be used to alter the state of an instantiated object but instead, they are capable of changing the class state which is shared amongst all the instances of that class.
Class methods are typically useful when we need to access the class itself — for example, when we want to create a factory method, that is a method that creates instances of the class. In other words, class methods can serve as alternative constructors.
In our example code, an instance of Employee can be constructed by providing three arguments; first_name , last_name and salary.
employee_1 = Employee('Andrew', 'Brown', 85000)
print(employee_1.first_name)
print(employee_1.salary)
'Andrew'
85000
Now let’s assume that there’s a chance that the name of an Employee can be provided in a single field in which the first and last names are separated by a whitespace. In this case, we could possibly use our class method called employee_from_full_name that accepts three arguments in total. The first one, is the class itself, which is an implicit argument which means that it won’t be provided when calling the method — Python will automatically do this for us:
employee_2 = Employee.employee_from_full_name('John Black', 95000)
print(employee_2.first_name)
print(employee_2.salary)
'John'
95000
Note that it is also possible to call employee_from_full_name from object instances although in this context it doesn’t make a lot of sense:
employee_1 = Employee('Andrew', 'Brown', 85000)
employee_2 = employee_1.employee_from_full_name('John Black', 95000)
Another reason why we might want to create a class method, is when we need to change the state of the class. In our example, the class variable NO_OF_EMPLOYEES keeps track of the number of employees currently working for the company. This method is called every time a new instance of Employee is created and it updates the count accordingly:
employee_1 = Employee('Andrew', 'Brown', 85000)
print(f'Number of employees: {Employee.NO_OF_EMPLOYEES}')
employee_2 = Employee.employee_from_full_name('John Black', 95000)
print(f'Number of employees: {Employee.NO_OF_EMPLOYEES}')
Number of employees: 1
Number of employees: 2
Static methods
On the other hand, in static methods neither the instance (i.e. self) nor the class itself (i.e. cls) is passed as an implicit argument. This means that such methods, are not capable of accessing the class itself or its instances.
Now one could argue that static methods are not useful in the context of classes as they can also be placed in helper modules instead of adding them as members of the class. In object oriented programming, it is important to structure your classes into logical chunks and thus, static methods are quite useful when we need to add a method under a class simply because it logically belongs to the class.
In our example, the static method named get_employee_legal_obligations_txt simply returns a string that contains the legal obligations of every single employee of a company. This function, does not interact with the class itself nor with any instance. It could have been placed into a different helper module however, it is only relevant to this class and therefore we have to place it under the Employee class.
A static method can be access directly from the class itself
print(Employee.get_employee_legal_obligations_txt())
1. An employee must complete 8 hours per working day
2. ...
or from an instance of the class:
employee_1 = Employee('Andrew', 'Brown', 85000)
print(employee_1.get_employee_legal_obligations_txt())
1. An employee must complete 8 hours per working day
2. ...
References
What's the difference between static and class methods in Python?

Python meta class - verifying the name of class where instance of other class with meta was initialized

I'm not sure if it is possible. I need to verify (with meta), the name of class which initializes the instance of class enhanced with meta in order to allow (or not) to use its class methods. I have prepared pseudo code to show what I need.
class meta:
__call__(cls, name, base, dict)
if some_class_name != "b":
del dict("some_func")
class a:
def some_func():
code code
class b:
a_instance = a()
a_instance.some_func()
a.some_func() can be only used when the class name's where it was called is b. Otherwise this method should be deleted from object.
This could be made with a metaclass, but it would not be needed, and might be a bit less effective
Normally people worry about metaclasses to modify a class as it is created, and then customize the __new__ or __init__ methods. By customizing the metaclass __getattribute__ you could, yes, control which methods are visible in each instance of the class - but that can also be made directly from your base class __getattribute__. At class creation time, we need to annotate the guarded methods and that can be done with the __init_subclass__ class method.
Now, to check the class name where one instance is bould as an attribute, that is more complicated: when code is run, it won't "know" where it was called from -
One could do a().some_func() at top level code: that creates an instance of "a", and runs the method without the instance ever being bound to any local or global variable.
Python do allow introspection of running code, so the called code (either the code in some_func or code in a.__getattribute__ can "see" the local variables where that code was called. Since the caller could "forge" local variables to whatever they'd like, this alone would not restrict code to be run - but it could be used as a simple mechanism to avoid someone that would unknowingly call the functions from "wrong places" at runtime.
Note that this code aditionally requieres that the calling method has a local self variable that points to the instance - and that is used to check the class name (as I said, this is easy to circunvent, but will avoid people calling your methods by mistake)
The restriction should be documented - and be cunted upon just as a safeguard.
So instead of a metaclass, there is a "RestrictorBase" class that should be inherited from any classes that have the protected methods. And them, for each method you want to protect by name in this way, you should apply the allowed_from decorator as bellow. As the comparison is with the in operator,
it will automatically work with single strings or with tuples of strings (say, you want it to work from "a" and "b").
import inspect
RestrictedRegistry = {}
class RestrictorBase:
def __init_subclass__(cls, **kwargs):
for attrname, attr in cls.__dict__.items():
if allowed:=getattr(attr, "_allowed_from", None):
RestrictedRegistry[attr] = allowed
super().__init_subclass__(**kwargs)
def __getattribute__(self, attrname):
attr = super().__getattribute__(attrname)
if (func:=getattr(attr, "__func__", None)) and func in RestrictedRegistry:
caller_vars = inspect.currentframe().f_back.f_locals
if "self" in caller_vars and caller_vars["self"].__class__.__name__ in RestrictedRegistry[func]:
return attr
raise AttributeError(f" {self.__class__.__name__!r} object has no attribute {attrname}")
return attr
def allowed_from(names):
def deco(func):
func._allowed_from = names
return func
return deco
def hashable(obj):
try:
hash(obj)
except TypeError:
return False
return True
class a(RestrictorBase):
#allowed_from('b')
def some_func(self):
print("Restricted function running")
class b:
def __init__(self):
self.a_instance = a()
def other_method(self):
self.a_instance.some_func()
And this is the code above running in the interactive interpreter:
In [121]: b().other_method()
Restricted function running
In [122]: class c(b): pass
In [123]: c().other_method()
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-123-6746b8f5138c> in <module>
----> 1 c().other_method()
[...]
AttributeError: 'a' object has no attribute some_func
In [124]:

How to Inherit multiple classes in python dynamically [duplicate]

This article has a snippet showing usage of __bases__ to dynamically change the inheritance hierarchy of some Python code, by adding a class to an existing classes collection of classes from which it inherits. Ok, that's hard to read, code is probably clearer:
class Friendly:
def hello(self):
print 'Hello'
class Person: pass
p = Person()
Person.__bases__ = (Friendly,)
p.hello() # prints "Hello"
That is, Person doesn't inherit from Friendly at the source level, but rather this inheritance relation is added dynamically at runtime by modification of the __bases__attribute of the Person class. However, if you change Friendly and Person to be new style classes (by inheriting from object), you get the following error:
TypeError: __bases__ assignment: 'Friendly' deallocator differs from 'object'
A bit of Googling on this seems to indicate some incompatibilities between new-style and old style classes in regards to changing the inheritance hierarchy at runtime. Specifically: "New-style class objects don't support assignment to their bases attribute".
My question, is it possible to make the above Friendly/Person example work using new-style classes in Python 2.7+, possibly by use of the __mro__ attribute?
Disclaimer: I fully realise that this is obscure code. I fully realize that in real production code tricks like this tend to border on unreadable, this is purely a thought experiment, and for funzies to learn something about how Python deals with issues related to multiple inheritance.
Ok, again, this is not something you should normally do, this is for informational purposes only.
Where Python looks for a method on an instance object is determined by the __mro__ attribute of the class which defines that object (the M ethod R esolution O rder attribute). Thus, if we could modify the __mro__ of Person, we'd get the desired behaviour. Something like:
setattr(Person, '__mro__', (Person, Friendly, object))
The problem is that __mro__ is a readonly attribute, and thus setattr won't work. Maybe if you're a Python guru there's a way around that, but clearly I fall short of guru status as I cannot think of one.
A possible workaround is to simply redefine the class:
def modify_Person_to_be_friendly():
# so that we're modifying the global identifier 'Person'
global Person
# now just redefine the class using type(), specifying that the new
# class should inherit from Friendly and have all attributes from
# our old Person class
Person = type('Person', (Friendly,), dict(Person.__dict__))
def main():
modify_Person_to_be_friendly()
p = Person()
p.hello() # works!
What this doesn't do is modify any previously created Person instances to have the hello() method. For example (just modifying main()):
def main():
oldperson = Person()
ModifyPersonToBeFriendly()
p = Person()
p.hello()
# works! But:
oldperson.hello()
# does not
If the details of the type call aren't clear, then read e-satis' excellent answer on 'What is a metaclass in Python?'.
I've been struggling with this too, and was intrigued by your solution, but Python 3 takes it away from us:
AttributeError: attribute '__dict__' of 'type' objects is not writable
I actually have a legitimate need for a decorator that replaces the (single) superclass of the decorated class. It would require too lengthy a description to include here (I tried, but couldn't get it to a reasonably length and limited complexity -- it came up in the context of the use by many Python applications of an Python-based enterprise server where different applications needed slightly different variations of some of the code.)
The discussion on this page and others like it provided hints that the problem of assigning to __bases__ only occurs for classes with no superclass defined (i.e., whose only superclass is object). I was able to solve this problem (for both Python 2.7 and 3.2) by defining the classes whose superclass I needed to replace as being subclasses of a trivial class:
## T is used so that the other classes are not direct subclasses of object,
## since classes whose base is object don't allow assignment to their __bases__ attribute.
class T: pass
class A(T):
def __init__(self):
print('Creating instance of {}'.format(self.__class__.__name__))
## ordinary inheritance
class B(A): pass
## dynamically specified inheritance
class C(T): pass
A() # -> Creating instance of A
B() # -> Creating instance of B
C.__bases__ = (A,)
C() # -> Creating instance of C
## attempt at dynamically specified inheritance starting with a direct subclass
## of object doesn't work
class D: pass
D.__bases__ = (A,)
D()
## Result is:
## TypeError: __bases__ assignment: 'A' deallocator differs from 'object'
I can not vouch for the consequences, but that this code does what you want at py2.7.2.
class Friendly(object):
def hello(self):
print 'Hello'
class Person(object): pass
# we can't change the original classes, so we replace them
class newFriendly: pass
newFriendly.__dict__ = dict(Friendly.__dict__)
Friendly = newFriendly
class newPerson: pass
newPerson.__dict__ = dict(Person.__dict__)
Person = newPerson
p = Person()
Person.__bases__ = (Friendly,)
p.hello() # prints "Hello"
We know that this is possible. Cool. But we'll never use it!
Right of the bat, all the caveats of messing with class hierarchy dynamically are in effect.
But if it has to be done then, apparently, there is a hack that get's around the "deallocator differs from 'object" issue when modifying the __bases__ attribute for the new style classes.
You can define a class object
class Object(object): pass
Which derives a class from the built-in metaclass type.
That's it, now your new style classes can modify the __bases__ without any problem.
In my tests this actually worked very well as all existing (before changing the inheritance) instances of it and its derived classes felt the effect of the change including their mro getting updated.
I needed a solution for this which:
Works with both Python 2 (>= 2.7) and Python 3 (>= 3.2).
Lets the class bases be changed after dynamically importing a dependency.
Lets the class bases be changed from unit test code.
Works with types that have a custom metaclass.
Still allows unittest.mock.patch to function as expected.
Here's what I came up with:
def ensure_class_bases_begin_with(namespace, class_name, base_class):
""" Ensure the named class's bases start with the base class.
:param namespace: The namespace containing the class name.
:param class_name: The name of the class to alter.
:param base_class: The type to be the first base class for the
newly created type.
:return: ``None``.
Call this function after ensuring `base_class` is
available, before using the class named by `class_name`.
"""
existing_class = namespace[class_name]
assert isinstance(existing_class, type)
bases = list(existing_class.__bases__)
if base_class is bases[0]:
# Already bound to a type with the right bases.
return
bases.insert(0, base_class)
new_class_namespace = existing_class.__dict__.copy()
# Type creation will assign the correct ‘__dict__’ attribute.
del new_class_namespace['__dict__']
metaclass = existing_class.__metaclass__
new_class = metaclass(class_name, tuple(bases), new_class_namespace)
namespace[class_name] = new_class
Used like this within the application:
# foo.py
# Type `Bar` is not available at first, so can't inherit from it yet.
class Foo(object):
__metaclass__ = type
def __init__(self):
self.frob = "spam"
def __unicode__(self): return "Foo"
# … later …
import bar
ensure_class_bases_begin_with(
namespace=globals(),
class_name=str('Foo'), # `str` type differs on Python 2 vs. 3.
base_class=bar.Bar)
Use like this from within unit test code:
# test_foo.py
""" Unit test for `foo` module. """
import unittest
import mock
import foo
import bar
ensure_class_bases_begin_with(
namespace=foo.__dict__,
class_name=str('Foo'), # `str` type differs on Python 2 vs. 3.
base_class=bar.Bar)
class Foo_TestCase(unittest.TestCase):
""" Test cases for `Foo` class. """
def setUp(self):
patcher_unicode = mock.patch.object(
foo.Foo, '__unicode__')
patcher_unicode.start()
self.addCleanup(patcher_unicode.stop)
self.test_instance = foo.Foo()
patcher_frob = mock.patch.object(
self.test_instance, 'frob')
patcher_frob.start()
self.addCleanup(patcher_frob.stop)
def test_instantiate(self):
""" Should create an instance of `Foo`. """
instance = foo.Foo()
The above answers are good if you need to change an existing class at runtime. However, if you are just looking to create a new class that inherits by some other class, there is a much cleaner solution. I got this idea from https://stackoverflow.com/a/21060094/3533440, but I think the example below better illustrates a legitimate use case.
def make_default(Map, default_default=None):
"""Returns a class which behaves identically to the given
Map class, except it gives a default value for unknown keys."""
class DefaultMap(Map):
def __init__(self, default=default_default, **kwargs):
self._default = default
super().__init__(**kwargs)
def __missing__(self, key):
return self._default
return DefaultMap
DefaultDict = make_default(dict, default_default='wug')
d = DefaultDict(a=1, b=2)
assert d['a'] is 1
assert d['b'] is 2
assert d['c'] is 'wug'
Correct me if I'm wrong, but this strategy seems very readable to me, and I would use it in production code. This is very similar to functors in OCaml.
This method isn't technically inheriting during runtime, since __mro__ can't be changed. But what I'm doing here is using __getattr__ to be able to access any attributes or methods from a certain class. (Read comments in order of numbers placed before the comments, it makes more sense)
class Sub:
def __init__(self, f, cls):
self.f = f
self.cls = cls
# 6) this method will pass the self parameter
# (which is the original class object we passed)
# and then it will fill in the rest of the arguments
# using *args and **kwargs
def __call__(self, *args, **kwargs):
# 7) the multiple try / except statements
# are for making sure if an attribute was
# accessed instead of a function, the __call__
# method will just return the attribute
try:
return self.f(self.cls, *args, **kwargs)
except TypeError:
try:
return self.f(*args, **kwargs)
except TypeError:
return self.f
# 1) our base class
class S:
def __init__(self, func):
self.cls = func
def __getattr__(self, item):
# 5) we are wrapping the attribute we get in the Sub class
# so we can implement the __call__ method there
# to be able to pass the parameters in the correct order
return Sub(getattr(self.cls, item), self.cls)
# 2) class we want to inherit from
class L:
def run(self, s):
print("run" + s)
# 3) we create an instance of our base class
# and then pass an instance (or just the class object)
# as a parameter to this instance
s = S(L) # 4) in this case, I'm using the class object
s.run("1")
So this sort of substitution and redirection will simulate the inheritance of the class we wanted to inherit from. And it even works with attributes or methods that don't take any parameters.

How to implement virtual methods in Python?

I know virtual methods from PHP or Java.
How can they be implemented in Python?
Or have I to define an empty method in an abstract class and override it?
Sure, and you don't even have to define a method in the base class. In Python methods are better than virtual - they're completely dynamic, as the typing in Python is duck typing.
class Dog:
def say(self):
print "hau"
class Cat:
def say(self):
print "meow"
pet = Dog()
pet.say() # prints "hau"
another_pet = Cat()
another_pet.say() # prints "meow"
my_pets = [pet, another_pet]
for a_pet in my_pets:
a_pet.say()
Cat and Dog in Python don't even have to derive from a common base class to allow this behavior - you gain it for free. That said, some programmers prefer to define their class hierarchies in a more rigid way to document it better and impose some strictness of typing. This is also possible - see for example the abc standard module.
raise NotImplementedError() (dynamic type checking)
This is the recommended exception to raise on "pure virtual methods" of "abstract" base classes that don't implement a method.
https://docs.python.org/3.5/library/exceptions.html#NotImplementedError says:
This exception is derived from RuntimeError. In user defined base classes, abstract methods should raise this exception when they require derived classes to override the method.
As others said, this is mostly a documentation convention and is not required, but this way you get a more meaningful exception than a missing attribute error.
dynamic.py
class Base(object):
def virtualMethod(self):
raise NotImplementedError()
def usesVirtualMethod(self):
return self.virtualMethod() + 1
class Derived(Base):
def virtualMethod(self):
return 1
print Derived().usesVirtualMethod()
Base().usesVirtualMethod()
gives:
2
Traceback (most recent call last):
File "./dynamic.py", line 13, in <module>
Base().usesVirtualMethod()
File "./dynamic.py", line 6, in usesVirtualMethod
return self.virtualMethod() + 1
File "./dynamic.py", line 4, in virtualMethod
raise NotImplementedError()
NotImplementedError
typing.Protocol (static type checking, Python 3.8)
Python 3.8 added typing.Protocol which now allows us to also statically type check that a virtual method is implemented on a subclass.
protocol.py
from typing import Protocol
class CanFly(Protocol):
def fly(self) -> str:
pass
def fly_fast(self) -> str:
return 'CanFly.fly_fast'
class Bird(CanFly):
def fly(self):
return 'Bird.fly'
def fly_fast(self):
return 'Bird.fly_fast'
class FakeBird(CanFly):
pass
assert Bird().fly() == 'Bird.fly'
assert Bird().fly_fast() == 'Bird.fly_fast'
# mypy error
assert FakeBird().fly() is None
# mypy error
assert FakeBird().fly_fast() == 'CanFly.fly_fast'
If we run this file, the asserts pass, as we didn't add any dynamic typechecking:
python protocol.py
but if we typecheck if mypy:
python -m pip install --user mypy
mypy protocol.py
we get an error as expected:
protocol.py:22: error: Cannot instantiate abstract class "FakeBird" with abstract attribute "fly"
protocol.py:24: error: Cannot instantiate abstract class "FakeBird" with abstract attribute "fly"
It is a bit unfortunate however that the error checking only picks up the error on instantiation, and not at class definition.
typing.Protocol counts methods as abstract when their body is "empty"
I'm not sure what they count as empty, but both all of the following count as empty:
pass
... ellipsis object
raise NotImplementedError()
So the best possibility is likely:
protocol_empty.py
from typing import Protocol
class CanFly(Protocol):
def fly(self) -> None:
raise NotImplementedError()
class Bird(CanFly):
def fly(self):
return None
class FakeBird(CanFly):
pass
Bird().fly()
FakeBird().fly()
which fails as desired:
protocol_empty.py:15: error: Cannot instantiate abstract class "FakeBird" with abstract attribute "fly"
protocol_empty.py:15: note: The following method was marked implicitly abstract because it has an empty function body: "fly". If it is not meant to be abstract, explicitly return None.
but if e.g. we replace the:
raise NotImplementedError()
with some random "non-empty" statement such as:
x = 1
then mypy does not count them as virtual and gives no errors.
#abc.abstractmethod: metaclass syntax changed in Python 3
In Python 3 metaclasses are declared as:
class C(metaclass=abc.ABCMeta):
instead of the Python 2:
class C:
__metaclass__=abc.ABCMeta
so now to use #abc.abstractmethod which was previously mentioned at https://stackoverflow.com/a/19316077/895245 you need:
abc_cheat.py
class C(metaclass=abc.ABCMeta):
#abc.abstractmethod
def m(self, i):
pass
try:
c = C()
except TypeError:
pass
else:
assert False
But TODO: what is the advantage of ABCMeta over just raise NotImplementedError? It has a disadvantage that you are forced to define a metaclass, so more work, but I don't see an advantage. https://peps.python.org/pep-0544 does mention both approaches in passing.
Outro
Bibiography:
https://peps.python.org/pep-0544 the typing.Protocol PEP
Is it possible to make abstract classes?
What to use in replacement of an interface/protocol in python
Tested on Python 3.10.7, mypy 0.982, Ubuntu 21.10.
Python methods are always virtual.
Actually, in version 2.6 python provides something called abstract base classes and you can explicitly set virtual methods like this:
from abc import ABCMeta
from abc import abstractmethod
...
class C:
__metaclass__ = ABCMeta
#abstractmethod
def my_abstract_method(self, ...):
It works very well, provided the class does not inherit from classes that already use metaclasses.
source: http://docs.python.org/2/library/abc.html
Python methods are always virtual
like Ignacio said yet
Somehow class inheritance may be a better approach to implement what you want.
class Animal:
def __init__(self,name,legs):
self.name = name
self.legs = legs
def getLegs(self):
return "{0} has {1} legs".format(self.name, self.legs)
def says(self):
return "I am an unknown animal"
class Dog(Animal): # <Dog inherits from Animal here (all methods as well)
def says(self): # <Called instead of Animal says method
return "I am a dog named {0}".format(self.name)
def somethingOnlyADogCanDo(self):
return "be loyal"
formless = Animal("Animal", 0)
rover = Dog("Rover", 4) #<calls initialization method from animal
print(formless.says()) # <calls animal say method
print(rover.says()) #<calls Dog says method
print(rover.getLegs()) #<calls getLegs method from animal class
Results should be:
I am an unknown animal
I am a dog named Rover
Rover has 4 legs
Something like a virtual method in C++ (calling method implementation of a derived class through a reference or pointer to the base class) doesn't make sense in Python, as Python doesn't have typing. (I don't know how virtual methods work in Java and PHP though.)
But if by "virtual" you mean calling the bottom-most implementation in the inheritance hierarchy, then that's what you always get in Python, as several answers point out.
Well, almost always...
As dplamp pointed out, not all methods in Python behave like that. Dunder method don't. And I think that's a not so well known feature.
Consider this artificial example
class A:
def prop_a(self):
return 1
def prop_b(self):
return 10 * self.prop_a()
class B(A):
def prop_a(self):
return 2
Now
>>> B().prop_b()
20
>>> A().prob_b()
10
However, consider this one
class A:
def __prop_a(self):
return 1
def prop_b(self):
return 10 * self.__prop_a()
class B(A):
def __prop_a(self):
return 2
Now
>>> B().prop_b()
10
>>> A().prob_b()
10
The only thing we've changes was making prop_a() a dunder method.
A problem with the first behavior can be that you can't change the behavior of prop_a() in the derived class without impacting the behavior of prop_b(). This very nice talk by Raymond Hettinger gives an example for a use case where this is inconvenient.
Python 3.6 introduced __init_subclass__ and this let you simply do this:
class A:
def method(self):
'''method needs to be overwritten'''
return NotImplemented
def __init_subclass__(cls):
if cls.method is A.method:
raise NotImplementedError(
'Subclass has not overwritten method {method}!')
The benefit of this solution is that you avoid the abc metaclass and give the user a direct imperative how to do it right. In addition to another answer here that raises NotImplementedError when calling the method. This solution is checked on runtime and not only IF the user calls the method.

Categories