We Keep Coding

Dynamically override all parent class methods

I want to be able to dynamically override all calls sent to methods on the parent class. So far I've only been able to do it for methods not defined in a parent class.
I.e. Given a parent class:
class A:
def foo(self):
print("foo")
def bar(self):
print("bar")
I want to create a Spy class that inherits from A will print "This is a spy" before calling any method on A, including foo.
class Spy(A):
pass # What do I do here?
s = Spy()
>>> s.foo()
This is a spy
foo
Current implementation
My current implementation of Spy is this:
class Spy(A):
def __getattr__(self, method):
print("This is a spy")
return getattr(super(A, self), method)
However, this only works for methods not defined in the parent class:
>>> s.baz()
This is a spy
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 4, in __getattr__
AttributeError: 'super' object has no attribute 'baz'
When I call a method that exists already, it doesn't work:
>>> s.foo()
foo
>>> s.bar()
bar

The following code snippet should do what you want. I have excluded methods starting with __ because these can be problematic (for example, overriding __class__ with a function will cause an error).
class A:
def foo(self):
print("foo")
def bar(self, x):
print(x)
class Spy(A):
def functionwrapper(self, functionname):
originalfunc = getattr(super(), functionname)
def wrap(*args, **kwargs):
print("This is a spy: ", end="")
originalfunc(*args, **kwargs)
return wrap
def __init__(self):
for methodname in [method for method in dir(A) if (method[0:2] != "__")]:
setattr(self, methodname, self.functionwrapper(methodname))
s = Spy()
s.foo()
s.bar("variable")
Output
This is a spy: foo
This is a spy: variable

When I call a method that exists already, it doesn't work
That's because __getattr__ is only gonna get called when the default attribute access fails. So if the parent has that method, it gets found and your __getattr__ won't get called.
You need to intercept the __getattribute__ instead:
from types import MethodType
from inspect import isfunction
class A:
def foo(self):
print("foo")
class Spy(A):
def bar(self):
print("barrrrrrrrr")
def __getattribute__(self, name):
# check it instance's namespace
instance_dict = object.__getattribute__(self, "__dict__")
if name in instance_dict:
return instance_dict[name]
# check its class' namespace
if name in type(self).__dict__:
return object.__getattribute__(self, name)
# check parents
for cls in type(self).mro()[1:]:
if name in cls.__dict__:
member = cls.__dict__[name]
if isfunction(member):
# your code here
print("This is a spy")
return MethodType(cls.__dict__[name], self)
return member
raise AttributeError(f"{type(self)} object has no attribute '{name}'")
s = Spy()
s.foo()
s.bar()
print("-----------------------------------")
s.boo()
output:
This is a spy
foo
barrrrrrrrr
-----------------------------------
Traceback (most recent call last):
File "...", line 32, in <module>
s.boo()
File "...", line 25, in __getattribute__
raise AttributeError(f"{type(self)} object has no attribute '{name}'")
AttributeError: <class '__main__.Spy'> object has no attribute 'boo'
So this is one way of doing it and I agree that it might be overkill.
You do that in three steps:
Check to see it is in the instance's namespace: like the data attributes you usually set in __init__.
Check to see it is in its class' namespace: You don't want yout print statement here because it's not sent to the parent.
Check its parents

test.py:
import functools
import inspect
TXT = "This is a spy"
def deco(func):
#functools.wraps(func)
def wrapper(*args, **kwargs):
print(TXT)
return func(*args, **kwargs)
return wrapper
class A:
def foo(self):
print("foo")
def bar(self):
print("bar")
class Spy(A):
def __init__(self):
self.patch()
#classmethod
def patch(cls):
superclass = cls.__bases__[0]
for name, value in inspect.getmembers(superclass, inspect.isfunction):
setattr(cls, name, deco(value))
def main():
s = Spy()
s.foo()
s.bar()
if __name__ == "__main__":
main()
Test:
$ python test.py
This is a spy
foo
This is a spy
bar

Using setattr() and getattr() in Python descriptors

While trying to create descriptors a few different ways I noticed some strange behavior that I'm trying to understand. Below are the three different ways I have gone about creating descriptors:
>>> class NumericValueOne():
... def __init__(self, name):
... self.name = name
... def __get__(self, obj, type=None) -> object:
... return obj.__dict__.get(self.name) or 0
... def __set__(self, obj, value) -> None:
... obj.__dict__[self.name] = value
>>> class NumericValueTwo():
... def __init__(self, name):
... self.name = name
... self.internal_name = '_' + self.name
... def __get__(self, obj, type=None) -> object:
... return getattr(obj, self.internal_name, 0)
... def __set__(self, obj, value) -> None:
... setattr(obj, self.internal_name, value)
>>> class NumericValueThree():
... def __init__(self, name):
... self.name = name
... def __get__(self, obj, type=None) -> object:
... return getattr(obj, self.name, 0)
... def __set__(self, obj, value) -> None:
... setattr(obj, self.name, value)
I then use them in the Foo classes, like below:
>>> class FooOne():
... number = NumericValueOne("number")
>>> class FooTwo():
... number = NumericValueTwo("number")
>>> class FooThree():
... number = NumericValueThree("number")
my_foo_object_one = FooOne()
my_foo_object_two = FooTwo()
my_foo_object_three = FooThree()
my_foo_object_one.number = 3
my_foo_object_two.number = 3
my_foo_object_three.number = 3
While FooOne and FooTwo work as expected when both setting & getting values. FooThree throws the following error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 7, in __set__
File "<stdin>", line 7, in __set__
File "<stdin>", line 7, in __set__
[Previous line repeated 497 more times]
RecursionError: maximum recursion depth exceeded while calling a Python object
It looks like setattr() is calling the __set__() method? But why should it be doing that if setattr() is modifying the obj __dict__? And why does this work if we use internal_name?
Why is that we NEED to use a private variable in order to use the built-in getattr() and setattr() methods correctly? Also, how is this different from just directly modifying the obj __dict__ like in NumericValueOne?

But why should it be doing that if setattr() is modifying the obj __dict__?
setattr doesn't just modify the __dict__. It sets attributes, exactly like x.y = z would, and for the attribute you're trying to set, "set this attribute" means "call the setter you're already in". Hence, infinite recursion.
And why does this work if we use internal_name?
That name doesn't correspond to a property, so it just gets a __dict__ entry.

Calling a method vs setattr

Let's say I need to call a method or change an attribute of the class, and I don't know exactly which it's going to be in advance. For now my code looks like this:
def change(obj, attr_name, val):
attr = getattr(obj, attr_name, None)
# check if attr exists
if type(attr) == 'NoneType':
print('no such attribute')
return
if callable(attr):
attr(val)
else:
setattr(obj, attr_name, val)
However, it doesn't seem very elegant to me.
I've tried googling this for a couple of days and can't find an answer.
I was using this to call methods of an object which names I didn't know in advance. As I understand from reading python literature it is valid to use getattr in this case. However, later I started to decorate some of my methods with #property and when I try to call them with getattr(obj, attr_name)(value) I get an error. Here's the code snippet to illustrate my problem:
class myclass1:
def __init__(self):
self._a = 'hello'
def getA(self):
return self._a
def setA(self, val):
self._a = val
class myclass2:
def __init__(self):
self._a = 'hello'
#property
def A(self):
return self._a
#A.setter
def A(self, val):
self._a = val
def changeOLD(obj, attr_name, val):
attr = getattr(obj, attr_name, None)(val)
obj1 = myclass1()
obj2 = myclass2()
changeOLD(obj1, 'setA', 'goodbye')
print(obj1.getA())
changeOLD(obj2, 'A', 'goodbye')
print(obj2.A)
When I run it:
$ python test.py
goodbye
Traceback (most recent call last):
File "test.py", line 45, in <module>
changeOLD(obj2, 'A', 'goodbye')
File "test.py", line 37, in changeOLD
attr = getattr(obj, attr_name, None)(val)
TypeError: 'str' object is not callable
So I figured I need a way to differentiate between attribute and method.

There's no major problems with the code you've provided for the use case you've described. There are some changes you could do though.
I'd change so you check against None instead of the type of None. And printing is usually not done in a method that handles logic. You'd might also want to return the value from the function you're calling. Lastly, change the method name to something more fitting.
def set_or_call(obj, name, value):
attr_or_function = getattr(obj, name, None)
if attr_or_function is None:
return None # Or raise an exception if this is suppose to be an error.
elif callable(attr_or_function):
return attr_or_function(value)
else:
setattr(obj, attr_or_function, value)

Specialized #property decorators in python

I have a few classes each of which has a number of attributes. What all of the attributes have in common is that they should be numeric properties. This seems to be an ideal place to use python's decorators, but I can't seem to wrap my mind around what the correct implementation would be. Here is a simple example:
class Junk(object):
def __init__(self, var):
self._var = var
#property
def var(self):
"""A numeric variable"""
return self._var
#var.setter
def size(self, value):
# need to make sure var is an integer
if not isinstance(value, int):
raise ValueError("var must be an integer, var = {}".format(value))
self._var = value
#var.deleter
def size(self):
raise RuntimeError("You can't delete var")
It seems to me that it should be possible to write a decorator that does everything so that the above can be transformed into:
def numeric_property(*args, **kwargs):
...
class Junk(object):
def __init__(self, var):
self._var = var
#numeric_property
def var(self):
"""A numeric variable"""
return self._var
That way the new numeric_property decorator can be used in many classes.

A #property is just a special case of Python's descriptor protocol, so you can certainly build your own custom versions. For your case:
class NumericProperty:
"""A property that must be numeric.
Args:
attr (str): The name of the backing attribute.
"""
def __init__(self, attr):
self.attr = attr
def __get__(self, obj, type=None):
return getattr(obj, self.attr)
def __set__(self, obj, value):
if not isinstance(value, int):
raise ValueError("{} must be an integer, var = {!r}".format(self.attr, value))
setattr(obj, self.attr, value)
def __delete__(self, obj):
raise RuntimeError("You can't delete {}".format(self.attr))
class Junk:
var = NumericProperty('_var')
def __init__(self, var):
self.var = var
In use:
>>> j = Junk('hi')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/jonrsharpe/test.py", line 29, in __init__
self.var = var
File "/Users/jonrsharpe/test.py", line 17, in __set__
raise ValueError("{} must be an integer, var = {!r}".format(self.attr, value))
ValueError: _var must be an integer, var = 'hi'
>>> j = Junk(1)
>>> del j.var
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/jonrsharpe/test.py", line 21, in __delete__
raise RuntimeError("You can't delete {}".format(self.attr))
RuntimeError: You can't delete _var
>>> j.var = 'hello'
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/jonrsharpe/test.py", line 17, in __set__
raise ValueError("{} must be an integer, var = {!r}".format(self.attr, value))
ValueError: _var must be an integer, var = 'hello'
>>> j.var = 2
>>> j.var
2

Option 1: inherit from property
property is a descriptor. See Descriptor HowTo on python.org.
So, can inherit from property and override the relevant methods.
For example, to enforce int on setter:
class numeric_property(property):
def __set__(self, obj, value):
assert isinstance(value, int), "numeric_property requires an int"
super(numeric_property, self).__set__(obj, value)
class A(object):
#numeric_property
def x(self):
return self._x
#x.setter
def x(self, value):
self._x = value
And now you have integers enforced:
>>> a = A()
>>> a.x = 'aaa'
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 3, in __set__
AssertionError: numeric_property requires an int
Option 2: Create a better descriptor
On the other hand, it may be even better to implement a brand new descriptor which does not inherit from property, which would enable you to define the property in one go.
It would be nicer to have this kind of interface:
class A(object):
x = numeric_property('_x')
For that you would implement a descriptor which takes the attribute name:
class numeric_property(object):
def __init__(self, private_attribute_name, default=0):
self.private_attribute_name = private_attribute_name
self.default = default
def __get__(self, obj, typ):
if not obj: return self
return getattr(obj, self.private_attribute_name, self.default)
def __set__(self, obj, value):
assert isinstance(value, int), "numeric_property requires an int"
setattr(obj, self.private_attribute_name, value)
Disclaimer :)
I would rather not enforce strict typing in Pyhon, because Python is much more powerful without it.

You may just create a function that does it for you . As simple as it can get, no need to create a custom descriptor:
def numprop(name, privname):
#property
def _numprop(self):
return getattr(self, privname)
#_numprop.setter
def _numprop(self, value):
if not isinstance(value, int):
raise ValueError("{name} must be an integer, {name} = {}".format(value, name=name))
setattr(self, privname, value)
#_numprop.deleter
def _numprop(self):
raise RuntimeError("You can't delete var")
return _numprop
class Junk(object):
def __init__(self, var):
self._var = var
var = numprop("var", "_var")

Python inheritance - how to disable a function

In C++ you can disable a function in parent's class by declaring it as private in the child class. How can this be done in Python? I.E. How can I hide parent's function from child's public interface?

There really aren't any true "private" attributes or methods in Python. One thing you can do is simply override the method you don't want in the subclass, and raise an exception:
>>> class Foo( object ):
... def foo( self ):
... print 'FOO!'
...
>>> class Bar( Foo ):
... def foo( self ):
... raise AttributeError( "'Bar' object has no attribute 'foo'" )
...
>>> b = Bar()
>>> b.foo()
Traceback (most recent call last):
File "<interactive input>", line 1, in <module>
File "<interactive input>", line 3, in foo
AttributeError: 'Bar' object has no attribute 'foo'

kurosch's method of solving the problem isn't quite correct, because you can still use b.foo without getting an AttributeError. If you don't invoke the function, no error occurs. Here are two ways that I can think to do this:
import doctest
class Foo(object):
"""
>>> Foo().foo()
foo
"""
def foo(self): print 'foo'
def fu(self): print 'fu'
class Bar(object):
"""
>>> b = Bar()
>>> b.foo()
Traceback (most recent call last):
...
AttributeError
>>> hasattr(b, 'foo')
False
>>> hasattr(b, 'fu')
True
"""
def __init__(self): self._wrapped = Foo()
def __getattr__(self, attr_name):
if attr_name == 'foo': raise AttributeError
return getattr(self._wrapped, attr_name)
class Baz(Foo):
"""
>>> b = Baz()
>>> b.foo() # doctest: +ELLIPSIS
Traceback (most recent call last):
...
AttributeError...
>>> hasattr(b, 'foo')
False
>>> hasattr(b, 'fu')
True
"""
foo = property()
if __name__ == '__main__':
doctest.testmod()
Bar uses the "wrap" pattern to restrict access to the wrapped object. Martelli has a good talk dealing with this. Baz uses the property built-in to implement the descriptor protocol for the attribute to override.

A variation on the answer of kurosch:
class Foo( object ):
def foo( self ):
print 'FOO!'
class Bar( Foo ):
#property
def foo( self ):
raise AttributeError( "'Bar' object has no attribute 'foo'" )
b = Bar()
b.foo
This raises an AttributeError on the property instead of when the method is called.
I would have suggested it in a comment but unfortunately do not have the reputation for it yet.

class X(object):
def some_function(self):
do_some_stuff()
class Y(object):
some_function = None
This may lead to some nasty and hard to find exceptions being thrown though, so you might try this:
class X(object):
def some_function(self):
do_some_stuff()
class Y(object):
def some_function(self):
raise NotImplementedError("function some_function not implemented")

That could be even simpler.
#property
def private(self):
raise AttributeError
class A:
def __init__(self):
pass
def hello(self):
print("Hello World")
class B(A):
hello = private # that short, really
def hi(self):
A.hello(self)
obj = A()
obj.hello()
obj = B()
obj.hi() # works
obj.hello() # raises AttributeError

This is the cleanest way I know to do it.
Override the methods and have each of the overridden methods call your disabledmethods() method. Like this:
class Deck(list):
...
#staticmethod
def disabledmethods():
raise Exception('Function Disabled')
def pop(self): Deck.disabledmethods()
def sort(self): Deck.disabledmethods()
def reverse(self): Deck.disabledmethods()
def __setitem__(self, loc, val): Deck.disabledmethods()

Another approach is define an descriptor that errors on access.
class NotHereDescriptor:
def __get__(self, obj, type=None):
raise AttributeError
class Bar:
foo = NotHereDescriptor()
This is similar in nature to the property approach a few people have used above. However it has the advantage that hasattr(Bar, 'foo') will return False as one would expect if the function really didn't exist. Which further reduces the chance of weird bugs. Although it does still show up in dir(Bar).
If you are interested in what this is doing and why it works check out the descriptor section of the data model page https://docs.python.org/3/reference/datamodel.html#descriptors and the descriptor how to https://docs.python.org/3/howto/descriptor.html