This one seems a bit tricky to me. Sometime ago I already managed to overwrite an instance's method with something like:
def my_method(self, attr):
pass
instancemethod = type(self.method_to_overwrite)
self.method_to_overwrite = instancemethod(my_method, self, self.__class__)
which worked very well for me; but now I'm trying to overwrite an instance's __getattribute__() function, which doesn't work for me for the reason the method seems to be
<type 'method-wrapper'>
Is it possible to do anything about that? I couldn't find any decent Python documentation on method-wrapper.
You want to override the attribute lookup algorithm on an per instance basis? Without knowing why you are trying to do this, I would hazard a guess that there is a cleaner less convoluted way of doing what you need to do. If you really need to then as Aaron said, you'll need to install a redirecting __getattribute__ handler on the class because Python looks up special methods only on the class, ignoring anything defined on the instance.
You also have to be extra careful about not getting into infinite recursion:
class FunkyAttributeLookup(object):
def __getattribute__(self, key):
try:
# Lookup the per instance function via objects attribute lookup
# to avoid infinite recursion.
getter = object.__getattribute__(self, 'instance_getattribute')
return getter(key)
except AttributeError:
return object.__getattribute__(self, key)
f = FunkyAttributeLookup()
f.instance_getattribute = lambda attr: attr.upper()
print(f.foo) # FOO
Also, if you are overriding methods on your instance, you don't need to instanciate the method object yourself, you can either use the descriptor protocol on functions that generates the methods or just curry the self argument.
#descriptor protocol
self.method_to_overwrite = my_method.__get__(self, type(self))
# or curry
from functools import partial
self.method_to_overwrite = partial(my_method, self)
You can't overwrite special methods at instance level. For new-style classes, implicit invocations of special methods are only guaranteed to work correctly if defined on an object’s type, not in the object’s instance dictionary.
There are a couple of methods which you can't overwrite and __getattribute__() is one of them.
I believe method-wrapper is a wrapper around a method written in C.
Related
I have a class
class A:
def sample_method():
I would like to decorate class A sample_method() and override the contents of sample_method()
class DecoratedA(A):
def sample_method():
The setup above resembles inheritance, but I need to keep the preexisting instance of class A when the decorated function is used.
a # preexisting instance of class A
decorated_a = DecoratedA(a)
decorated_a.functionInClassA() #functions in Class A called as usual with preexisting instance
decorated_a.sample_method() #should call the overwritten sample_method() defined in DecoratedA
What is the proper way to go about this?
There isn't a straightforward way to do what you're asking. Generally, after an instance has been created, it's too late to mess with the methods its class defines.
There are two options you have, as far as I see it. Either you create a wrapper or proxy object for your pre-existing instance, or you modify the instance to change its behavior.
A proxy defers most behavior to the object itself, while only adding (or overriding) some limited behavior of its own:
class Proxy:
def __init__(self, obj):
self.obj = obj
def overridden_method(self): # add your own limited behavior for a few things
do_stuff()
def __getattr__(self, name): # and hand everything else off to the other object
return getattr(self.obj, name)
__getattr__ isn't perfect here, it can only work for regular methods, not special __dunder__ methods that are often looked up directly in the class itself. If you want your proxy to match all possible behavior, you probably need to add things like __add__ and __getitem__, but that might not be necessary in your specific situation (it depends on what A does).
As for changing the behavior of the existing object, one approach is to write your subclass, and then change the existing object's class to be the subclass. This is a little sketchy, since you won't have ever initialized the object as the new class, but it might work if you're only modifying method behavior.
class ModifiedA(A):
def overridden_method(self): # do the override in a normal subclass
do_stuff()
def modify_obj(obj): # then change an existing object's type in place!
obj.__class__ = ModifiedA # this is not terribly safe, but it can work
You could also consider adding an instance variable that would shadow the method you want to override, rather than modifying __class__. Writing the function could be a little tricky, since it won't get bound to the object automatically when called (that only happens for functions that are attributes of a class, not attributes of an instance), but you could probably do the binding yourself (with partial or lambda if you need to access self.
First, why not just define it from the beginning, how you want it, instead of decorating it?
Second, why not decorate the method itself?
To answer the question:
You can reassign it
class A:
def sample_method(): ...
pass
A.sample_method = DecoratedA.sample_method;
but that affects every instance.
Another solution is to reassign the method for just one object.
import functools;
a.sample_method = functools.partial(DecoratedA.sample_method, a);
Another solution is to (temporarily) change the type of an existing object.
a = A();
a.__class__ = DecoratedA;
a.sample_method();
a.__class__ = A;
This question answers how to implement __getattr__ for static/class attributes - using a metaclass. However, I would like to implement __getattr__ and __getattribute__ for a class generated by type() and to make things even more interesting, the class inherits a class which has a custom metaclass which must be executed properly.
The code summarizing the paragraph above:
class Inherited(metaclass=SomeFancyMetaclass):
...
generated_class = type("GeneratedClass", (Inherited,), {})
def __class_getattr__(cls, name): # __getattr__ for class, not sure how the code shall look exactly like
return getattr(cls, name)
setattr(generated_class, "__getattr__", __class_getattr__) # similarly for __getattribute__
The question: is this possible, and if so, how? Could someone provide a minimal working example?
Just make your metaclass inherit from SomeFancyMetaclass, implement the __getattr__ (and __getattribute__) there properly, and use this metaclass, rather than a call to type to generate your inheited, dynamic class.
Although you are using a lot of seldon used stuff, there are no special mechanisms in the way - it should be plain Python -
Of course, you did not tell what you want to do in the metaclass special methods - there might be some black magic to be performed there - and if you are doing __getattribute__, you always have to be extra careful, and redirect all attrbiutes that you don't care about to the super-call, otherwise, nothing works.
Also, keep in mind that the attribute-access ustomization possible with both methods won't work to "create magic dunder methods" - that is: your class won't magically have an __add__ or __dir__ method because your metaclass __getattribute__ generates one - rather, these are fixed in spcial slots by the Python runtime, and their checking and calling bypasses normal attribute lookup in Python.
Otherwise:
class Inherited(metaclass=SomeFancyMetaclass):
...
class MagicAttrsMeta(Inherited.__class__):
def __getattr__(self, attr):
if attr in ("flying", "circus", "brian", "king_arthur"):
return "coconut"
raise AttributeError()
generated_class = MagicAttrsMeta("GeneratedClass", (Inherited,), {})
I need to override the __getattr__ method, but how to I get properties there?
I can access the attributes from the __getattr__ method like this:
class Someclass:
#property
def some_prop(self):
return something
def __getattr__(self, attr_name):
return self.__dict__[attr_name]
but it won't work if attr_name == 'some_prop', how to access some_prop there?
EDIT: this is not the question about how to solve '2+2 = x', this is the question about metaprogramming in python, the provided example is just for illustration of the problem
Have you tried self.some_prop()?
Because it works.
But also self.__class__.some_prop.__get__(self, self.__class__) if you just want to do it differently.
Besides that, you are aware that __getattr__ won't be called for a reference for instance.some_prop, right? Since the descriptor exists, __getattr__ is not called. (Also, that is why it works).
If you need to intercept the references for instance.some_prop itself, you will need to implement __getattribute__ instead - that is unconditionally called for all your attribute references.
In that case, I'd recommend simply call super().__getattribute__ unconditionally, and just check afterwards if the attribute was some_prop. If not, just return the original value.
Consider:
class Parent():
def __init__(self, last_name, eye_color):
self.last_name = last_name
self.eye_color = eye_color
def show_info(self):
print("Last Name - "+self.last_name)
print("Eye Color - "+self.eye_color)
billy_cyrus = Parent("Cyrus", "blue")
The above is from the Udacity Python course. I discovered I'm able to call show_info for instance billy_cyrus using either of the following:
billy_cyrus.show_info()
Parent.show_info(billy_cyrus)
I'm curious as to why. Is there a difference between the two methods? If so when would one be used vs. the other? I'm using Python 3.6 if that matters.
In terms of just calling the method, there is no difference most of the time. In terms of how the underlying machinery, works, there is a bit of a difference.
Since show_info is a method, it is a descriptor in the class. That means that when you access it through an instance in which it is not shadowed by another attribute, the . operator calls __get__ on the descriptor to create a bound method for that instance. A bound method is basically a closure that passes in the self parameter for you before any of the other arguments you supply. You can see the binding happen like this:
>>> billy_cyrus.show_info
<bound method Parent.show_info of <__main__.Parent object at 0x7f7598b14be0>>
A different closure is created every time you use the . operator on a class method.
If you access the method through the class object, on the other hand, it does not get bound. The method is a descriptor, which is just a regular attribute of the class:
>>> Parent.show_info
<function __main__.Parent.show_info>
You can simulate the exact behavior of binding a method before calling it by calling its __get__ yourself:
>>> bound_meth = Parent.show_info.__get__(billy_cyrus, type(billy_cyrus))
>>> bound_meth
<bound method Parent.show_info of <__main__.Parent object at 0x7f7598b14be0>>
Again, this will not make any difference to you in 99.99% of cases, since functionally bound_meth() and Parent.bound_meth(billy_cyrus) end up calling the same underlying function object with the same parameters.
Where it matters
There are a couple of places where it matters how you call a class method. One common use case is when you override a method, but want to use the definition provided in the parent class. For example, say I have a class that I made "immutable" by overriding __setattr__. I can still set attributes on the instance, as in the __init__ method shown below:
class Test:
def __init__(self, a):
object.__setattr__(self, 'a', a)
def __setattr__(self, name, value):
raise ValueError('I am immutable!')
If I tried to do a normal call to __setattr__ in __init__ by doing self.a = a, a ValueError would be raised every time. But by using object.__setattr__, I can bypass this limitation. Alternatively, I could do super().__setattr__('a', a) for the same effect, or self.__dict__['a'] = a for a very similar one.
#Silvio Mayolo's answer has another good example, where you would deliberately want to use the class method as a function that could be applied to many objects.
Another place it matters (although not in terms of calling methods), is when you use other common descriptors like property. Unlike methods, properties are data-descriptors. This means that they define a __set__ method (and optionally __delete__) in addition to __get__. A property creates a virtual attribute whose getter and setter are arbitrarily complex functions instead of just simple assignments. To properly use a property, you have to do it through the instance. For example:
class PropDemo:
def __init__(self, x=0):
self.x = x
#property
def x(self):
return self.__dict__['x']
#x.setter
def x(self, value):
if value < 0:
raise ValueError('Not negatives, please!')
self.__dict__['x'] = value
Now you can do something like
>>> inst = PropDemo()
>>> inst.x
0
>>> inst.x = 3
>>> inst.x
3
If you try to access the property through the class, you can get the underlying descriptor object since it will be an unbound attribute:
>>> PropDemo.x
<property at 0x7f7598af00e8>
On a side note, hiding attributes with the same name as a property in __dict__ is a neat trick that works because data descriptors in a class __dict__ trump entries in the instance __dict__, even though instance __dict__ entries trump non-data-descriptors in a class.
Where it can Get Weird
You can override a class method with an instance method in Python. That would mean that type(foo).bar(foo) and foo.bar() don't call the same underlying function at all. This is irrelevant for magic methods because they always use the former invocation, but it can make a big difference for normal method calls.
There are a few ways to override a method on an instance. The one I find most intuitive is to set the instance attribute to a bound method. Here is an example of a modified billy_cyrus, assuming the definition of Parent in the original question:
def alt_show_info(self):
print('Another version of', self)
billy_cyrus.show_info = alt_show_info.__get__(billy_cyrus, Parent)
In this case, calling the method on the instance vs the class would have completely different results. This only works because methods are non-data descriptors by the way. If they were data descriptors (with a __set__ method), the assignment billy_cyrus.show_info = alt_show_info.__get__(billy_cyrus, Parent) would not override anything but would instead just redirect to __set__, and manually setting it in b
billy_cyrus's __dict__ would just get it ignored, as happens with a property.
Additional Resources
Here are a couple of resources on descriptors:
Python Reference - Descriptor Protocol: http://python-reference.readthedocs.io/en/latest/docs/dunderdsc/
(Official?) Descriptor HowTo Guide: https://docs.python.org/3/howto/descriptor.html
There is no semantic difference between the two. It's entirely a matter of style. You would generally use billy_cyrus.show_info() in normal use, but the fact that the second approach is allowed permits you to use Parent.show_info to get the method as a first-class object itself. If that was not allowed, then it would not be possible (or at least, it would be fairly difficult) to do something like this.
function = Parent.show_info
so_many_billy_cyrus = [billy_cyrus, billy_cyrus, billy_cyrus]
map(function, so_many_billy_cyrus)
How can I quickly disable all methods in a class instance based on a condition? My naive solution is to override using the __getattr__ but this is not called when the function name exists already.
class my():
def method1(self):
print 'method1'
def method2(self):
print 'method2'
def __getattr__(self, name):
print 'Fetching '+str(name)
if self.isValid():
return getattr(self, name)
def isValid(self):
return False
if __name__ == '__main__':
m=my()
m.method1()
The equivalent of what you want to do is actually to override __getattribute__, which is going to be called for every attribute access. Besides it being very slow, take care: by definition of every, that includes e.g. the call to self.isValid within __getattribute__'s own body, so you'll have to use some circuitous route to access that attribute (type(self).isValid(self) should work, for example, as it gets the attribute from the class, not from the instance).
This points to a horrible terminological confusion: this is not disabling "method from a class", but from an instance, and in particular has nothing to do with classmethods. If you do want to work in a similar way on a class basis, rather than an instance basis, you'll need to make a custom metaclass and override __getattribute__ on the metaclass (that's the one that's called when you access attributes on the class -- as you're asking in your title and text -- rather than on the instance -- as you in fact appear to be doing, which is by far the more normal and usual case).
Edit: a completely different approach might be to use a peculiarly Pythonic pathway to implementing the State design pattern: class-switching. E.g.:
class _NotValid(object):
def isValid(self):
return False
def setValid(self, yesno):
if yesno:
self.__class__ = TheGoodOne
class TheGoodOne(object):
def isValid(self):
return True
def setValid(self, yesno):
if not yesno:
self.__class__ = _NotValid
# write all other methods here
As long as you can call setValid appropriately, so that the object's __class__ is switched appropriately, this is very fast and simple -- essentially, the object's __class__ is where all the object's methods are found, so by switching it you switch, en masse, the set of methods that exist on the object at a given time. However, this does not work if you absolutely insist that validity checking must be performed "just in time", i.e. at the very instant the object's method is being looked up.
An intermediate approach between this and the __getattribute__ one would be to introduce an extra level of indirection (which is popularly held to be the solution to all problems;-), along the lines of:
class _Valid(object):
def __init__(self, actualobject):
self._actualobject = actualobject
# all actual methods go here
# keeping state in self._actualobject
class Wrapit(object):
def __init__(self):
self._themethods = _Valid(self)
def isValid(self):
# whatever logic you want
# (DON'T call other self. methods!-)
return False
def __getattr__(self, n):
if self.isValid():
return getattr(self._themethods, n)
raise AttributeError(n)
This is more idiomatic than __getattribute__ because it relies on the fact that __getattr__ is only called for attributes that aren't found in other ways -- so the object can hold normal state (data) in its __dict__, and that will be accessed without any big overhead; only method calls pay the extra overhead of indiretion. The _Valid class instances can keep some or all state in their respective self._actualobject, if any of the state needs to stay accessible on invalid objects (so that the invalid state disable methods, but not data attributes access; it's not clear from your Q if that's needed, but it's a free extra possibility offered by this approach). This idiom is less error-prone than __getattribute__, since state can be accessed more directly in the methods (without triggering validity checks).
As presented, the solution creates a circular reference loop, which may impose a bit of overhead in terms of garbage collection. If that's a problem in your application, use the weakref module from the standard Python library, of course -- that module is generally the simplest way to remove circular loops of references, if and when they're a problem.
(E.g., make the _actualobject attribute of _Valid class instances a weak reference to the object that holds that instance as its _themethods attribute).