Is there any way to determine the difference between a method and an attribute call using __getattr__?
I.e. in:
class Bar(object):
def __getattr__(self, name):
if THIS_IS_A_METHOD_CALL:
# Handle method call
def method(**kwargs):
return 'foo'
return method
else:
# Handle attribute call
return 'bar'
foo=Bar()
print(foo.test_method()) # foo
print(foo.test_attribute) # bar
The methods are not local so it's not possible to determine it using getattr/callable. I also understand that methods are attributes, and that there might not be a solution. Just hoping there is one.
You cannot tell how an object is going to used in the __getattr__ hook, at all. You can access methods without calling them, store them in a variable, and later call them, for example.
Return an object with a __call__ method, it'll be invoked when called:
class CallableValue(object):
def __init__(self, name):
self.name = name
def __call__(self, *args, **kwargs):
print "Lo, {} was called!".format(self.name)
class Bar(object):
def __getattr__(self, name):
return CallableValue(name)
but instances of this will not be the same thing as a string or a list at the same time.
Demo:
>>> class CallableValue(object):
... def __init__(self, name):
... self.name = name
... def __call__(self, *args, **kwargs):
... print "Lo, {} was called!".format(self.name)
...
>>> class Bar(object):
... def __getattr__(self, name):
... return CallableValue(name)
...
>>> b = Bar()
>>> something = b.test_method
>>> something
<__main__.CallableValue object at 0x10ac3c290>
>>> something()
Lo, test_method was called!
In short, no, there is no reliable way - the issue is that a method is an attribute in Python - there is no distinction made. It just happens to be an attribute that is a bound method.
You can check if the attribute is a method, but there is no guarantee that means it will be called, e.g:
class Test:
def test(self):
...
Test().test # This accesses the method, but doesn't call it!
There is no way for the call accessing the function to know if it's going to be called when it is returned - that's a future event that hasn't yet been processed.
If you are willing to assume that a method being accessed is a method being called, you can determine that it is a method being accessed with a check like this:
hasattr(value, "__self__") and value.__self__ is self
Where value is the attribute you want to check to see if it is a method or some other attribute, and self is the instance you want to see if it's a method for.
If you need something to happen when it is called, you could use this moment to decorate the function.
An solid code example of this can be found here.
It is not possible to know whether an attribute is subsequently called from __getattr__, which Martijn Pieters explains.
Although it does not determine whether an attribute is called, it is possible to know whether an attribute can be called with callable. Another way is to use type to keep track of the various objects, or make a list of attribute names.
class Foo(object):
bar_attribute = 'callable'
def __getattr__(self, name):
instanceOrValue = getattr(self, "bar_%s" %name)
if callable(instanceOrValue):
# Handle object that can be called
def wrap(**kwargs):
return "is %s" %instanceOrValue(**kwargs)
return wrap
# Handle object that can not be called
return 'not %s' %instanceOrValue
def bar_method(self, **kwargs):
return 'callable';
foo=Foo()
print(foo.method()) # is callable
print(foo.attribute) # not callable
__getattr__ can only keep track of certain things, but it can be the right solution in many situations because change calling (__call__) has an impact on calling in all situations and not only when the Foo class is used.
Difference between calling a method and accessing an attribute
What is a "callable"?
Python __call__ special method practical example
Related
I have a decorator in my library which takes a user's class and creates a new version of it, with a new metaclass, it is supposed to completely replace the original class. Everything works; except for super() calls:
class NewMeta(type):
pass
def deco(cls):
cls_dict = dict(cls.__dict__)
if "__dict__" in cls_dict:
del cls_dict["__dict__"]
if "__weakref__" in cls_dict:
del cls_dict["__weakref__"]
return NewMeta(cls.__name__, cls.__bases__, cls_dict)
#deco
class B:
def x(self):
print("Hi there")
#deco
class A(B):
def x(self):
super().x()
Using this code like so, yields an error:
>>> a = A()
>>> a.x()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 4, in x
TypeError: super(type, obj): obj must be an instance or subtype of type
Some terminology:
The source code class A as produced by class A(B):.
The produced class A*, as produced by NewMeta(cls.__name__, cls.__bases__, cls_dict).
A is established by Python to be the type when using super inside of the methods of A*. How can I correct this?
There's some suboptimal solutions like calling super(type(self), self).x, or passing cls.__mro__ instead of cls.__bases__ into the NewMeta call (so that obj=self always inherits from the incorrect type=A). The first is unsustainable for end users, the 2nd pollutes the inheritance chains and is confusing as the class seems to inherit from itself.
Python seems to introspect the source code, or maybe stores some information to automatically establish the type, and in this case, I'd say it is failing to do so;
How could I make sure that inside of the methods of A A* is established as the type argument of argumentless super calls?
The argument-free super uses the __class__ cell, which is a regular function closure.
Data Model: Creating the class object
__class__ is an implicit closure reference created by the compiler if any methods in a class body refer to either __class__ or super.
>>> class E:
... def x(self):
... return __class__ # return the __class__ cell
...
>>> E().x()
__main__.E
>>> # The cell is stored as a __closure__
>>> E.x.__closure__[0].cell_contents is E().x() is E
True
Like any other closure, this is a lexical relation: it refers to class scope in which the method was literally defined. Replacing the class with a decorator still has the methods refer to the original class.
The simplest fix is to explicitly refer to the name of the class, which gets rebound to the newly created class by the decorator.
#deco
class A(B):
def x(self):
super(A, self).x()
Alternatively, one can change the content of the __class__ cell to point to the new class:
def deco(cls):
cls_dict = dict(cls.__dict__)
cls_dict.pop("__dict__", None)
cls_dict.pop("__weakref__", None)
new_cls = NewMeta(cls.__name__, cls.__bases__, cls_dict)
for method in new_cls.__dict__.values():
if getattr(method, "__closure__", None) and method.__closure__[0].cell_contents is cls:
method.__closure__[0].cell_contents = new_cls
return new_cls
I was writing a simple implementation of classmethod decorator for a better understanding of both decorator and classmethod. Here is the problem I faced. When I called the class method with an instance of my class, everything was fine, but calling the method with the class object fails with error:
>>**TypeError: wrapper() missing 1 required positional argument: 'cls'**
and when I called the method with the explicit class object as parameter it succeed, but a call to classmethod from class object should pass the class object itself as the first parameter, right?
import functools
import inspect
def myclassmethod(meth):
#functools.wraps(meth)
def wrapper(cls, *args, **kwargs):
#print(f'obj:{cls}, cls:{cls.__class__}, isclass:{inspect.isclass(cls)}')
return meth(cls if inspect.isclass(cls) else cls.__class__, *args, **kwargs)
return wrapper
class MyDecoratedMethods(object):
_name = 'ClassName'
def __init__(self):
self._name = 'InstanceName'
def __repr__(self):
return f'{self.__class__.__name__}({self._name!r})'
#myclassmethod
def classname(cls):
return cls._name
MyDecoratedMethods().classname()
#MyDecoratedMethods.classname()
MyDecoratedMethods.classname(MyDecoratedMethods) # This works
To see what's going on, I removed the #functools.wraps(meth) line and then ran:
print(MyDecoratedMethods.classname)
# <function __main__.myclassmethod.<locals>.wrapper(cls, *args, **kwargs)>
This shows us, that MyDecoratedMethods.classname is simply the function you created inside your decorator. And this function knows nothing about the class it is called from.
However, we can override this behavior with Descriptors. A descriptor "knows" when it is accessed from a class or an instance and, most importantly, can distinguish between those cases (which is how regular methods are created).
Here is a first try:
class ClassMethod:
def __init__(self, function):
self.function = function
def __get__(self, instance, cls):
print(cls, instance)
class MyDecoratedMethods(object):
...
#ClassMethod
def classname(cls):
...
MyDecoratedMethods.classname
# prints <class '__main__.MyDecoratedMethods'> None
MyDecoratedMethods().classname
# prints <class '__main__.MyDecoratedMethods'> <__main__.MyDecoratedMethods object ...>
So we see that accessing the class method from the class sets instance to None and accessing it from an instance sets instance to that very object.
But actually we don't need the instance at all. We can implement the logic without it.
from functools import partial
class ClassMethod:
def __init__(self, function):
self.function = function
def __get__(self, instance, cls):
# create a new function and set cls to the first argument
return partial(self.function, cls)
...
MyDecoratedMethods().classname()
# "ClassName"
MyDecoratedMethods.classname()
# "ClassName"
And we are done. Our custom descriptor accomplished two things:
It prevented the function from binding the instance to the first argument when calling it from an instance of the class (like a function would normally do to become a method)
It always binds the class to the first argument when it is accessed from either the class or an instance.
Side note: Your approach to check whether an instance or the class called the function is also flawed (inspect.isclass(cls)). It will work for "normal" classes but not for meta classes, because then inspect.isclass returns True for the class and its instances.
Say I have a class which contains several functions.
class Person:
def __init__(self): pass
def say(self, speech): pass
def walk(self, destination): pass
def jump(self): pass
When the user instantiates a Person, I'd like them to be able to call any method of the class. If the requested method does not exist (e.g. Person.dance()), a default function should be called instead.
I imagine that this could be done via a theoretical magic method -
class Person:
def __init__(self): pass
def say(self, speech): pass
def walk(self, destination): pass
def jump(self): pass
def sleep(self): print("Zzz")
def __method__(self, func):
if func.__name__ not in ['say','walk','jump']:
return self.sleep
else
return func
billy = Person()
billy.dance()
>> "Zzz"
However, I know of no such magic method.
Is there a way to make non-existent methods within a class redirect to another class?
It's important that the end-user doesn't have to do anything - from their perspective, it should just work.
The standard way to catch an undefined attribute is to use __getattr__:
# But see the end of the answer for an afterthought
def __getattr__(self, attr):
return self.sleep
Python does not differentiate between "regular" attributes and methods; a method call starts with an ordinary attribute lookup, whose result just happens to be callable. That is,
billy.run()
is the same as
f = billy.run
f()
This means that __getattr__ will be invoked for any undefined attribute; there is no way to tell at lookup time whether the result is going to be called or not.
However, if all you want is to define "aliases" for a common method, you can do that with a loop after the class statement.
class Person:
def __init__(self): pass
def say(self, speech): pass
def walk(self, destination): pass
def jump(self): pass
def do_all(self): pass
for alias in ["something", "something_else", "other"]:
setattr(Person, alias, Person.do_all)
You can also make hard-coded assignments in the class statement, but that would be unwieldy if there are, as you mention, hundreds of such cases:
class Person:
def do_all(self): pass
something = do_all
something_else = do_all
(I did not experiment with using exec to automate such assignments in a loop; it might be possible, though not recommended.)
You can also embed the list of aliases in the definition of __getattr__, come to think of it:
def __getattr__(self, attr):
if attr in ["something", "something_else", "other"]:
return self.sleep
else:
raise AttributeError(f"type object 'Person' has no attribute '{attr}'")
Your users might find the API behavior confusing. However, if you're sure you need this pattern, you can try something like
# getattr will get the function attribute by a string name version
# None is returned if no function is found
my_func = getattr(self, 'my_func', None)
# callable ensures `my_func` is actually a function and not a generic attribute
# Add your if-else logic here
if callable(my_func):
my_func(args)
else:
...
You could nest your "default" function inside __getattr__ in order to gain access to the called non-existent method's name and arguments.
class Test:
def __getattr__(self, attr):
def default(*args, **kwargs):
return attr, args, kwargs
return default
test = Test()
print(test.test('test'))
# ('test', ('test',), {})
I have a class who's job is to wrap another class (code I don't control), intercept all calls to the wrapped class, perform some logic, and pass along the call to the underlying class. Here's an example:
class GithubRepository(object):
def get_commit(self, sha):
return 'Commit {}'.format(sha)
def get_contributors(self):
return ['bobbytables']
class LoggingGithubRepositoryWrapper(object):
def __init__(self, github_repository):
self._github_repository = github_repository
def __getattr__(self, name):
base_func = getattr(self._github_repository, name)
def log_wrap(*args, **kwargs):
print "Calling {}".format(name)
return base_func(*args, **kwargs)
return log_wrap
if __name__ == '__main__':
git_client = LoggingGithubRepositoryWrapper(GithubRepository())
print git_client.get_commit('abcdef1245')
print git_client.get_contributors()
As you can see, the way that I do this is by implementing __getattr__ on the wrapping class and delegating to the underlying class. The downside to this approach is that users of LoggingGithubRepositoryWrapper don't know which attributes/methods the underlying GithubRepository actually has.
This leads me to my question: is there a way to define or document the calls handled by __getattr__? Ideally, I'd like to be able to autocomplete on git_client. and be provided a list of supported methods. Thanks for your help in advance!
You can do this a few different ways, but they wont involve the use of __getattr__.
What you really need to do is dynamically create your class, or at least dynamically create the wrapped functions on your class. There are a few ways to do this in python.
You could build the class definition using type() or metaclasses, or build it on class instantiation using the __new__ method.
Every time you call LoggingGithubRepositoryWrapper(), the __new__ method will be called. Here, it looks at all the attributes on the github_repository argument and finds all the non-private methods. It then creates a function on the instantiated LoggingGithubRepositoryWrapper class instance that wraps the repo call in a logging statement.
At the end, it passes back the modified class instance. Then __init__ is called.
from types import MethodType
class LoggingGithubRepositoryWrapper(object):
def __new__(cls, github_repository):
self = super(LoggingGithubRepositoryWrapper, cls).__new__(cls)
for name in dir(github_repository):
if name.startswith('__'):
continue
func = getattr(github_repository, name)
if isinstance(func, MethodType):
setattr(self, name, cls.log_wrap(func))
return self
#staticmethod
def log_wrap(func):
def wrap(*args, **kwargs):
print 'Calling {0}'.format(func.__name__)
return func(*args, **kwargs)
return wrap
def __init__(self, github_repository):
... # this is all the same
This doesn't work:
def register_method(name=None):
def decorator(method):
# The next line assumes the decorated method is bound (which of course it isn't at this point)
cls = method.im_class
cls.my_attr = 'FOO BAR'
def wrapper(*args, **kwargs):
method(*args, **kwargs)
return wrapper
return decorator
Decorators are like the movie Inception; the more levels in you go, the more confusing they are. I'm trying to access the class that defines a method (at definition time) so that I can set an attribute (or alter an attribute) of the class.
Version 2 also doesn't work:
def register_method(name=None):
def decorator(method):
# The next line assumes the decorated method is bound (of course it isn't bound at this point).
cls = method.__class__ # I don't really understand this.
cls.my_attr = 'FOO BAR'
def wrapper(*args, **kwargs):
method(*args, **kwargs)
return wrapper
return decorator
The point of putting my broken code above when I already know why it's broken is that it conveys what I'm trying to do.
I don't think you can do what you want to do with a decorator (quick edit: with a decorator of the method, anyway). The decorator gets called when the method gets constructed, which is before the class is constructed. The reason your code isn't working is because the class doesn't exist when the decorator is called.
jldupont's comment is the way to go: if you want to set an attribute of the class, you should either decorate the class or use a metaclass.
EDIT: okay, having seen your comment, I can think of a two-part solution that might work for you. Use a decorator of the method to set an attribute of the method, and then use a metaclass to search for methods with that attribute and set the appropriate attribute of the class:
def TaggingDecorator(method):
"Decorate the method with an attribute to let the metaclass know it's there."
method.my_attr = 'FOO BAR'
return method # No need for a wrapper, we haven't changed
# what method actually does; your mileage may vary
class TaggingMetaclass(type):
"Metaclass to check for tags from TaggingDecorator and add them to the class."
def __new__(cls, name, bases, dct):
# Check for tagged members
has_tag = False
for member in dct.itervalues():
if hasattr(member, 'my_attr'):
has_tag = True
break
if has_tag:
# Set the class attribute
dct['my_attr'] = 'FOO BAR'
# Now let 'type' actually allocate the class object and go on with life
return type.__new__(cls, name, bases, dct)
That's it. Use as follows:
class Foo(object):
__metaclass__ = TaggingMetaclass
pass
class Baz(Foo):
"It's enough for a base class to have the right metaclass"
#TaggingDecorator
def Bar(self):
pass
>> Baz.my_attr
'FOO BAR'
Honestly, though? Use the supported_methods = [...] approach. Metaclasses are cool, but people who have to maintain your code after you will probably hate you.
Rather than use a metaclass, in python 2.6+ you should use a class decorator. You can wrap the function and class decorators up as methods of a class, like this real-world example.
I use this example with djcelery; the important aspects for this problem are the "task" method and the line "args, kw = self.marked[klass.dict[attr]]" which implicitly checks for "klass.dict[attr] in self.marked". If you want to use #methodtasks.task instead of #methodtasks.task() as a decorator, you could remove the nested def and use a set instead of a dict for self.marked. The use of self.marked, instead of setting a marking attribute on the function as the other answer did, allows this to work for classmethods and staticmethods which, because they use slots, won't allow setting arbitrary attributes. The downside of doing it this way is that the function decorator MUST go above other decorators, and the class decorator MUST go below, so that the functions are not modified / re=wrapped between one and the other.
class DummyClass(object):
"""Just a holder for attributes."""
pass
class MethodTasksHolder(object):
"""Register tasks with class AND method decorators, then use as a dispatcher, like so:
methodtasks = MethodTasksHolder()
#methodtasks.serve_tasks
class C:
#methodtasks.task()
##other_decorators_come_below
def some_task(self, *args):
pass
#methodtasks.task()
#classmethod
def classmethod_task(self, *args):
pass
def not_a_task(self):
pass
#..later
methodtasks.C.some_task.delay(c_instance,*args) #always treat as unbound
#analagous to c_instance.some_task(*args) (or C.some_task(c_instance,*args))
#...
methodtasks.C.classmethod_task.delay(C,*args) #treat as unbound classmethod!
#analagous to C.classmethod_task(*args)
"""
def __init__(self):
self.marked = {}
def task(self, *args, **kw):
def mark(fun):
self.marked[fun] = (args,kw)
return fun
return mark
def serve_tasks(self, klass):
setattr(self, klass.__name__, DummyClass())
for attr in klass.__dict__:
try:
args, kw = self.marked[klass.__dict__[attr]]
setattr(getattr(self, klass.__name__), attr, task(*args,**kw)(getattr(klass, attr)))
except KeyError:
pass
#reset for next class
self.marked = {}
return klass