Is it possible to make a decorator that makes attributes lazy that do not eval when you try to access it with hasattr()? I worked out how to make it lazy, but hasattr() makes it evaluate prematurely. E.g.,
class lazyattribute:
# Magic.
class A:
#lazyattribute
def bar(self):
print("Computing")
return 5
>>> a = A()
>>> print(a.bar)
'Computing'
5
>>> print(a.bar)
5
>>> b = A()
>>> hasattr(b, 'bar')
'Computing'
5
# Wanted output: 5
It may be difficult to do. From the hasattr documentation:
hasattr(object, name)
The arguments are an object and a string. The result is True if the string is the name of one of the object’s attributes, False if not. (This is implemented by calling getattr(object, name) and seeing whether it raises an exception or not.)
Since attributes may be generated dynamically by a __getattr__ method, there's no other way to reliably check for their presence. For your special situation, maybe testing the dictionaries explicitly would be enough:
any('bar' in d for d in (b.__dict__, b.__class__.__dict__))
What nobody seems to have addressed so far is that perhaps the best thing to do is not to use hasattr(). Instead, go for EAFP (Easier to Ask Forgiveness than Permission).
try:
x = foo.bar
except AttributeError:
# what went in your else-block
...
else:
# what went in your if hasattr(foo, "bar") block
...
This is obviously not a drop-in replacement, and you might have to move stuff around a bit, but it's possibly the "nicest" solution (subjectively, of course).
The problem is that hasattr uses getattr so your attribute is always going to be evaluated when you use hasattr. If you post the code for your lazyattribute magic hopefully someone can suggest an alternative way of testing the presence of the attribute which doesn't require hasattr or getattr. See the help for hasattr:
>>> help(hasattr)
Help on built-in function hasattr in module __builtin__:
hasattr(...)
hasattr(object, name) -> bool
Return whether the object has an attribute with the given name.
(This is done by calling getattr(object, name) and catching exceptions.)
I'm curious why you need something like this. If hasattr ends up calling your "compute function", then so be it. Just how lazy does your property need to be anyway?
Still, here's a rather inelegeant way of doing it by examining the calling function's name. It could probably be coded a little better, but I don't think it should ever be used seriously.
import inspect
class lazyattribute(object):
def __init__(self, func):
self.func = func
def __get__(self, obj, kls=None):
if obj is None or inspect.stack()[1][4][0].startswith('hasattr'):
return None
value = self.func(obj)
setattr(obj, self.func.__name__, value)
return value
class Foo(object):
#lazyattribute
def bar(self):
return 42
orip's answer will be enough only if your object's inheritance has one level of depth.
You should iterate over the method resolution order of object's class to have a complete solution:
from itertools import chain
def lazy_hasattr(obj, name):
# checks for the attribute without triggering __get__
return any(name in d for d in chain((obj.__dict__,),
(c.__dict__ for c in obj.__class__.mro())))
# Usage:
lazy_hasattr(b,'bar')
well the fix is a little hackish but it consists of the following
rename hasattr (say as _hasattr)
rebind hasattr as the following:
def hasattr(obj, name):
try:
return obj._hasattr(name) or _hasattr(obj, name)
except:
return _hasattr(obj, name)
implement class method _hasattr by checking some data structure (ie. array) which is populated with all the lazy attribute names (for an array you'd say: name in lazyAttrArray)
finally somehow have your #lazyattribute decorator add items into some sort of structure (like the array we mentioned above), and then when you call _hasattr then you look in that structure
this is the step that I'm not quite sure how you'd implement 'cause I haven't worked with creating my own decorators
Related
How do I check if an object has some attribute? For example:
>>> a = SomeClass()
>>> a.property
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: SomeClass instance has no attribute 'property'
How do I tell if a has the attribute property before using it?
Try hasattr():
if hasattr(a, 'property'):
a.property
See zweiterlinde's answer below, who offers good advice about asking forgiveness! A very pythonic approach!
The general practice in python is that, if the property is likely to be there most of the time, simply call it and either let the exception propagate, or trap it with a try/except block. This will likely be faster than hasattr. If the property is likely to not be there most of the time, or you're not sure, using hasattr will probably be faster than repeatedly falling into an exception block.
As Jarret Hardie answered, hasattr will do the trick. I would like to add, though, that many in the Python community recommend a strategy of "easier to ask for forgiveness than permission" (EAFP) rather than "look before you leap" (LBYL). See these references:
EAFP vs LBYL (was Re: A little disappointed so far)
EAFP vs. LBYL #Code Like a Pythonista: Idiomatic Python
ie:
try:
doStuff(a.property)
except AttributeError:
otherStuff()
... is preferred to:
if hasattr(a, 'property'):
doStuff(a.property)
else:
otherStuff()
You can use hasattr() or catch AttributeError, but if you really just want the value of the attribute with a default if it isn't there, the best option is just to use getattr():
getattr(a, 'property', 'default value')
I think what you are looking for is hasattr. However, I'd recommend something like this if you want to detect python properties-
try:
getattr(someObject, 'someProperty')
except AttributeError:
print "Doesn't exist"
else
print "Exists"
The disadvantage here is that attribute errors in the properties __get__ code are also caught.
Otherwise, do-
if hasattr(someObject, 'someProp'):
#Access someProp/ set someProp
pass
Docs:http://docs.python.org/library/functions.html
Warning:
The reason for my recommendation is that hasattr doesn't detect properties.
Link:http://mail.python.org/pipermail/python-dev/2005-December/058498.html
According to pydoc, hasattr(obj, prop) simply calls getattr(obj, prop) and catches exceptions. So, it is just as valid to wrap the attribute access with a try statement and catch AttributeError as it is to use hasattr() beforehand.
a = SomeClass()
try:
return a.fake_prop
except AttributeError:
return default_value
I would like to suggest avoid this:
try:
doStuff(a.property)
except AttributeError:
otherStuff()
The user #jpalecek mentioned it: If an AttributeError occurs inside doStuff(), you are lost.
Maybe this approach is better:
try:
val = a.property
except AttributeError:
otherStuff()
else:
doStuff(val)
For objects other than dictonary:
if hasattr(a, 'property'):
a.property
For dictionary, hasattr() will not work.
Many people are telling to use has_key() for dictionary, but it is depreciated.
So for dictionary, you have to use has_attr()
if a.has_attr('property'):
a['property']
Or you can also use
if 'property' in a:
hasattr() is the right answer. What I want to add is that hasattr() can be used well in conjunction with assert (to avoid unnecessary if statements and make the code more readable):
assert hasattr(a, 'property'), 'object lacks property'
print(a.property)
In case that the property is missing, the program will exit with an AssertionError and printing out the provided error message (object lacks property in this case).
As stated in another answer on SO:
Asserts should be used to test conditions that should never happen.
The purpose is to crash early in the case of a corrupt program state.
Often this is the case when a property is missing and then assert is very appropriate.
EDIT:This approach has serious limitation. It should work if the object is an iterable one. Please check the comments below.
If you are using Python 3.6 or higher like me there is a convenient alternative to check whether an object has a particular attribute:
if 'attr1' in obj1:
print("attr1 = {}".format(obj1["attr1"]))
However, I'm not sure which is the best approach right now. using hasattr(), using getattr() or using in. Comments are welcome.
Hope you expecting hasattr(), but try to avoid hasattr() and please prefer getattr(). getattr() is faster than hasattr()
using hasattr():
if hasattr(a, 'property'):
print a.property
same here i am using getattr to get property if there is no property it return none
property = getattr(a,"property",None)
if property:
print property
Depending on the situation you can check with isinstance what kind of object you have, and then use the corresponding attributes. With the introduction of abstract base classes in Python 2.6/3.0 this approach has also become much more powerful (basically ABCs allow for a more sophisticated way of duck typing).
One situation were this is useful would be if two different objects have an attribute with the same name, but with different meaning. Using only hasattr might then lead to strange errors.
One nice example is the distinction between iterators and iterables (see this question). The __iter__ methods in an iterator and an iterable have the same name but are semantically quite different! So hasattr is useless, but isinstance together with ABC's provides a clean solution.
However, I agree that in most situations the hasattr approach (described in other answers) is the most appropriate solution.
Here's a very intuitive approach :
if 'property' in dir(a):
a.property
If a is a dictionary, you can check normally
if 'property' in a:
a.property
This is super simple, just use dir(object)
This will return a list of every available function and attribute of the object.
You can check whether object contains attribute by using hasattr builtin method.
For an instance if your object is a and you want to check for attribute stuff
>>> class a:
... stuff = "something"
...
>>> hasattr(a,'stuff')
True
>>> hasattr(a,'other_stuff')
False
The method signature itself is hasattr(object, name) -> bool which mean if object has attribute which is passed to second argument in hasattr than it gives boolean True or False according to the presence of name attribute in object.
You can use hasattr() to check if object or class has an attribute in Python.
For example, there is Person class as shown below:
class Person:
greeting = "Hello"
def __init__(self, name, age):
self.name = name
self.age = age
def test(self):
print("Test")
Then, you can use hasattr() for object as shown below:
obj = Person("John", 27)
obj.gender = "Male"
print("greeting:", hasattr(obj, 'greeting'))
print("name:", hasattr(obj, 'name'))
print("age:", hasattr(obj, 'age'))
print("gender:", hasattr(obj, 'gender'))
print("test:", hasattr(obj, 'test'))
print("__init__:", hasattr(obj, '__init__'))
print("__str__:", hasattr(obj, '__str__'))
print("__module__:", hasattr(obj, '__module__'))
Output:
greeting: True
name: True
age: True
gender: True
test: True
__init__: True
__str__: True
__module__: True
And, you can also use hasattr() directly for class name as shown below:
print("greeting:", hasattr(Person, 'greeting'))
print("name:", hasattr(Person, 'name'))
print("age:", hasattr(Person, 'age'))
print("gender:", hasattr(Person, 'gender'))
print("test:", hasattr(Person, 'test'))
print("__init__:", hasattr(Person, '__init__'))
print("__str__:", hasattr(Person, '__str__'))
print("__module__:", hasattr(Person, '__module__'))
Output:
greeting: True
name: False
age: False
gender: False
test: True
__init__: True
__str__: True
__module__: True
Another possible option, but it depends if what you mean by before:
undefined = object()
class Widget:
def __init__(self):
self.bar = 1
def zoom(self):
print("zoom!")
a = Widget()
bar = getattr(a, "bar", undefined)
if bar is not undefined:
print("bar:%s" % (bar))
foo = getattr(a, "foo", undefined)
if foo is not undefined:
print("foo:%s" % (foo))
zoom = getattr(a, "zoom", undefined)
if zoom is not undefined:
zoom()
output:
bar:1
zoom!
This allows you to even check for None-valued attributes.
But! Be very careful you don't accidentally instantiate and compare undefined multiple places because the is will never work in that case.
Update:
because of what I was warning about in the above paragraph, having multiple undefineds that never match, I have recently slightly modified this pattern:
undefined = NotImplemented
NotImplemented, not to be confused with NotImplementedError, is a built-in: it semi-matches the intent of a JS undefined and you can reuse its definition everywhere and it will always match. The drawbacks is that it is "truthy" in booleans and it can look weird in logs and stack traces (but you quickly get over it when you know it only appears in this context).
class a_class:
def __getattr__(self, name):
# if called by hasattr(a, 'b') not by a.b
# print("I am called by hasattr")
print(name)
a = a_class()
a.b_attr
hasattr(a, 'c_attr')
Please take a look the comment inside __getattr__. How do I do that? I am using Python 3. The reason is I want to create attribute dynamically but I don't want to do that when using hasattr. Thanks.
You can't, without cheating. As the documentation says:
This [that is, hasattr] is implemented by calling getattr(object, name) and seeing whether it raises an exception or not.
In other words, you can't block hasattr without also blocking getattr, which basically means you can't block hasattr at all if you care about accessing attributes.
By "cheating" I mean one of these solutions that clever people like to post on here that involve an end-run around essentially all of Python. They typically involve reassigning builtins, inspecting/manipulating the call stack, using introspection to peek at the literal source code, modifying "secret" internal attributes of objects, and so on. For instance, you could look at the call stack to see if hasattr is in the call chain. This type of solution is possible, but extremely fragile, with possibility of breaking in future Python versions, on non-CPython implementations, or in situations where another equally ugly and devious hack is also being used.
You can see a similar question and some discussion here.
This discussion applies to Python 3. (turns out it works on Python 2.7 as well)
Not exactly the way you described but the following points might help:
__getattr__ will only be accessed when attribute is not found under normal way
hasattr() check if AttributeError is raised
See if the following code help!
>>> class A:
... def __init__(self, a=1, b=2):
... self.a = a
... self.b = b
...
... def __getattr__(self, name):
... print('calling __getattr__')
... print('This is instance attributes: {}'.format(self.__dict__))
...
... if name not in ('c', 'd'):
... raise AttributeError()
... else:
... return 'My Value'
... return 'Default'
>>>
>>> a = A()
>>> print('a = {}'.format(a.a))
a = 1
>>> print('c = {}'.format(a.c))
calling __getattr__
This is instance attributes: {'a': 1, 'b': 2}
c = My Value
>>> print('hasattr(a, "e") returns {}'.format(hasattr(a, 'e')))
calling __getattr__
This is instance attributes: {'a': 1, 'b': 2}
hasattr(a, "e") returns False
>>>
This question is meant to be more about __dir__ than about numpy.
I have a subclass of numpy.recarray (in python 2.7, numpy 1.6.2), and I noticed recarray's field names are not listed when diring the object (and therefore ipython's autocomplete doesn't work).
Trying to fix it, I tried overriding __dir__ in my subclass, like this:
def __dir__(self):
return sorted(set(
super(MyRecArray, self).__dir__() + \
self.__dict__.keys() + self.dtype.fields.keys()))
which resulted with: AttributeError: 'super' object has no attribute '__dir__'.
(I found here this should actually work in python 3.3...)
As a workaround, I tried:
def __dir__(self):
return sorted(set(
dir(type(self)) + \
self.__dict__.keys() + self.dtype.fields.keys()))
As far as I can tell, this one works, but of course, not as elegantly.
Questions:
Is the latter solution correct in my case, i.e. for a subclass of recarray?
Is there a way to make it work in the general case? It seems to me it wouldn't work with multiple inheritance (breaking the super-call chain), and of course, for objects with no __dict__...
Do you know why recarray does not support listing its field names to begin with? mere oversight?
Python 2.7+, 3.3+ class mixin that simplifies implementation of __dir__ method in subclasses. Hope it will help. Gist.
import six
class DirMixIn:
""" Mix-in to make implementing __dir__ method in subclasses simpler
"""
def __dir__(self):
if six.PY3:
return super(DirMixIn, self).__dir__()
else:
# code is based on
# http://www.quora.com/How-dir-is-implemented-Is-there-any-PEP-related-to-that
def get_attrs(obj):
import types
if not hasattr(obj, '__dict__'):
return [] # slots only
if not isinstance(obj.__dict__, (dict, types.DictProxyType)):
raise TypeError("%s.__dict__ is not a dictionary"
"" % obj.__name__)
return obj.__dict__.keys()
def dir2(obj):
attrs = set()
if not hasattr(obj, '__bases__'):
# obj is an instance
if not hasattr(obj, '__class__'):
# slots
return sorted(get_attrs(obj))
klass = obj.__class__
attrs.update(get_attrs(klass))
else:
# obj is a class
klass = obj
for cls in klass.__bases__:
attrs.update(get_attrs(cls))
attrs.update(dir2(cls))
attrs.update(get_attrs(obj))
return list(attrs)
return dir2(self)
Have you tried:
def __dir__(self):
return sorted(set(
dir(super(MyRecArray, self)) + \
self.__dict__.keys() + self.dtype.fields.keys()))
and 3: Yes your solution is correct. recarray does not define __dir__ simply because the default implementation was okay, so they didn't bother implementing it, and numpy's devs did not design the class to be subclassed, so I don't see why they should have bothered.
It's often a bad idea to subclass built-in types or classes that are not specifically designed for inheritance, thus I'd suggest you to use delegation/composition instead of inheritance, except if there is a particular reason(e.g. you want to pass it to a numpy function that excplicitly checks with isinstance).
No. As you pointed out in python3 they changed the implementation so that there is an object.__dir__, but on other python versions I can't see anything that you can do. Also, again, using recarray with multiple-inheritance is simply crazy, things will break. Multiple-inheritance should be carefully designed, and usually classes are specifically designed to be used with it(e.g. mix-ins). So I wouldn't bother treating this case, since whoever tries it will be bitten by other problems.
I don't see why you should care for classes that do not have __dict__... since your subclass has it how should it break? When you'll change the subclass implementation, e.g. using __slots__ you could easily change the __dir__ also. If you want to avoid redefining __dir__ you can simply define a function that checks for __dict__ then for __slots__ etc. Note however that attributes can be generated in subtle ways with __getattr__ and __getattribute__ and thus you simply can't reliably catch all of them.
Official docs says:
If the object has a method named __dir__(), this method will be called
and must return the list of attributes. This allows objects that
implement a custom __getattr__() or __getattribute__() function to
customize the way dir() reports their attributes.
If custom __dir__ implemented, results, returning by another function, inspect.getmembers(), also affected.
For example:
class С(object):
__slots__ = ['atr']
def __dir__(self):
return ['nothing']
def method(self):
pass
def __init__(self):
self.atr = 'string'
c = C()
print dir(f) #If we try this - well get ['nothing'] returned by custom __dir__()
print inspect.getmembers(f) #Here we get []
print f.__dict__ #And here - exception will be raised because of __slots__
How in this case list of names of object might be getted?
Answer to original question- does inspect.getmembers() use __dir__() like dir() does?
Here's the source code for inspect.getmembers() so we can see what it's really doing:
def getmembers(object, predicate=None):
"""Return all members of an object as (name, value) pairs sorted by name.
Optionally, only return members that satisfy a given predicate."""
results = []
for key in dir(object):
try:
value = getattr(object, key)
except AttributeError:
continue
if not predicate or predicate(value):
results.append((key, value))
results.sort()
return results
From this we see that it is using dir() and just filtering the results a bit.
How to get attributes with an overridden __dir__()?
According to this answer, it isn't possible to always get a complete list of attributes, but we can still definitely get them in some cases/get enough to be useful.
From the docs:
If the object does not provide __dir__(), the function tries its best
to gather information from the object’s __dict__ attribute, if
defined, and from its type object. The resulting list is not
necessarily complete, and may be inaccurate when the object has a
custom __getattr__().
So if you are not using __slots__, you could look at your object's __dict__ (and it's type object's) to get basically the same info that dir() would normally give you. So, just like with dir(), you would have to use a more rigorous method to get metaclass methods.
If you are using __slots__, then getting class attributes is, in a way, a bit more simple. Yes, there's no dict, but there is __slots__ itself, which contains the names of all of the attributes. For example, adding print c.__slots__ to your example code yields ['atr']. (Again, a more rigorous approach is needed to get the attributes of superclasses as well.)
How to get methods
You might need a different solution depending on the use case, but if you just want to find out the methods easily, you can simply use the builtin help().
Modified PyPy dir()
Here's an alternative to some of the above: To get a version of dir() that ignores user-defined __dir__ methods, you could just take PyPy's implementation of dir() and delete the parts that reference __dir__ methods.
As Matthew pointed out in the other answer, getmembers apparently returns the subset of dir results that are actual attributes.
>>> class C:
>>> def foo(self):
>>> pass
>>> def __dir__(self):
>>> return ['test']
>>>
>>> import inspect
>>> c = C()
>>> dir(c)
['test']
>>> inspect.getmembers(c)
[]
How can I get the list of class functions from within __getattr__ function?
Python v2.7 if it matters.
Trying to use dir within __getattr__ leads to infinite recursion.
class Hal(object):
def __getattr__(self, name):
print 'I don\'t have a %s function' % name
names = dir(self) # <-- infinite recursion happens here
print 'My functions are: %s' % ', '.join(names)
exit()
def close_door(self):
pass
x = Hal()
x.open_door()
Here's the output I want:
I don't have a open_door function
My functions are: close_door, __getattr__, __init__, __doc__, ...
Any other solution which gets me the output I want will work fine. I want to do fuzzy string matching in the case when a function doesn't exist to try to suggest what the user might have meant.
This works I think:
import types
class Hal(object):
def __getattr__(self, name):
print ('I don\'t have a %s function' % name)
funcs = (name for name,func in self.__class__.__dict__.items() if isinstance(func,types.FunctionType))
#names = dir(self) # <-- infinite recursion happens here
print ('My functions are: %s' % ', '.join(str(f) for f in funcs))
exit()
#staticmethod
def foo():
pass
#classmethod
def bar(cls):
pass
def qux(self):
pass
def close_door(self):
pass
x = Hal()
x.foo = 'bar'
x.open_door()
is there any reason why you can't do this?
names = dir(self.__class__)
are you expecting consumers to extend instances of Hal to have custom methods?
if you only want methods you've implemented, with no built-ins listed, you could try this too:
names = [prop for prop in dir(self.__class__) if prop[1] != "_"]
names = self.__class__.__dict__
possibly?
>>> class A:
... def hello(self,x):
... print "hello ",x
... def my_dir(self):
... print self.__class__.__dict__
...
>>> A().my_dir()
{'__module__': '__main__', 'my_dir': <function my_dir at 0x029A5AB0>, 'hello': <
function hello at 0x029A5CB0>, '__doc__': None}
One solution is to make a copy of the dir before adding the __getattr__ method:
class Hal(object):
def __init__(self):
self._names = dir(self)
def __getattr__(self, name):
print self.names
self.__getattr__ = __getattr__
However, for simple cases, you can just call dir (and likewise getattr, or inspect.getmembers, or whatever) on your class object to solve the problem. This doesn't work if instance can have methods added after construction, but if that's not an issue, it's easy:
names = dir(self.__class__)
However you get the names, to filter for methods, there are a few things to do.
First, you can use isinstance on getattr(self, name) and make sure it's a method-wrapper (or get the type of the bound version and make sure it's an instancemethod). If you get the values directly out of self.__class__.__dict__, you don't get exactly the same thing as if you get the names in your favorite way and call either getattr(self, name) or getattr(self.__class__, name). In particular, an instance method will show up as a function, which is easier to test for than a method-wrapper. Although some of the other cases now get harder to detect.
At any rate, nothing based on type will find things that act like methods but aren't (e.g., because you've assigned a built-in function directly to the object, wrapped something in certain kinds of decorators, written custom descriptors, used a class with a __callable__ method as a function, etc.). If you're doing anything fancy (or worry that someone might later add something fancy), you really need to test whether you can explicitly bind the member (or fake-bind it to None), and then check if the result is callable, and then possibly do further tests to make sure it's callable properly (because otherwise you'll get fooled by #staticmethods and similar things). Really, if this comes up (and you've really thought through your design and convinced yourself and at least one other person that it isn't insane…), you should test everything you can think of against every case you have…
If you want to know if the methods are defined in Hal or the instance as opposed to object or another base class, there are a few ways to do this, but the simplest is to just subtract out the members of the base classes. (Of course if you don't care about methods defined in the instance, Hal.__dict__ already has what you want.)