I want to be able use python descriptors in a class which has the slots optimization:
class C(object):
__slots__ = ['a']
a = MyDescriptor('a')
def __init__(self, val):
self.a = val
The problem I have is how to implement the descriptor class in order to be able to store values in the class instance which invokes the descriptor object. The usual solution would look like the one below but will not work since "dict" is no longer defined when "slots" is invoked in the C class:
class MyDescriptor(object):
__slots__ = ['name']
def __init__(self, name_):
self.name = name_
def __get__(self, instance, owner):
if self.name not in instance.__dict__:
raise AttributeError, self.name
return instance.__dict__[self.name]
def __set__(self, instance, value):
instance.__dict__[self.name] = value
Don't declare the same name as a slot and as an instance method. Use different names, and access the slot as an attribute, not via __dict__.
class MyDescriptor(object):
__slots__ = ['name']
def __init__(self, name_):
self.name = name_
def __get__(self, instance, owner):
return getattr(instance, self.name)
def __set__(self, instance, value):
setattr(instance, self.name, value)
class C(object):
__slots__ = ['_a']
a = MyDescriptor('_a')
def __init__(self, val):
self.a = val
foo = C(1)
print foo.a
foo.a = 2
print foo.a
Though of dubious value, the following trick will work, if it is ok to put the first __slots__ in a subclass.
class A( object ):
__slots__ = ( 'a', )
class B( A ):
__slots__ = ()
#property
def a( self ):
try:
return A.a.__get__( self )
except AttributeError:
return 'no a set'
#a.setter
def a( self, val ):
A.a.__set__( self, val )
(You can use your own descriptor rather than property.) With these definitions:
>>> b = B()
>>> b.a
'no a set'
>>> b.a = 'foo'
>>> b.a
'foo'
As far as I understand, __slots__ is implemented with its own descriptor, so another descriptor after __slots__ in the same class would just overwrite. If you want to elaborate this technique, you could search for a candidate descriptor in self.__class__.__mro__ (or starting with instance in your own __get__).
Postscript
Ok ... well if you really want to use one class, you can use the following adaptation:
class C( object ):
__slots__ = ( 'c', )
class MyDescriptor( object ):
def __init__( self, slots_descriptor ):
self.slots_descriptor = slots_descriptor
def __get__( self, inst, owner = None ):
try:
return self.slots_descriptor.__get__( inst, owner )
except AttributeError:
return 'no c'
def __set__( self, inst, val ):
self.slots_descriptor.__set__( inst, val )
C.c = MyDescriptor( C.c )
If you insist on inscrutability, you can make the assignment in a metaclass or a class decorator.
The #Glenn Maynard's answer is the good one.
But I would like to point at a problem in the OP's question (I can't add a comment to his question since I havn't enough reputation yet):
The following test is raising an error when the instance hasn't a __dict__ variable:
if self.name not in instance.__dict__:
So, here is an a generic solution that tries to acces to the __dict__ variable first (which is the default anyway) and, if it fails, use getattr and setattr:
class WorksWithDictAndSlotsDescriptor:
def __init__(self, attr_name):
self.attr_name = attr_name
def __get__(self, instance, owner):
try:
return instance.__dict__[self.attr_name]
except AttributeError:
return getattr(instance, self.attr_name)
def __set__(self, instance, value):
try:
instance.__dict__[self.attr_name] = value
except AttributeError:
setattr(instance, self.attr_name, value)
(Works only if the attr_name is not the same as the real instance variable's name, or you will have a RecursionError as pointed to in the accepted answer)
(Won't work as expected if there is both __slots__ AND __dict__)
Hope this helps.
Related
I trying to implement a class property where the setter can only be called once and am wondering how best to achieve this ? And how to make it most 'pythonic'?
Options I have considered:
Subclass and extend the builtin property.
Decorate the setter of a property.
Add an attribute which persists how often each setter has been set.
Any other ideas?
And suggestions how to best implement?
If you are using it a lot, along with other property functionality, subclassing property is appropriate.
It is a bit tricky, due to the way property works - when one calls
#prop.setter, a new instance of the property is created. The subclass bellow will work.
class FuseProperty(property):
def setter(self, func):
def fuse(instance, value):
name = f"_fuse_{self.fget.__name__}"
if not getattr(instance, name, False):
func(instance, value)
setattr(instance, name, True)
return super().setter(lambda instance, value: fuse(instance, value))
Here is it in use.
In [24]: class A:
...: #FuseProperty
...: def a(self):
...: return self._a
...: #a.setter
...: def a(self, value):
...: self._a = value
...:
In [25]: a = A()
In [26]: a.a = 23
In [27]: a.a
Out[27]: 23
In [28]: a.a = 5
In [29]: a.a
Out[29]: 23
However, if this "fuse" property is all you need, and no other code is being added to the getters and setters, it can be much simpler: you can create a brand new "Descriptor" class, using the same mechanisms used by property - this can be much better, as your "fuse" properties can be built in a single line, with no need for a setter and getter methods.
All that is needed is a class with __get__ and __set__ methods - we can add __set_name__ to get the new property name automatically (which property itself does not, so we get the name from the fget method above)
class FuseAttribute:
def __set_name__(self, owner, name):
self.name = name
def __get__(self, instance, owner):
if instance is None:
return self
return getattr(instance, f"_{self.name}")
def __set__(self, instance, value):
if not getattr(instance, f"_fuse_{self.name}", False):
setattr(instance, f"_{self.name}", value)
# add an else clause for optionally raising an error
setattr(instance, f"_fuse_{self.name}", True)
And using it:
In [36]: class A:
...: a = FuseAttribute()
...:
In [37]: a = A()
In [38]: a.a = 23
In [39]: a.a
Out[39]: 23
In [40]: a.a = 5
In [41]: a.a
Out[41]: 23
Properties in Python are just descriptors, and it's relatively easy to implement your own that does exactly what you want:
class SetOnceProperty:
def __init__(self, name):
self.storage_name = '_' + name
def __get__(self, obj, owner=None):
return getattr(obj, self.storage_name)
def __set__(self, obj, value):
if hasattr(obj, self.storage_name):
raise RuntimeError(f'{self.storage_name[1:]!r} property already set.')
setattr(obj, self.storage_name, value)
def __delete___(self, obj):
delattr(obj, self.storage_name)
class Test:
test_attr = SetOnceProperty('test_attr')
def __init__(self, value):
self.test_attr = value*2 # Sets property.
test = Test(21)
print(test.test_attr) # -> 42
test.test_attr = 13 # -> RuntimeError: 'test_attr' property already set.
I have often preferred this way; "explicit is better than implicit":
class MyError(Exception):
...
NOT_SET = object()
class C:
def set_my_property(self, spam, eggs, cheese):
"""This sets the property.
If it's already set, you'll get an error. Donna do dat.
"""
if getattr(self, "_my_property", NOT_SET) is NOT_SET:
self._my_property = spam, eggs, cheese
return
raise MyError("I said, Donna do dat.")
#property
def my_property(self):
return self._my_property
Testing:
c=C()
c.set_my_property("spam", "eggs", "cheese")
assert c.my_property == ("spam", "eggs", "cheese")
try:
c.set_my_property("bacon", "butter", "coffee")
except MyError:
pass
This is simple enough and more general. A decorator for a function to be called only once, and ignore subsecuent calls.
def onlyonce(func):
#functools.wraps(func)
def decorated(*args):
if not decorated.called:
decorated.called = True
return self.func(*args)
decorated.called = False
return decorated
use like this
class A:
#property
def x(self):
...
#x.setter
#onlyonce
def x(self, val):
...
Or you can define a descriptor:
class Desc:
def __get__(self, inst, own):
return self._value
def __set__(self, inst, value):
if not hasattr(self, _value):
self._value = value
And use like this:
class A:
x = Desc()
It sounds like the attrs project would be helpful for your use case. You can realize "frozen" attributes in the following way:
import attr
#attr.s
class Test:
constant = attr.ib(on_setattr=attr.setters.frozen)
test = Test('foo')
test.constant = 'bar' # raises `attr.exceptions.FrozenAttributeError`
Note that it also supports validators via #constant.validator (see the example at the end of attr.ib docs).
I'm trying to create a class that will have attributes which have a display name, i.e,
class MainClass:
def __init__(self, value):
self.ip24xs = Attribute(value = value, display_name="Attribute X")
This Attribute object I thought of implementing like a descriptor, but once I call for example mc.ip24xs.display_name I get that the str object has no attribute display_name, which makes sense, because it's the return value of the __get__ method. What is the correct way of implementing such functionality?
#dataclass
class Attribute:
value : Any = None
display_name : str = "var"
def __get__(self, obj, objtype = None):
print(f"Retrieving {self.display_name}")
return self.value
def __set__(self, obj, val):
print(f"Setting {self.display_name}")
self.value = val
I copy/pasted your code, and I don't get any errors. Only, ip24xs is not working as a descriptor but as a regular attribute or MainClass, because descriptors work in a different way. This will do, however:
#dataclass
class Attribute:
display_name : str
def __get__(self, obj, objtype = None):
print(f"Retrieving {self.display_name}")
return obj.value
def __set__(self, obj, val):
print(f"Setting {self.display_name}")
obj.value = val
class MainClass:
ip24xs = Attribute(display_name="Attribute X")
def __init__(self,value):
self.value = value
Note the difference: value is a regular attribute of MainClass, not of Attribute, so you reference it in __get__() and __set__() by obj.value, not self.value. You may also want to make it "private" as in _value just to make (more or less) sure it is not accessed directly
I have known the use of setter and getter for several properties, how could I trigger a same function when any property changes?
For example, the following codes add a setter to property a.
class AAA(object):
def __init__(self):
...
#property
def a(self):
...
#a.setter
def a(self, value):
...
If the class has a lot of properties like a, b, ... , z, and I want to print something like property xxx is modified when any property changes.
It is stupid to add the similar getter and setter one by one.
I have read some related questions and answers, but I do not find the solution for many properties.
How to trigger function on value change?
Using #property versus getters and setters
Metaprogramming, using __setattr__ to intercept modification:
class AAA(object):
def __setattr__(self, attr, value):
print("set %s to %s" % (attr, value))
super().__setattr__(attr, value)
aaa = AAA()
aaa.x = 17
# => set x to 17
print(aaa.x)
# => 17
You can do similarly with __getattr__ for reading access.
You can use descriptors. Descriptors are, in layman's terms, reusable properties. The advantage over the __getattr__ and __setattr__ hooks is that you have more fine-grained control over what attributes are managed by descriptors.
class MyDescriptor:
def __init__(self, default='default'):
self.default = default
def __set_name__(self, owner, name): # new in Python3.6
self.name = name
def __get__(self, instance, owner):
print('getting {} on {}'.format(self.name, instance))
# your getter logic here
# dummy implementation:
if instance is not None:
try:
return vars(instance)[self.name]
except KeyError:
return self.default
return self
def __set__(self, instance, value):
print('setting {} on {}'.format(self.name, instance))
# your getter logic here
# dummy implementation:
vars(instance)[self.name] = value
class MyClass:
a = MyDescriptor()
b = MyDescriptor()
_id = 1
# some logic for demo __repr__
def __init__(self):
self.c = 'non-descriptor-handled'
self.id = MyClass._id
MyClass._id += 1
def __repr__(self):
return 'MyClass #{}'.format(self.id)
Demo:
>>> m1 = MyClass()
>>> m2 = MyClass()
>>> m1.c
'non-descriptor-handled'
>>> m1.a
getting a on MyClass #1
'default'
>>> m1.b
getting b on MyClass #1
'default'
>>> m1.b = 15
setting b on MyClass #1
>>> m1.b
getting b on MyClass #1
15
>>> m2.b
getting b on MyClass #2
'default'
One year after asking this question, I find a more elgant way to add getter and setter to multiple similar properties.
Just make a more 'abstract' function which returns decorated property. And pass each of these properties to this function with a for loop. Then the getter and setter of all these properties are added.
def propABC(arg):
# arg: 'a', 'b', 'c'
#property
def prop(self):
_arg = '_' + arg
return getattr(self, _arg)
#prop.setter
def prop(self, val):
_arg = '_' + arg
setattr(self, _arg, val)
print(f"Set prop {_arg}")
return prop
for key in ['a', 'b', 'c']:
exec(f"{key} = propABC('{key}')")
Or does the attribute have to be defined outside of any class methods?
So my descriptor object is this. The IDN object already has some information about the UserNameField, so I want to use it.
class UserNameElement(basePageElement):
_testMethodName="UserNameElement Test method"
def __init__(self, IDN, PTF):
print "creating UserNameElement"
self.locator = IDN.UserNameField()
And here is my calling class. Where I want to instantiate the UserNameElement object
class LoginPageObject(basePageObject):
_testMethodName="LoginPageObject Test method"
print "creating LoginPageObject"
def __init__(self, BaseURL):
super(LoginPageObject, self).__init__()
self.username=UserNameElement(IDN=self.IDN, PTF=self.PTF)
It seems that the standard process would put the username= in in the general class definition, like this:
class LoginPageObject(basePageObject):
_testMethodName="LoginPageObject Test method"
username=UserNameElement()
print "creating LoginPageObject"
def __init__(self, BaseURL):
super(LoginPageObject, self).__init__()
But then I don't have the PTF and IDN that I define in the basePageObject class.
What can I do to make those available when the username attribute is created?
Thanks
I am afraid that will not be possible, as your attribute username will be resolved via normal attribute access see http://docs.python.org/howto/descriptor.html#invoking-descriptors
May be you can get away by overriding __getattribute__ and simulating what type.__getattribute__() does
class MyD(object):
def __init__(self, val):
self.val = val
def __get__(self, obj, objtype):
return self.val
def __set__(self, obj, val):
self.val = val
class C(object):
a = MyD(42)
def __init__(self):
self.d = MyD(42)
def __getattribute__(self, name):
attr = super(C, self).__getattribute__(name)
if hasattr(attr, '__get__'):
return attr.__get__(self, C)
return attr
c = C()
print c.d
print c.a
Output:
42
42
Since you probably won't need the username until the object has been instantiated, it's probably best to just make it a property and write a getter for it.
This question already has answers here:
Using property() on classmethods
(19 answers)
Closed 3 years ago.
In python I can add a method to a class with the #classmethod decorator. Is there a similar decorator to add a property to a class? I can better show what I'm talking about.
class Example(object):
the_I = 10
def __init__( self ):
self.an_i = 20
#property
def i( self ):
return self.an_i
def inc_i( self ):
self.an_i += 1
# is this even possible?
#classproperty
def I( cls ):
return cls.the_I
#classmethod
def inc_I( cls ):
cls.the_I += 1
e = Example()
assert e.i == 20
e.inc_i()
assert e.i == 21
assert Example.I == 10
Example.inc_I()
assert Example.I == 11
Is the syntax I've used above possible or would it require something more?
The reason I want class properties is so I can lazy load class attributes, which seems reasonable enough.
Here's how I would do this:
class ClassPropertyDescriptor(object):
def __init__(self, fget, fset=None):
self.fget = fget
self.fset = fset
def __get__(self, obj, klass=None):
if klass is None:
klass = type(obj)
return self.fget.__get__(obj, klass)()
def __set__(self, obj, value):
if not self.fset:
raise AttributeError("can't set attribute")
type_ = type(obj)
return self.fset.__get__(obj, type_)(value)
def setter(self, func):
if not isinstance(func, (classmethod, staticmethod)):
func = classmethod(func)
self.fset = func
return self
def classproperty(func):
if not isinstance(func, (classmethod, staticmethod)):
func = classmethod(func)
return ClassPropertyDescriptor(func)
class Bar(object):
_bar = 1
#classproperty
def bar(cls):
return cls._bar
#bar.setter
def bar(cls, value):
cls._bar = value
# test instance instantiation
foo = Bar()
assert foo.bar == 1
baz = Bar()
assert baz.bar == 1
# test static variable
baz.bar = 5
assert foo.bar == 5
# test setting variable on the class
Bar.bar = 50
assert baz.bar == 50
assert foo.bar == 50
The setter didn't work at the time we call Bar.bar, because we are calling
TypeOfBar.bar.__set__, which is not Bar.bar.__set__.
Adding a metaclass definition solves this:
class ClassPropertyMetaClass(type):
def __setattr__(self, key, value):
if key in self.__dict__:
obj = self.__dict__.get(key)
if obj and type(obj) is ClassPropertyDescriptor:
return obj.__set__(self, value)
return super(ClassPropertyMetaClass, self).__setattr__(key, value)
# and update class define:
# class Bar(object):
# __metaclass__ = ClassPropertyMetaClass
# _bar = 1
# and update ClassPropertyDescriptor.__set__
# def __set__(self, obj, value):
# if not self.fset:
# raise AttributeError("can't set attribute")
# if inspect.isclass(obj):
# type_ = obj
# obj = None
# else:
# type_ = type(obj)
# return self.fset.__get__(obj, type_)(value)
Now all will be fine.
If you define classproperty as follows, then your example works exactly as you requested.
class classproperty(object):
def __init__(self, f):
self.f = f
def __get__(self, obj, owner):
return self.f(owner)
The caveat is that you can't use this for writable properties. While e.I = 20 will raise an AttributeError, Example.I = 20 will overwrite the property object itself.
[answer written based on python 3.4; the metaclass syntax differs in 2 but I think the technique will still work]
You can do this with a metaclass...mostly. Dappawit's almost works, but I think it has a flaw:
class MetaFoo(type):
#property
def thingy(cls):
return cls._thingy
class Foo(object, metaclass=MetaFoo):
_thingy = 23
This gets you a classproperty on Foo, but there's a problem...
print("Foo.thingy is {}".format(Foo.thingy))
# Foo.thingy is 23
# Yay, the classmethod-property is working as intended!
foo = Foo()
if hasattr(foo, "thingy"):
print("Foo().thingy is {}".format(foo.thingy))
else:
print("Foo instance has no attribute 'thingy'")
# Foo instance has no attribute 'thingy'
# Wha....?
What the hell is going on here? Why can't I reach the class property from an instance?
I was beating my head on this for quite a while before finding what I believe is the answer. Python #properties are a subset of descriptors, and, from the descriptor documentation (emphasis mine):
The default behavior for attribute access is to get, set, or delete the
attribute from an object’s dictionary. For instance, a.x has a lookup chain
starting with a.__dict__['x'], then type(a).__dict__['x'], and continuing
through the base classes of type(a) excluding metaclasses.
So the method resolution order doesn't include our class properties (or anything else defined in the metaclass). It is possible to make a subclass of the built-in property decorator that behaves differently, but (citation needed) I've gotten the impression googling that the developers had a good reason (which I do not understand) for doing it that way.
That doesn't mean we're out of luck; we can access the properties on the class itself just fine...and we can get the class from type(self) within the instance, which we can use to make #property dispatchers:
class Foo(object, metaclass=MetaFoo):
_thingy = 23
#property
def thingy(self):
return type(self).thingy
Now Foo().thingy works as intended for both the class and the instances! It will also continue to do the right thing if a derived class replaces its underlying _thingy (which is the use case that got me on this hunt originally).
This isn't 100% satisfying to me -- having to do setup in both the metaclass and object class feels like it violates the DRY principle. But the latter is just a one-line dispatcher; I'm mostly okay with it existing, and you could probably compact it down to a lambda or something if you really wanted.
If you use Django, it has a built in #classproperty decorator.
from django.utils.decorators import classproperty
For Django 4, use:
from django.utils.functional import classproperty
I think you may be able to do this with the metaclass. Since the metaclass can be like a class for the class (if that makes sense). I know you can assign a __call__() method to the metaclass to override calling the class, MyClass(). I wonder if using the property decorator on the metaclass operates similarly.
Wow, it works:
class MetaClass(type):
def getfoo(self):
return self._foo
foo = property(getfoo)
#property
def bar(self):
return self._bar
class MyClass(object):
__metaclass__ = MetaClass
_foo = 'abc'
_bar = 'def'
print MyClass.foo
print MyClass.bar
Note: This is in Python 2.7. Python 3+ uses a different technique to declare a metaclass. Use: class MyClass(metaclass=MetaClass):, remove __metaclass__, and the rest is the same.
As far as I can tell, there is no way to write a setter for a class property without creating a new metaclass.
I have found that the following method works. Define a metaclass with all of the class properties and setters you want. IE, I wanted a class with a title property with a setter. Here's what I wrote:
class TitleMeta(type):
#property
def title(self):
return getattr(self, '_title', 'Default Title')
#title.setter
def title(self, title):
self._title = title
# Do whatever else you want when the title is set...
Now make the actual class you want as normal, except have it use the metaclass you created above.
# Python 2 style:
class ClassWithTitle(object):
__metaclass__ = TitleMeta
# The rest of your class definition...
# Python 3 style:
class ClassWithTitle(object, metaclass = TitleMeta):
# Your class definition...
It's a bit weird to define this metaclass as we did above if we'll only ever use it on the single class. In that case, if you're using the Python 2 style, you can actually define the metaclass inside the class body. That way it's not defined in the module scope.
def _create_type(meta, name, attrs):
type_name = f'{name}Type'
type_attrs = {}
for k, v in attrs.items():
if type(v) is _ClassPropertyDescriptor:
type_attrs[k] = v
return type(type_name, (meta,), type_attrs)
class ClassPropertyType(type):
def __new__(meta, name, bases, attrs):
Type = _create_type(meta, name, attrs)
cls = super().__new__(meta, name, bases, attrs)
cls.__class__ = Type
return cls
class _ClassPropertyDescriptor(object):
def __init__(self, fget, fset=None):
self.fget = fget
self.fset = fset
def __get__(self, obj, owner):
if self in obj.__dict__.values():
return self.fget(obj)
return self.fget(owner)
def __set__(self, obj, value):
if not self.fset:
raise AttributeError("can't set attribute")
return self.fset(obj, value)
def setter(self, func):
self.fset = func
return self
def classproperty(func):
return _ClassPropertyDescriptor(func)
class Bar(metaclass=ClassPropertyType):
__bar = 1
#classproperty
def bar(cls):
return cls.__bar
#bar.setter
def bar(cls, value):
cls.__bar = value
bar = Bar()
assert Bar.bar==1
Bar.bar=2
assert bar.bar==2
nbar = Bar()
assert nbar.bar==2
I happened to come up with a solution very similar to #Andrew, only DRY
class MetaFoo(type):
def __new__(mc1, name, bases, nmspc):
nmspc.update({'thingy': MetaFoo.thingy})
return super(MetaFoo, mc1).__new__(mc1, name, bases, nmspc)
#property
def thingy(cls):
if not inspect.isclass(cls):
cls = type(cls)
return cls._thingy
#thingy.setter
def thingy(cls, value):
if not inspect.isclass(cls):
cls = type(cls)
cls._thingy = value
class Foo(metaclass=MetaFoo):
_thingy = 23
class Bar(Foo)
_thingy = 12
This has the best of all answers:
The "metaproperty" is added to the class, so that it will still be a property of the instance
Don't need to redefine thingy in any of the classes
The property works as a "class property" in for both instance and class
You have the flexibility to customize how _thingy is inherited
In my case, I actually customized _thingy to be different for every child, without defining it in each class (and without a default value) by:
def __new__(mc1, name, bases, nmspc):
nmspc.update({'thingy': MetaFoo.services, '_thingy': None})
return super(MetaFoo, mc1).__new__(mc1, name, bases, nmspc)
If you only need lazy loading, then you could just have a class initialisation method.
EXAMPLE_SET = False
class Example(object):
#classmethod
def initclass(cls):
global EXAMPLE_SET
if EXAMPLE_SET: return
cls.the_I = 'ok'
EXAMPLE_SET = True
def __init__( self ):
Example.initclass()
self.an_i = 20
try:
print Example.the_I
except AttributeError:
print 'ok class not "loaded"'
foo = Example()
print foo.the_I
print Example.the_I
But the metaclass approach seems cleaner, and with more predictable behavior.
Perhaps what you're looking for is the Singleton design pattern. There's a nice SO QA about implementing shared state in Python.