We Keep Coding

what does self refer to in Property class creating code?

In the below code that emulates property creation code:
class Property:
"Emulate PyProperty_Type() in Objects/descrobject.c"
def __init__(self, fget=None, fset=None, fdel=None, doc=None):
self.fget = fget
self.fset = fset
self.fdel = fdel
if doc is None and fget is not None:
doc = fget.__doc__
self.__doc__ = doc
def __get__(self, obj, objtype=None):
if obj is None:
return self
if self.fget is None:
raise AttributeError("unreadable attribute")
return self.fget(obj)
def __set__(self, obj, value):
if self.fset is None:
raise AttributeError("can't set attribute")
self.fset(obj, value)
def __delete__(self, obj):
if self.fdel is None:
raise AttributeError("can't delete attribute")
self.fdel(obj)
def getter(self, fget):
return type(self)(fget, self.fset, self.fdel, self.__doc__)
def setter(self, fset):
return type(self)(self.fget, fset, self.fdel, self.__doc__)
def deleter(self, fdel):
return type(self)(self.fget, self.fset, fdel, self.__doc__)
in __get__, self is the instance of the property class?
Also, what does obj refer to? Why is obj passed to self.fget?
When I test it I get the following error:
class A:
func = Property(fget=func)
a = A()
a.func
TypeError: 'Property' object is not callable
How does it work then if it's not callable? I was under the impression that when defining a decorator as an instance you need to make sure it implements the __call__ method?
What am I missing?

self is the instance of Property itself. Its main purpose is to simply hold references to the getter, setter, and/or deleter so that you can call them when the property is accessed. When those functions are called, they get a reference to the object from which the property was accessed, so in a.func, self is the instance of Property and obj is a.
The Property instance itself doesn't need to be callable, because it's not what gets called. Property.__get__ only returns self if you access it from the class, not an instance of the class. The built-in property works the same way.
>>> class A:
... func = property(lambda self: 3)
...
>>> A.func
<property object at 0x10a33bb88>
>>> A().func
3
func is a class attribute whose value is an instance of property. The result of accessing the attribute differs depending on whether you look it up on the class itself or an instance of the class.
When accessed via the class, you get back the property itself.
A.func == A.__dict__['func'].__get__(None, A)
== A.__dict__['func']
When accessed via an instance, you get back the result of calling the original function passed to property.
a = A()
a.func == A.__dict__['func'].__get__(a, A)
== A.__dict__['func'].fget(a)
== (lambda self: 3)(a)
== 3

Python descriptor returning itself not value when called from class

I want to make an attribute accessible from instance and class that cannot be reassigned. I have the "prevent from reassigned" part tackled thanks to a metaclass. However, now the attribute cannot be read from the class. How to make it so.
Having this:
class Meta(type):
def __setattr__(cls, attr, val):
if attr =="frozen":
print "You can't unfreeze the frozen"
else:
cls.name = val
class Fixed(object):
__metaclass__ = Meta
frzn = 'I AM AWESOME'
def holdset(_,val):
_.frzn = _.frzn
print "frozen is frozen not setting to ", val
def get(_):
return _.frzn
frozen = property(fset=holdset,fget=get)
When calling
print Fixed.frozen
print Fixed().frozen
Gives
<property object at 0x106dbeba8>
I AM AWESOME
Why doesn't it give the same? How to make it give the same?

A property normally only works on an instance. To make a property work on a class too, you'll need to create your own descriptor instead:
class ClassAndInstanceProperty(object):
def __init__(self, fget=None, fset=None, fdel=None, doc=None):
self.fget = fget
self.fset = fset
self.fdel = fdel
if doc is None and fget is not None:
doc = fget.__doc__
self.__doc__ = doc
def __get__(self, obj, objtype=None):
if obj is None:
obj = objtype
if self.fget is None:
raise AttributeError("unreadable attribute")
return self.fget(obj)
def __set__(self, obj, value):
if self.fset is None:
raise AttributeError("can't set attribute")
self.fset(obj, value)
def __delete__(self, obj):
if self.fdel is None:
raise AttributeError("can't delete attribute")
self.fdel(obj)
def getter(self, fget):
return type(self)(fget, self.fset, self.fdel, self.__doc__)
def setter(self, fset):
return type(self)(self.fget, fset, self.fdel, self.__doc__)
def deleter(self, fdel):
return type(self)(self.fget, self.fset, fdel, self.__doc__)
The only difference between the above and a regular property is in the __get__ method; a normal property object does (in C) the equivalent of:
def __get__(self, obj, objtype=None):
if obj is None:
return self
if self.fget is None:
raise AttributeError("unreadable attribute")
return self.fget(obj)
e.g. on a class the descriptor returns self, while my version sets obj to objtype in that case.
Also note that a descriptor __set__ method is only called when working with an instance; Fixed.frozen = 'something' would not invoke the descriptor __set__, only Fixed().frozen = 'something' would. Your metaclass __setattr__ intercepts the attribute assignment on the class instead.
You could also put a descriptor object on the metaclass, which would then be given the opportunity to have it's __set__ called for attribute assignment on the class itself.

By convention a Python descriptor returns itself when not called with an instance. Properties are just a shortcut for a descriptor, to do this you need to implement your own descriptor. The following should do what you want, without needing a metaclass:
class Frozen(object):
def __init__(self, fixed_value):
self.fixed_value = fixed_value
def __get__(self, instance, owner=None):
return self.fixed_value
def __set__(self, instance, value):
print "You can't unfreeze the frozen"
class Fixed(object):
frzn = Frozen("I AM AWESOME")
You would also probably want to override the delete method which is part of the descriptor interface. There are some good articles explaining exactly how descriptors work.

How to make a class property? [duplicate]

This question already has answers here:
Using property() on classmethods
(19 answers)
Closed 3 years ago.
In python I can add a method to a class with the #classmethod decorator. Is there a similar decorator to add a property to a class? I can better show what I'm talking about.
class Example(object):
the_I = 10
def __init__( self ):
self.an_i = 20
#property
def i( self ):
return self.an_i
def inc_i( self ):
self.an_i += 1
# is this even possible?
#classproperty
def I( cls ):
return cls.the_I
#classmethod
def inc_I( cls ):
cls.the_I += 1
e = Example()
assert e.i == 20
e.inc_i()
assert e.i == 21
assert Example.I == 10
Example.inc_I()
assert Example.I == 11
Is the syntax I've used above possible or would it require something more?
The reason I want class properties is so I can lazy load class attributes, which seems reasonable enough.

Here's how I would do this:
class ClassPropertyDescriptor(object):
def __init__(self, fget, fset=None):
self.fget = fget
self.fset = fset
def __get__(self, obj, klass=None):
if klass is None:
klass = type(obj)
return self.fget.__get__(obj, klass)()
def __set__(self, obj, value):
if not self.fset:
raise AttributeError("can't set attribute")
type_ = type(obj)
return self.fset.__get__(obj, type_)(value)
def setter(self, func):
if not isinstance(func, (classmethod, staticmethod)):
func = classmethod(func)
self.fset = func
return self
def classproperty(func):
if not isinstance(func, (classmethod, staticmethod)):
func = classmethod(func)
return ClassPropertyDescriptor(func)
class Bar(object):
_bar = 1
#classproperty
def bar(cls):
return cls._bar
#bar.setter
def bar(cls, value):
cls._bar = value
# test instance instantiation
foo = Bar()
assert foo.bar == 1
baz = Bar()
assert baz.bar == 1
# test static variable
baz.bar = 5
assert foo.bar == 5
# test setting variable on the class
Bar.bar = 50
assert baz.bar == 50
assert foo.bar == 50
The setter didn't work at the time we call Bar.bar, because we are calling
TypeOfBar.bar.__set__, which is not Bar.bar.__set__.
Adding a metaclass definition solves this:
class ClassPropertyMetaClass(type):
def __setattr__(self, key, value):
if key in self.__dict__:
obj = self.__dict__.get(key)
if obj and type(obj) is ClassPropertyDescriptor:
return obj.__set__(self, value)
return super(ClassPropertyMetaClass, self).__setattr__(key, value)
# and update class define:
# class Bar(object):
# __metaclass__ = ClassPropertyMetaClass
# _bar = 1
# and update ClassPropertyDescriptor.__set__
# def __set__(self, obj, value):
# if not self.fset:
# raise AttributeError("can't set attribute")
# if inspect.isclass(obj):
# type_ = obj
# obj = None
# else:
# type_ = type(obj)
# return self.fset.__get__(obj, type_)(value)
Now all will be fine.

If you define classproperty as follows, then your example works exactly as you requested.
class classproperty(object):
def __init__(self, f):
self.f = f
def __get__(self, obj, owner):
return self.f(owner)
The caveat is that you can't use this for writable properties. While e.I = 20 will raise an AttributeError, Example.I = 20 will overwrite the property object itself.

[answer written based on python 3.4; the metaclass syntax differs in 2 but I think the technique will still work]
You can do this with a metaclass...mostly. Dappawit's almost works, but I think it has a flaw:
class MetaFoo(type):
#property
def thingy(cls):
return cls._thingy
class Foo(object, metaclass=MetaFoo):
_thingy = 23
This gets you a classproperty on Foo, but there's a problem...
print("Foo.thingy is {}".format(Foo.thingy))
# Foo.thingy is 23
# Yay, the classmethod-property is working as intended!
foo = Foo()
if hasattr(foo, "thingy"):
print("Foo().thingy is {}".format(foo.thingy))
else:
print("Foo instance has no attribute 'thingy'")
# Foo instance has no attribute 'thingy'
# Wha....?
What the hell is going on here? Why can't I reach the class property from an instance?
I was beating my head on this for quite a while before finding what I believe is the answer. Python #properties are a subset of descriptors, and, from the descriptor documentation (emphasis mine):
The default behavior for attribute access is to get, set, or delete the
attribute from an object’s dictionary. For instance, a.x has a lookup chain
starting with a.__dict__['x'], then type(a).__dict__['x'], and continuing
through the base classes of type(a) excluding metaclasses.
So the method resolution order doesn't include our class properties (or anything else defined in the metaclass). It is possible to make a subclass of the built-in property decorator that behaves differently, but (citation needed) I've gotten the impression googling that the developers had a good reason (which I do not understand) for doing it that way.
That doesn't mean we're out of luck; we can access the properties on the class itself just fine...and we can get the class from type(self) within the instance, which we can use to make #property dispatchers:
class Foo(object, metaclass=MetaFoo):
_thingy = 23
#property
def thingy(self):
return type(self).thingy
Now Foo().thingy works as intended for both the class and the instances! It will also continue to do the right thing if a derived class replaces its underlying _thingy (which is the use case that got me on this hunt originally).
This isn't 100% satisfying to me -- having to do setup in both the metaclass and object class feels like it violates the DRY principle. But the latter is just a one-line dispatcher; I'm mostly okay with it existing, and you could probably compact it down to a lambda or something if you really wanted.

If you use Django, it has a built in #classproperty decorator.
from django.utils.decorators import classproperty
For Django 4, use:
from django.utils.functional import classproperty

I think you may be able to do this with the metaclass. Since the metaclass can be like a class for the class (if that makes sense). I know you can assign a __call__() method to the metaclass to override calling the class, MyClass(). I wonder if using the property decorator on the metaclass operates similarly.
Wow, it works:
class MetaClass(type):
def getfoo(self):
return self._foo
foo = property(getfoo)
#property
def bar(self):
return self._bar
class MyClass(object):
__metaclass__ = MetaClass
_foo = 'abc'
_bar = 'def'
print MyClass.foo
print MyClass.bar
Note: This is in Python 2.7. Python 3+ uses a different technique to declare a metaclass. Use: class MyClass(metaclass=MetaClass):, remove __metaclass__, and the rest is the same.

As far as I can tell, there is no way to write a setter for a class property without creating a new metaclass.
I have found that the following method works. Define a metaclass with all of the class properties and setters you want. IE, I wanted a class with a title property with a setter. Here's what I wrote:
class TitleMeta(type):
#property
def title(self):
return getattr(self, '_title', 'Default Title')
#title.setter
def title(self, title):
self._title = title
# Do whatever else you want when the title is set...
Now make the actual class you want as normal, except have it use the metaclass you created above.
# Python 2 style:
class ClassWithTitle(object):
__metaclass__ = TitleMeta
# The rest of your class definition...
# Python 3 style:
class ClassWithTitle(object, metaclass = TitleMeta):
# Your class definition...
It's a bit weird to define this metaclass as we did above if we'll only ever use it on the single class. In that case, if you're using the Python 2 style, you can actually define the metaclass inside the class body. That way it's not defined in the module scope.

def _create_type(meta, name, attrs):
type_name = f'{name}Type'
type_attrs = {}
for k, v in attrs.items():
if type(v) is _ClassPropertyDescriptor:
type_attrs[k] = v
return type(type_name, (meta,), type_attrs)
class ClassPropertyType(type):
def __new__(meta, name, bases, attrs):
Type = _create_type(meta, name, attrs)
cls = super().__new__(meta, name, bases, attrs)
cls.__class__ = Type
return cls
class _ClassPropertyDescriptor(object):
def __init__(self, fget, fset=None):
self.fget = fget
self.fset = fset
def __get__(self, obj, owner):
if self in obj.__dict__.values():
return self.fget(obj)
return self.fget(owner)
def __set__(self, obj, value):
if not self.fset:
raise AttributeError("can't set attribute")
return self.fset(obj, value)
def setter(self, func):
self.fset = func
return self
def classproperty(func):
return _ClassPropertyDescriptor(func)
class Bar(metaclass=ClassPropertyType):
__bar = 1
#classproperty
def bar(cls):
return cls.__bar
#bar.setter
def bar(cls, value):
cls.__bar = value
bar = Bar()
assert Bar.bar==1
Bar.bar=2
assert bar.bar==2
nbar = Bar()
assert nbar.bar==2

I happened to come up with a solution very similar to #Andrew, only DRY
class MetaFoo(type):
def __new__(mc1, name, bases, nmspc):
nmspc.update({'thingy': MetaFoo.thingy})
return super(MetaFoo, mc1).__new__(mc1, name, bases, nmspc)
#property
def thingy(cls):
if not inspect.isclass(cls):
cls = type(cls)
return cls._thingy
#thingy.setter
def thingy(cls, value):
if not inspect.isclass(cls):
cls = type(cls)
cls._thingy = value
class Foo(metaclass=MetaFoo):
_thingy = 23
class Bar(Foo)
_thingy = 12
This has the best of all answers:
The "metaproperty" is added to the class, so that it will still be a property of the instance
Don't need to redefine thingy in any of the classes
The property works as a "class property" in for both instance and class
You have the flexibility to customize how _thingy is inherited
In my case, I actually customized _thingy to be different for every child, without defining it in each class (and without a default value) by:
def __new__(mc1, name, bases, nmspc):
nmspc.update({'thingy': MetaFoo.services, '_thingy': None})
return super(MetaFoo, mc1).__new__(mc1, name, bases, nmspc)

If you only need lazy loading, then you could just have a class initialisation method.
EXAMPLE_SET = False
class Example(object):
#classmethod
def initclass(cls):
global EXAMPLE_SET
if EXAMPLE_SET: return
cls.the_I = 'ok'
EXAMPLE_SET = True
def __init__( self ):
Example.initclass()
self.an_i = 20
try:
print Example.the_I
except AttributeError:
print 'ok class not "loaded"'
foo = Example()
print foo.the_I
print Example.the_I
But the metaclass approach seems cleaner, and with more predictable behavior.
Perhaps what you're looking for is the Singleton design pattern. There's a nice SO QA about implementing shared state in Python.

How to create decorator for lazy initialization of a property

I want to create a decorator that works like a property, only it calls the decorated function only once, and on subsequent calls always return the result of the first call. An example:
def SomeClass(object):
#LazilyInitializedProperty
def foo(self):
print "Now initializing"
return 5
>>> x = SomeClass()
>>> x.foo
Now initializing
5
>>> x.foo
5
My idea was to write a custom decorator for this. So i started, and this is how far I came:
class LazilyInitializedProperty(object):
def __init__(self, function):
self._function = function
def __set__(self, obj, value):
raise AttributeError("This property is read-only")
def __get__(self, obj, type):
# problem: where to store the value once we have calculated it?
As you can see, I do not know where to store the cached value. The simplest solution seems to be to just maintain a dictionary, but I am wondering if there is a more elegant solution for this.
EDIT Sorry for that, I forgot to mention that I want the property to be read-only.

Denis Otkidach's CachedAttribute is a method decorator which makes attributes lazy (computed once, accessible many). To make it also read-only, I added a __set__ method. To retain the ability to recalculate (see below) I added a __delete__ method:
class ReadOnlyCachedAttribute(object):
'''Computes attribute value and caches it in the instance.
Source: Python Cookbook
Author: Denis Otkidach https://stackoverflow.com/users/168352/denis-otkidach
This decorator allows you to create a property which can be computed once and
accessed many times. Sort of like memoization
'''
def __init__(self, method, name=None):
self.method = method
self.name = name or method.__name__
self.__doc__ = method.__doc__
def __get__(self, inst, cls):
if inst is None:
return self
elif self.name in inst.__dict__:
return inst.__dict__[self.name]
else:
result = self.method(inst)
inst.__dict__[self.name]=result
return result
def __set__(self, inst, value):
raise AttributeError("This property is read-only")
def __delete__(self,inst):
del inst.__dict__[self.name]
For example:
if __name__=='__main__':
class Foo(object):
#ReadOnlyCachedAttribute
# #read_only_lazyprop
def bar(self):
print 'Calculating self.bar'
return 42
foo=Foo()
print(foo.bar)
# Calculating self.bar
# 42
print(foo.bar)
# 42
try:
foo.bar=1
except AttributeError as err:
print(err)
# This property is read-only
del(foo.bar)
print(foo.bar)
# Calculating self.bar
# 42
One of the beautiful things about CachedAttribute (and
ReadOnlyCachedAttribute) is that if you del foo.bar, then the next time you
access foo.bar, the value is re-calculated. (This magic is made possible by
the fact that del foo.bar removes 'bar' from foo.__dict__ but the property
bar remains in Foo.__dict__.)
If you don't need or don't want this ability to recalculate,
then the following (based on Mike Boers' lazyprop) is a simpler way to make a read-only lazy property.
def read_only_lazyprop(fn):
attr_name = '_lazy_' + fn.__name__
#property
def _lazyprop(self):
if not hasattr(self, attr_name):
setattr(self, attr_name, fn(self))
return getattr(self, attr_name)
#_lazyprop.setter
def _lazyprop(self,value):
raise AttributeError("This property is read-only")
return _lazyprop

How to create a read-only class property in Python? [duplicate]

This question already has answers here:
Using property() on classmethods
(19 answers)
Closed 3 years ago.
Essentially I want to do something like this:
class foo:
x = 4
#property
#classmethod
def number(cls):
return x
Then I would like the following to work:
>>> foo.number
4
Unfortunately, the above doesn't work. Instead of given me 4 it gives me <property object at 0x101786c58>. Is there any way to achieve the above?

This will make Foo.number a read-only property:
class MetaFoo(type):
#property
def number(cls):
return cls.x
class Foo(object, metaclass=MetaFoo):
x = 4
print(Foo.number)
# 4
Foo.number = 6
# AttributeError: can't set attribute
Explanation: The usual scenario when using #property looks like this:
class Foo(object):
#property
def number(self):
...
foo = Foo()
A property defined in Foo is read-only with respect to its instances. That is, foo.number = 6 would raise an AttributeError.
Analogously, if you want Foo.number to raise an AttributeError you would need to setup a property defined in type(Foo). Hence the need for a metaclass.
Note that this read-onlyness is not immune from hackers.
The property can be made writable by changing Foo's
class:
class Base(type): pass
Foo.__class__ = Base
# makes Foo.number a normal class attribute
Foo.number = 6
print(Foo.number)
prints
6
or, if you wish to make Foo.number a settable property,
class WritableMetaFoo(type):
#property
def number(cls):
return cls.x
#number.setter
def number(cls, value):
cls.x = value
Foo.__class__ = WritableMetaFoo
# Now the assignment modifies `Foo.x`
Foo.number = 6
print(Foo.number)
also prints
6

The property descriptor always returns itself when accessed from a class (ie. when instance is None in its __get__ method).
If that's not what you want, you can write a new descriptor that always uses the class object (owner) instead of the instance:
>>> class classproperty(object):
... def __init__(self, getter):
... self.getter= getter
... def __get__(self, instance, owner):
... return self.getter(owner)
...
>>> class Foo(object):
... x= 4
... #classproperty
... def number(cls):
... return cls.x
...
>>> Foo().number
4
>>> Foo.number
4

I agree with unubtu's answer; it seems to work, however, it doesn't work with this precise syntax on Python 3 (specifically, Python 3.4 is what I struggled with). Here's how one must form the pattern under Python 3.4 to make things work, it seems:
class MetaFoo(type):
#property
def number(cls):
return cls.x
class Foo(metaclass=MetaFoo):
x = 4
print(Foo.number)
# 4
Foo.number = 6
# AttributeError: can't set attribute

Problem with solutions above is that it wouldn't work for accessing class variables from instance variable:
print(Foo.number)
# 4
f = Foo()
print(f.number)
# 'Foo' object has no attribute 'number'
Moreover, using metaclass explicit is not so nice, as using regular property decorator.
I tried to solve this problems. Here how it works now:
#classproperty_support
class Bar(object):
_bar = 1
#classproperty
def bar(cls):
return cls._bar
#bar.setter
def bar(cls, value):
cls._bar = value
# #classproperty should act like regular class variable.
# Asserts can be tested with it.
# class Bar:
# bar = 1
assert Bar.bar == 1
Bar.bar = 2
assert Bar.bar == 2
foo = Bar()
baz = Bar()
assert foo.bar == 2
assert baz.bar == 2
Bar.bar = 50
assert baz.bar == 50
assert foo.bar == 50
As you see, we have #classproperty that works same way as #property for class variables. Only thing we will need is additional #classproperty_support class decorator.
Solution also works for read-only class properties.
Here's implementation:
class classproperty:
"""
Same as property(), but passes obj.__class__ instead of obj to fget/fset/fdel.
Original code for property emulation:
https://docs.python.org/3.5/howto/descriptor.html#properties
"""
def __init__(self, fget=None, fset=None, fdel=None, doc=None):
self.fget = fget
self.fset = fset
self.fdel = fdel
if doc is None and fget is not None:
doc = fget.__doc__
self.__doc__ = doc
def __get__(self, obj, objtype=None):
if obj is None:
return self
if self.fget is None:
raise AttributeError("unreadable attribute")
return self.fget(obj.__class__)
def __set__(self, obj, value):
if self.fset is None:
raise AttributeError("can't set attribute")
self.fset(obj.__class__, value)
def __delete__(self, obj):
if self.fdel is None:
raise AttributeError("can't delete attribute")
self.fdel(obj.__class__)
def getter(self, fget):
return type(self)(fget, self.fset, self.fdel, self.__doc__)
def setter(self, fset):
return type(self)(self.fget, fset, self.fdel, self.__doc__)
def deleter(self, fdel):
return type(self)(self.fget, self.fset, fdel, self.__doc__)
def classproperty_support(cls):
"""
Class decorator to add metaclass to our class.
Metaclass uses to add descriptors to class attributes, see:
http://stackoverflow.com/a/26634248/1113207
"""
class Meta(type):
pass
for name, obj in vars(cls).items():
if isinstance(obj, classproperty):
setattr(Meta, name, property(obj.fget, obj.fset, obj.fdel))
class Wrapper(cls, metaclass=Meta):
pass
return Wrapper
Note: code isn't tested much, feel free to note if it doesn't work as you expect.

The solution of Mikhail Gerasimov is quite complete. Unfortunately, it was one drawback. If you have a class using his classproperty, no child class can use it due to an
TypeError: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases with class Wrapper.
Fortunately, this can be fixed. Just inherit from the metaclass of the given class when creating class Meta.
def classproperty_support(cls):
"""
Class decorator to add metaclass to our class.
Metaclass uses to add descriptors to class attributes, see:
http://stackoverflow.com/a/26634248/1113207
"""
# Use type(cls) to use metaclass of given class
class Meta(type(cls)):
pass
for name, obj in vars(cls).items():
if isinstance(obj, classproperty):
setattr(Meta, name, property(obj.fget, obj.fset, obj.fdel))
class Wrapper(cls, metaclass=Meta):
pass
return Wrapper