Related
I want to be able to create a class (in Python) that once initialized with __init__, does not accept new attributes, but accepts modifications of existing attributes. There's several hack-ish ways I can see to do this, for example having a __setattr__ method such as
def __setattr__(self, attribute, value):
if not attribute in self.__dict__:
print "Cannot set %s" % attribute
else:
self.__dict__[attribute] = value
and then editing __dict__ directly inside __init__, but I was wondering if there is a 'proper' way to do this?
I wouldn't use __dict__ directly, but you can add a function to explicitly "freeze" a instance:
class FrozenClass(object):
__isfrozen = False
def __setattr__(self, key, value):
if self.__isfrozen and not hasattr(self, key):
raise TypeError( "%r is a frozen class" % self )
object.__setattr__(self, key, value)
def _freeze(self):
self.__isfrozen = True
class Test(FrozenClass):
def __init__(self):
self.x = 42#
self.y = 2**3
self._freeze() # no new attributes after this point.
a,b = Test(), Test()
a.x = 10
b.z = 10 # fails
Slots is the way to go:
The pythonic way is to use slots instead of playing around with the __setter__. While it may solve the problem, it does not give any performance improvement. The attributes of objects are stored in a dictionary "__dict__", this is the reason, why you can dynamically add attributes to objects of classes that we have created so far. Using a dictionary for attribute storage is very convenient, but it can mean a waste of space for objects, which have only a small amount of instance variables.
Slots are a nice way to work around this space consumption problem. Instead of having a dynamic dict that allows adding attributes to objects dynamically, slots provide a static structure which prohibits additions after the creation of an instance.
When we design a class, we can use slots to prevent the dynamic creation of attributes. To define slots, you have to define a list with the name __slots__. The list has to contain all the attributes, you want to use. We demonstrate this in the following class, in which the slots list contains only the name for an attribute "val".
class S(object):
__slots__ = ['val']
def __init__(self, v):
self.val = v
x = S(42)
print(x.val)
x.new = "not possible"
=> It fails to create an attribute "new":
42
Traceback (most recent call last):
File "slots_ex.py", line 12, in <module>
x.new = "not possible"
AttributeError: 'S' object has no attribute 'new'
Notes:
Since Python 3.3 the advantage optimizing the space consumption is not as impressive any more. With Python 3.3 Key-Sharing Dictionaries are used for the storage of objects. The attributes of the instances are capable of sharing part of their internal storage between each other, i.e. the part which stores the keys and their corresponding hashes. This helps to reduce the memory consumption of programs, which create many instances of non-builtin types. But still is the way to go to avoid dynamically created attributes.
Using slots come also with it's own cost. It will break serialization (e.g. pickle). It will also break multiple inheritance. A class can't inherit from more than one class that either defines slots or has an instance layout defined in C code (like list, tuple or int).
If someone is interested in doing that with a decorator, here is a working solution:
from functools import wraps
def froze_it(cls):
cls.__frozen = False
def frozensetattr(self, key, value):
if self.__frozen and not hasattr(self, key):
print("Class {} is frozen. Cannot set {} = {}"
.format(cls.__name__, key, value))
else:
object.__setattr__(self, key, value)
def init_decorator(func):
#wraps(func)
def wrapper(self, *args, **kwargs):
func(self, *args, **kwargs)
self.__frozen = True
return wrapper
cls.__setattr__ = frozensetattr
cls.__init__ = init_decorator(cls.__init__)
return cls
Pretty straightforward to use:
#froze_it
class Foo(object):
def __init__(self):
self.bar = 10
foo = Foo()
foo.bar = 42
foo.foobar = "no way"
Result:
>>> Class Foo is frozen. Cannot set foobar = no way
Actually, you don't want __setattr__, you want __slots__. Add __slots__ = ('foo', 'bar', 'baz') to the class body, and Python will make sure that there's only foo, bar and baz on any instance. But read the caveats the documentation lists!
The proper way is to override __setattr__. That's what it's there for.
I like very much the solution that uses a decorator, because it's easy to use it for many classes across a project, with minimum additions for each class. But it doesn't work well with inheritance.
So here is my version: It only overrides the __setattr__ function - if the attribute doesn't exist and the caller function is not __init__, it prints an error message.
import inspect
def froze_it(cls):
def frozensetattr(self, key, value):
if not hasattr(self, key) and inspect.stack()[1][3] != "__init__":
print("Class {} is frozen. Cannot set {} = {}"
.format(cls.__name__, key, value))
else:
self.__dict__[key] = value
cls.__setattr__ = frozensetattr
return cls
#froze_it
class A:
def __init__(self):
self._a = 0
a = A()
a._a = 1
a._b = 2 # error
What about this:
class A():
__allowed_attr=('_x', '_y')
def __init__(self,x=0,y=0):
self._x=x
self._y=y
def __setattr__(self,attribute,value):
if not attribute in self.__class__.__allowed_attr:
raise AttributeError
else:
super().__setattr__(attribute,value)
Here is approach i came up with that doesn't need a _frozen attribute or method to freeze() in init.
During init i just add all class attributes to the instance.
I like this because there is no _frozen, freeze(), and _frozen also does not show up in the vars(instance) output.
class MetaModel(type):
def __setattr__(self, name, value):
raise AttributeError("Model classes do not accept arbitrary attributes")
class Model(object):
__metaclass__ = MetaModel
# init will take all CLASS attributes, and add them as SELF/INSTANCE attributes
def __init__(self):
for k, v in self.__class__.__dict__.iteritems():
if not k.startswith("_"):
self.__setattr__(k, v)
# setattr, won't allow any attributes to be set on the SELF/INSTANCE that don't already exist
def __setattr__(self, name, value):
if not hasattr(self, name):
raise AttributeError("Model instances do not accept arbitrary attributes")
else:
object.__setattr__(self, name, value)
# Example using
class Dog(Model):
name = ''
kind = 'canine'
d, e = Dog(), Dog()
print vars(d)
print vars(e)
e.junk = 'stuff' # fails
I like the "Frozen" of Jochen Ritzel. The inconvenient is that the isfrozen variable then appears when printing a Class.__dict
I went around this problem this way by creating a list of authorized attributes (similar to slots):
class Frozen(object):
__List = []
def __setattr__(self, key, value):
setIsOK = False
for item in self.__List:
if key == item:
setIsOK = True
if setIsOK == True:
object.__setattr__(self, key, value)
else:
raise TypeError( "%r has no attributes %r" % (self, key) )
class Test(Frozen):
_Frozen__List = ["attr1","attr2"]
def __init__(self):
self.attr1 = 1
self.attr2 = 1
The FrozenClass by Jochen Ritzel is cool, but calling _frozen() when initialing a class every time is not so cool (and you need to take the risk of forgetting it). I added a __init_slots__ function:
class FrozenClass(object):
__isfrozen = False
def _freeze(self):
self.__isfrozen = True
def __init_slots__(self, slots):
for key in slots:
object.__setattr__(self, key, None)
self._freeze()
def __setattr__(self, key, value):
if self.__isfrozen and not hasattr(self, key):
raise TypeError( "%r is a frozen class" % self )
object.__setattr__(self, key, value)
class Test(FrozenClass):
def __init__(self):
self.__init_slots__(["x", "y"])
self.x = 42#
self.y = 2**3
a,b = Test(), Test()
a.x = 10
b.z = 10 # fails
None of the answers mention the performance impact of overriding __setattr__, which can be an issue when creating many small objects. (And __slots__ would be the performant solution but limits pickle/inheritance).
So I came up with this variant which installs our slower settatr after init:
class FrozenClass:
def freeze(self):
def frozen_setattr(self, key, value):
if not hasattr(self, key):
raise TypeError("Cannot set {}: {} is a frozen class".format(key, self))
object.__setattr__(self, key, value)
self.__setattr__ = frozen_setattr
class Foo(FrozenClass): ...
If you don't want to call freeze at the end of __init__, if inheritance is an issue, or if you don't want it in vars(), it can also be adapted: for example here is a decorator version based on the pystrict answer:
import functools
def strict(cls):
cls._x_setter = getattr(cls, "__setattr__", object.__setattr__)
cls._x_init = cls.__init__
#functools.wraps(cls.__init__)
def wrapper(self, *args, **kwargs):
cls._x_init(self, *args, **kwargs)
def frozen_setattr(self, key, value):
if not hasattr(self, key):
raise TypeError("Class %s is frozen. Cannot set '%s'." % (cls.__name__, key))
cls._x_setter(self, key, value)
cls.__setattr__ = frozen_setattr
cls.__init__ = wrapper
return cls
#strict
class Foo: ...
I wrote pystrict as a solution to this problem. It's too large to paste all of the code in stackoverflow.
pystrict is a pypi installable decorator that can be used with classes to freeze them. Many solutions here don't properly support inheritance.
If __slots__ doesn't work for you (because of inheritance issues), this is a good alternative.
There is an example to the README that shows why a decorator like this is needed even if you have mypy and pylint running on your project:
pip install pystrict
Then just use the #strict decorator:
from pystrict import strict
#strict
class Blah
def __init__(self):
self.attr = 1
#dataclass(slots=True) Nirvana (Python 3.10)
I'm in love with this #dataclass thing:
main.py
from dataclasses import dataclass
#dataclass(slots=True)
class C:
n: int
s: str
c = C(n=1, s='one')
assert c.n == 1
assert c.s == 'one'
c.n == 2
c.s == 'two'
c.asdf = 2
Outcome:
Traceback (most recent call last):
File "/home/ciro/main.py", line 15, in <module>
c.asdf = 2
AttributeError: 'C' object has no attribute 'asdf'
Note how #dataclass only requires use to define our attributes once with type annotations
n: int
s: str
and then, without any repetition we get for free:
def __init__(n, s):
self.n = n
self.s = s
__slots__ = ['n', 's']
Other free things not shown in this example:
__str__
__eq__: Compare object instances for equality by their attributes
__hash__ if you also use frozen=True: Object of custom type as dictionary key
Tested on Python 3.10.7, Ubuntu 22.10.
I want to be able to create a class (in Python) that once initialized with __init__, does not accept new attributes, but accepts modifications of existing attributes. There's several hack-ish ways I can see to do this, for example having a __setattr__ method such as
def __setattr__(self, attribute, value):
if not attribute in self.__dict__:
print "Cannot set %s" % attribute
else:
self.__dict__[attribute] = value
and then editing __dict__ directly inside __init__, but I was wondering if there is a 'proper' way to do this?
I wouldn't use __dict__ directly, but you can add a function to explicitly "freeze" a instance:
class FrozenClass(object):
__isfrozen = False
def __setattr__(self, key, value):
if self.__isfrozen and not hasattr(self, key):
raise TypeError( "%r is a frozen class" % self )
object.__setattr__(self, key, value)
def _freeze(self):
self.__isfrozen = True
class Test(FrozenClass):
def __init__(self):
self.x = 42#
self.y = 2**3
self._freeze() # no new attributes after this point.
a,b = Test(), Test()
a.x = 10
b.z = 10 # fails
Slots is the way to go:
The pythonic way is to use slots instead of playing around with the __setter__. While it may solve the problem, it does not give any performance improvement. The attributes of objects are stored in a dictionary "__dict__", this is the reason, why you can dynamically add attributes to objects of classes that we have created so far. Using a dictionary for attribute storage is very convenient, but it can mean a waste of space for objects, which have only a small amount of instance variables.
Slots are a nice way to work around this space consumption problem. Instead of having a dynamic dict that allows adding attributes to objects dynamically, slots provide a static structure which prohibits additions after the creation of an instance.
When we design a class, we can use slots to prevent the dynamic creation of attributes. To define slots, you have to define a list with the name __slots__. The list has to contain all the attributes, you want to use. We demonstrate this in the following class, in which the slots list contains only the name for an attribute "val".
class S(object):
__slots__ = ['val']
def __init__(self, v):
self.val = v
x = S(42)
print(x.val)
x.new = "not possible"
=> It fails to create an attribute "new":
42
Traceback (most recent call last):
File "slots_ex.py", line 12, in <module>
x.new = "not possible"
AttributeError: 'S' object has no attribute 'new'
Notes:
Since Python 3.3 the advantage optimizing the space consumption is not as impressive any more. With Python 3.3 Key-Sharing Dictionaries are used for the storage of objects. The attributes of the instances are capable of sharing part of their internal storage between each other, i.e. the part which stores the keys and their corresponding hashes. This helps to reduce the memory consumption of programs, which create many instances of non-builtin types. But still is the way to go to avoid dynamically created attributes.
Using slots come also with it's own cost. It will break serialization (e.g. pickle). It will also break multiple inheritance. A class can't inherit from more than one class that either defines slots or has an instance layout defined in C code (like list, tuple or int).
If someone is interested in doing that with a decorator, here is a working solution:
from functools import wraps
def froze_it(cls):
cls.__frozen = False
def frozensetattr(self, key, value):
if self.__frozen and not hasattr(self, key):
print("Class {} is frozen. Cannot set {} = {}"
.format(cls.__name__, key, value))
else:
object.__setattr__(self, key, value)
def init_decorator(func):
#wraps(func)
def wrapper(self, *args, **kwargs):
func(self, *args, **kwargs)
self.__frozen = True
return wrapper
cls.__setattr__ = frozensetattr
cls.__init__ = init_decorator(cls.__init__)
return cls
Pretty straightforward to use:
#froze_it
class Foo(object):
def __init__(self):
self.bar = 10
foo = Foo()
foo.bar = 42
foo.foobar = "no way"
Result:
>>> Class Foo is frozen. Cannot set foobar = no way
Actually, you don't want __setattr__, you want __slots__. Add __slots__ = ('foo', 'bar', 'baz') to the class body, and Python will make sure that there's only foo, bar and baz on any instance. But read the caveats the documentation lists!
The proper way is to override __setattr__. That's what it's there for.
I like very much the solution that uses a decorator, because it's easy to use it for many classes across a project, with minimum additions for each class. But it doesn't work well with inheritance.
So here is my version: It only overrides the __setattr__ function - if the attribute doesn't exist and the caller function is not __init__, it prints an error message.
import inspect
def froze_it(cls):
def frozensetattr(self, key, value):
if not hasattr(self, key) and inspect.stack()[1][3] != "__init__":
print("Class {} is frozen. Cannot set {} = {}"
.format(cls.__name__, key, value))
else:
self.__dict__[key] = value
cls.__setattr__ = frozensetattr
return cls
#froze_it
class A:
def __init__(self):
self._a = 0
a = A()
a._a = 1
a._b = 2 # error
What about this:
class A():
__allowed_attr=('_x', '_y')
def __init__(self,x=0,y=0):
self._x=x
self._y=y
def __setattr__(self,attribute,value):
if not attribute in self.__class__.__allowed_attr:
raise AttributeError
else:
super().__setattr__(attribute,value)
Here is approach i came up with that doesn't need a _frozen attribute or method to freeze() in init.
During init i just add all class attributes to the instance.
I like this because there is no _frozen, freeze(), and _frozen also does not show up in the vars(instance) output.
class MetaModel(type):
def __setattr__(self, name, value):
raise AttributeError("Model classes do not accept arbitrary attributes")
class Model(object):
__metaclass__ = MetaModel
# init will take all CLASS attributes, and add them as SELF/INSTANCE attributes
def __init__(self):
for k, v in self.__class__.__dict__.iteritems():
if not k.startswith("_"):
self.__setattr__(k, v)
# setattr, won't allow any attributes to be set on the SELF/INSTANCE that don't already exist
def __setattr__(self, name, value):
if not hasattr(self, name):
raise AttributeError("Model instances do not accept arbitrary attributes")
else:
object.__setattr__(self, name, value)
# Example using
class Dog(Model):
name = ''
kind = 'canine'
d, e = Dog(), Dog()
print vars(d)
print vars(e)
e.junk = 'stuff' # fails
I like the "Frozen" of Jochen Ritzel. The inconvenient is that the isfrozen variable then appears when printing a Class.__dict
I went around this problem this way by creating a list of authorized attributes (similar to slots):
class Frozen(object):
__List = []
def __setattr__(self, key, value):
setIsOK = False
for item in self.__List:
if key == item:
setIsOK = True
if setIsOK == True:
object.__setattr__(self, key, value)
else:
raise TypeError( "%r has no attributes %r" % (self, key) )
class Test(Frozen):
_Frozen__List = ["attr1","attr2"]
def __init__(self):
self.attr1 = 1
self.attr2 = 1
The FrozenClass by Jochen Ritzel is cool, but calling _frozen() when initialing a class every time is not so cool (and you need to take the risk of forgetting it). I added a __init_slots__ function:
class FrozenClass(object):
__isfrozen = False
def _freeze(self):
self.__isfrozen = True
def __init_slots__(self, slots):
for key in slots:
object.__setattr__(self, key, None)
self._freeze()
def __setattr__(self, key, value):
if self.__isfrozen and not hasattr(self, key):
raise TypeError( "%r is a frozen class" % self )
object.__setattr__(self, key, value)
class Test(FrozenClass):
def __init__(self):
self.__init_slots__(["x", "y"])
self.x = 42#
self.y = 2**3
a,b = Test(), Test()
a.x = 10
b.z = 10 # fails
None of the answers mention the performance impact of overriding __setattr__, which can be an issue when creating many small objects. (And __slots__ would be the performant solution but limits pickle/inheritance).
So I came up with this variant which installs our slower settatr after init:
class FrozenClass:
def freeze(self):
def frozen_setattr(self, key, value):
if not hasattr(self, key):
raise TypeError("Cannot set {}: {} is a frozen class".format(key, self))
object.__setattr__(self, key, value)
self.__setattr__ = frozen_setattr
class Foo(FrozenClass): ...
If you don't want to call freeze at the end of __init__, if inheritance is an issue, or if you don't want it in vars(), it can also be adapted: for example here is a decorator version based on the pystrict answer:
import functools
def strict(cls):
cls._x_setter = getattr(cls, "__setattr__", object.__setattr__)
cls._x_init = cls.__init__
#functools.wraps(cls.__init__)
def wrapper(self, *args, **kwargs):
cls._x_init(self, *args, **kwargs)
def frozen_setattr(self, key, value):
if not hasattr(self, key):
raise TypeError("Class %s is frozen. Cannot set '%s'." % (cls.__name__, key))
cls._x_setter(self, key, value)
cls.__setattr__ = frozen_setattr
cls.__init__ = wrapper
return cls
#strict
class Foo: ...
I wrote pystrict as a solution to this problem. It's too large to paste all of the code in stackoverflow.
pystrict is a pypi installable decorator that can be used with classes to freeze them. Many solutions here don't properly support inheritance.
If __slots__ doesn't work for you (because of inheritance issues), this is a good alternative.
There is an example to the README that shows why a decorator like this is needed even if you have mypy and pylint running on your project:
pip install pystrict
Then just use the #strict decorator:
from pystrict import strict
#strict
class Blah
def __init__(self):
self.attr = 1
#dataclass(slots=True) Nirvana (Python 3.10)
I'm in love with this #dataclass thing:
main.py
from dataclasses import dataclass
#dataclass(slots=True)
class C:
n: int
s: str
c = C(n=1, s='one')
assert c.n == 1
assert c.s == 'one'
c.n == 2
c.s == 'two'
c.asdf = 2
Outcome:
Traceback (most recent call last):
File "/home/ciro/main.py", line 15, in <module>
c.asdf = 2
AttributeError: 'C' object has no attribute 'asdf'
Note how #dataclass only requires use to define our attributes once with type annotations
n: int
s: str
and then, without any repetition we get for free:
def __init__(n, s):
self.n = n
self.s = s
__slots__ = ['n', 's']
Other free things not shown in this example:
__str__
__eq__: Compare object instances for equality by their attributes
__hash__ if you also use frozen=True: Object of custom type as dictionary key
Tested on Python 3.10.7, Ubuntu 22.10.
I have been studying Python for a little while now, and I've come to understand that overriding __setattr__ correctly can be troublesome (to say the least!).
What are some effective ways to ensure/prove to myself the override has been done correctly? I'm specifically concerned about ensuring the override remains consistent with the descriptor protocol and MRO.
(Tagged as Python 3.x since that's what I am using, but the question is certainly applicable to other versions as well.)
Example code in which the "override" exhibits default behavior (but how can I prove it?):
class MyClass():
def __setattr__(self,att,val):
print("I am exhibiting default behavior!")
super().__setattr__(att,val)
Contrived example in which the override violates the descriptor protocol (instance storage lookup occurs prior to the descriptor lookup - but how can I test it?):
class MyClass():
def __init__(self,mydict):
self.__dict__['mydict'] = mydict
#property
def mydict(self):
return self._mydict
def __setattr__(self,att,val):
if att in self.mydict:
self.mydict[att] = val
else:
super().__setattr__(att, val)
The ideal answer will provide a general test that will succeed when __setattr__ has been overridden correctly, and fail otherwise.
In this case there's a simple solution: add a binding descriptor with a name that's in mydict and test that assigning to that name goes thru the descriptor (NB : Python 2.x code, I don't have a Python 3 install here):
class MyBindingDescriptor(object):
def __init__(self, key):
self.key = key
def __get__(self, obj, cls=None):
if not obj:
return self
return obj.__dict__[self.key]
def __set__(self, obj, value):
obj.__dict__[self.key] = value
sentinel = object()
class MyClass(object):
test = MyBindingDescriptor("test")
def __init__(self, mydict):
self.__dict__['mydict'] = mydict
self.__dict__["test"] = sentinel
def __setattr__(self, att, val):
if att in self.mydict:
self.mydict[att] = val
else:
super(MyClass, self).__setattr__(att, val)
# first test our binding descriptor
instance1 = MyClass({})
# sanity check
assert instance1.test is sentinel, "instance1.test should be sentinel, got '%s' instead" % instance1.test
# this one should pass ok
instance1.test = NotImplemented
assert instance1.test is NotImplemented, "instance1.test should be NotImplemented, got '%s' instead" % instance1.test
# now demonstrate that the current implementation is broken:
instance2 = MyClass({"test":42})
instance2.test = NotImplemented
assert instance2.test is NotImplemented, "instance2.test should be NotImplemented, got '%s' instead" % instance2.test
If you define overriding __setattr__ correctly as calling the __setattr__ of the parent class then you could graft your method into a class hierarchy that defines its own custom __setattr__:
def inject_tester_class(cls):
def __setattr__(self, name, value):
self._TesterClass__setattr_args.append((name, value))
super(intermediate, self).__setattr__(name, value)
def assertSetAttrDelegatedFor(self, name, value):
assert \
[args for args in self._TesterClass__setattr_args if args == (name, value)], \
'__setattr__(name, value) was never delegated'
body = {
'__setattr__': __setattr__,
'assertSetAttrDelegatedFor': assertSetAttrDelegatedFor,
'_TesterClass__setattr_args': []
}
intermediate = type('TesterClass', cls.__bases__, body)
testclass = type(cls.__name__, (intermediate,), vars(cls).copy())
# rebind the __class__ closure
def closure():
testclass
osa = testclass.__setattr__
new_closure = tuple(closure.__closure__[0] if n == '__class__' else c
for n, c in zip(osa.__code__.co_freevars, osa.__closure__))
testclass.__setattr__ = type(osa)(
osa.__code__, osa.__globals__, osa.__name__,
osa.__defaults__, new_closure)
return testclass
This function jumps through a few hoops to insert an intermediate class that'll intercept any properly delegated __setattr__ call. It'll work even if you don't have any base classes other than the default object (which wouldn't let us replace __setattr__ for an easier path to test this).
It does make the assumption that you are using super().__setattr__() to delegate, where you used super() without arguments. It also assumes there is no meta class involved.
The extra __setattr__ is injected in a manner consistent with the existing MRO; the extra intermediate class is injected between the original class and the rest of the MRO, and delegates the __setattr__ call onwards.
To use this in a test, you'd produce a new class with the above function, create an instance then set attributes on that instance:
MyTestClass = inject_tester_class(MyClass)
my_test_instance = MyTestClass()
my_test_instance.foo = 'bar'
my_test_instance.assertSetAttrDelegatedFor('foo', 'bar')
If setting foo is not delegated, an AssertionError exception is raised, which the unittest test runner records as a test failure.
Note: Python version is 2.7
Problem
I want to have {hierarchy of descriptor classes} integrated within {hierarchy of widget classes}, and overriding descriptor behaviour must be as easy as defining nested derived class. Example:
class A(object):
class width(Attribute):
def changed(self, obj, value, old):
print 'A.width changed:', value
class B(A):
class width(A.width):
def changed(self, obj, value, old):
super(B.width, self).changed(obj, value, old)
print 'B.width changed:', value
B().width = 10
# must print:
# A.width.changed: 10
# B.width.changed: 10
Here is my custom descriptor class:
class Attribute(object):
def __init__(self):
if not hasattr(self, '_name'):
self._name = self.__class__.__name__
def __get__(self, obj, cls):
if obj is None:
print 'Attribute getter (class):', cls
return self
print 'Attribute getter (class, inst):', (cls, obj)
print 'Attribute getter returning:', self.get(obj)
return self.get(obj)
def __set__(self, obj, value):
print 'Attribute setter (inst, value):', (obj, value)
self.set(obj, value)
def get(self, obj):
try:
return obj.__dict__[self._name]
except KeyError:
raise AttributeError("attribute '%s' referenced before assigment" % (self._name))
def set(self, obj, value):
try:
old = obj.__dict__[self._name]
except KeyError:
obj.__dict__[self._name] = value
self.changed(obj, value, None)
else:
obj.__dict__[self._name] = value
if old != value:
self.changed(obj, value, old)
def changed(self, obj, value, old):
pass
Problem is that Python don't want to use __get__ and __set__, while they are attributes of class. It may be seen from this test:
# `A` and `B` were defined above
A.width_ = A.width()
B.width_ = B.width()
to_test = (
# good:
'Aw_ = A.width_',
'Bw_ = B.width_',
'B().width_ = 10',
# silent:
'Aw = A.width',
'Bw = B.width',
'B().width = 10',
)
for expr in to_test:
print "Testing:", expr
exec expr
So, my Attribute works only when instantiated.
What I have tried already
Decorating __get__ and __set__ with either staticmethod or classmethod. No changes in silent section. Good section fails: methods aren't callable. Wat.
Adding __get__ and __set__ to class Attribute from outside, as methods bound to Attribute. Nothing changed.
Code:
# `__get__` was renamed to `_unbound__get__`
Attribute.__get__ = Attribute._unbound__get__.__get__(Attribute, Attribute.__class__)
# `__set__` was renamed to `_unbound__set__`
Attribute.__set__ = Attribute._unbound__set__.__get__(Attribute, Attribute.__class__)
Using instantiated descriptors. This approach requires 2 symbols: one for descriptor (sub)class and one for descriptor. It also requires instantiating descriptor after subclassing it.
Code:
class B(A):
class Width(A.width):
def changed(self, obj, value, old):
super(B.width, self).changed(obj, value, old)
print 'B.width.changed:', value
B.width = B.Width()
More background.
I have growing hierarchy of widgets, where some propertries must be tracked for changes, and response to change may be extended in subclassses. So I am trying to create automated approach, with modular machinery. Because keeping relevant variables and methods per each propertry inside actual widgets is just horrible annoying mess.
Question
Are there workarounds to achieve my needs? Or may be I am doing something wrong?
Descriptors require instances, so you cannot achieve exactly what you want. In particular you want A.width to be, at the same time, both a class and an instance of that class, which is impossible. You must have a way to access the class and the instance separately.
It's pretty simple to automatically create the instances with a known naming scheme. For example using a class decorator:
def track_attributes(cls):
for attr in dir(cls):
value = getattr(cls, attr)
if isinstance(value, type) and issubclass(value, Attribute):
# probably require more checks and/or different naming scheme
setattr(cls, attr.lower(), value())
return cls
Used as:
#track_attributes
class A(object):
class Width(Attribute):
def changed(self, obj, value, old):
print 'A.width changed:', value
#track_attributes
class B(A):
class Width(A.Width):
def changed(self, obj, value, old):
super(B.Width, self).changed(obj, value, old)
print 'B.width changed:', value
B().width = 10
output:
$python test_attributes.py
Attribute getter (class): <class '__main__.B'>
Attribute setter (inst, value): (<__main__.B object at 0x7fc225cd7d50>, 10)
A.width changed: 10
B.width changed: 10
An other way to achieve this without explicit decorators is to create a metaclass and have a Widget base class for all widgets.
As mata says, descriptors needs to be instances. One possible solution would be to use a class decorator or a custom metaclass (in your case more probably a custom metaclass) that would lookup the class namespace for Attribute subclasses and instanciate them. The whole thing smells a bit like overengineering to me but sometimes well you just need that level of complexity.
I want to be able to create a class (in Python) that once initialized with __init__, does not accept new attributes, but accepts modifications of existing attributes. There's several hack-ish ways I can see to do this, for example having a __setattr__ method such as
def __setattr__(self, attribute, value):
if not attribute in self.__dict__:
print "Cannot set %s" % attribute
else:
self.__dict__[attribute] = value
and then editing __dict__ directly inside __init__, but I was wondering if there is a 'proper' way to do this?
I wouldn't use __dict__ directly, but you can add a function to explicitly "freeze" a instance:
class FrozenClass(object):
__isfrozen = False
def __setattr__(self, key, value):
if self.__isfrozen and not hasattr(self, key):
raise TypeError( "%r is a frozen class" % self )
object.__setattr__(self, key, value)
def _freeze(self):
self.__isfrozen = True
class Test(FrozenClass):
def __init__(self):
self.x = 42#
self.y = 2**3
self._freeze() # no new attributes after this point.
a,b = Test(), Test()
a.x = 10
b.z = 10 # fails
Slots is the way to go:
The pythonic way is to use slots instead of playing around with the __setter__. While it may solve the problem, it does not give any performance improvement. The attributes of objects are stored in a dictionary "__dict__", this is the reason, why you can dynamically add attributes to objects of classes that we have created so far. Using a dictionary for attribute storage is very convenient, but it can mean a waste of space for objects, which have only a small amount of instance variables.
Slots are a nice way to work around this space consumption problem. Instead of having a dynamic dict that allows adding attributes to objects dynamically, slots provide a static structure which prohibits additions after the creation of an instance.
When we design a class, we can use slots to prevent the dynamic creation of attributes. To define slots, you have to define a list with the name __slots__. The list has to contain all the attributes, you want to use. We demonstrate this in the following class, in which the slots list contains only the name for an attribute "val".
class S(object):
__slots__ = ['val']
def __init__(self, v):
self.val = v
x = S(42)
print(x.val)
x.new = "not possible"
=> It fails to create an attribute "new":
42
Traceback (most recent call last):
File "slots_ex.py", line 12, in <module>
x.new = "not possible"
AttributeError: 'S' object has no attribute 'new'
Notes:
Since Python 3.3 the advantage optimizing the space consumption is not as impressive any more. With Python 3.3 Key-Sharing Dictionaries are used for the storage of objects. The attributes of the instances are capable of sharing part of their internal storage between each other, i.e. the part which stores the keys and their corresponding hashes. This helps to reduce the memory consumption of programs, which create many instances of non-builtin types. But still is the way to go to avoid dynamically created attributes.
Using slots come also with it's own cost. It will break serialization (e.g. pickle). It will also break multiple inheritance. A class can't inherit from more than one class that either defines slots or has an instance layout defined in C code (like list, tuple or int).
If someone is interested in doing that with a decorator, here is a working solution:
from functools import wraps
def froze_it(cls):
cls.__frozen = False
def frozensetattr(self, key, value):
if self.__frozen and not hasattr(self, key):
print("Class {} is frozen. Cannot set {} = {}"
.format(cls.__name__, key, value))
else:
object.__setattr__(self, key, value)
def init_decorator(func):
#wraps(func)
def wrapper(self, *args, **kwargs):
func(self, *args, **kwargs)
self.__frozen = True
return wrapper
cls.__setattr__ = frozensetattr
cls.__init__ = init_decorator(cls.__init__)
return cls
Pretty straightforward to use:
#froze_it
class Foo(object):
def __init__(self):
self.bar = 10
foo = Foo()
foo.bar = 42
foo.foobar = "no way"
Result:
>>> Class Foo is frozen. Cannot set foobar = no way
Actually, you don't want __setattr__, you want __slots__. Add __slots__ = ('foo', 'bar', 'baz') to the class body, and Python will make sure that there's only foo, bar and baz on any instance. But read the caveats the documentation lists!
The proper way is to override __setattr__. That's what it's there for.
I like very much the solution that uses a decorator, because it's easy to use it for many classes across a project, with minimum additions for each class. But it doesn't work well with inheritance.
So here is my version: It only overrides the __setattr__ function - if the attribute doesn't exist and the caller function is not __init__, it prints an error message.
import inspect
def froze_it(cls):
def frozensetattr(self, key, value):
if not hasattr(self, key) and inspect.stack()[1][3] != "__init__":
print("Class {} is frozen. Cannot set {} = {}"
.format(cls.__name__, key, value))
else:
self.__dict__[key] = value
cls.__setattr__ = frozensetattr
return cls
#froze_it
class A:
def __init__(self):
self._a = 0
a = A()
a._a = 1
a._b = 2 # error
What about this:
class A():
__allowed_attr=('_x', '_y')
def __init__(self,x=0,y=0):
self._x=x
self._y=y
def __setattr__(self,attribute,value):
if not attribute in self.__class__.__allowed_attr:
raise AttributeError
else:
super().__setattr__(attribute,value)
Here is approach i came up with that doesn't need a _frozen attribute or method to freeze() in init.
During init i just add all class attributes to the instance.
I like this because there is no _frozen, freeze(), and _frozen also does not show up in the vars(instance) output.
class MetaModel(type):
def __setattr__(self, name, value):
raise AttributeError("Model classes do not accept arbitrary attributes")
class Model(object):
__metaclass__ = MetaModel
# init will take all CLASS attributes, and add them as SELF/INSTANCE attributes
def __init__(self):
for k, v in self.__class__.__dict__.iteritems():
if not k.startswith("_"):
self.__setattr__(k, v)
# setattr, won't allow any attributes to be set on the SELF/INSTANCE that don't already exist
def __setattr__(self, name, value):
if not hasattr(self, name):
raise AttributeError("Model instances do not accept arbitrary attributes")
else:
object.__setattr__(self, name, value)
# Example using
class Dog(Model):
name = ''
kind = 'canine'
d, e = Dog(), Dog()
print vars(d)
print vars(e)
e.junk = 'stuff' # fails
I like the "Frozen" of Jochen Ritzel. The inconvenient is that the isfrozen variable then appears when printing a Class.__dict
I went around this problem this way by creating a list of authorized attributes (similar to slots):
class Frozen(object):
__List = []
def __setattr__(self, key, value):
setIsOK = False
for item in self.__List:
if key == item:
setIsOK = True
if setIsOK == True:
object.__setattr__(self, key, value)
else:
raise TypeError( "%r has no attributes %r" % (self, key) )
class Test(Frozen):
_Frozen__List = ["attr1","attr2"]
def __init__(self):
self.attr1 = 1
self.attr2 = 1
The FrozenClass by Jochen Ritzel is cool, but calling _frozen() when initialing a class every time is not so cool (and you need to take the risk of forgetting it). I added a __init_slots__ function:
class FrozenClass(object):
__isfrozen = False
def _freeze(self):
self.__isfrozen = True
def __init_slots__(self, slots):
for key in slots:
object.__setattr__(self, key, None)
self._freeze()
def __setattr__(self, key, value):
if self.__isfrozen and not hasattr(self, key):
raise TypeError( "%r is a frozen class" % self )
object.__setattr__(self, key, value)
class Test(FrozenClass):
def __init__(self):
self.__init_slots__(["x", "y"])
self.x = 42#
self.y = 2**3
a,b = Test(), Test()
a.x = 10
b.z = 10 # fails
None of the answers mention the performance impact of overriding __setattr__, which can be an issue when creating many small objects. (And __slots__ would be the performant solution but limits pickle/inheritance).
So I came up with this variant which installs our slower settatr after init:
class FrozenClass:
def freeze(self):
def frozen_setattr(self, key, value):
if not hasattr(self, key):
raise TypeError("Cannot set {}: {} is a frozen class".format(key, self))
object.__setattr__(self, key, value)
self.__setattr__ = frozen_setattr
class Foo(FrozenClass): ...
If you don't want to call freeze at the end of __init__, if inheritance is an issue, or if you don't want it in vars(), it can also be adapted: for example here is a decorator version based on the pystrict answer:
import functools
def strict(cls):
cls._x_setter = getattr(cls, "__setattr__", object.__setattr__)
cls._x_init = cls.__init__
#functools.wraps(cls.__init__)
def wrapper(self, *args, **kwargs):
cls._x_init(self, *args, **kwargs)
def frozen_setattr(self, key, value):
if not hasattr(self, key):
raise TypeError("Class %s is frozen. Cannot set '%s'." % (cls.__name__, key))
cls._x_setter(self, key, value)
cls.__setattr__ = frozen_setattr
cls.__init__ = wrapper
return cls
#strict
class Foo: ...
I wrote pystrict as a solution to this problem. It's too large to paste all of the code in stackoverflow.
pystrict is a pypi installable decorator that can be used with classes to freeze them. Many solutions here don't properly support inheritance.
If __slots__ doesn't work for you (because of inheritance issues), this is a good alternative.
There is an example to the README that shows why a decorator like this is needed even if you have mypy and pylint running on your project:
pip install pystrict
Then just use the #strict decorator:
from pystrict import strict
#strict
class Blah
def __init__(self):
self.attr = 1
#dataclass(slots=True) Nirvana (Python 3.10)
I'm in love with this #dataclass thing:
main.py
from dataclasses import dataclass
#dataclass(slots=True)
class C:
n: int
s: str
c = C(n=1, s='one')
assert c.n == 1
assert c.s == 'one'
c.n == 2
c.s == 'two'
c.asdf = 2
Outcome:
Traceback (most recent call last):
File "/home/ciro/main.py", line 15, in <module>
c.asdf = 2
AttributeError: 'C' object has no attribute 'asdf'
Note how #dataclass only requires use to define our attributes once with type annotations
n: int
s: str
and then, without any repetition we get for free:
def __init__(n, s):
self.n = n
self.s = s
__slots__ = ['n', 's']
Other free things not shown in this example:
__str__
__eq__: Compare object instances for equality by their attributes
__hash__ if you also use frozen=True: Object of custom type as dictionary key
Tested on Python 3.10.7, Ubuntu 22.10.