I want to be able to create a class (in Python) that once initialized with __init__, does not accept new attributes, but accepts modifications of existing attributes. There's several hack-ish ways I can see to do this, for example having a __setattr__ method such as
def __setattr__(self, attribute, value):
if not attribute in self.__dict__:
print "Cannot set %s" % attribute
else:
self.__dict__[attribute] = value
and then editing __dict__ directly inside __init__, but I was wondering if there is a 'proper' way to do this?
I wouldn't use __dict__ directly, but you can add a function to explicitly "freeze" a instance:
class FrozenClass(object):
__isfrozen = False
def __setattr__(self, key, value):
if self.__isfrozen and not hasattr(self, key):
raise TypeError( "%r is a frozen class" % self )
object.__setattr__(self, key, value)
def _freeze(self):
self.__isfrozen = True
class Test(FrozenClass):
def __init__(self):
self.x = 42#
self.y = 2**3
self._freeze() # no new attributes after this point.
a,b = Test(), Test()
a.x = 10
b.z = 10 # fails
Slots is the way to go:
The pythonic way is to use slots instead of playing around with the __setter__. While it may solve the problem, it does not give any performance improvement. The attributes of objects are stored in a dictionary "__dict__", this is the reason, why you can dynamically add attributes to objects of classes that we have created so far. Using a dictionary for attribute storage is very convenient, but it can mean a waste of space for objects, which have only a small amount of instance variables.
Slots are a nice way to work around this space consumption problem. Instead of having a dynamic dict that allows adding attributes to objects dynamically, slots provide a static structure which prohibits additions after the creation of an instance.
When we design a class, we can use slots to prevent the dynamic creation of attributes. To define slots, you have to define a list with the name __slots__. The list has to contain all the attributes, you want to use. We demonstrate this in the following class, in which the slots list contains only the name for an attribute "val".
class S(object):
__slots__ = ['val']
def __init__(self, v):
self.val = v
x = S(42)
print(x.val)
x.new = "not possible"
=> It fails to create an attribute "new":
42
Traceback (most recent call last):
File "slots_ex.py", line 12, in <module>
x.new = "not possible"
AttributeError: 'S' object has no attribute 'new'
Notes:
Since Python 3.3 the advantage optimizing the space consumption is not as impressive any more. With Python 3.3 Key-Sharing Dictionaries are used for the storage of objects. The attributes of the instances are capable of sharing part of their internal storage between each other, i.e. the part which stores the keys and their corresponding hashes. This helps to reduce the memory consumption of programs, which create many instances of non-builtin types. But still is the way to go to avoid dynamically created attributes.
Using slots come also with it's own cost. It will break serialization (e.g. pickle). It will also break multiple inheritance. A class can't inherit from more than one class that either defines slots or has an instance layout defined in C code (like list, tuple or int).
If someone is interested in doing that with a decorator, here is a working solution:
from functools import wraps
def froze_it(cls):
cls.__frozen = False
def frozensetattr(self, key, value):
if self.__frozen and not hasattr(self, key):
print("Class {} is frozen. Cannot set {} = {}"
.format(cls.__name__, key, value))
else:
object.__setattr__(self, key, value)
def init_decorator(func):
#wraps(func)
def wrapper(self, *args, **kwargs):
func(self, *args, **kwargs)
self.__frozen = True
return wrapper
cls.__setattr__ = frozensetattr
cls.__init__ = init_decorator(cls.__init__)
return cls
Pretty straightforward to use:
#froze_it
class Foo(object):
def __init__(self):
self.bar = 10
foo = Foo()
foo.bar = 42
foo.foobar = "no way"
Result:
>>> Class Foo is frozen. Cannot set foobar = no way
Actually, you don't want __setattr__, you want __slots__. Add __slots__ = ('foo', 'bar', 'baz') to the class body, and Python will make sure that there's only foo, bar and baz on any instance. But read the caveats the documentation lists!
The proper way is to override __setattr__. That's what it's there for.
I like very much the solution that uses a decorator, because it's easy to use it for many classes across a project, with minimum additions for each class. But it doesn't work well with inheritance.
So here is my version: It only overrides the __setattr__ function - if the attribute doesn't exist and the caller function is not __init__, it prints an error message.
import inspect
def froze_it(cls):
def frozensetattr(self, key, value):
if not hasattr(self, key) and inspect.stack()[1][3] != "__init__":
print("Class {} is frozen. Cannot set {} = {}"
.format(cls.__name__, key, value))
else:
self.__dict__[key] = value
cls.__setattr__ = frozensetattr
return cls
#froze_it
class A:
def __init__(self):
self._a = 0
a = A()
a._a = 1
a._b = 2 # error
What about this:
class A():
__allowed_attr=('_x', '_y')
def __init__(self,x=0,y=0):
self._x=x
self._y=y
def __setattr__(self,attribute,value):
if not attribute in self.__class__.__allowed_attr:
raise AttributeError
else:
super().__setattr__(attribute,value)
Here is approach i came up with that doesn't need a _frozen attribute or method to freeze() in init.
During init i just add all class attributes to the instance.
I like this because there is no _frozen, freeze(), and _frozen also does not show up in the vars(instance) output.
class MetaModel(type):
def __setattr__(self, name, value):
raise AttributeError("Model classes do not accept arbitrary attributes")
class Model(object):
__metaclass__ = MetaModel
# init will take all CLASS attributes, and add them as SELF/INSTANCE attributes
def __init__(self):
for k, v in self.__class__.__dict__.iteritems():
if not k.startswith("_"):
self.__setattr__(k, v)
# setattr, won't allow any attributes to be set on the SELF/INSTANCE that don't already exist
def __setattr__(self, name, value):
if not hasattr(self, name):
raise AttributeError("Model instances do not accept arbitrary attributes")
else:
object.__setattr__(self, name, value)
# Example using
class Dog(Model):
name = ''
kind = 'canine'
d, e = Dog(), Dog()
print vars(d)
print vars(e)
e.junk = 'stuff' # fails
I like the "Frozen" of Jochen Ritzel. The inconvenient is that the isfrozen variable then appears when printing a Class.__dict
I went around this problem this way by creating a list of authorized attributes (similar to slots):
class Frozen(object):
__List = []
def __setattr__(self, key, value):
setIsOK = False
for item in self.__List:
if key == item:
setIsOK = True
if setIsOK == True:
object.__setattr__(self, key, value)
else:
raise TypeError( "%r has no attributes %r" % (self, key) )
class Test(Frozen):
_Frozen__List = ["attr1","attr2"]
def __init__(self):
self.attr1 = 1
self.attr2 = 1
The FrozenClass by Jochen Ritzel is cool, but calling _frozen() when initialing a class every time is not so cool (and you need to take the risk of forgetting it). I added a __init_slots__ function:
class FrozenClass(object):
__isfrozen = False
def _freeze(self):
self.__isfrozen = True
def __init_slots__(self, slots):
for key in slots:
object.__setattr__(self, key, None)
self._freeze()
def __setattr__(self, key, value):
if self.__isfrozen and not hasattr(self, key):
raise TypeError( "%r is a frozen class" % self )
object.__setattr__(self, key, value)
class Test(FrozenClass):
def __init__(self):
self.__init_slots__(["x", "y"])
self.x = 42#
self.y = 2**3
a,b = Test(), Test()
a.x = 10
b.z = 10 # fails
None of the answers mention the performance impact of overriding __setattr__, which can be an issue when creating many small objects. (And __slots__ would be the performant solution but limits pickle/inheritance).
So I came up with this variant which installs our slower settatr after init:
class FrozenClass:
def freeze(self):
def frozen_setattr(self, key, value):
if not hasattr(self, key):
raise TypeError("Cannot set {}: {} is a frozen class".format(key, self))
object.__setattr__(self, key, value)
self.__setattr__ = frozen_setattr
class Foo(FrozenClass): ...
If you don't want to call freeze at the end of __init__, if inheritance is an issue, or if you don't want it in vars(), it can also be adapted: for example here is a decorator version based on the pystrict answer:
import functools
def strict(cls):
cls._x_setter = getattr(cls, "__setattr__", object.__setattr__)
cls._x_init = cls.__init__
#functools.wraps(cls.__init__)
def wrapper(self, *args, **kwargs):
cls._x_init(self, *args, **kwargs)
def frozen_setattr(self, key, value):
if not hasattr(self, key):
raise TypeError("Class %s is frozen. Cannot set '%s'." % (cls.__name__, key))
cls._x_setter(self, key, value)
cls.__setattr__ = frozen_setattr
cls.__init__ = wrapper
return cls
#strict
class Foo: ...
I wrote pystrict as a solution to this problem. It's too large to paste all of the code in stackoverflow.
pystrict is a pypi installable decorator that can be used with classes to freeze them. Many solutions here don't properly support inheritance.
If __slots__ doesn't work for you (because of inheritance issues), this is a good alternative.
There is an example to the README that shows why a decorator like this is needed even if you have mypy and pylint running on your project:
pip install pystrict
Then just use the #strict decorator:
from pystrict import strict
#strict
class Blah
def __init__(self):
self.attr = 1
#dataclass(slots=True) Nirvana (Python 3.10)
I'm in love with this #dataclass thing:
main.py
from dataclasses import dataclass
#dataclass(slots=True)
class C:
n: int
s: str
c = C(n=1, s='one')
assert c.n == 1
assert c.s == 'one'
c.n == 2
c.s == 'two'
c.asdf = 2
Outcome:
Traceback (most recent call last):
File "/home/ciro/main.py", line 15, in <module>
c.asdf = 2
AttributeError: 'C' object has no attribute 'asdf'
Note how #dataclass only requires use to define our attributes once with type annotations
n: int
s: str
and then, without any repetition we get for free:
def __init__(n, s):
self.n = n
self.s = s
__slots__ = ['n', 's']
Other free things not shown in this example:
__str__
__eq__: Compare object instances for equality by their attributes
__hash__ if you also use frozen=True: Object of custom type as dictionary key
Tested on Python 3.10.7, Ubuntu 22.10.
The following program is unable to create a function of a class
class MyClass(object):
def __init__(self, name=""):
self.name = name
def read_name(self):
return self.name
# First argument should be a ref to class
def callback(fcn, arg):
fcn.name=arg
# Create a instance of class
a = MyClass("Blue")
# Lets add new member functions
setattr(a, 'callback1', callback)
setattr(a, 'callback2', callback)
print a.read_name()
print a.callback1("purple") #! FAILS
print a.callback2("cyan") #! FAILS
What is the right way of creating a class member function automatically?
I want to create 'N' callback functions, they all will modify some common/uncommon class data (A shared dict)
EDIT 1
I wish to collect information from 'N' separate/parallel threads by passing callback functions. I do not know beforehand how many callback functions I need thus I want to create them on fly.
EDIT 2
I have a dictionary(d) where I am storing the information of different processes. The dictionary(d) is accessed within the callback. But because the same callback function is called at different threads, the dictionary data gets garbled. As a quickfix, I thought of creating separate callbacks.
If you know what you're doing, you'd want to try
import types
setattr(a, 'callback1', types.MethodType(callback, a, MyClass))
In short: when grafting a method, assign it to the class, not to the instance.
Here's an elucidating example.
class A(object):
"""As trivial as a class can get."""
def foo(self):
return self.bar(1) + self.baz()
# Rework everything!
def new_bar(self, x):
return 'I got %r' % x
def new_baz(self):
return ' and I\'m okay!'
A.bar = new_bar
A.baz = new_baz
print A().foo()
Now grafting method to an instance.
a = A()
# An instance attribute is a bound method;
# when we replace it with a function, we lose access to self.
a.bar = lambda x: x * 100
A.baz = lambda self: 42
assert a.foo() == 142
# We can do better, though.
from types import MethodType
a2 = A()
a2.foo = MethodType(lambda self: 'I know myself, my class is %s' % self.__class__.__name__, a2)
print a2.foo()
Note how you don't need setattr to set an attribute, even an unknown attribute. You may remember that you don't use setattr in __init__ either.
You can't add a class method to an instance; you have to add it to the class:
setattr(MyClass, 'callback1', callback)
But it's still a terrible idea. Why would you want this functionality?
Edit: keep your callbacks in a container instead:
class MyClass(object):
def __init__(self, name=""):
self.name = name
self.callbacks = []
def callback(self, idx, arg):
self.callbacks[idx](self, arg)
# First argument should be a ref to class
def callback(fcn, arg):
fcn.name=arg
# Create a instance of class
a = MyClass("Blue")
# Lets add new member functions
a.callbacks.append(callback)
a.callbacks.append(callback)
print a.name
a.callback(0, "purple")
print a.name
a.callback(1, "cyan")
print a.name
I have a dictionary which stores objects of a class foo. Class foo has an attribute Name. For every instance, I want the Name attribute to be the key to the instance in the dictionary. All instances of the class will be defined inside the dictionary.
class foo:
def __init__(self):
self.Name = None #self.Name should equal "self"
foo_dict = {
'foo1' = foo()
}
#(foo.Name should equal 'foo1')
How can I set the Name attribute to be the key to the instance in the dictionary?
Comment if specifications are needed.
I can't possibly stress enough how BAD this is... Please, please, use this only for educational purposes. It's crumbly, unreliable... BAD If you change anything in your code, it'll stop working. It is dirty. It is possibly non portable... OMG... I think a few kittens were killed when I hit Post Your Answer
import inspect
import re
class Foo(object):
def __init__(self):
r = re.compile(
r"\W+['\"](?P<name>\w+)['\"]\W+%s\W+"
% self.__class__.__name__
)
caller_frame = inspect.currentframe().f_back
code_context = inspect.getframeinfo(caller_frame).code_context
match = r.match(''.join(code_context))
if match:
self.name = match.groupdict()['name']
print "Assigned name: %s" % self.name
else:
raise Exception("This wasn't called as it was supposed to")
if __name__ == "__main__":
foo_dict = {
'foo1': Foo(),
'foo2': Foo(),
}
But it does what you seem to be asking:
borrajax#borrajax:/tmp$ python ./test.py
Assigned name: foo1
Assigned name: foo2
Now, what I would do is:
Option 1:
Pass the name in the initialization.
Possibly the simplest, most maintainable and that leaves the code in a much clearer state (important if someone else reads your code)
class Foo(object):
def __init__(self, name):
self.name = name
print "Assigned name: %s" % self.name
if __name__ == "__main__":
foo_dict = {
'foo1': Foo('foo1'),
'foo2': Foo('foo2'),
}
Option 2:
Create your own dict class and overwrite the __setitem__ method (see also Subclassing Python dictionary to override __setitem__ and How to "perfectly" override a dict?):
class Foo(object):
pass
class MyDict(dict):
def __setitem__(self, key, val):
if not isinstance(val, Foo):
raise TypeError("My dict only accepts %s" % Foo)
val.name = key
print "Assigned name: %s" % val.name
return super(MyDict, self).__setitem__(key, val)
if __name__ == "__main__":
foo_dict = MyDict()
foo_dict['foo1'] = Foo()
foo_dict['foo2'] = Foo()
foo_dict['foo3'] = 1
Prints:
borrajax#borrajax:/tmp$ python ./test.py
Assigned name: foo1
Assigned name: foo2
Traceback (most recent call last):
File "./stack64.py", line 17, in <module>
foo_dict['foo3'] = 1
File "./stack64.py", line 8, in __setitem__
raise TypeError("My dict only accepts %s" % Foo)
TypeError: My dict only accepts <class '__main__.Foo'>
This has the disadvantage of magically adding attributes (the .name) to the instances of Foo when assigned to the dictionary, which can cause name conflicts (if your Foo class already had a .name, this method would change its value). In general, I'd stay away of methods that magically add attributes to instances in the middle of the execution.
Option 3:
Use #Daniel's answer to this question. Clean and understandable for someone else reading your code.
Seems like you need a reference to the instance to do what you want. If you build the dictionary with a comprehension, you can create instance references and use them.
class Foo(object):
def __init__(self, n = None):
self.name = n
d = {f.name:f for f in (Foo(n) for n in 'abcd')}
>>> d
{'a': <__main__.Foo object at 0x03DF9710>, 'c': <__main__.Foo object at 0x03E01250>, 'b': <__main__.Foo object at 0x03DF9A50>, 'd': <__main__.Foo object at 0x03E01290>}
>>>
>>> d = {f.name:f for f in (Foo(n) for n in [1])}
>>> d
{1: <__main__.Foo object at 0x03E01B50>}
>>> foo_dict = {}
>>> foo_dict.update(d)
>>> foo_dict
{1: <__main__.Foo object at 0x03E01B50>}
>>>
I stumbled upon this SO answer the other day. Using that class decorator/descriptor, you could create a class factory that produces Foo objects and keeps track of the current object and a counter for the next object.
class InnerClassDescriptor(object):
'''allows access to the outer class and its attributes
decorator/descriptor
an instance of a nested inner class can access the outer class and its attributes
'''
def __init__(self, cls):
self.cls = cls
def __get__(self, instance, outerclass):
class Wrapper(self.cls):
outer = instance
Wrapper.__name__ = self.cls.__name__
return Wrapper
class FooFactory(object):
next_foo = 0
this_foo = None
#InnerClassDescriptor
class Foo(object):
def __init__(self):
# print 'Foo,__init__, next_foo = ', self.outer.next_foo
self.name = 'Foo' + str(self.outer.next_foo)
self.outer.next_foo += 1
self.outer.this_foo = self
Usage:
ff = FooFactory()
d = {ff.this_foo.name:ff.Foo()}
for k, v in d.items():
print k, v.name
>>>
Foo0 Foo0
>>>
This relies on the dictionary item value being evaluated before the key - which seems to be the case for Python 2.7
Do it the other way round:
class Foo:
def __init__(self, name=None):
self.name = name
foo1 = Foo('foo1')
foo_dict = {
foo1.name: foo1
}
I have a dict of different types for which I want to add a simple getter based on the name of the actual parameter.
For example, for three storage parameters, let's say:
self.storage = {'total':100,'used':88,'free':1}
I am looking now for a way (if possible?) to generate a function on the fly with some meta-programming magic.
Instead of
class spaceObj(object):
def getSize(what='total'):
return storage[what]
or hard coding
#property
def getSizeTotal():
return storage['total']
but
class spaceObj(object):
# manipulting the object's index and magic
#property
def getSize:
return ???
so that calling mySpaceObj.getSizeFree would be derived - with getSize only defined once in the object and related functions derived from it by manipulating the objects function list.
Is something like that possible?
While certainly possible to get an unknown attribute from a class as a property, this is not a pythonic approach (__getattr__ magic methods are rather rubyist)
class spaceObj(object):
storage = None
def __init__(self): # this is for testing only
self.storage = {'total':100,'used':88,'free':1}
def __getattr__(self, item):
if item[:7] == 'getSize': # check if an undefined attribute starts with this
return self.getSize(item[7:])
def getSize(self, what='total'):
return self.storage[what.lower()]
print (spaceObj().getSizeTotal) # 100
You can put the values into the object as properties:
class SpaceObj(object):
def __init__(self, **kwargs):
self.__dict__.update(kwargs)
storage = {'total':100,'used':88,'free':1}
o = SpaceObj(**storage)
print o.total
or
o = SpaceObj(total=100, used=88, free=1)
print o.total
or using __getattr__:
class SpaceObj(object):
def __init__(self, **kwargs):
self.storage = kwargs
def __getattr__(self,name):
return self.storage[name]
o = SpaceObj(total=100, used=88, free=1)
print o.total
The latter approach takes a bit more code but it's more safe; if you have a method foo and someone create the instance with SpaceObj(foo=1), then the method will be overwritten with the first approach.
>>> import new
>>> funcstr = "def wat(): print \"wat\";return;"
>>> funcbin = compile(funcstr,'','exec')
>>> ns = {}
>>> exec funcbin in ns
>>> watfunction = new.function(ns["wat"].func_code,globals(),"wat")
>>> globals()["wat"]=watfunction
>>> wat()
wat
I have this (Py2.7.2):
class MyClass(object):
def __init__(self, dict_values):
self.values = dict_values
self.changed_values = {} #this should track changes done to the values{}
....
I can use it like this:
var = MyClass()
var.values['age'] = 21
var.changed_values['age'] = 21
But I want to use it like this:
var.age = 21
print var.changed_values #prints {'age':21}
I suspect I can use properties to do that, but how?
UPDATE:
I don't know the dict contents at the design time. It will be known at run-time only. And it will likely to be not empty
You can create a class that inherits from a dict and override the needed functions
class D(dict):
def __init__(self):
self.changed_values = {}
self.__initialized = True
def __setitem__(self, key, value):
self.changed_values[key] = value
super(D, self).__setitem__(key, value)
def __getattr__(self, item):
"""Maps values to attributes.
Only called if there *isn't* an attribute with this name
"""
try:
return self.__getitem__(item)
except KeyError:
raise AttributeError(item)
def __setattr__(self, item, value):
"""Maps attributes to values.
Only if we are initialised
"""
if not self.__dict__.has_key('_D__initialized'): # this test allows attributes to be set in the __init__ method
return dict.__setattr__(self, item, value)
elif self.__dict__.has_key(item): # any normal attributes are handled normally
dict.__setattr__(self, item, value)
else:
self.__setitem__(item, value)
a = D()
a['hi'] = 'hello'
print a.hi
print a.changed_values
a.hi = 'wow'
print a.hi
print a.changed_values
a.test = 'test1'
print a.test
print a.changed_values
output
>>hello
>>{'hi': 'hello'}
>>wow
>>{'hi': 'wow'}
>>test1
>>{'hi': 'wow', 'test': 'test1'}
Properties (descriptors, really) will only help if the set of attributes to monitor is bounded. Simply file the new value away in the __set__() method of the descriptor.
If the set of attributes is arbitrary or unbounded then you will need to overrive MyClass.__setattr__() instead.
You can use the property() built-in function.
This is preferred to overriding __getattr__ and __setattr__, as explained here.
class MyClass:
def __init__(self):
self.values = {}
self.changed_values = {}
def set_age( nr ):
self.values['age'] = nr
self.changed_values['age'] = nr
def get_age():
return self.values['age']
age = property(get_age,set_age)