While looking over some code in Think Complexity, I noticed their Graph class assigning values to itself. I've copied a few important lines from that class and written an example class, ObjectChild, that fails at this behavior.
class Graph(dict):
def __init__(self, vs=[], es=[]):
for v in vs:
self.add_vertex(v)
for e in es:
self.add_edge(e)
def add_edge(self, e):
v, w = e
self[v][w] = e
self[w][v] = e
def add_vertex(self, v):
self[v] = {}
class ObjectChild(object):
def __init__(self, name):
self['name'] = name
I'm sure the different built in types all have their own way of using this, but I'm not sure whether this is something I should try to build into my classes. Is it possible, and how? Is this something I shouldn't bother with, relying instead on simple composition, e.g. self.l = [1, 2, 3]? Should it be avoided outside built in types?
I ask because I was told "You should almost never inherit from the builtin python collections"; advice I'm hesitant to restrict myself to.
To clarify, I know that ObjectChild won't "work", and I could easily make it "work", but I'm curious about the inner workings of these built in types that makes their interface different from a child of object.
In Python 3 and later, just add these simple functions to your class:
class some_class(object):
def __setitem__(self, key, value):
setattr(self, key, value)
def __getitem__(self, key):
return getattr(self, key)
They are accomplishing this magic by inheriting from dict. A better way of doing this is to inherit from UserDict or the newer collections.MutableMapping
You could accomplish a similar result by doing the same:
import collections
class ObjectChild(collections.MutableMapping):
def __init__(self, name):
self['name'] = name
You can also define two special functions to make your class dictionary-like: __getitem__(self, key) and __setitem__(self, key, value). You can see an example of this at Dive Into Python - Special Class Methods.
Disclaimer : I might be wrong.
the notation :
self[something]
is legit in the Graph class because it inherits fro dict. This notation is from the dictionnaries ssyntax not from the class attribute declaration syntax.
Although all namespaces associated with a class are dictionnaries, in your class ChildObject, self isn't a dictionnary. Therefore you can't use that syntax.
Otoh, in your class Graph, self IS a dictionnary, since it is a graph, and all graphs are dictionnaries because they inherit from dict.
Is using something like this ok?
def mk_opts_dict(d):
''' mk_options_dict(dict) -> an instance of OptionsDict '''
class OptionsDict(object):
def __init__(self, d):
self.__dict__ = d
def __setitem__(self, key, value):
self.__dict__[key] = value
def __getitem__(self, key):
return self.__dict__[key]
return OptionsDict(d)
I realize this is an old post, but I was looking for some details around item assignment and stumbled upon the answers here. Ted's post wasn't completely wrong. To avoid inheritance from dict, you can make a class inherit from MutableMapping, and then provide methods for __setitem__ and __getitem__.
Additionally, the class will need to support methods for __delitem__, __iter__, __len__, and (optionally) other inherited mixin methods, like pop. The documentation has more info on the details.
from collections.abc import MutableMapping
class ItemAssign(MutableMapping):
def __init__(self, a, b):
self.a = a
self.b = b
def __setitem__(self, k, v):
setattr(self, k, v)
def __getitem__(self, k):
getattr(self, k)
def __len__(self):
return 2
def __delitem__(self, k):
self[k] = None
def __iter__(self):
yield self.a
yield self.b
Example use:
>>> x = ItemAssign("banana","apple")
>>> x["a"] = "orange"
>>> x.a
'orange'
>>> del x["a"]
>>> print(x.a)
None
>>> x.pop("b")
'apple'
>>> print(x.b)
None
Hope this serves to clarify how to properly implement item assignment for others stumbling across this post :)
Your ObjectChild doesn't work because it's not a subclass of dict. Either of these would work:
class ObjectChild(dict):
def __init__(self, name):
self['name'] = name
or
class ObjectChild(object):
def __init__(self, name):
self.name = name
You don't need to inherit from dict. If you provide setitem and getitem methods, you also get the desired behavior I believe.
class a(object):
def __setitem__(self, k, v):
self._data[k] = v
def __getitem__(self, k):
return self._data[k]
_data = {}
Little memo about <dict> inheritance
For those who want to inherit dict.
In this case MyDict will have a shallow copy of original dict in it.
class MyDict(dict):
...
d = {'a': 1}
md = MyDict(d)
print(d['a']) # 1
print(md['a']) # 1
md['a'] = 'new'
print(d['a']) # 1
print(md['a']) # new
This could lead to problem when you have a tree of nested dicts and you want to covert part of it to an object. Changing this object will not affect its parent
root = {
'obj': {
'a': 1,
'd': {'x': True}
}
}
obj = MyDict(root['obj'])
obj['a'] = 2
print(root) # {'obj': {'a': 1, 'd': {'x': True}}} # 'a' is the same
obj['d']['x'] = False
print(root) # {'obj': {'a': 1, 'd': {'x': True}}} # 'x' chanded
Related
I want to be able to create a class (in Python) that once initialized with __init__, does not accept new attributes, but accepts modifications of existing attributes. There's several hack-ish ways I can see to do this, for example having a __setattr__ method such as
def __setattr__(self, attribute, value):
if not attribute in self.__dict__:
print "Cannot set %s" % attribute
else:
self.__dict__[attribute] = value
and then editing __dict__ directly inside __init__, but I was wondering if there is a 'proper' way to do this?
I wouldn't use __dict__ directly, but you can add a function to explicitly "freeze" a instance:
class FrozenClass(object):
__isfrozen = False
def __setattr__(self, key, value):
if self.__isfrozen and not hasattr(self, key):
raise TypeError( "%r is a frozen class" % self )
object.__setattr__(self, key, value)
def _freeze(self):
self.__isfrozen = True
class Test(FrozenClass):
def __init__(self):
self.x = 42#
self.y = 2**3
self._freeze() # no new attributes after this point.
a,b = Test(), Test()
a.x = 10
b.z = 10 # fails
Slots is the way to go:
The pythonic way is to use slots instead of playing around with the __setter__. While it may solve the problem, it does not give any performance improvement. The attributes of objects are stored in a dictionary "__dict__", this is the reason, why you can dynamically add attributes to objects of classes that we have created so far. Using a dictionary for attribute storage is very convenient, but it can mean a waste of space for objects, which have only a small amount of instance variables.
Slots are a nice way to work around this space consumption problem. Instead of having a dynamic dict that allows adding attributes to objects dynamically, slots provide a static structure which prohibits additions after the creation of an instance.
When we design a class, we can use slots to prevent the dynamic creation of attributes. To define slots, you have to define a list with the name __slots__. The list has to contain all the attributes, you want to use. We demonstrate this in the following class, in which the slots list contains only the name for an attribute "val".
class S(object):
__slots__ = ['val']
def __init__(self, v):
self.val = v
x = S(42)
print(x.val)
x.new = "not possible"
=> It fails to create an attribute "new":
42
Traceback (most recent call last):
File "slots_ex.py", line 12, in <module>
x.new = "not possible"
AttributeError: 'S' object has no attribute 'new'
Notes:
Since Python 3.3 the advantage optimizing the space consumption is not as impressive any more. With Python 3.3 Key-Sharing Dictionaries are used for the storage of objects. The attributes of the instances are capable of sharing part of their internal storage between each other, i.e. the part which stores the keys and their corresponding hashes. This helps to reduce the memory consumption of programs, which create many instances of non-builtin types. But still is the way to go to avoid dynamically created attributes.
Using slots come also with it's own cost. It will break serialization (e.g. pickle). It will also break multiple inheritance. A class can't inherit from more than one class that either defines slots or has an instance layout defined in C code (like list, tuple or int).
If someone is interested in doing that with a decorator, here is a working solution:
from functools import wraps
def froze_it(cls):
cls.__frozen = False
def frozensetattr(self, key, value):
if self.__frozen and not hasattr(self, key):
print("Class {} is frozen. Cannot set {} = {}"
.format(cls.__name__, key, value))
else:
object.__setattr__(self, key, value)
def init_decorator(func):
#wraps(func)
def wrapper(self, *args, **kwargs):
func(self, *args, **kwargs)
self.__frozen = True
return wrapper
cls.__setattr__ = frozensetattr
cls.__init__ = init_decorator(cls.__init__)
return cls
Pretty straightforward to use:
#froze_it
class Foo(object):
def __init__(self):
self.bar = 10
foo = Foo()
foo.bar = 42
foo.foobar = "no way"
Result:
>>> Class Foo is frozen. Cannot set foobar = no way
Actually, you don't want __setattr__, you want __slots__. Add __slots__ = ('foo', 'bar', 'baz') to the class body, and Python will make sure that there's only foo, bar and baz on any instance. But read the caveats the documentation lists!
The proper way is to override __setattr__. That's what it's there for.
I like very much the solution that uses a decorator, because it's easy to use it for many classes across a project, with minimum additions for each class. But it doesn't work well with inheritance.
So here is my version: It only overrides the __setattr__ function - if the attribute doesn't exist and the caller function is not __init__, it prints an error message.
import inspect
def froze_it(cls):
def frozensetattr(self, key, value):
if not hasattr(self, key) and inspect.stack()[1][3] != "__init__":
print("Class {} is frozen. Cannot set {} = {}"
.format(cls.__name__, key, value))
else:
self.__dict__[key] = value
cls.__setattr__ = frozensetattr
return cls
#froze_it
class A:
def __init__(self):
self._a = 0
a = A()
a._a = 1
a._b = 2 # error
What about this:
class A():
__allowed_attr=('_x', '_y')
def __init__(self,x=0,y=0):
self._x=x
self._y=y
def __setattr__(self,attribute,value):
if not attribute in self.__class__.__allowed_attr:
raise AttributeError
else:
super().__setattr__(attribute,value)
Here is approach i came up with that doesn't need a _frozen attribute or method to freeze() in init.
During init i just add all class attributes to the instance.
I like this because there is no _frozen, freeze(), and _frozen also does not show up in the vars(instance) output.
class MetaModel(type):
def __setattr__(self, name, value):
raise AttributeError("Model classes do not accept arbitrary attributes")
class Model(object):
__metaclass__ = MetaModel
# init will take all CLASS attributes, and add them as SELF/INSTANCE attributes
def __init__(self):
for k, v in self.__class__.__dict__.iteritems():
if not k.startswith("_"):
self.__setattr__(k, v)
# setattr, won't allow any attributes to be set on the SELF/INSTANCE that don't already exist
def __setattr__(self, name, value):
if not hasattr(self, name):
raise AttributeError("Model instances do not accept arbitrary attributes")
else:
object.__setattr__(self, name, value)
# Example using
class Dog(Model):
name = ''
kind = 'canine'
d, e = Dog(), Dog()
print vars(d)
print vars(e)
e.junk = 'stuff' # fails
I like the "Frozen" of Jochen Ritzel. The inconvenient is that the isfrozen variable then appears when printing a Class.__dict
I went around this problem this way by creating a list of authorized attributes (similar to slots):
class Frozen(object):
__List = []
def __setattr__(self, key, value):
setIsOK = False
for item in self.__List:
if key == item:
setIsOK = True
if setIsOK == True:
object.__setattr__(self, key, value)
else:
raise TypeError( "%r has no attributes %r" % (self, key) )
class Test(Frozen):
_Frozen__List = ["attr1","attr2"]
def __init__(self):
self.attr1 = 1
self.attr2 = 1
The FrozenClass by Jochen Ritzel is cool, but calling _frozen() when initialing a class every time is not so cool (and you need to take the risk of forgetting it). I added a __init_slots__ function:
class FrozenClass(object):
__isfrozen = False
def _freeze(self):
self.__isfrozen = True
def __init_slots__(self, slots):
for key in slots:
object.__setattr__(self, key, None)
self._freeze()
def __setattr__(self, key, value):
if self.__isfrozen and not hasattr(self, key):
raise TypeError( "%r is a frozen class" % self )
object.__setattr__(self, key, value)
class Test(FrozenClass):
def __init__(self):
self.__init_slots__(["x", "y"])
self.x = 42#
self.y = 2**3
a,b = Test(), Test()
a.x = 10
b.z = 10 # fails
None of the answers mention the performance impact of overriding __setattr__, which can be an issue when creating many small objects. (And __slots__ would be the performant solution but limits pickle/inheritance).
So I came up with this variant which installs our slower settatr after init:
class FrozenClass:
def freeze(self):
def frozen_setattr(self, key, value):
if not hasattr(self, key):
raise TypeError("Cannot set {}: {} is a frozen class".format(key, self))
object.__setattr__(self, key, value)
self.__setattr__ = frozen_setattr
class Foo(FrozenClass): ...
If you don't want to call freeze at the end of __init__, if inheritance is an issue, or if you don't want it in vars(), it can also be adapted: for example here is a decorator version based on the pystrict answer:
import functools
def strict(cls):
cls._x_setter = getattr(cls, "__setattr__", object.__setattr__)
cls._x_init = cls.__init__
#functools.wraps(cls.__init__)
def wrapper(self, *args, **kwargs):
cls._x_init(self, *args, **kwargs)
def frozen_setattr(self, key, value):
if not hasattr(self, key):
raise TypeError("Class %s is frozen. Cannot set '%s'." % (cls.__name__, key))
cls._x_setter(self, key, value)
cls.__setattr__ = frozen_setattr
cls.__init__ = wrapper
return cls
#strict
class Foo: ...
I wrote pystrict as a solution to this problem. It's too large to paste all of the code in stackoverflow.
pystrict is a pypi installable decorator that can be used with classes to freeze them. Many solutions here don't properly support inheritance.
If __slots__ doesn't work for you (because of inheritance issues), this is a good alternative.
There is an example to the README that shows why a decorator like this is needed even if you have mypy and pylint running on your project:
pip install pystrict
Then just use the #strict decorator:
from pystrict import strict
#strict
class Blah
def __init__(self):
self.attr = 1
#dataclass(slots=True) Nirvana (Python 3.10)
I'm in love with this #dataclass thing:
main.py
from dataclasses import dataclass
#dataclass(slots=True)
class C:
n: int
s: str
c = C(n=1, s='one')
assert c.n == 1
assert c.s == 'one'
c.n == 2
c.s == 'two'
c.asdf = 2
Outcome:
Traceback (most recent call last):
File "/home/ciro/main.py", line 15, in <module>
c.asdf = 2
AttributeError: 'C' object has no attribute 'asdf'
Note how #dataclass only requires use to define our attributes once with type annotations
n: int
s: str
and then, without any repetition we get for free:
def __init__(n, s):
self.n = n
self.s = s
__slots__ = ['n', 's']
Other free things not shown in this example:
__str__
__eq__: Compare object instances for equality by their attributes
__hash__ if you also use frozen=True: Object of custom type as dictionary key
Tested on Python 3.10.7, Ubuntu 22.10.
I want to be able to create a class (in Python) that once initialized with __init__, does not accept new attributes, but accepts modifications of existing attributes. There's several hack-ish ways I can see to do this, for example having a __setattr__ method such as
def __setattr__(self, attribute, value):
if not attribute in self.__dict__:
print "Cannot set %s" % attribute
else:
self.__dict__[attribute] = value
and then editing __dict__ directly inside __init__, but I was wondering if there is a 'proper' way to do this?
I wouldn't use __dict__ directly, but you can add a function to explicitly "freeze" a instance:
class FrozenClass(object):
__isfrozen = False
def __setattr__(self, key, value):
if self.__isfrozen and not hasattr(self, key):
raise TypeError( "%r is a frozen class" % self )
object.__setattr__(self, key, value)
def _freeze(self):
self.__isfrozen = True
class Test(FrozenClass):
def __init__(self):
self.x = 42#
self.y = 2**3
self._freeze() # no new attributes after this point.
a,b = Test(), Test()
a.x = 10
b.z = 10 # fails
Slots is the way to go:
The pythonic way is to use slots instead of playing around with the __setter__. While it may solve the problem, it does not give any performance improvement. The attributes of objects are stored in a dictionary "__dict__", this is the reason, why you can dynamically add attributes to objects of classes that we have created so far. Using a dictionary for attribute storage is very convenient, but it can mean a waste of space for objects, which have only a small amount of instance variables.
Slots are a nice way to work around this space consumption problem. Instead of having a dynamic dict that allows adding attributes to objects dynamically, slots provide a static structure which prohibits additions after the creation of an instance.
When we design a class, we can use slots to prevent the dynamic creation of attributes. To define slots, you have to define a list with the name __slots__. The list has to contain all the attributes, you want to use. We demonstrate this in the following class, in which the slots list contains only the name for an attribute "val".
class S(object):
__slots__ = ['val']
def __init__(self, v):
self.val = v
x = S(42)
print(x.val)
x.new = "not possible"
=> It fails to create an attribute "new":
42
Traceback (most recent call last):
File "slots_ex.py", line 12, in <module>
x.new = "not possible"
AttributeError: 'S' object has no attribute 'new'
Notes:
Since Python 3.3 the advantage optimizing the space consumption is not as impressive any more. With Python 3.3 Key-Sharing Dictionaries are used for the storage of objects. The attributes of the instances are capable of sharing part of their internal storage between each other, i.e. the part which stores the keys and their corresponding hashes. This helps to reduce the memory consumption of programs, which create many instances of non-builtin types. But still is the way to go to avoid dynamically created attributes.
Using slots come also with it's own cost. It will break serialization (e.g. pickle). It will also break multiple inheritance. A class can't inherit from more than one class that either defines slots or has an instance layout defined in C code (like list, tuple or int).
If someone is interested in doing that with a decorator, here is a working solution:
from functools import wraps
def froze_it(cls):
cls.__frozen = False
def frozensetattr(self, key, value):
if self.__frozen and not hasattr(self, key):
print("Class {} is frozen. Cannot set {} = {}"
.format(cls.__name__, key, value))
else:
object.__setattr__(self, key, value)
def init_decorator(func):
#wraps(func)
def wrapper(self, *args, **kwargs):
func(self, *args, **kwargs)
self.__frozen = True
return wrapper
cls.__setattr__ = frozensetattr
cls.__init__ = init_decorator(cls.__init__)
return cls
Pretty straightforward to use:
#froze_it
class Foo(object):
def __init__(self):
self.bar = 10
foo = Foo()
foo.bar = 42
foo.foobar = "no way"
Result:
>>> Class Foo is frozen. Cannot set foobar = no way
Actually, you don't want __setattr__, you want __slots__. Add __slots__ = ('foo', 'bar', 'baz') to the class body, and Python will make sure that there's only foo, bar and baz on any instance. But read the caveats the documentation lists!
The proper way is to override __setattr__. That's what it's there for.
I like very much the solution that uses a decorator, because it's easy to use it for many classes across a project, with minimum additions for each class. But it doesn't work well with inheritance.
So here is my version: It only overrides the __setattr__ function - if the attribute doesn't exist and the caller function is not __init__, it prints an error message.
import inspect
def froze_it(cls):
def frozensetattr(self, key, value):
if not hasattr(self, key) and inspect.stack()[1][3] != "__init__":
print("Class {} is frozen. Cannot set {} = {}"
.format(cls.__name__, key, value))
else:
self.__dict__[key] = value
cls.__setattr__ = frozensetattr
return cls
#froze_it
class A:
def __init__(self):
self._a = 0
a = A()
a._a = 1
a._b = 2 # error
What about this:
class A():
__allowed_attr=('_x', '_y')
def __init__(self,x=0,y=0):
self._x=x
self._y=y
def __setattr__(self,attribute,value):
if not attribute in self.__class__.__allowed_attr:
raise AttributeError
else:
super().__setattr__(attribute,value)
Here is approach i came up with that doesn't need a _frozen attribute or method to freeze() in init.
During init i just add all class attributes to the instance.
I like this because there is no _frozen, freeze(), and _frozen also does not show up in the vars(instance) output.
class MetaModel(type):
def __setattr__(self, name, value):
raise AttributeError("Model classes do not accept arbitrary attributes")
class Model(object):
__metaclass__ = MetaModel
# init will take all CLASS attributes, and add them as SELF/INSTANCE attributes
def __init__(self):
for k, v in self.__class__.__dict__.iteritems():
if not k.startswith("_"):
self.__setattr__(k, v)
# setattr, won't allow any attributes to be set on the SELF/INSTANCE that don't already exist
def __setattr__(self, name, value):
if not hasattr(self, name):
raise AttributeError("Model instances do not accept arbitrary attributes")
else:
object.__setattr__(self, name, value)
# Example using
class Dog(Model):
name = ''
kind = 'canine'
d, e = Dog(), Dog()
print vars(d)
print vars(e)
e.junk = 'stuff' # fails
I like the "Frozen" of Jochen Ritzel. The inconvenient is that the isfrozen variable then appears when printing a Class.__dict
I went around this problem this way by creating a list of authorized attributes (similar to slots):
class Frozen(object):
__List = []
def __setattr__(self, key, value):
setIsOK = False
for item in self.__List:
if key == item:
setIsOK = True
if setIsOK == True:
object.__setattr__(self, key, value)
else:
raise TypeError( "%r has no attributes %r" % (self, key) )
class Test(Frozen):
_Frozen__List = ["attr1","attr2"]
def __init__(self):
self.attr1 = 1
self.attr2 = 1
The FrozenClass by Jochen Ritzel is cool, but calling _frozen() when initialing a class every time is not so cool (and you need to take the risk of forgetting it). I added a __init_slots__ function:
class FrozenClass(object):
__isfrozen = False
def _freeze(self):
self.__isfrozen = True
def __init_slots__(self, slots):
for key in slots:
object.__setattr__(self, key, None)
self._freeze()
def __setattr__(self, key, value):
if self.__isfrozen and not hasattr(self, key):
raise TypeError( "%r is a frozen class" % self )
object.__setattr__(self, key, value)
class Test(FrozenClass):
def __init__(self):
self.__init_slots__(["x", "y"])
self.x = 42#
self.y = 2**3
a,b = Test(), Test()
a.x = 10
b.z = 10 # fails
None of the answers mention the performance impact of overriding __setattr__, which can be an issue when creating many small objects. (And __slots__ would be the performant solution but limits pickle/inheritance).
So I came up with this variant which installs our slower settatr after init:
class FrozenClass:
def freeze(self):
def frozen_setattr(self, key, value):
if not hasattr(self, key):
raise TypeError("Cannot set {}: {} is a frozen class".format(key, self))
object.__setattr__(self, key, value)
self.__setattr__ = frozen_setattr
class Foo(FrozenClass): ...
If you don't want to call freeze at the end of __init__, if inheritance is an issue, or if you don't want it in vars(), it can also be adapted: for example here is a decorator version based on the pystrict answer:
import functools
def strict(cls):
cls._x_setter = getattr(cls, "__setattr__", object.__setattr__)
cls._x_init = cls.__init__
#functools.wraps(cls.__init__)
def wrapper(self, *args, **kwargs):
cls._x_init(self, *args, **kwargs)
def frozen_setattr(self, key, value):
if not hasattr(self, key):
raise TypeError("Class %s is frozen. Cannot set '%s'." % (cls.__name__, key))
cls._x_setter(self, key, value)
cls.__setattr__ = frozen_setattr
cls.__init__ = wrapper
return cls
#strict
class Foo: ...
I wrote pystrict as a solution to this problem. It's too large to paste all of the code in stackoverflow.
pystrict is a pypi installable decorator that can be used with classes to freeze them. Many solutions here don't properly support inheritance.
If __slots__ doesn't work for you (because of inheritance issues), this is a good alternative.
There is an example to the README that shows why a decorator like this is needed even if you have mypy and pylint running on your project:
pip install pystrict
Then just use the #strict decorator:
from pystrict import strict
#strict
class Blah
def __init__(self):
self.attr = 1
#dataclass(slots=True) Nirvana (Python 3.10)
I'm in love with this #dataclass thing:
main.py
from dataclasses import dataclass
#dataclass(slots=True)
class C:
n: int
s: str
c = C(n=1, s='one')
assert c.n == 1
assert c.s == 'one'
c.n == 2
c.s == 'two'
c.asdf = 2
Outcome:
Traceback (most recent call last):
File "/home/ciro/main.py", line 15, in <module>
c.asdf = 2
AttributeError: 'C' object has no attribute 'asdf'
Note how #dataclass only requires use to define our attributes once with type annotations
n: int
s: str
and then, without any repetition we get for free:
def __init__(n, s):
self.n = n
self.s = s
__slots__ = ['n', 's']
Other free things not shown in this example:
__str__
__eq__: Compare object instances for equality by their attributes
__hash__ if you also use frozen=True: Object of custom type as dictionary key
Tested on Python 3.10.7, Ubuntu 22.10.
My class has a dict, for example:
class MyClass(object):
def __init__(self):
self.data = {'a': 'v1', 'b': 'v2'}
Then I want to use the dict's key with MyClass instance to access the dict, for example:
ob = MyClass()
v = ob.a # Here I expect ob.a returns 'v1'
I know this should be implemented by __getattr__, but I'm new to Python, I don't exactly know how to implement it.
class MyClass(object):
def __init__(self):
self.data = {'a': 'v1', 'b': 'v2'}
def __getattr__(self, attr):
return self.data[attr]
>>> ob = MyClass()
>>> v = ob.a
>>> v
'v1'
Be careful when implementing __setattr__ though, you will need to make a few modifications:
class MyClass(object):
def __init__(self):
# prevents infinite recursion from self.data = {'a': 'v1', 'b': 'v2'}
# as now we have __setattr__, which will call __getattr__ when the line
# self.data[k] tries to access self.data, won't find it in the instance
# dictionary and return self.data[k] will in turn call __getattr__
# for the same reason and so on.... so we manually set data initially
super(MyClass, self).__setattr__('data', {'a': 'v1', 'b': 'v2'})
def __setattr__(self, k, v):
self.data[k] = v
def __getattr__(self, k):
# we don't need a special call to super here because getattr is only
# called when an attribute is NOT found in the instance's dictionary
try:
return self.data[k]
except KeyError:
raise AttributeError
>>> ob = MyClass()
>>> ob.c = 1
>>> ob.c
1
If you don't need to set attributes just use a namedtuple
eg.
>>> from collections import namedtuple
>>> MyClass = namedtuple("MyClass", ["a", "b"])
>>> ob = MyClass(a=1, b=2)
>>> ob.a
1
If you want the default arguments you can just write a wrapper class around it:
class MyClass(namedtuple("MyClass", ["a", "b"])):
def __new__(cls, a="v1", b="v2"):
return super(MyClass, cls).__new__(cls, a, b)
or maybe it looks nicer as a function:
def MyClass(a="v1", b="v2", cls=namedtuple("MyClass", ["a", "b"])):
return cls(a, b)
>>> ob = MyClass()
>>> ob.a
'v1'
Late to the party, but found two really good resources that explain this better (IMHO).
As explained here, you should use self.__dict__ to access fields from within __getattr__, in order to avoid infinite recursion. The example provided is:
def __getattr__(self, attrName):
if not self.__dict__.has_key(attrName):
value = self.fetchAttr(attrName) # computes the value
self.__dict__[attrName] = value
return self.__dict__[attrName]
Note: in the second line (above), a more Pythonic way would be (has_key apparently was even removed in Python 3):
if attrName not in self.__dict__:
The other resource explains that the __getattr__ is invoked only when the attribute is not found in the object, and that hasattr always returns True if there is an implementation for __getattr__. It provides the following example, to demonstrate:
class Test(object):
def __init__(self):
self.a = 'a'
self.b = 'b'
def __getattr__(self, name):
return 123456
t = Test()
print 'object variables: %r' % t.__dict__.keys()
#=> object variables: ['a', 'b']
print t.a
#=> a
print t.b
#=> b
print t.c
#=> 123456
print getattr(t, 'd')
#=> 123456
print hasattr(t, 'x')
#=> True
class A(object):
def __init__(self):
self.data = {'a': 'v1', 'b': 'v2'}
def __getattr__(self, attr):
try:
return self.data[attr]
except Exception:
return "not found"
>>>a = A()
>>>print a.a
v1
>>>print a.c
not found
I like to take this therefore.
I took it from somewhere, but I don't remember where.
class A(dict):
def __init__(self, *a, **k):
super(A, self).__init__(*a, **k)
self.__dict__ = self
This makes the __dict__ of the object the same as itself, so that attribute and item access map to the same dict:
a = A()
a['a'] = 2
a.b = 5
print a.a, a['b'] # prints 2 5
I figured out an extension to #glglgl's answer that handles nested dictionaries and dictionaries insides lists that are in the original dictionary:
class d(dict):
def __init__(self, *a, **k):
super(d, self).__init__(*a, **k)
self.__dict__ = self
for k in self.__dict__:
if isinstance(self.__dict__[k], dict):
self.__dict__[k] = d(self.__dict__[k])
elif isinstance(self.__dict__[k], list):
for i in range(len(self.__dict__[k])):
if isinstance(self.__dict__[k][i], dict):
self.__dict__[k][i] = d(self.__dict__[k][i])
A simple approach to solving your __getattr__()/__setattr__() infinite recursion woes
Implementing one or the other of these magic methods can usually be easy. But when overriding them both, it becomes trickier. This post's examples apply mostly to this more difficult case.
When implementing both these magic methods, it's not uncommon to get stuck figuring out a strategy to get around recursion in the __init__() constructor of classes. This is because variables need to be initialized for the object, but every attempt to read or write those variables go through __get/set/attr__(), which could have more unset variables in them, incurring more futile recursive calls.
Up front, a key point to remember is that __getattr__() only gets called by the runtime if the attribute can't be found on the object already. The trouble is to get attributes defined without tripping these functions recursively.
Another point is __setattr__() will get called no matter what. That's an important distinction between the two functions, which is why implementing both attribute methods can be tricky.
This is one basic pattern that solves the problem.
class AnObjectProxy:
_initialized = False # *Class* variable 'constant'.
def __init__(self):
self._any_var = "Able to access instance vars like usual."
self._initialized = True # *instance* variable.
def __getattr__(self, item):
if self._initialized:
pass # Provide the caller attributes in whatever ways interest you.
else:
try:
return self.__dict__[item] # Transparent access to instance vars.
except KeyError:
raise AttributeError(item)
def __setattr__(self, key, value):
if self._initialized:
pass # Provide caller ways to set attributes in whatever ways.
else:
self.__dict__[key] = value # Transparent access.
While the class is initializing and creating it's instance vars, the code in both attribute functions permits access to the object's attributes via the __dict__ dictionary transparently - your code in __init__() can create and access instance attributes normally. When the attribute methods are called, they only access self.__dict__ which is already defined, thus avoiding recursive calls.
In the case of self._any_var, once it's assigned, __get/set/attr__() won't be called to find it again.
Stripped of extra code, these are the two pieces that are most important.
... def __getattr__(self, item):
... try:
... return self.__dict__[item]
... except KeyError:
... raise AttributeError(item)
...
... def __setattr__(self, key, value):
... self.__dict__[key] = value
Solutions can build around these lines accessing the __dict__ dictionary. To implement an object proxy, two modes were implemented: initialization and post-initialization in the code before this - a more detailed example of the same is below.
There are other examples in answers that may have differing levels of effectiveness in dealing with all aspects of recursion. One effective approach is accessing __dict__ directly in __init__() and other places that need early access to instance vars. This works but can be a little verbose. For instance,
self.__dict__['_any_var'] = "Setting..."
would work in __init__().
My posts tend to get a little long-winded.. after this point is just extra. You should already have the idea with the examples above.
A drawback to some other approaches can be seen with debuggers in IDE's. They can be overzealous in their use of introspection and produce warning and error recovery messages as you're stepping through code. You can see this happening even with solutions that work fine standalone. When I say all aspects of recursion, this is what I'm talking about.
The examples in this post only use a single class variable to support 2-modes of operation, which is very maintainable.
But please NOTE: the proxy class required two modes of operation to set up and proxy for an internal object. You don't have to have two modes of operation.
You could simply incorporate the code to access the __dict__ as in these examples in whatever ways suit you.
If your requirements don't include two modes of operation, you may not need to declare any class variables at all. Just take the basic pattern and customize it.
Here's a closer to real-world (but by no means complete) example of a 2-mode proxy that follows the pattern:
>>> class AnObjectProxy:
... _initialized = False # This class var is important. It is always False.
... # The instances will override this with their own,
... # set to True.
... def __init__(self, obj):
... # Because __getattr__ and __setattr__ access __dict__, we can
... # Initialize instance vars without infinite recursion, and
... # refer to them normally.
... self._obj = obj
... self._foo = 123
... self._bar = 567
...
... # This instance var overrides the class var.
... self._initialized = True
...
... def __setattr__(self, key, value):
... if self._initialized:
... setattr(self._obj, key, value) # Proxying call to wrapped obj.
... else:
... # this block facilitates setting vars in __init__().
... self.__dict__[key] = value
...
... def __getattr__(self, item):
... if self._initialized:
... attr = getattr(self._obj, item) # Proxying.
... return attr
... else:
... try:
... # this block facilitates getting vars in __init__().
... return self.__dict__[item]
... except KeyError:
... raise AttributeError(item)
...
... def __call__(self, *args, **kwargs):
... return self._obj(*args, **kwargs)
...
... def __dir__(self):
... return dir(self._obj) + list(self.__dict__.keys())
The 2-mode proxy only needs a bit of "bootstrapping" to access vars in its own scope at initialization before any of its vars are set. After initialization, the proxy has no reason to create more vars for itself, so it will fare fine by deferring all attribute calls to it's wrapped object.
Any attribute the proxy itself owns will still be accessible to itself and other callers since the magic attribute functions only get called if an attribute can't be found immediately on the object.
Hopefully this approach can be of benefit to anyone who appreciates a direct approach to resolving their __get/set/attr__() __init__() frustrations.
You can initialize your class dictionary through the constructor:
def __init__(self,**data):
And call it as follows:
f = MyClass(**{'a': 'v1', 'b': 'v2'})
All of the instance attributes being accessed (read) in __setattr__, need to be declared using its parent (super) method, only once:
super().__setattr__('NewVarName1', InitialValue)
Or
super().__setattr__('data', dict())
Thereafter, they can be accessed or assigned to in the usual manner:
self.data = data
And instance attributes not being accessed in __setattr__, can be declared in the usual manner:
self.x = 1
The overridden __setattr__ method must now call the parent method inside itself, for new variables to be declared:
super().__setattr__(key,value)
A complete class would look as follows:
class MyClass(object):
def __init__(self, **data):
# The variable self.data is used by method __setattr__
# inside this class, so we will need to declare it
# using the parent __setattr__ method:
super().__setattr__('data', dict())
self.data = data
# These declarations will jump to
# super().__setattr__('data', dict())
# inside method __setattr__ of this class:
self.x = 1
self.y = 2
def __getattr__(self, name):
# This will callback will never be called for instance variables
# that have beed declared before being accessed.
if name in self.data:
# Return a valid dictionary item:
return self.data[name]
else:
# So when an instance variable is being accessed, and
# it has not been declared before, nor is it contained
# in dictionary 'data', an attribute exception needs to
# be raised.
raise AttributeError
def __setattr__(self, key, value):
if key in self.data:
# Assign valid dictionary items here:
self.data[key] = value
else:
# Assign anything else as an instance attribute:
super().__setattr__(key,value)
Test:
f = MyClass(**{'a': 'v1', 'b': 'v2'})
print("f.a = ", f.a)
print("f.b = ", f.b)
print("f.data = ", f.data)
f.a = 'c'
f.d = 'e'
print("f.a = ", f.a)
print("f.b = ", f.b)
print("f.data = ", f.data)
print("f.d = ", f.d)
print("f.x = ", f.x)
print("f.y = ", f.y)
# Should raise attributed Error
print("f.g = ", f.g)
Output:
f.a = v1
f.b = v2
f.data = {'a': 'v1', 'b': 'v2'}
f.a = c
f.b = v2
f.data = {'a': 'c', 'b': 'v2'}
f.d = e
f.x = 1
f.y = 2
Traceback (most recent call last):
File "MyClass.py", line 49, in <module>
print("f.g = ", f.g)
File "MyClass.py", line 25, in __getattr__
raise AttributeError
AttributeError
I think this implement is cooler
class MyClass(object):
def __init__(self):
self.data = {'a': 'v1', 'b': 'v2'}
def __getattr__(self,key):
return self.data.get(key,None)
I have a dictionary-like object which store descriptors inside:
class MyDict(object):
def __init__(self):
dict.__init__(self)
def __getitem__(self, key):
v = dict.__getitem__(self, key)
if hasattr(v, '__get__'):
return v.__get__(None, self)
return v
class MyDescriptor(object):
def __init__(self, value, attrib={}):
self.__value = value
self.attrib= attrib
def __get__(self, instance, owner):
return self.__value
def __set__(self, instance, value):
self.__value = value
I want to be able to do the following:
d = MyDict()
d['1'] = MyDescriptor("123", {"name": "val"})
print d['1'] # prints "123"
print d['1'].attrib["name"] # prints "val"
My classes don't work. Could you please help me?
Your solution looks unnecessarily complicated to solve your problem, unless there's more to it that is shown. Why not simply do this:
class MyObject(object):
def __init__(value, attrib=None):
self.__value = value
self.attrib = {} if attrib is None else attrib
def __str__(self):
return __value
d = {}
d['1'] = MyObject("123", {"name": "val"})
print d['1'] # prints "123"
print d['1'].attrib["name"] # prints "val"
As for why your code doesn't work, there are a few obvious problems.
From your calls in various special methods of __dict__, it appears that MyDict is meant to subclass dict, so the definition should be:
class MyDict(dict):
...
While not incorrect, it is better practice to use super rather than referring to the
base class directly, so dict.__init__(self) would become super(MyDict, self).__init__() and dict.__getitem__(self, key) becomes super(MyDict, dict).__getitem__(key).
Your call to get sill work, but doesn't match the method specification. You should
call it as v.__get__(self, MyDict). However, the way you are using it actually makes __get__ redundant, and I think that this usage it where the main problem lies.
In class MyDescriptor, early binding will give you unexpected results for attrib. See my example above for a better way for declaring it.
I suspect that instead of a description, what you actually want is an object which looks like a string (for some definition of "looks like"), but has an attribute attrib. To do this, there is no need to try to create a descriptor, which is intended for a different use case altogether. My example above gives a class which satisfies the requirement of an object which "looks like" a string, where "looks like" means it prints a string, but here is another which may be more like what you want:
class MyString(str):
def __init__(self, value, attrib=None):
super(MyString, self).__init__(value)
self.attrib = {} if attrib is None else attrib
I'm not sure if this solves your use case, but in terms of achieving the results you've stated you could simply remove your MyDict class and use an ordinary dict:
d = {}
Then, add a __str__ method to the MyDescriptor class returning self.__value and you'll achieve the results you've described.
>>> d['1'] = MyDescriptor("123", {"name": "val"})
>>> d['1']
123
>>> d['1'].attrib["name"]
val
I want to write a custom class that behaves like dict - so, I am inheriting from dict.
My question, though, is: Do I need to create a private dict member in my __init__() method?. I don't see the point of this, since I already have the dict behavior if I simply inherit from dict.
Can anyone point out why most of the inheritance snippets look like the one below?
class CustomDictOne(dict):
def __init__(self):
self._mydict = {}
# other methods follow
Instead of the simpler...
class CustomDictTwo(dict):
def __init__(self):
# initialize my other stuff here ...
# other methods follow
Actually, I think I suspect the answer to the question is so that users cannot directly access your dictionary (i.e. they have to use the access methods that you have provided).
However, what about the array access operator []? How would one implement that? So far, I have not seen an example that shows how to override the [] operator.
So if a [] access function is not provided in the custom class, the inherited base methods will be operating on a different dictionary?
I tried the following snippet to test out my understanding of Python inheritance:
class myDict(dict):
def __init__(self):
self._dict = {}
def add(self, id, val):
self._dict[id] = val
md = myDict()
md.add('id', 123)
print md[id]
I got the following error:
KeyError: < built-in function id>
What is wrong with the code above?
How do I correct the class myDict so that I can write code like this?
md = myDict()
md['id'] = 123
[Edit]
I have edited the code sample above to get rid of the silly error I made before I dashed away from my desk. It was a typo (I should have spotted it from the error message).
class Mapping(dict):
def __setitem__(self, key, item):
self.__dict__[key] = item
def __getitem__(self, key):
return self.__dict__[key]
def __repr__(self):
return repr(self.__dict__)
def __len__(self):
return len(self.__dict__)
def __delitem__(self, key):
del self.__dict__[key]
def clear(self):
return self.__dict__.clear()
def copy(self):
return self.__dict__.copy()
def has_key(self, k):
return k in self.__dict__
def update(self, *args, **kwargs):
return self.__dict__.update(*args, **kwargs)
def keys(self):
return self.__dict__.keys()
def values(self):
return self.__dict__.values()
def items(self):
return self.__dict__.items()
def pop(self, *args):
return self.__dict__.pop(*args)
def __cmp__(self, dict_):
return self.__cmp__(self.__dict__, dict_)
def __contains__(self, item):
return item in self.__dict__
def __iter__(self):
return iter(self.__dict__)
def __unicode__(self):
return unicode(repr(self.__dict__))
o = Mapping()
o.foo = "bar"
o['lumberjack'] = 'foo'
o.update({'a': 'b'}, c=44)
print 'lumberjack' in o
print o
In [187]: run mapping.py
True
{'a': 'b', 'lumberjack': 'foo', 'foo': 'bar', 'c': 44}
Like this
class CustomDictOne(dict):
def __init__(self,*arg,**kw):
super(CustomDictOne, self).__init__(*arg, **kw)
Now you can use the built-in functions, like dict.get() as self.get().
You do not need to wrap a hidden self._dict. Your class already is a dict.
Check the documentation on emulating container types. In your case, the first parameter to add should be self.
UserDict from the Python standard library is designed for this purpose.
Here is an alternative solution:
class AttrDict(dict):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.__dict__ = self
a = AttrDict()
a.a = 1
a.b = 2
This is my best solution. I used this many times.
class DictLikeClass:
...
def __getitem__(self, key):
return getattr(self, key)
def __setitem__(self, key, value):
setattr(self, key, value)
...
You can use like:
>>> d = DictLikeClass()
>>> d["key"] = "value"
>>> print(d["key"])
A python class that acts like dict
What's wrong with this?
Can anyone point out why most of the inheritance snippets look like the one below?
class CustomDictOne(dict):
def __init__(self):
self._mydict = {}
Presumably there's a good reason to inherit from dict (maybe you're already passing one around and you want a more specific kind of dict) and you have a good reason to instantiate another dict to delegate to (because this will instantiate two dicts per instance of this class.) But doesn't that sound incorrect?
I never run into this use-case myself. I do like the idea of typing dicts where you are using dicts that are type-able. But in that case I like the idea of typed class attributes even moreso - and the whole point of a dict is you can give it keys of any hashable type, and values of any type.
So why do we see snippets like this? I personally think it's an easily made mistake that went uncorrected and thus perpetuated over time.
I would rather see, in these snippets, this, to demonstrate code reuse through inheritance:
class AlternativeOne(dict):
__slots__ = ()
def __init__(self):
super().__init__()
# other init code here
# new methods implemented here
or, to demonstrate re-implementing the behavior of dicts, this:
from collections.abc import MutableMapping
class AlternativeTwo(MutableMapping):
__slots__ = '_mydict'
def __init__(self):
self._mydict = {}
# other init code here
# dict methods reimplemented and new methods implemented here
By request - adding slots to a dict subclass.
Why add slots? A builtin dict instance doesn't have arbitrary attributes:
>>> d = dict()
>>> d.foo = 'bar'
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'dict' object has no attribute 'foo'
If we create a subclass the way most are doing it here on this answer, we see we don't get the same behavior, because we'll have a __dict__ attribute, causing our dicts to take up to potentially twice the space:
my_dict(dict):
"""my subclass of dict"""
md = my_dict()
md.foo = 'bar'
Since there's no error created by the above, the above class doesn't actually act, "like dict."
We can make it act like dict by giving it empty slots:
class my_dict(dict):
__slots__ = ()
md = my_dict()
So now attempting to use arbitrary attributes will fail:
>>> md.foo = 'bar'
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: 'my_dict' object has no attribute 'foo'
And this Python class acts more like a dict.
For more on how and why to use slots, see this Q&A: Usage of __slots__?
I really don't see the right answer to this anywhere
class MyClass(dict):
def __init__(self, a_property):
self[a_property] = a_property
All you are really having to do is define your own __init__ - that really is all that there is too it.
Another example (little more complex):
class MyClass(dict):
def __init__(self, planet):
self[planet] = planet
info = self.do_something_that_returns_a_dict()
if info:
for k, v in info.items():
self[k] = v
def do_something_that_returns_a_dict(self):
return {"mercury": "venus", "mars": "jupiter"}
This last example is handy when you want to embed some kind of logic.
Anyway... in short class GiveYourClassAName(dict) is enough to make your class act like a dict. Any dict operation you do on self will be just like a regular dict.
The problem with this chunk of code:
class myDict(dict):
def __init__(self):
self._dict = {}
def add(id, val):
self._dict[id] = val
md = myDict()
md.add('id', 123)
...is that your 'add' method (...and any method you want to be a member of a class) needs to have an explicit 'self' declared as its first argument, like:
def add(self, 'id', 23):
To implement the operator overloading to access items by key, look in the docs for the magic methods __getitem__ and __setitem__.
Note that because Python uses Duck Typing, there may actually be no reason to derive your custom dict class from the language's dict class -- without knowing more about what you're trying to do (e.g, if you need to pass an instance of this class into some code someplace that will break unless isinstance(MyDict(), dict) == True), you may be better off just implementing the API that makes your class sufficiently dict-like and stopping there.
Don’t inherit from Python built-in dict, ever! for example update method woldn't use __setitem__, they do a lot for optimization. Use UserDict.
from collections import UserDict
class MyDict(UserDict):
def __delitem__(self, key):
pass
def __setitem__(self, key, value):
pass