Just learning about properties and setters in python, and seems fair enough when we have a mutable attribute. But what happens when I want to validate a .append() on a list for example? In the below, I can validate the setting of the attribute and it works as expected. But I can bypass its effect by simply appending to get more cookies onto the tray...
class CookieTray:
def __init__(self):
self.cookies = []
#property
def cookies(self):
return self._cookies
#cookies.setter
def cookies(self, cookies):
if len(cookies) > 8:
raise ValueError("Too many cookies in the tray!")
self._cookies = cookies
if __name__ == '__main__':
tray = CookieTray()
print("Cookies: ", tray.cookies)
try:
tray.cookies = [1,1,0,0,0,1,1,0,1] # too many
except Exception as e:
print(e)
tray.cookies = [1,0,1,0,1,0]
print(tray.cookies)
tray.cookies.append(0)
tray.cookies.append(0)
tray.cookies.append(1) # too many, but can still append
print(tray.cookies)
Silly example, but I hope it illustrates my question. Should I just be avoiding the setter and making a "setter" method, like add_cookie(self, cookie_type) and then do my validation in there?
The setter only applies when assigning to the attribute. As you've seen mutating the attribute bypasses this.
To apply the validation to the object being mutated we can use a custom type. Here's an example which just wraps a normal list:
import collections
class SizedList(collections.abc.MutableSequence):
def __init__(self, maxlen):
self.maxlen = maxlen
self._list = []
def check_length(self):
if len(self._list) >= self.maxlen:
raise OverflowError("Max length exceeded")
def __setitem__(self, i, v):
self.check_length()
self._list[i] = v
def insert(self, i, v):
self.check_length()
self._list.insert(i, v)
def __getitem__(self, i): return self._list[i]
def __delitem__(self, i): del self._list[i]
def __len__(self): return len(self._list)
def __repr__(self): return f"{self._list!r}"
When overriding container types collections.abc can be useful - we can see the abstract methods that must be implemented: __getitem__, __setitem__, __delitem__, __len__, and insert in this case. All of them are delegated to the list object that's being wrapped, with the two that add items having the added length check.
The repr isn't needed, but makes it easier to check the contents - once again just delegating to the wrapped list.
With this you can simply replace the self.cookies = [] line with self.cookies = SizedList(8).
You would need to create a ValidatingList class that overrides list.append, something like this:
class ValidatingList(list):
def append(self, value):
if len(self) >= 8:
raise ValueError("Too many items in the list")
else:
super().append(value)
You then convert your cookies to a ValidatingList:
class CookieTray:
def __init__(self):
self.cookies = []
#property
def cookies(self):
return self._cookies
#cookies.setter
def cookies(self, cookies):
if len(cookies) > 8:
raise ValueError("Too many cookies in the tray!")
self._cookies = ValidatingList(cookies)
Related
I'm working from a book, very much newbie stuff, and the code below is from the book, and defines a simple class. But for some reason, the author has decided to put a "helper method" called check_index outside the class. I cannot for the life of me figure out why he would do this, as the method seems integral to teh operation of the class. He writes:
The index checking is taken care of by a utility function I’ve written
for the purpose, check_index.
I have tried putting it inside the class (the code below is as it is in the book), but the runtime refuses to find the method - it falls over with
NameError: name 'check_index' is not defined
My questions are, why did the author put this "helper method" outside the class, and why does the code not work when I move the method inside the class:
class ArithmeticSequence:
def __init__(self, start=0, step=1):
self.start = start # Store the start value
self.step = step # Store the step value
self.changed = {} # No items have been modified
def __getitem__(self, key):
check_index(key)
try: return self.changed[key] # Modified?
except KeyError: # otherwise ...
return self.start + key * self.step # ... calculate the value
def __setitem__(self, key, value):
check_index(key)
self.changed[key] = value # Store the changed value
def check_index(key):
if not isinstance(key, int): raise TypeError
if key < 0: raise IndexError
When I move the method inside the class, I just slot it in with the other methods. But it is not found by the runtime. Why?
class ArithmeticSequence:
def __init__(self, start=0, step=1):
self.start = start # Store the start value
self.step = step # Store the step value
self.changed = {} # No items have been modified
def check_index(key):
if not isinstance(key, int): raise TypeError
if key < 0: raise IndexError
def __getitem__(self, key):
check_index(key)
try: return self.changed[key] # Modified?
except KeyError: # otherwise ...
return self.start + key * self.step # ... calculate the value
def __setitem__(self, key, value):
check_index(key)
self.changed[key] = value # Store the changed value
You need to use self
Ex:
class ArithmeticSequence:
def __init__(self, start=0, step=1):
self.start = start # Store the start value
self.step = step # Store the step value
self.changed = {} # No items have been modified
def check_index(self, key):
if not isinstance(key, int): raise TypeError
if key < 0: raise IndexError
def __getitem__(self, key):
self.check_index(key)
try: return self.changed[key] # Modified?
except KeyError: # otherwise ...
return self.start + key * self.step # ... calculate the value
def __setitem__(self, key, value):
self.check_index(key)
self.changed[key] = value # Store the changed value
And call the function with self Ex: self.check_index
Your def check_index(key) still defines a method of ArithmeticSequence, regardless of what you call the first argument, which means you have to call it like a regular instance method (self.check_index()), and if you want to pass it an argument you have to add it after self. If you want to define a method on the class itself, you can use #staticmethod or #classmethod:
class Foo:
#staticmethod
def bar(key):
return key
#classmethod
def baz(cls, key):
return key
def quux(self):
print(Foo.bar("abcd"), Foo.baz("abcd"))
Foo().quux()
ArithmeticSequence is not a new-style class. Check this
You have 2 options:
Add self to check_index. In the class, you will use it as self.check_index(key). You will need to instantiate an ArithmeticSequence class object.
Add #staticmethod before check_index. You will use it as ArithmeticSequence.check_index(key)
Consider following example
class Key:
def __init__(self, s):
self.s = s
d = {}
for x in range(1, 10000):
t = Key(x)
d[t] = x
This will create 10000 keys. Is it possible to control the object creation of class key, for example we cannot create more than 5 objects of key class. The loop should not be changed in any ways.
You can control how, or how many objects are created by giving your class a __new__ method:
class Key(object):
_count = 0
def __new__(cls, s):
if cls._count == 5:
raise TypeError('Too many keys created')
cls._count += 1
return super(Key, cls).__new__(cls, s)
def __init__(self,s):
self.s = s
Key.__new__() is called to create a new instance; here I keep a count of how many are created, and if there are too many, an exception is raised. You could also keep a pool of instances in a dictionary, or control creating of new instance in other ways.
Note that this only works for new-style classes, inheriting from object.
You can also use a metaclass approach
import weakref
import random
class FiveElementType(type):
def __init__(self, name, bases, d):
super(FiveElementType, self).__init__(name, bases, d)
self.__cache = weakref.WeakValueDictionary()
def __call__(self, *args):
if len(self.__cache) == 5:
return random.choice(self.__cache.values())
else:
obj = super(FiveElementType, self).__call__(*args)
self.__cache[len(self.__cache)] = obj
return obj
class Key(object):
__metaclass__ = FiveElementType
def __init__(self, s):
self.s = s
You can choose a random element or select it on the base of stored index. In this approach your loop don't fail with an exception that can be right or not, depending on your intention.
I have a class like:
class A:
def __init__(self):
self.data = {}
and at some moment I want to prohibit self.data fields modification.
I've read in PEP-416 rejection notice that there are a lot of ways to do it. So I'd like to find what they are.
I tried this:
a = A()
a.data = types.MappingProxyType(a.data)
That should work but first, its python3.3+ and second, when I do this "prohibition" multiple times I get this:
>>> a.data = types.MappingProxyType(a.data)
>>> a.data = types.MappingProxyType(a.data)
>>> a.data
mappingproxy(mappingproxy({}))
though it would be much better to get just mappingproxy({}) as I am going to "prohibit" a lot of times. Check of isinstance(MappingProxyType) is an option, but I think that other options can exist.
Thanks
Use collections.Mapping e.g.
import collections
class DictWrapper(collections.Mapping):
def __init__(self, data):
self._data = data
def __getitem__(self, key):
return self._data[key]
def __len__(self):
return len(self._data)
def __iter__(self):
return iter(self._data)
This is the full implementation of fast (shallow-)read-only dict:
def _readonly(self, *args, **kwargs):
raise RuntimeError("Cannot modify ReadOnlyDict")
class ReadOnlyDict(dict):
__setitem__ = _readonly
__delitem__ = _readonly
pop = _readonly
popitem = _readonly
clear = _readonly
update = _readonly
setdefault = _readonly
My previous (worse) implementation was as follows (thanks #mtraceur for the great remarks!):
class ReadOnlyDict(dict):
def __readonly__(self, *args, **kwargs):
raise RuntimeError("Cannot modify ReadOnlyDict")
__setitem__ = __readonly__
__delitem__ = __readonly__
pop = __readonly__
popitem = __readonly__
clear = __readonly__
update = __readonly__
setdefault = __readonly__
del __readonly__
Very easy, you just override default dict's methods!
Here is an example:
class ReadOnlyDict(dict):
__readonly = False
def readonly(self, allow=1):
"""Allow or deny modifying dictionary"""
self.__readonly = bool(allow)
def __setitem__(self, key, value):
if self.__readonly:
raise TypeError, "__setitem__ is not supported"
return dict.__setitem__(self, key, value)
def __delitem__(self, key):
if self.__readonly:
raise TypeError, "__delitem__ is not supported"
return dict.__delitem__(self, key)
BTW, you can also remove .pop, .update and other methods you need. Just play around with it.
The best way is to derive from UserDict like this:
from collections import UserDict
class MyReadOnlyDict(UserDict):
def my_set(self, key, val, more_params):
# do something special
# custom logic etc
self.data[key] = val
def __setitem__(self, key, val):
raise NotImplementedError('This dictionary cannot be updated')
def __delitem__(self, key):
raise NotImplementedError('This dictionary does not allow delete')
The advantage of this method is that you can still have internal methods in your class that can update dictionary by accessing self.data.
How about? It is the update of #mouad 's answer.
import json
from collections import OrderedDict
from collections.abc import Mapping
class ReadOnlyJsonObject(Mapping):
def __init__(self, data, dumps_kw: dict=None, loads_kw: dict=None):
if dumps_kw is None:
dumps_kw = dict()
if loads_kw is None:
self._loads_kw = dict(object_pairs_hook=OrderedDict)
else:
self._loads_kw = loads_kw
if isinstance(data, str):
self._json_string = data
else:
self._json_string = json.dumps(data, **dumps_kw)
#property
def _data(self):
return json.loads(self._json_string, **self._loads_kw)
def __getitem__(self, key):
return self._data[key]
def __len__(self):
return len(self._data)
def __iter__(self):
return iter(self._data)
def __str__(self):
return self._json_string
Not sure about the performance, though. I use this one in a real project https://github.com/patarapolw/AnkiTools/blob/master/AnkiTools/tools/defaults.py
I am interested in using the python list object, but with slightly altered functionality. In particular, I would like the list to be 1-indexed instead of 0-indexed. E.g.:
>> mylist = MyList()
>> mylist.extend([1,2,3,4,5])
>> print mylist[1]
output should be: 1
But when I changed the __getitem__() and __setitem__() methods to do this, I was getting a RuntimeError: maximum recursion depth exceeded error. I tinkered around with these methods a lot but this is basically what I had in there:
class MyList(list):
def __getitem__(self, key):
return self[key-1]
def __setitem__(self, key, item):
self[key-1] = item
I guess the problem is that self[key-1] is itself calling the same method it's defining. If so, how do I make it use the list() method instead of the MyList() method? I tried using super[key-1] instead of self[key-1] but that resulted in the complaint TypeError: 'type' object is unsubscriptable
Any ideas? Also if you could point me at a good tutorial for this that'd be great!
Thanks!
Use the super() function to call the method of the base class, or invoke the method directly:
class MyList(list):
def __getitem__(self, key):
return list.__getitem__(self, key-1)
or
class MyList(list):
def __getitem__(self, key):
return super(MyList, self).__getitem__(key-1)
However, this will not change the behavior of other list methods. For example, index remains unchanged, which can lead to unexpected results:
numbers = MyList()
numbers.append("one")
numbers.append("two")
print numbers.index('one')
>>> 1
print numbers[numbers.index('one')]
>>> 'two'
Instead, subclass integer using the same method to define all numbers to be minus one from what you set them to. Voila.
Sorry, I had to. It's like the joke about Microsoft defining dark as the standard.
You can avoid violating the Liskov Substitution principle by creating a class that inherits from collections.MutableSequence, which is an abstract class. It would look something like this:
def indexing_decorator(func):
def decorated(self, index, *args):
if index == 0:
raise IndexError('Indices start from 1')
elif index > 0:
index -= 1
return func(self, index, *args)
return decorated
class MyList(collections.MutableSequence):
def __init__(self):
self._inner_list = list()
def __len__(self):
return len(self._inner_list)
#indexing_decorator
def __delitem__(self, index):
self._inner_list.__delitem__(index)
#indexing_decorator
def insert(self, index, value):
self._inner_list.insert(index, value)
#indexing_decorator
def __setitem__(self, index, value):
self._inner_list.__setitem__(index, value)
#indexing_decorator
def __getitem__(self, index):
return self._inner_list.__getitem__(index)
def append(self, value):
self.insert(len(self) + 1, value)
class ListExt(list):
def extendX(self, l):
if l:
self.extend(l)
class a(object):
w='www'
def __init__(self):
for i in self.keys():
print i
def __iter__(self):
for k in self.keys():
yield k
a() # why is there an error here?
Thanks.
Edit: The following class also doesn't extend any class;
why it can use keys?
class DictMixin:
# Mixin defining all dictionary methods for classes that already have
# a minimum dictionary interface including getitem, setitem, delitem,
# and keys. Without knowledge of the subclass constructor, the mixin
# does not define __init__() or copy(). In addition to the four base
# methods, progressively more efficiency comes with defining
# __contains__(), __iter__(), and iteritems().
# second level definitions support higher levels
def __iter__(self):
for k in self.keys():
yield k
def has_key(self, key):
try:
value = self[key]
except KeyError:
return False
return True
def __contains__(self, key):
return self.has_key(key)
# third level takes advantage of second level definitions
def iteritems(self):
for k in self:
yield (k, self[k])
def iterkeys(self):
return self.__iter__()
# fourth level uses definitions from lower levels
def itervalues(self):
for _, v in self.iteritems():
yield v
def values(self):
return [v for _, v in self.iteritems()]
def items(self):
return list(self.iteritems())
def clear(self):
for key in self.keys():
del self[key]
def setdefault(self, key, default=None):
try:
return self[key]
except KeyError:
self[key] = default
return default
def pop(self, key, *args):
if len(args) > 1:
raise TypeError, "pop expected at most 2 arguments, got "\
+ repr(1 + len(args))
try:
value = self[key]
except KeyError:
if args:
return args[0]
raise
del self[key]
return value
def popitem(self):
try:
k, v = self.iteritems().next()
except StopIteration:
raise KeyError, 'container is empty'
del self[k]
return (k, v)
def update(self, other=None, **kwargs):
# Make progressively weaker assumptions about "other"
if other is None:
pass
elif hasattr(other, 'iteritems'): # iteritems saves memory and lookups
for k, v in other.iteritems():
self[k] = v
elif hasattr(other, 'keys'):
for k in other.keys():
self[k] = other[k]
else:
for k, v in other:
self[k] = v
if kwargs:
self.update(kwargs)
def get(self, key, default=None):
try:
return self[key]
except KeyError:
return default
def __repr__(self):
return repr(dict(self.iteritems()))
def __cmp__(self, other):
if other is None:
return 1
if isinstance(other, DictMixin):
other = dict(other.iteritems())
return cmp(dict(self.iteritems()), other)
def __len__(self):
return len(self.keys())
Why would you expect it to have keys? You didn't define such a method in your class. Did you intend to inherit from a dictionary?
To do that declare class a(dict)
Or maybe you meant a.__dict__.keys()?
As for the large snippet you've posted in the update, read the comment above the class again:
# Mixin defining all dictionary methods for classes that already have
# a minimum dictionary interface including getitem, setitem, delitem,
# and keys
Note that "already have ... keys" part.
The DictMixin class comes from the UserDict module, which says:
class UserDict.DictMixin Mixin
defining all dictionary methods for
classes that already have a minimum
dictionary interface including
getitem(), setitem(), delitem(), and keys().
This mixin should be used as a
superclass. Adding each of the above
methods adds progressively more
functionality. For instance, defining
all but delitem() will preclude
only pop() and popitem() from the full
interface.
In addition to the four base methods,
progressively more efficiency comes
with defining contains(),
iter(), and iteritems().
Since the mixin has no knowledge of
the subclass constructor, it does not
define init() or copy().
Starting with Python version 2.6, it
is recommended to use
collections.MutableMapping instead of
DictMixin.
Note the recommendation in the last part - use collections.MutableMapping instead.
To iterate over attributes of an object:
class A(object):
def __init__(self):
self.myinstatt1 = 'one'
self.myinstatt2 = 'two'
def mymethod(self):
pass
a = A()
for attr, value in a.__dict__.iteritems():
print attr, value