I am interested in using the python list object, but with slightly altered functionality. In particular, I would like the list to be 1-indexed instead of 0-indexed. E.g.:
>> mylist = MyList()
>> mylist.extend([1,2,3,4,5])
>> print mylist[1]
output should be: 1
But when I changed the __getitem__() and __setitem__() methods to do this, I was getting a RuntimeError: maximum recursion depth exceeded error. I tinkered around with these methods a lot but this is basically what I had in there:
class MyList(list):
def __getitem__(self, key):
return self[key-1]
def __setitem__(self, key, item):
self[key-1] = item
I guess the problem is that self[key-1] is itself calling the same method it's defining. If so, how do I make it use the list() method instead of the MyList() method? I tried using super[key-1] instead of self[key-1] but that resulted in the complaint TypeError: 'type' object is unsubscriptable
Any ideas? Also if you could point me at a good tutorial for this that'd be great!
Thanks!
Use the super() function to call the method of the base class, or invoke the method directly:
class MyList(list):
def __getitem__(self, key):
return list.__getitem__(self, key-1)
or
class MyList(list):
def __getitem__(self, key):
return super(MyList, self).__getitem__(key-1)
However, this will not change the behavior of other list methods. For example, index remains unchanged, which can lead to unexpected results:
numbers = MyList()
numbers.append("one")
numbers.append("two")
print numbers.index('one')
>>> 1
print numbers[numbers.index('one')]
>>> 'two'
Instead, subclass integer using the same method to define all numbers to be minus one from what you set them to. Voila.
Sorry, I had to. It's like the joke about Microsoft defining dark as the standard.
You can avoid violating the Liskov Substitution principle by creating a class that inherits from collections.MutableSequence, which is an abstract class. It would look something like this:
def indexing_decorator(func):
def decorated(self, index, *args):
if index == 0:
raise IndexError('Indices start from 1')
elif index > 0:
index -= 1
return func(self, index, *args)
return decorated
class MyList(collections.MutableSequence):
def __init__(self):
self._inner_list = list()
def __len__(self):
return len(self._inner_list)
#indexing_decorator
def __delitem__(self, index):
self._inner_list.__delitem__(index)
#indexing_decorator
def insert(self, index, value):
self._inner_list.insert(index, value)
#indexing_decorator
def __setitem__(self, index, value):
self._inner_list.__setitem__(index, value)
#indexing_decorator
def __getitem__(self, index):
return self._inner_list.__getitem__(index)
def append(self, value):
self.insert(len(self) + 1, value)
class ListExt(list):
def extendX(self, l):
if l:
self.extend(l)
Related
Just learning about properties and setters in python, and seems fair enough when we have a mutable attribute. But what happens when I want to validate a .append() on a list for example? In the below, I can validate the setting of the attribute and it works as expected. But I can bypass its effect by simply appending to get more cookies onto the tray...
class CookieTray:
def __init__(self):
self.cookies = []
#property
def cookies(self):
return self._cookies
#cookies.setter
def cookies(self, cookies):
if len(cookies) > 8:
raise ValueError("Too many cookies in the tray!")
self._cookies = cookies
if __name__ == '__main__':
tray = CookieTray()
print("Cookies: ", tray.cookies)
try:
tray.cookies = [1,1,0,0,0,1,1,0,1] # too many
except Exception as e:
print(e)
tray.cookies = [1,0,1,0,1,0]
print(tray.cookies)
tray.cookies.append(0)
tray.cookies.append(0)
tray.cookies.append(1) # too many, but can still append
print(tray.cookies)
Silly example, but I hope it illustrates my question. Should I just be avoiding the setter and making a "setter" method, like add_cookie(self, cookie_type) and then do my validation in there?
The setter only applies when assigning to the attribute. As you've seen mutating the attribute bypasses this.
To apply the validation to the object being mutated we can use a custom type. Here's an example which just wraps a normal list:
import collections
class SizedList(collections.abc.MutableSequence):
def __init__(self, maxlen):
self.maxlen = maxlen
self._list = []
def check_length(self):
if len(self._list) >= self.maxlen:
raise OverflowError("Max length exceeded")
def __setitem__(self, i, v):
self.check_length()
self._list[i] = v
def insert(self, i, v):
self.check_length()
self._list.insert(i, v)
def __getitem__(self, i): return self._list[i]
def __delitem__(self, i): del self._list[i]
def __len__(self): return len(self._list)
def __repr__(self): return f"{self._list!r}"
When overriding container types collections.abc can be useful - we can see the abstract methods that must be implemented: __getitem__, __setitem__, __delitem__, __len__, and insert in this case. All of them are delegated to the list object that's being wrapped, with the two that add items having the added length check.
The repr isn't needed, but makes it easier to check the contents - once again just delegating to the wrapped list.
With this you can simply replace the self.cookies = [] line with self.cookies = SizedList(8).
You would need to create a ValidatingList class that overrides list.append, something like this:
class ValidatingList(list):
def append(self, value):
if len(self) >= 8:
raise ValueError("Too many items in the list")
else:
super().append(value)
You then convert your cookies to a ValidatingList:
class CookieTray:
def __init__(self):
self.cookies = []
#property
def cookies(self):
return self._cookies
#cookies.setter
def cookies(self, cookies):
if len(cookies) > 8:
raise ValueError("Too many cookies in the tray!")
self._cookies = ValidatingList(cookies)
recently i needed to implement some data structures that could tell if they have been modified since a given point in time. Currently i have a ChangeDetectable class that implements the check but the implementation of setting the flag is left for the child classes. Here's a minimal example:
ChangeDetectable class:
class ChangeDetectable:
def __init__(self):
self._changed = False
def save_state(self):
self._changed = False
def has_changed(self) -> bool:
return self._changed
List-like class:
class MyList(MutableSequence, ChangeDetectable):
def __init__(self):
super().__init__()
self._list = list()
def __getitem__(self, item):
return self._list.__getitem__(item)
def __setitem__(self, key, value):
self._list.__setitem__(key, value)
self._changed = True
def __delitem__(self, key):
self._list.__delitem__(key)
self._changed = True
def __len__(self):
return self._list.__len__()
def insert(self, index, item):
self._list.insert(index, item)
self._changed = True
Dict-like class:
class MyDict(MutableMapping, ChangeDetectable):
def __init__(self):
super().__init__()
self._dict = dict()
def __getitem__(self, key):
return self._dict.__getitem__(key)
def __setitem__(self, key, value):
self._dict.__setitem__(key, value)
self._changed = True
def __delitem__(self, key):
self._dict.__delitem__(key)
self._changed = True
def __iter__(self):
return self._dict.__iter__()
def __len__(self):
return self._dict.__len__()
So my question is: right now children have to implement the right write operation. For instance MyList needs insert method and MyDict does not. Is there a way to implement all the methods i could possibly need in the parent and then only inherit in the children the ones i need?
This could make the code cleaner because it would have super() but i wouldn't want to have insert in MyDict.
Thank you.
Let me give you a general answer.
Abstract Base Classes
The official documentation for this is quite good: https://docs.python.org/3/library/abc.html.
Also please take a look at this SO question: What are metaclasses in Python?
You create a class like MyABC and then you define all the methods must be implemented and mark them as #abstractmethod. For example, if MyList and MyDict must implement values, then you should define a method values in MyABC. MyList and MyDict inherit from MyABC and must implement values, as seen in this answer.
If MyDict implements something specific to it, as a dictionary, then it simply defines and implements myKeys.
This is answered quite fully at Why use Abstract Base Classes in Python?.
Consider for example that we have a class 'Agent' as below:
class Agent:
def __init__(self, number):
self.position = []
self.number = number
for i in range(number):
self.position.append([0, 0])
I can make an instance of the class by:
agent = Agent(10)
and then access the i'th agent's position by:
agent.position[i]
However, this does not seem elegant enough and to me it's a bit counter-intuitive. Instead I want to index the class instance itself. For example:
pos_i = agent[i].position
which should return the same answer as the one-line code above. Is there a way to accomplish this?
If you want to do that, you just need a class-level container, with all instances.
Since your positions, given your example, are created in an arbitrary order, I'd suggest using a dictionary.
You can just fill the class-level "position" dictionary. You could then just implement the __getitem__ method to retrieve elements from this dictionary:
class Agent:
position = {}
def __new__(cls, pos):
if pos in cls.position:
return cls.position[pos]
instance = super().__new__(cls)
cls.position[pos] = instance
return instance
def __getitem__(self, item):
return self.position[pos]
This, however, will only allow you to retrieve an instance given the position from an instance - i.e.:
agent_5 = Agent(5)
agent_10 = agent_5[10]
would work, but not:
agent_10 = Agent[10]
If you want that, you have to use a custom metaclass, and put the __getitem__ method there:
class MAgent(type):
def __getitem__(cls, item):
return cls.position[pos]
class Agent(metaclass=MAgent):
position = {}
def __new__(cls, pos):
if pos in cls.position:
return cls.position[pos]
instance = super().__new__(cls)
cls.position[pos] = instance
return instance
If you want to overload the indexing operator just overload the __getitem__ method in the class.
class Agent:
def __getitem__(self, key):
return self.position[key]
>>> myobj = MyClass()
>>> myobj[3]
I am trying to master (begin ;)) to understand how to properly work with decorators and descriptors on Python 3. I came up with an idea that i´m trying to figure how to code it.
I want to be able to create a class A decorated with certain "function" B or "class" B that allows me to create a instance of A, after delaring properties on A to be a component of certain type and assigning values on A __init__ magic function. For instance:
componentized is certain "function B" or "class B" that allows me to declarate a class Vector. I declare x and y to be a component(float) like this:
#componentized
class Vector:
x = component(float)
y = component(float)
def __init__ (self, x, y):
self.x = x
self.y = y
What I have in mind is to be able to this:
v = Vector(1,2)
v.x #returns 1
But the main goal is that I want do this for every marked component(float) property:
v.xy #returns a tuple (1,2)
v.xy = (3,4) #assigns to x the value 3 and y the value 4
My idea is to create a decorator #componentized that overrides the __getattr__ and __setattr__ magic methods. Sort of this:
def componentized(cls):
class Wrapper(object):
def __init__(self, *args):
self.wrapped = cls(*args)
def __getattr__(self, name):
print("Getting :", name)
if(len(name) == 1):
return getattr(self.wrapped, name)
t = []
for x in name:
t.append(getattr(self.wrapped, x))
return tuple(t)
#componentized
class Vector(object):
def __init__(self, x, y):
self.x = x
self.y = y
And it kind of worked, but i don't think I quite understood what happened. Cause when I tried to do an assign and override the __setattr__ magic method it gets invoked even when I am instantiating the class. Two times in the following example:
vector = Vector(1,2)
vector.x = 1
How would could I achieve that sort of behavior?
Thanks in advance! If more info is needed don't hesitate to ask!
EDIT:
Following #Diego's answer I manage to do this:
def componentized(cls):
class wrappedClass(object):
def __init__(self, *args, **kwargs):
t = cls(*args,**kwargs)
self.wrappedInstance = t
def __getattr__(self, item):
if(len(item) == 1):
return self.wrappedInstance.__getattribute__(item)
else:
return tuple(self.wrappedInstance.__getattribute__(char) for char in item)
def __setattr__(self, attributeName, value):
if isinstance(value, tuple):
for char, val in zip(attributeName, value):
self.wrappedInstance.__setattr__(char, val)
elif isinstance(value, int): #EMPHASIS HERE
for char in attributeName:
self.wrappedInstance.__setattr__(char, value)
else:
object.__setattr__(self, attributeName, value)
return wrappedClass
And Having a class Vector like this:
#componentized
class Vector:
def __init__ (self, x, y):
self.x = x
self.y = y
It kind of behave like I wanted, but I still have no idea how to achieve:
x = component(float)
y = component(float)
inside the Vector class to somehow subscribe x and y of type float, so when I do the #EMPHASIS LINE(in the line I hardcoded a specific type) on the code I can check whether the value someone is assigning a value to x and/or y for an instance of Vector is of type I defined it with:
x = component(float)
So I tried this (a component (descriptor) class):
class component(object):
def __init__(self, t, initval=None):
self.val = initval
self.type = t
def __get__(self, obj, objtype):
return self.val
def __set__(self, obj, val):
self.val = val
To use component like a descriptor, but I couldn't managed to do a workaround to handle the type. I tried to do an array to hold val and type, but then didn't know how to get it from the decorator __setattr__ method.
Can you point me into the right direction?
PS: Hope you guys understand what I am trying to do and lend me a hand with it. Thanks in advance
Solution
Well, using #Diego´s answer (which I will be accepting) and some workarounds to achieve my personal needs I managed to this:
Decorator (componentized)
def componentized(cls):
class wrappedClass(object):
def __init__(self, *args):
self.wrappedInstance = cls(*args)
def __getattr__(self, name):
#Checking if we only request for a single char named value
#and return the value using getattr() for the wrappedInstance instance
#If not, then we return a tuple getting every wrappedInstance attribute
if(len(name) == 1):
return getattr(self.wrappedInstance, name)
else:
return tuple(getattr(self.wrappedInstance, char) for char in name)
def __setattr__(self, attributeName, value):
try:
#We check if there is not an instance created on the wrappedClass __dict__
#Meaning we are initializing the class
if len(self.__dict__) == 0:
self.__dict__[attributeName] = value
elif isinstance(value, tuple): # We get a Tuple assign
self.__checkMultipleAssign(attributeName)
for char, val in zip(attributeName, value):
setattr(self.wrappedInstance, char, val)
else:
#We get a value assign to every component
self.__checkMultipleAssign(attributeName)
for char in attributeName:
setattr(self.wrappedInstance, char, value)
except Exception as e:
print(e)
def __checkMultipleAssign(self, attributeName):
#With this we avoid assigning multiple values to the same property like this
# instance.xx = (2,3) => Exception
for i in range(0,len(attributeName)):
for j in range(i+1,len(attributeName)):
if attributeName[i] == attributeName[j]:
raise Exception("Multiple component assignment not allowed")
return wrappedClass
component (descriptor class)
class component(object):
def __init__(self, t):
self.type = t #We store the type
self.value = None #We set an initial value to None
def __get__(self, obj, objtype):
return self.value #Return the value
def __set__(self, obj, value):
try:
#We check whether the type of the component is diferent to the assigned value type and raise an exeption
if self.type != type(value):
raise Exception("Type \"{}\" do not match \"{}\".\n\t--Assignation never happened".format(type(value), self.type))
except Exception as e:
print(e)
else:
#If the type match we set the value
self.value = value
(The code comments are self explanatories)
With this design I can achieve what I wanted (explained above)
Thanks you all for your help.
I thing there is an easiest way to achive the behavior : overloading __getattr__and __setattr__ functions.
Getting vector.xy :
class Vector:
...
def __getattr__(self, item):
return tuple(object.__getattribute__(self, char) for char in item)
The __getattr__ function is called only when "normal" ways of accessing an atribute fails, as stated in the Python documentation.
So, when python doesn't find vector.xy, the __getattr__method is called and we return a tuple of every value (ie. x and y).
We use object.__getattribute__ to avoid infinite recurtion.
Setting vector.abc :
def __setattr__(self, key, value):
if isinstance(value, tuple) and len(key) == len(value):
for char, val in zip(key, value):
object.__setattr__(self, char, val)
else:
object.__setattr__(self, key, value)
The __setattr__ method is always called unlike __getattr__, so we set each value separately only when the item we want to set is of the same lenght as the tuple of value.
>>> vector = Vector(4, 2)
>>> vector.x
4
>>> vector.xy
(4, 2)
>>> vector.xyz = 1, 2, 3
>>> vector.xyxyxyzzz
(1, 2, 1, 2, 1, 2, 3, 3, 3)
The only drawback is that if you really want to asign a tuple like (suppose you have an attribute called size):
vector.size = (1, 2, 3, 4)
Then s, i, z and e will by assigned separately, and that's obviously not what you want !
FWIW, I've done something similar by abusing __slots__. I created an Abstract Base Class which read the subclass's slots and then used that for pickling (with __getstate__ and __setstate__). You could do something similar with get-set-attr but you will still need to muck about with the class's actual attr's vs the ones you want to use as get/set properties.
Previous answer:
Why not just use the #property decorator? See the third example in the docs. You would apply it by first changing the attr names to something different and private (like _x) and then use the actual name x as the property.
class Vector(object):
def __init__(self, x, y):
self._x = x
self._y = y
#property
def x(self):
return self._x
#x.setter
def x(self, value):
self._x = value
#property
def xy(self):
return (self._x, self._y) # returns a tuple
#xy.setter
def xy(self, value):
self._x, self._y = value # splits out `value` to _x and _y
And if you want this to happen with every attr automatically, then you will need to use a metaclass, as #kasramvd commented. If you don't have many such different classes where you want to do this or many properties, may not be worth the effort.
Currently when I want to define a setter and leave getter alone I do this:
#property
def my_property(self):
return self._my_property
#my_property.setter
def my_property(self, value):
value.do_some_magic()
self._my_property = value
Is there any way to make it shorter? I'd like to skip this part as it always look the same:
#property
def my_property(self):
return self._my_property
There's no out of the box solution, but you can try something like this:
def defprop(name):
def getter(self):
return getattr(self, name)
return property(getter)
class C(object):
# ...
my_dictionary = defprop('_my_dictionary')
# ...
That does not save you that many keystrokes though, you still have to duplicate the attribute name. Besides it's less explicit.
Update: after thinking a bit, I've come up with this descriptor-based hackish trick (disclaimer: this is done just for a demonstration, I don't imply it's a good practice unless you have a damn good reason to do so):
class with_default_getter(object):
def __init__(self, func):
self._attr_name = '_{0.__name__}'.format(func)
self._setter = func
def __get__(self, obj, type):
return getattr(obj, self._attr_name)
def __set__(self, obj, value):
return self._setter(obj, value)
Usage:
class C(object):
#with_default_getter
def my_property(self, value):
print 'setting %s'
self._my_property = value
>>> c = C()
>>> c.my_property = 123
setting 123
>>> c.my_property
123
This is pretty much the same as #georg suggests, just unfolds the implementation down to descriptors.
You can make a decorator that auto-creates the getter, following the underscores convention:
def setter(fn):
def _get(self):
return getattr(self, '_' + fn.__name__)
def _set(self, val):
return fn(self, val)
return property(_get, _set)
or more concisely, if you like this style more:
def setter(fn):
return property(
lambda self: getattr(self, '_' + fn.__name__),
fn)
Usage:
class X(object):
#setter
def my_property(self, value):
self._my_property = value + 1
x = X()
x.my_property = 42
print x.my_property # 43
There is no shortcut that I am aware of- remember explicit is better than implicit (from the Zen of python).
It could be that in your code so far, a property is always like that - but you could at some point write a a property getter which fetches an entirely calculated value - in which case your property getter and setter wont look like that at all.
Haveing said that you could write a wrapper which provides those simple default methods as part of the wrapper, if you wish.
def set_my_property(self, value):
value.do_some_magic()
self._my_property = value
my_property = property(fset=set_my_property)