How can every set method in python's list object be overridden in a derived object such that every item in that list is of a specific class?
Consider
class Index(int):
pass
class IndexList(list):
def __init__(self, int_list):
for el in int_list:
self.append(Index(el))
def __setitem__(self, key, val):
super(IndexList, self).__setitem__(key, Index(val))
# override append insert etc...
Can this be done without directly overriding every single function that adds elements to the list? I expected simply overriding __setitem__ was enough.
Example, if append is not overridden.
ilist = IndexList([1,2])
ilist.append(3)
for i in ilist:
print(isinstance(i, Index)) # True, True, False
You'll have to implement the various directly; the underlying C implementation does not call __setitem__ for each and every change, as it is far more efficient to directly manipulate the (dynamically grown) C array.
Take a look at the collections abstract base classes, specifically at the MutableSequence ABC, to get an idea of what methods all can mutate your list, to maintain your type invariant you'd need to implement insert, append, extend and __iadd__.
Better still, you can use the collections.MutableSequence() class as an alternative base class to list; this is a pure-python implementation that does cast many of those methods as calls to a core set of methods; you'd only need to provide implementations for __len__, __getitem__, __setitem__, __delitem__ and insert; any method named in the Abstract Methods column of the table.
class IndexList(collections.MutableSequence):
def __init__(self, int_list):
self._list = []
for el in int_list:
self.append(Index(el))
def __len__(self): return len(self._list)
def __getitem__(self, item): return self._list[item]
def __delitem__(self, item): del self._list[item]
def __setitem__(self, index, value):
self._list.key[index] = Index(value)
def insert(self, index, value):
self._list.insert(index, Index(value))
Related
I know Python will simply return the key list when I put a dict in for...in... syntax.
But what what happens to the dict?
When we use help(dict), we can not see __next()__ method in the method list. So if I want to make a derived class based on dict:
class MyDict(dict)
def __init__(self, *args, **kwargs):
super(MyDict, self).__init__(*args, **kwargs)
and return the value list with for...in...
d = Mydict({'a': 1, 'b': 2})
for value in d:
what should I do?
Naively, if all you want is for iteration over an instance of MyClass to yield the values instead of the keys, then in MyClass define:
def __iter__(self):
return self.itervalues()
In Python 3:
def __iter__(self):
return iter(self.values())
But beware! By doing this your class no longer implements the contract of collections.MutableMapping, even though issubclass(MyClass, collections.MutableMapping) is True. You might be better off not subclassing dict, if this is the behaviour you want, but instead have an attribute of type dict to hold the data, and implement only the functions and operators you need.
When using Python properties (setters and getters), usually following is used:
class MyClass(object):
...
#property
def my_attr(self):
...
#my_attr.setter
def my_attr(self, value):
...
However, is there any similar approach for appending / removing arrays? For example, in a bi-directional relationship between two objects, when removing object A, it would be nice to dereference the relationship to A in object B. I know that SQLAlchemy has implemeneted a similar function.
I also know that I can implement methods like
def add_element_to_some_array(element):
some_array.append(element)
element.some_parent(self)
but I would prefer to do it like "properties" in Python.. do you know some way?
To make your class act array-like (or dict-like), you can override __getitem__ and __setitem__.
class HappyArray(object):
#
def __getitem__(self, key):
# We skip the real logic and only demo the effect
return 'We have an excellent %r for you!' % key
#
def __setitem__(self, key, value):
print('From now on, %r maps to %r' % (key, value))
>>> h = HappyArray()
>>> h[3]
'We have an excellent 3 for you!'
>>> h[3] = 'foo'
From now on, 3 maps to 'foo'
If you want several attributes of your object to exhibit such behavior, you need several array-like objects, one for each attribute, constructed and linked at your master object's creation time.
The getter property will return reference to the array. You may do array operations with that. Like this
class MyClass(object):
...
#property
def my_attr(self):
...
#my_attr.setter
def my_attr(self, value):
...
m = MyClass()
m.my_attr.append(0) # <- array operations like this
What's the proper way to subclass object.__getitem__() but fall back on the default behavior? I want to make this look like a dict that behaves dynamically (but consistently) for any possible key.
class MyObj(object):
"""MyObj()[k] returns 1 for any key k that ends in '_q',
default item lookup behavior otherwise"""
mypattern = re.compile(r'_q$')
def __getitem__(self, k):
if self.mypattern.match(k):
return 1
else:
???
I tried calling return super(TFConverter, self).__getitem__(k) and it tells me AttributeError: 'super' object has no attribute '__getitem__'
If you're looking to emulate a dict-like interface, you should raise a KeyError when the item doesn't exist, just like dict does. This tends to be the case when your keys are strings.
def __getitem__(self, k):
if self.mypattern.match(k):
return 1
else:
raise KeyError("Key {} does not match pattern {}".format(k, self.mypattern))
If you want your object to work like a list (generally the case if it has integer keys), you should raise an IndexError.
Calling super().__getitem__ doesn't work because object doesn't define __getitem__. __getitem__ only exists on objects which support subscripting. So if you want to delegate to a superclass's implementation of __getitem__, you have to make sure you're subclassing something which defines it (such as dict).
An alternative use-case is to return a default value when the key is not found. For example, here's a pure Python paraphrase of collections.defaultdict (the real version is written in C):
class defaultdict(dict):
# default_factory is a function which supplies the default value when a key is not found
def __init__(self, default_factory):
self.default_factory = default_factory
def __getitem__(self, key):
if key in self:
# the super() call works here because we are
# subclassing dict, which supports __getitem__
return super().__getitem__(key)
if self.default_factory is not None:
val = self.default_factory(key)
self[key] = val
return val
raise KeyError(key)
There is no object.__getitem__. To be able to call the base class implementation of __getitem__, you need to inherit from a class that actually has one. Given that, your super invocation will work correctly.
I am dealing with classes whose attributes are sometimes list whose elements can be dictionaries or further nested objects with attributes etc. I would like to perform some slicing that with my grasp of python is only doable with what feels profoundly un-Pythonic.
My minimal code looks like this:
class X(object):
def __init__(self):
self.a = []
x=X()
x.a.append({'key1':'v1'})
x.a.append({'key1':'v2'})
x.a.append({'key1':'v3'})
# this works as desired
x.a[0]['key1'] # 'v1'
I would like to perform an access to a key in the nested dictionary but make that call for all elements of the list containing that dictionary. The standard python way of doing this would be a list comprehension a la:
[v['key1'] for v in x.a]
However, my minimal example doesn't quite convey the full extent of nesting in my real-world scenario: The attribute list a in class X might contain objects, whose attributes are objects, whose attributes are dictionaries whose keys I want to select on while iterating over the outer list.
# I would like something like
useful_list = x.a[:]['key1'] # TypeError: list indices must be integers, not str
# or even better
cool_list = where(x.a[:]['key1'] == 'v2') # same TypeError
If I start list comprehending for every interesting key it quickly doesn't look all that Pythonic. Is there a nice way of doing this or do I have to code 'getter' methods for all conceivable pairings of lists and dictionary keys?
UPDATE:
I have been reading about overloading lists. Apparently one can mess with the getitem method which is used for indeces for lists and keys for dict. Maybe a custom class that iterates over list members. This is starting to sound contrived...
So, you want to create an hierarchical structure, with an operation which means
different things for different types, and is defined recursively.
Polymorphism to the rescue.
You could override __getitem__ instead of my get_items below, but in your case it might be better to define a non-builtin operation to avoid risking ambiguity. It's up to you really.
class ItemsInterface(object):
def get_items(self, key):
raise NotImplementedError
class DictItems(ItemsInterface, dict):
def __init__(self, *args, **kwargs):
dict.__init__(self, *args, **kwargs)
def get_items(self, key):
res = self[key]
# apply recursively
try:
res = res.get_items(key)
except AttributeError:
pass
return res
class ListItems(ItemsInterface, list):
def __init__(self, *args, **kwargs):
list.__init__(self, *args, **kwargs)
def get_items(self, key):
return [ x.get_items(key) for x in self ]
x = ListItems()
x.append(DictItems({'key1':'v1'}))
x.append(DictItems({'key1':'v2'}))
x.append(DictItems({'key1':'v3'}))
y = DictItems({'key1':'v999'})
x.append(ListItems([ y ]))
x.get_items('key1')
=> ['v1', 'v2', 'v3', ['v999']]
Of course, this solution might not be exactly what you need (you didn't explain what it should do if the key is missing, etc.)
but you can easily modify it to suit your needs.
This solution also supports ListItems as values of the DictItems. the get_items operation is applied recursively.
Edit: There was some confusion, but I want to ask a general question about object oriented design in Python.
Consider a class that lets you map data values to counts or frequencies:
class DataMap(dict):
pass
Now consider a subclass that allows you to construct a histogram from a list of data:
class Histogram(DataMap):
def __init__(self, list_of_values):
# 1. Put appropriate super(...) call here if necessary
# 2. Build the map of values to counts in self
pass
Now consider a class that lets you make a smoothed probability mass table rather than a Histogram.
class ProbabilityMass(DataMap):
pass
What is the best way to allow a ProbabilityMass to be constructed from either a Histogram or a list of values?
I "grew up" programming in C++, and in this case I would use an overloaded constructor. In Python I've thought of doing this with:
The constructor takes multiple arguments (all but one of these should == None)
I define from_Histogram and from_list methods
In the second case (which I believe is better), what is the best way to allow the from_list method to use the shared code from the Histogram constructor? A ProbabilityMass table is nearly identical to a Histogram table, but it is scaled so that the sum of all value is 1.0.
If you have come across a similar problem, please share your expertise!
To start with, if you think you want #staticmethod, you almost always don't. Either the function is not part of the class, in which case it should just be a free function, or it is part of the class, but not tied to an instance, and it should be a #classmethod. Your named constructor is a good candidate for a #classmethod.
Also note that you should invoke A.__init__ from B via super(), otherwise multiple inheritance can bite you bad.
class A:
def __init__(self, data):
self.values_to_counts = {}
for val in data:
if val in self.values_to_counts:
self.values_to_counts[val] += 1
else:
self.values_to_counts[val] = 1
#classmethod
def from_values_to_counts(cls, values_to_counts):
self = cls([])
self.values_to_counts = values_to_counts
return self
class B(A):
def __init__(self, data, parameter):
super(B, self).__init__(data)
self.parameter = parameter
def print_parameter(self):
print self.parameter
In this case, you don't need a B.from_values_to_counts, it inherits from A, and it will return an instance of B, since that's how it was called.
If you need to do more complex initialization in B, you can, using super(), which looks very similar to the way it would when you use it with instances. after all, a classmethod really isn't anything more complex than an instancemethod where the im_self attribute is assigned to the class itself.
class A:
def __init__(self, data):
self.values_to_counts = {}
for val in data:
if val in self.values_to_counts:
self.values_to_counts[val] += 1
else:
self.values_to_counts[val] = 1
#classmethod
def from_values_to_counts(cls, values_to_counts):
self = cls([])
self.values_to_counts = values_to_counts
return self
class B(A):
def __init__(self, data, parameter):
super(B, self).__init__(data)
self.parameter = parameter
def print_parameter(self):
print self.parameter
#classmethod
def from_values_to_counts(cls, values_to_counts):
self = super(B, cls).from_values_to_counts(values_to_counts)
do_more_initialization(self)
return self