I built a custom list-like class based on collections.MutableSequence:
class MyList(collections.MutableSequence):
etc... behave mostly like a list...
value = MyList([1,2,3])
Before processing list data, a third-party library runs a check:
def check_correct_type(value):
assert isinstance(value, list)
I do not wish to convert my custom list-like object to a built-in list before passing it to the third-party library.
Is there an elegant way to make an instance of MyList appear as though it was an instance of list in the isinstance(MyList([1,2,3]), list) check?
No, there is no way instances of your class can pass that test without inheriting from list. You have to subclass list to pass that test.
You can try inheriting from both MutableSequence and list; any method or attribute not implemented by your class or by MutableSequence will then be looked up on list, so you may get extra methods that you don't want this way and those may behave unexpectedly:
class MyList(collections.MutableSequence, list):
You could also monkeypatch the check_correct_type() function, provided it really is a stand-alone function like that:
def my_check_correct_type(value):
assert isinstance(value, collections.MutableSequence)
third_party_library.check_correct_type = my_check_correct_type
Related
I’m building a class that extends the list data structure in Python, called a Partitional. I’m adding a few methods that I find myself using frequently when dividing a list into partitions.
The class is initialized with a (nullable) list, which exists as an attribute on the class.
class Partitional(list):
"""Extends the list data type. Adds methods for dividing a list into partition sets
and returning data about those partition sets"""
def __init__(self, source_list: list=[]):
super().__init__()
self.source_list: list = source_list
self.n: int = len(source_list)
...
I want to be able to reliably replace list instances with Partitional instances without violating Liskov substitution. So for list’s methods, I wrote methods on the Partitional class that operate on self.source_list, e.g.
...
def remove(self, matched_item):
self.source_list.remove(matched_item)
self.__init__(self.source_list)
def pop(self, *args):
popped_item = self.source_list.pop(*args)
self.__init__(self.source_list)
return popped_item
def clear(self):
self.source_list.clear()
self.__init__(self.source_list)
...
(the __init__ call is there because the Partitional class builds some internal attributes based on self.source_list when it’s initialized, so these need to be rebuilt if source_list changes.)
And I also want Python’s built-in methods that take a list as an argument to work with a Partitional instance, so I set to work writing method overrides for those as well, e.g.
...
def __len__(self):
return len(self.source_list)
def __enumerate__(self):
return enumerate(self.source_list)
...
The relevant built-in methods are a finite set for any given Python version, but... is there not a simpler way to do this?
My question:
Is there a way to write a class such that, if an instance of that class is used as the argument for a function, the class provides an attribute to the function instead, by default?
That way I’d only need to override this default behaviour for a subset of built-in methods.
So for example, if a use case involving a list instance looks like this:
example_list: list = [1,2,3,4,5]
length = len(example_list)
we substitute a Partitional instance built from the same list:
example_list: list = [1,2,3,4,5]
example_partitional = Partitional(example_list)
length = len(example_partitional)
and what’s “actually” happening is this:
length = len(example_partitional.source_list)
i.e.
length = len([1,2,3,4,5])
Other notes:
In working on this, I’ve realized that there are two broad categories of Liskov substitution violation possible:
Inherent violation, where the structure of the child class will make it incompatible with any use case where the child class is used in place of the parent class, e.g. if you override some fundamental property or structure of the parent.
Context-dependent violation, where, for any given piece of software, so long as you never use the child class in a way that would violate Liskov substitution, you’re fine. E.g. You override a method on the parent class that would change how a built-in function acts when it takes an instance of the class as an argument, but you never use that built-in method with the class instance in your system. Or any system that depends on your system. Or... (you see how relying on this caveat is not foolproof)
What I’m looking to do is come up with a technique that will protect against both categories of violation, without having to worry about use cases and context.
I am new to Python and I am trying to find a way to add new methods to pre-existing classes in Python. For example, to add a .print() method to the list class.
I know I can create a new class that inherits the list class and add this method like so:
import builtins
class list(list):
def print(self):
builtins.print(self)
But this doesn't modify the pre-existing 'list' class. I can do assignments like this: trial_list = list([3,4,6]) but not like this: trial_list = [3,4,6].
Also, is there a way to view the actual content of the list class besides dir() and help()?
The recommended approach to subclass built in container types, is to use the abstract base classes provided in the collections module:
In the case of list, you should use collections.UserList
from collections import UserList
class MySpecialList(UserList):
def print(self):
print(self)
seq = MySpecialList([1, 2, 3])
print(seq)
seq.print()
You can now create MySpecialList objects the way you do with a plain python list.
Subclassing requirements: Subclasses of UserList are expected to offer
a constructor which can be called with either no arguments or one
argument. List operations which return a new sequence attempt to
create an instance of the actual implementation class. To do so, it
assumes that the constructor can be called with a single parameter,
which is a sequence object used as a data source.
Of course you can override the methods of list to provide your MySpecialList with the behavior you need.
I've created a class that is a tuple wrapper and tuples doesn't support item mutations.
Should I leave __setitem__ and __delitem__ implementation or implement those methods like e.g. below (thus fall in kind of Refused Bequest code smell)? Which approach is more pythonic? Aren't custom exceptions better in such case?
def __setitem__(self, key, value):
"""
:raise: Always.
:raises: TypeError
"""
self.data_set[key] = value # Raise from tuple.
def __delitem__(self, key):
"""
:raise: Always.
:raises: TypeError
"""
raise TypeError("Item deletion is unsupported") # Custom exceptions thrown.
If your class is supposed to be a proper tuple subtype (according to Liskov substitution principle), then it should behave the same way as a tuple wrt/ to set/del - which as Guillaume mentions is the default behaviour if you just define neither __setitem__ nor __delitem__. I don't see how that would fall into the "Refused Bequest" category.
If your class uses a tuple as part of it's implementation but is NOT supposed to be a proper tuple subtype, then do whatever makes sense - but if you don't want to allow item assignment / deletion then again the simplest thing is to not implement them.
Although that is a matter of taste, I think you should not implement them at all. A class that has a __setitem__, __delitem__ implements the mutable collection protocol (either implicitly, or even explicitly by using collection abstract base classes). Your class just does not support this interface, that's it, and the user has neither reason nor right to assume it does
Implement one or the other or both if they make sense for your custom class.
If you implement __setitem__() you will be able to use yourobject[yourindex] = yourvalue syntax in your code (with the semantic that you choose to implement).
If you implement __delitem__() you will be able to use del yourobject[yourindex]
It makes no sense to explictly implement a method just to raise an Exception, Python will do it by default:
class Test(object):
pass
test = Test()
test['foo'] = 'bar' # will call Test.__setitem__() which is not explicitly defined
will give TypeError: 'Test' object does not support item assignment
I want to do something like this:
class Dictable:
def dict(self):
raise NotImplementedError
class Foo(Dictable):
def dict(self):
return {'bar1': self.bar1, 'bar2': self.bar2}
Is there a more pythonic way to do this? For example, is it possible to overload the built-in conversion dict(...)? Note that I don't necessarily want to return all the member variables of Foo, I'd rather have each class decide what to return.
Thanks.
The Pythonic way depends on what you want to do. If your objects shouldn't be regarded as mappings in their own right, then a dict method is perfectly fine, but you shouldn't "overload" dict to handle dictables. Whether or not you need the base class depends on whether you want to do isinstance(x, Dictable); note that hasattr(x, "dict") would serve pretty much the same purpose.
If the classes are conceptually mappings of keys to values, then implementing the Mapping protocol seems appropriate. I.e., you'd implement
__getitem__
__iter__
__len__
and inherit from collections.Mapping to get the other methods. Then you get dict(Foo()) for free. Example:
class Foo(Mapping):
def __getitem__(self, key):
if key not in ("bar1", "bar2"):
raise KeyError("{} not found".format(repr(key))
return getattr(self, key)
def __iter__(self):
yield "bar1"
yield "bar2"
def __len__(self):
return 2
Firstly, look at collections.ABC, which describes the Python abstract base class protocol (equivalent to interfaces in static languages).
Then, decide if you want to write your own ABC or make use of an existing one; in this case, Mapping might be what you want.
Note that although the dict constructor (i.e. dict(my_object)) is not overrideable, if it encounters an iterable object that yields a sequence of key-value pairs, it will construct a dict from that; i.e. (Python 2; for Python 3 replace items with iteritems):
def __iter__(self):
return {'bar1': self.bar1, 'bar2': self.bar2}.iteritems()
However, if your classes are intended to behave like a dict you shouldn't do this as it's different from the expected behaviour of a Mapping instance, which is to iterate over keys, not key-value pairs. In particular it would cause for .. in to behave incorrectly.
Most of the answers here are about making your class behave like a dict, which isn't actually what you asked. If you want to express the idea, "I am a class that can be turned into a dict," I would simply define a bunch of classes and have them each implement .dict(). Python favors duck-typing (what an object can do) over what an object is. The ABC doesn't add much. Documentation serves the same purpose.
You can certainly overload dict() but you almost never want to! Too many aspects of the standard library depend upon dict being available and you will break most of its functionality. You cab probably do something like this though:
class Dictable:
def dict(self):
return self.__dict__
I need a custom __reverse__ function for my class that I am deploying on App Engine, so it needs to work with Python 2.5. Is there a __future__ import or a workaround I could use?
Subclassing list won't work, as I need my class to be a subclass of dict.
EDIT:
Using OrderedDict will not solve the problems, because the dict keys are not the same the same as the list items.
This is the object I'm trying to create:
My object needs to provide the same attributes as a list, i.e. support iter(obj) and reverse(obj).
The elements must be instances of a special third party class.
Each elements is associated with a key.
Internally, need to access these objects using their keys. That's why I'd put them in a mapping.
I've revised my implementation to be a list subclass instead of a dict subclass, so here's what I have now:
class Foo(list):
pat = {}
def __init__(self):
for app in APPS: # these are strings
obj = SpecialClass(app)
self.append(obj)
self.pat[app] = obj
def __getitem__(self, item):
# Use object as a list
if isinstance(item, int):
return super(Foo, self).__getitem__(item)
# Use object as a dict
if item not in self.pat:
# Never raise a KeyError
self.pat[item] = SpecialClass(None)
return self.pat[item]
def __setitem__(self, item, value):
if isinstance(item, int):
return self.pat.__setitem__(item, value)
return super(Foo).__setitem__(item, value)
EDIT 2:
Now that my class is a subclass of list, my problem is resolved.
__reversed__ isn't supported in 2.5, so your only option if you really need to customize the reversed order of your collection, is to modify the places that you call reversed to use something else.
But I'm curious: if you are subclassing dict, then the order of items is arbitrary anyway, so what does reversed mean in this case?
Creating a custom __reversed__ is only possible since 2.6, so you can't simply implement that and have reversed working in 2.5. In 2.5 and below, you can however make your custom class still working with reversed by implementing the sequence protocol (i.e. implement both __len__ and __getitem__).
A different possibility would be to replace the built-in function reversed with a custom function that treats your custom class differently. This could work like this:
originalReversed = reversed
def myReversed ( seq ):
if isinstance( seq, MyCustomClass ):
# do something special
else:
return originalReversed( seq )
reversed = myReversed
However, I wouldn't recommend that as it changes the normal behaviour of built-in functions (obviously) and might confuse other users.. So you should rather implement the sequnce protocol to make reversed working.