Can you override a magic method when extending a builtin in python? - python

I am trying to extend a str and override the magic method __cmp__. The below example shows that the magic method __cmp__ is never called when > is used:
class MyStr(str):
def __cmp__(self, other):
print '(was called)',
return int(self).__cmp__(int(other))
print 'Testing that MyStr(16) > MyStr(7)'
print '---------------------------------'
print 'using _cmp__ :', MyStr(16).__cmp__(MyStr(7))
print 'using > :', MyStr(16) > MyStr(7)
when run results in:
Testing that MyStr(16) > MyStr(7)
---------------------------------
using __cmp__ : (was called) 1
using > : False
Obviously, when using the > the underlying "compare" functionality within the builtin is getting called, which in this case is an alphabetization ordering.
Is there a way to override the __cmp__ builtin with magic methods? And if you cannot directly - what is going on here that is different than non-magic methods where you can?

Comparison operators do not call __cmp__ if the corresponding magic method or its counterpart are defined and do not return NotImplemented:
class MyStr(str):
def __gt__(self, other):
print '(was called)',
return int(self) > int(other)
print MyStr(16) > MyStr(7) # True
P.S.: You probably don't want harmless comparisons to throw exceptions:
class MyStr(str):
def __gt__(self, other):
try:
return int(self) > int(other)
except ValueError:
return False

Related

Python: Raise error when comparing objects of different type for equality

As far as I understand, in Python, comparing the instances of two different classes for equality, does:
evaluate the __eq__ method of the first instance and return the result
Except if the result is NotImplemented, then evaluate the __eq__ method of the second instance and return the result
Except if the result is NotImplemented, then compare for object identity and return the result (which is False)
I encountered multiple times situations where a raised Exception would have helped me to spot a bug in my code. Martijn Pieters has sketched in this post a way to do that, but also mentioned that it's unpythonic.
Besides being unpythonic, are there actual problems arising from this approach?
Strictly speaking, it is, of course, "unpythonic" because Python encourages duck typing alongside strict subtyping. But I personally prefer strictly typed interfaces, so I throw TypeErrors. Never had any issues, but I can't difinitively say that you will never have any.
Theoretically, I can imagine it being a problem if you find yourself in need to use a mixed-type container like list[Optional[YourType]] and then compare its elements, maybe indirectly, like in constructing set(your_mixed_list).
It would cause problems for dictionaries with mixed keys that have the same hash. Dictionaries allow several keys to have the same hash, and then do an equality check to distinguish keys. See https://stackoverflow.com/a/9022835/1217284
The following script demonstrates this, it fails when the class Misbehaving1 is used:
#!/usr/bin/env python3
class Behaving1:
def __init__(self, value):
self.value = value
def __hash__(self):
return 7
class Behaving2:
def __init__(self, value):
self.value = value
def __hash__(self):
return 7
def __eq__(self, other):
if not (isinstance(other, type(self)) or isinstance(self, type(other))):
return NotImplemented
return self.value == other.value
class MisBehaving1:
def __init__(self, value):
self.value = value
def __hash__(self):
return 7
def __eq__(self, other):
if not (isinstance(other, type(self)) or isinstance(self, type(other))):
raise TypeError(f"Cannot compare {self} of type {type(self)} with {other} of type {type(other)}")
return self.value == other.value
d = {Behaving1(5): 'hello', Behaving2(4): 'world'}
print(d)
dd = {Behaving1(5): 'hello', MisBehaving1(4): 'world'}
print(dd)

How to selectively override '==' behaviour in Python?

Environment: Python 2.7 (It just might have something to do with this.)
First I understand the '==' is implemented as following (source):
if
type(b) is a new-style class,
type(b) is a subclass of type(a)
type(b) has overridden __eq__
then the result is b.__eq__(a)
If
type(a) has overridden __eq__ (that is, type(a).__eq__ isn't object.__eq__)
then the result is a.__eq__(b)
If
type(b) has overridden __eq__
then the result is b.__eq__(a).
If none of the above are the case, repeats the process above, but looking for __cmp__. If it exists, the objects are equal if it returns zero.
As a final fallback, Python calls object.__eq__(a, b), which is just testing if a and b are the same object.
.
Now I want to override __eq__ of an object, but falls back to the mechanism above when object has no custom __eq__ defined. How to achieve this?
I can't just save the original __eq__ method because '==' actually involves a complex mechanism described above.
Example Code (with target not achieved):
class A(object):
def __eq__(self, other):
try:
self.___eq___
return self.___eq___(other)
except AttributeError:
# trying to save default behaviour (Goal)
def custom_eq_bound_method(self, other):
return True
# overriding __eq__
a1 = A()
a1.___eq___ = MethodType(custom_eq_bound_method, a1, A)
# not overriding __eq__
a2 = A()
# comparing a1 == a2 in business logic....
I'm sure you've asked this before and had it answered, but it seems what you are looking for is to have the class to try to defer to your own alias for __eq__ since this method cannot be overridden at the instance level.
from types import MethodType
class A(object):
def __eq__(self, other):
try:
self._eq
return self._eq(other) # defer to _eq
except AttributeError:
return super().__eq__(other) # Here is the default behaviour
def custom_eq_bound_method(self, other):
return True
a1 = A()
a2 = A()
a3 = A()
print(a1==a2)
print(a1==a3)
print(a2==a3)
a1._eq = MethodType(custom_eq_bound_method, a1)
print('After override')
print(a1==a2)
print(a1==a3)
print(a2==a3)
Output:
False
False
False
After override
True
True
False

'Reversed' comparison operator in Python

class Inner():
def __init__(self, x):
self.x = x
def __eq__(self, other):
if isinstance(other, Inner):
return self.x == other.x
else:
raise TypeError("Incorrect type to compare")
class Outer():
def __init__(self, y):
self.y = Inner(y)
def __eq__(self, other):
if isinstance(other, Outer):
return self.y == other.y
elif isinstance(other, Inner):
return self.y == other
else:
raise TypeError("Incorrect type to compare")
if __name__ == "__main__":
a = Outer(1)
b = Inner(1)
print(a == b) # ok no problem
print(b == a) # This will raise a type error
In the example I have inner and outer class. I have no control over what Inner implements just wanted to simulate the situation. I have only control over Outer's behavior. I want Outer instances to be able to compare to Inner instances (not just equality). With the given implementation only the first comparison works because that is calling Outer's __eq__ method allowed to be compared to Outer and Inner instances but the second one is calling Inner's __eq__ which will not allow the comparison to Outer - heck it doesn't know Outer exists why should it bother to implement it.
Is there a way to get the second type of comparison to work, with something similar like the __radd__ and such functions.
I know for instance in C++ you resolve this with inline operator definitions, but we don't have such in Python.
Not to put too fine a point on it: Inner.__eq__ is broken. At the very least, rather than throwing an error it should return NotImplemented, which would allow Python to try the reverse comparison:
When NotImplemented is returned, the interpreter will then try the
reflected operation on the other type, or some other fallback,
depending on the operator. If all attempted operations return
NotImplemented, the interpreter will raise an appropriate exception.
Better yet it would use "duck typing", rather than insisting on a specific class (unless the class, rather than its interface, is an explicitly important part of the comparison):
def __eq__(self, other):
try:
return self.x == other.x
except AttributeError:
return NotImplemented
However, as you say you cannot control this, you will have to manually implement similar functionality, for example:
def compare(a, b):
"""'Safe' comparison between two objects."""
try:
return a == b
except TypeError:
return b == a
as there is no such thing as __req__ in Python's data model.

Operators for new-style classes

Could anyone explain to me why A()+A() does give an error, but B()+B() works as expected? I came across this when I was writing a larger piece of code, but this seems to be the smallest code necessary to reproduce it.
from types import MethodType
D = {'__add__': lambda x, y: "".join((repr(x), repr(y)))}
class A(object):
def __getattr__(self, item):
if item == '__coerce__':
raise AttributeError()
return MethodType(D[item], self)
def __repr__(self):
return "A"
class B():
def __getattr__(self, item):
if item == '__coerce__':
raise AttributeError()
return MethodType(D[item], self)
def __repr__(self):
return "B"
try:
A()+A()
except Exception as e:
print e
B()+B()
Does anyone have an explanation?
That's because new style classes never invoke __coerce__ with binary operators. Also for special methods __getattr__ is never invoked in new style classes: From Data model coercion rules:
New-style classes (those derived from object) never invoke the
__coerce__() method in response to a binary operator; the only time __coerce__() is invoked is when the built-in function coerce() is called.

Classes Python and developing Stack

I have created my own LIFO container class Stack that supports the methods of push, len, pop, and a check on isEmpty. All methods appear to be working in my example calls, except for when I call a created instance of this class(in my example s) I receive a memory location for the created object when I want to see the actual contents of that object.
class Stack:
x = []
def __init__(self, x=None):
if x == None:
self.x = []
else:
self.x = x
def isEmpty(self):
return len(self.x) == 0
def push(self,p):
self.x.append(p)
def pop(self):
return self.x.pop()
def __len__(self):
return(len(self.x))
s = Stack()
s.push('plate 1')
s.push('plate 2')
s.push('plate 3')
print(s)
print(s.isEmpty())
print(len(s))
print(s.pop())
print(s.pop())
print(s.pop())
print(s.isEmpty())
I get the result of running this line print(s) to be <__main__.Stack object at 0x00000000032CD748>t when I would expect and am looking for ['plate 1','plate 2','plate3']
You need to also override __str__ or __repr__ if you want your class to have a different representation when printing. Something like:
def __str__(self):
return str(self.x)
should do the trick. __str__ is what is called by the str function (and implicitly called by print). The default __str__ simply returns the result of __repr__ which defaults to that funny string with the type and the memory address.
You need to override the default implementation of __repr__. Otherwise it will use the default implementation which returns an informal string representation of the class, in this case the type and memory address.
def __repr__(self):
return str(self.x)
Yes override __str__ and/or __repr__
Remember __repr__ can can evaled and return the same object if possible

Categories