I am building up my understanding of Python, and recently I understood that functions must be classes(?), and that a def func(): just instantiates an object of class function. I was mindblown when I created an attribute of func honestly.
Lurking in dir(func) I noticed that indeed all the special methods such as .__eq__ are inherited, and I wanted to play around with it:
def func(n):
print(n)
def __eq__(self,func2):
print('hello')
However, it does not work:
>>> func.__eq__(print)
NotImplemented
What would it be the proper way to overload the equality operator for a function? I don't see how to overload it without having a proper class definition.
Because python treats all functions as objects, it might be worth thinking of creating functions as creating instances of the class function (which isn't documented) with the __call__ method being the body of the function created with def. The actual C source of the function class is on Github if you want to know implementation details.
With returning NotImplemented:
In Python, if objects do not override __eq__ or __hash__, there is default implementations where __hash__ = builtins.id and __eq__ is like lambda self, other: self is other. When the comparison operators return NotImplemented, this instructs the runtime to search for another method that does the same thing, like trying __ne__ instead of __eq__, or trying operators from the parent.
>>> def test(a):
... return a
...
>>> def test2(a):
... return a
...
>>> test == test2
False
>>> test.__eq__(test2)
NotImplemented
You can also test this by creating a dummy class that doesn't override __eq__ (like how the function class doesn't):
>>> class testcls:
... pass
...
>>> t1 = testcls()
>>> t2 = testcls()
>>> t1.__eq__(t2)
NotImplemented
>>> t1.__eq__(t1)
True
>>> t1 == t2
False
No, functions don't have to be classes (yet they are in Python), actually. But that is another story.
Seems you misunderstand that __eq__ is a method, i.e. you need a class that it belongs to in order to overload it. What you code does is defines some custom function within another function, which is valid Python code but has nothing to do with __eq__ overloading.
Since you can't inherit function, you can't overwrite it's eq. And even if you could do that, it won't work due to some purely theoretical reasons.
Related
If someone writes a class in python, and fails to specify their own __repr__() method, then a default one is provided for them. However, suppose we want to write a function which has the same, or similar, behavior to the default __repr__(). However, we want this function to have the behavior of the default __repr__() method even if the actual __repr__() for the class was overloaded. That is, suppose we want to write a function which has the same behavior as a default __repr__() regardless of whether someone overloaded the __repr__() method or not. How might we do it?
class DemoClass:
def __init__(self):
self.var = 4
def __repr__(self):
return str(self.var)
def true_repr(x):
# [magic happens here]
s = "I'm not implemented yet"
return s
obj = DemoClass()
print(obj.__repr__())
print(true_repr(obj))
Desired Output:
print(obj.__repr__()) prints 4, but print(true_repr(obj)) prints something like:
<__main__.DemoClass object at 0x0000000009F26588>
You can use object.__repr__(obj). This works because the default repr behavior is defined in object.__repr__.
Note, the best answer is probably just to use object.__repr__ directly, as the others have pointed out. But one could implement that same functionality roughly as:
>>> def true_repr(x):
... type_ = type(x)
... module = type_.__module__
... qualname = type_.__qualname__
... return f"<{module}.{qualname} object at {hex(id(x))}>"
...
So....
>>> A()
hahahahaha
>>> true_repr(A())
'<__main__.A object at 0x106549208>'
>>>
Typically we can use object.__repr__ for that, but this will to the "object repr for every item, so:
>>> object.__repr__(4)
'<int object at 0xa6dd20>'
Since an int is an object, but with the __repr__ overriden.
If you want to go up one level of overwriting, we can use super(..):
>>> super(type(4), 4).__repr__() # going up one level
'<int object at 0xa6dd20>'
For an int that thus again means that we will print <int object at ...>, but if we would for instance subclass the int, then it would use the __repr__ of int again, like:
class special_int(int):
def __repr__(self):
return 'Special int'
Then it will look like:
>>> s = special_int(4)
>>> super(type(s), s).__repr__()
'4'
What we here do is creating a proxy object with super(..). Super will walk the method resolution order (MRO) of the object and will try to find the first function (from a superclass of s) that has overriden the function. If we use single inheritance, that is the closest parent that overrides the function, but if it there is some multiple inheritance involved, then this is more tricky. We thus select the __repr__ of that parent, and call that function.
This is also a rather weird application of super since usually the class (here type(s)) is a fixed one, and does not depend on the type of s itself, since otherwise multiple such super(..) calls would result in an infinite loop.
But usually it is a bad idea to break overriding anyway. The reason a programmer overrides a function is to change the behavior. Not respecting this can of course sometimes result into some useful functions, but frequently it will result in the fact that the code contracts are no longer satisfied. For example if a programmer overrides __eq__, he/she will also override __hash__, if you use the hash of another class, and the real __eq__, then things will start breaking.
Calling magic function directly is also frequently seen as an antipattern, so you better avoid that as well.
I have a function that generally accepts lists, but on occasions needs to accept functions as well. There were several ways of dealing with this, but it would have been very very useful to be able to do len(foo) for a given function foo.
In the end, instead of passing in functions, I passed in callable classes that had a __len__ function defined. But it got me thinking, since in python everything is an object, and functions can have attributes etc. just as a curiosity...
Question
Is there any way to give a function a len? A quick google didn't bring up anything.
My attempt
def foo():
return True
def my_len(self):
return 5
foo.__len__ = my_len
len(foo)
Adding __len__ to an object is not working (see this link added by Aran-Fey why). A function is just an object defining a __call__ method. You can define a class like this:
class Foo:
def __call__(self):
return True
def __len__(self):
return 5
Using it:
>>> foo=Foo()
>>> foo()
True
>>> len(foo)
5
It is possible to create a function which is having a length, but you should consider the use case. Python gives you a lot of power, but not everything what's possible is actually a good idea.
Base on my understanding, magic methods such as __str__ , __next__, __setattr__ are built-in features in Python. They will automatically called when a instance object is created. It also plays a role of overridden. What else some important features of magic method do I omit or ignore?
"magic" methods in python do specific things in specific contexts.
For example, to "override" the addition operator (+), you'd define a __add__ method. subtraction is __sub__, etc.
Other methods are called during object creation (__new__, __init__). Other methods are used with specific language constructs (__enter__, __exit__ and you might argue __init__ and __next__).
Really, there's nothing special about magic methods other than they are guaranteed to be called by the language at specific times. As the programmer, you're given the power to hook into structure and change the way an object behaves in those circumstances.
For a near complete summary, have a look at the python data model.
There is a lot you can do with magic methods and since it can be hard finding the right way to get started, I'd like to give you some inspiration on what I'm using a lot.
While you're probably already using some of them (like __init__), I would start learning on the operator specific magic methods, which helped me a lot optimising classes and how I use them. The magic method __mul__ for example allows you to describe what should happen to your class in case it's getting called by the multiplication operator. In the following example you can see, that the interpreter first looks for a multiplicand's __mul__ method and if this doesn't exist (like in the second example) it tries to call the multiplier's __rmul__ method.
Example 1:
class a:
def __mul__(self, other):
print("__mul__ a")
def __rmul__(self, other):
print("__rmul__ a")
class b:
def __mul__(self, other):
print("__mul__ b")
def __rmul__(self, other):
print("__rmul__ b")
ia = a()
ib = b()
ia * ib
# prints __mul__ a
Example 2:
class a:
pass
class b:
def __mul__(self, other):
print("__mul__ b")
def __rmul__(self, other):
print("__rmul__ b")
ia = a()
ib = b()
ia * ib
# prints __rmul__ b
Any other operator works corresponding to this example. I hope that helps you getting started enhancing your magic method skills.
hint:
To make your classes comparable you can use __cmp__.
The method is called with the two classes which are compared.
Return a positive value if the first class is bigger.
Return a nagative value if the second class is bigger.
Return zero if they have the same size.
You don't have to use the magic methods for every possibility.
I'm writing a decorator, and for various annoying reasons[0] it would be expedient to check if the function it is wrapping is being defined stand-alone or as part of a class (and further which classes that new class is subclassing).
For example:
def my_decorator(f):
defined_in_class = ??
print "%r: %s" %(f, defined_in_class)
#my_decorator
def foo(): pass
class Bar(object):
#my_decorator
def bar(self): pass
Should print:
<function foo …>: False
<function bar …>: True
Also, please note:
At the point decorators are applied the function will still be a function, not an unbound method, so testing for instance/unbound method (using typeof or inspect) will not work.
Please only offer suggestions that solve this problem — I'm aware that there are many similar ways to accomplish this end (ex, using a class decorator), but I would like them to happen at decoration time, not later.
[0]: specifically, I'm writing a decorator that will make it easy to do parameterized testing with nose. However, nose will not run test generators on subclasses of unittest.TestCase, so I would like my decorator to be able to determine if it's being used inside a subclass of TestCase and fail with an appropriate error. The obvious solution - using isinstance(self, TestCase) before calling the wrapped function doesn't work, because the wrapped function needs to be a generator, which doesn't get executed at all.
Take a look at the output of inspect.stack() when you wrap a method. When your decorator's execution is underway, the current stack frame is the function call to your decorator; the next stack frame down is the # wrapping action that is being applied to the new method; and the third frame will be the class definition itself, which merits a separate stack frame because the class definition is its own namespace (that is wrapped up to create a class when it is done executing).
I suggest, therefore:
defined_in_class = (len(frames) > 2 and
frames[2][4][0].strip().startswith('class '))
If all of those crazy indexes look unmaintainable, then you can be more explicit by taking the frame apart piece by piece, like this:
import inspect
frames = inspect.stack()
defined_in_class = False
if len(frames) > 2:
maybe_class_frame = frames[2]
statement_list = maybe_class_frame[4]
first_statment = statement_list[0]
if first_statment.strip().startswith('class '):
defined_in_class = True
Note that I do not see any way to ask Python about the class name or inheritance hierarchy at the moment your wrapper runs; that point is "too early" in the processing steps, since the class creation is not yet finished. Either parse the line that begins with class yourself and then look in that frame's globals to find the superclass, or else poke around the frames[1] code object to see what you can learn — it appears that the class name winds up being frames[1][0].f_code.co_name in the above code, but I cannot find any way to learn what superclasses will be attached when the class creation finishes up.
A little late to the party here, but this has proven to be a reliable means of determining if a decorator is being used on a function defined in a class:
frames = inspect.stack()
className = None
for frame in frames[1:]:
if frame[3] == "<module>":
# At module level, go no further
break
elif '__module__' in frame[0].f_code.co_names:
className = frame[0].f_code.co_name
break
The advantage of this method over the accepted answer is that it works with e.g. py2exe.
Some hacky solution that I've got:
import inspect
def my_decorator(f):
args = inspect.getargspec(f).args
defined_in_class = bool(args and args[0] == 'self')
print "%r: %s" %(f, defined_in_class)
But it relays on the presence of self argument in function.
you can use the package wrapt to check for
- instance/class methods
- classes
- freestanding functions/static methods:
See the project page of wrapt: https://pypi.org/project/wrapt/
You could check if the decorator itself is being called at the module level or nested within something else.
defined_in_class = inspect.currentframe().f_back.f_code.co_name != "<module>"
I think the functions in the inspect module will do what you want, particularly isfunction and ismethod:
>>> import inspect
>>> def foo(): pass
...
>>> inspect.isfunction(foo)
True
>>> inspect.ismethod(foo)
False
>>> class C(object):
... def foo(self):
... pass
...
>>> inspect.isfunction(C.foo)
False
>>> inspect.ismethod(C.foo)
True
>>> inspect.isfunction(C().foo)
False
>>> inspect.ismethod(C().foo)
True
You can then follow the Types and Members table to access the function inside the bound or unbound method:
>>> C.foo.im_func
<function foo at 0x1062dfaa0>
>>> inspect.isfunction(C.foo.im_func)
True
>>> inspect.ismethod(C.foo.im_func)
False
The Python docs clearly state that x==y calls x.__eq__(y). However it seems that under many circumstances, the opposite is true. Where is it documented when or why this happens, and how can I work out for sure whether my object's __cmp__ or __eq__ methods are going to get called.
Edit: Just to clarify, I know that __eq__ is called in preferecne to __cmp__, but I'm not clear why y.__eq__(x) is called in preference to x.__eq__(y), when the latter is what the docs state will happen.
>>> class TestCmp(object):
... def __cmp__(self, other):
... print "__cmp__ got called"
... return 0
...
>>> class TestEq(object):
... def __eq__(self, other):
... print "__eq__ got called"
... return True
...
>>> tc = TestCmp()
>>> te = TestEq()
>>>
>>> 1 == tc
__cmp__ got called
True
>>> tc == 1
__cmp__ got called
True
>>>
>>> 1 == te
__eq__ got called
True
>>> te == 1
__eq__ got called
True
>>>
>>> class TestStrCmp(str):
... def __new__(cls, value):
... return str.__new__(cls, value)
...
... def __cmp__(self, other):
... print "__cmp__ got called"
... return 0
...
>>> class TestStrEq(str):
... def __new__(cls, value):
... return str.__new__(cls, value)
...
... def __eq__(self, other):
... print "__eq__ got called"
... return True
...
>>> tsc = TestStrCmp("a")
>>> tse = TestStrEq("a")
>>>
>>> "b" == tsc
False
>>> tsc == "b"
False
>>>
>>> "b" == tse
__eq__ got called
True
>>> tse == "b"
__eq__ got called
True
Edit: From Mark Dickinson's answer and comment it would appear that:
Rich comparison overrides __cmp__
__eq__ is it's own __rop__ to it's __op__ (and similar for __lt__, __ge__, etc)
If the left object is a builtin or new-style class, and the right is a subclass of it, the right object's __rop__ is tried before the left object's __op__
This explains the behaviour in theTestStrCmp examples. TestStrCmp is a subclass of str but doesn't implement its own __eq__ so the __eq__ of str takes precedence in both cases (ie tsc == "b" calls b.__eq__(tsc) as an __rop__ because of rule 1).
In the TestStrEq examples, tse.__eq__ is called in both instances because TestStrEq is a subclass of str and so it is called in preference.
In the TestEq examples, TestEq implements __eq__ and int doesn't so __eq__ gets called both times (rule 1).
But I still don't understand the very first example with TestCmp. tc is not a subclass on int so AFAICT 1.__cmp__(tc) should be called, but isn't.
You're missing a key exception to the usual behaviour: when the right-hand operand is an instance of a subclass of the class of the left-hand operand, the special method for the right-hand operand is called first.
See the documentation at:
http://docs.python.org/reference/datamodel.html#coercion-rules
and in particular, the following two paragraphs:
For objects x and y, first
x.__op__(y) is tried. If this is not
implemented or returns
NotImplemented, y.__rop__(x) is
tried. If this is also not implemented
or returns NotImplemented, a
TypeError exception is raised. But see
the following exception:
Exception to the previous item: if the
left operand is an instance of a
built-in type or a new-style class,
and the right operand is an instance
of a proper subclass of that type or
class and overrides the base’s
__rop__() method, the right
operand’s __rop__() method is tried
before the left operand’s __op__()
method.
Actually, in the docs, it states:
[__cmp__ is c]alled by comparison operations if rich comparison (see above) is not defined.
__eq__ is a rich comparison method and, in the case of TestCmp, is not defined, hence the calling of __cmp__
As I know, __eq__() is a so-called “rich comparison” method, and is called for comparison operators in preference to __cmp__() below. __cmp__() is called if "rich comparison" is not defined.
So in A == B:
If __eq__() is defined in A it will be called
Else __cmp__() will be called
__eq__() defined in 'str' so your __cmp__() function was not called.
The same rule is for __ne__(), __gt__(), __ge__(), __lt__() and __le__() "rich comparison" methods.
Is this not documented in the Language Reference? Just from a quick look there, it looks like __cmp__ is ignored when __eq__, __lt__, etc are defined. I'm understanding that to include the case where __eq__ is defined on a parent class. str.__eq__ is already defined so __cmp__ on its subclasses will be ignored. object.__eq__ etc are not defined so __cmp__ on its subclasses will be honored.
In response to the clarified question:
I know that __eq__ is called in
preferecne to __cmp__, but I'm not
clear why y.__eq__(x) is called in
preference to x.__eq__(y), when the
latter is what the docs state will
happen.
Docs say x.__eq__(y) will be called first, but it has the option to return NotImplemented in which case y.__eq__(x) is called. I'm not sure why you're confident something different is going on here.
Which case are you specifically puzzled about? I'm understanding you just to be puzzled about the "b" == tsc and tsc == "b" cases, correct? In either case, str.__eq__(onething, otherthing) is being called. Since you don't override the __eq__ method in TestStrCmp, eventually you're just relying on the base string method and it's saying the objects aren't equal.
Without knowing the implementation details of str.__eq__, I don't know whether ("b").__eq__(tsc) will return NotImplemented and give tsc a chance to handle the equality test. But even if it did, the way you have TestStrCmp defined, you're still going to get a false result.
So it's not clear what you're seeing here that's unexpected.
Perhaps what's happening is that Python is preferring __eq__ to __cmp__ if it's defined on either of the objects being compared, whereas you were expecting __cmp__ on the leftmost object to have priority over __eq__ on the righthand object. Is that it?