I would like to define my own operator. Does python support such a thing?
While technically you cannot define new operators in Python, this clever hack works around this limitation. It allows you to define infix operators like this:
# simple multiplication
x=Infix(lambda x,y: x*y)
print 2 |x| 4
# => 8
# class checking
isa=Infix(lambda x,y: x.__class__==y.__class__)
print [1,2,3] |isa| []
print [1,2,3] <<isa>> []
# => True
No, Python comes with a predefined, yet overridable, set of operators.
No, you can't create new operators. However, if you are just evaluating expressions, you could process the string yourself and calculate the results of the new operators.
Sage provides this functionality, essentially using the "clever hack" described by #Ayman Hourieh, but incorporated into a module as a decorator to give a cleaner appearance and additional functionality – you can choose the operator to overload and therefore the order of evaluation.
from sage.misc.decorators import infix_operator
#infix_operator('multiply')
def dot(a,b):
return a.dot_product(b)
u=vector([1,2,3])
v=vector([5,4,3])
print(u *dot* v)
# => 22
#infix_operator('or')
def plus(x,y):
return x*y
print(2 |plus| 4)
# => 6
See the Sage documentation and this enhancement tracking ticket for more information.
Python 3.5 introduces the symbol # for an extra operator.
PEP465 introduced this new operator for matrix multiplication, to simplify the notation of many numerical code. The operator will not be implemented for all types, but just for arrays-like-objects.
You can support the operator for your classes/objects by implementing __matmul__().
The PEP leaves space for a different usage of the operator for non-arrays-like objects.
Of course you can implement with # any sort of operation different from matrix multiplication also for arrays-like objects, but the user experience will be affected, because everybody will expect your data type to behave in a different way.
If you intend to apply the operation on a particular class of objects, you could just override the operator that matches your function the closest... for instance, overriding __eq__() will override the == operator to return whatever you want. This works for almost all the operators.
Related
Is it possible to overload arbitrary operators in Python? Or is one restricted to the list of operators which have associated magic methods as listed here: https://www.python-course.eu/python3_magic_methods.php ?
I'm asking because I noticed that Numpy uses the # operator to perform matrix multiplication e.g. C=A#B where A,B are Numpy arrays, and I was wondering how they did it.
Edit: The # operator is not in the list I linked to.
Could someone point me to the Numpy source where this is done?
In Python, you cannot create new operators, no. By defining those "magic" functions, you can affect what happens when objects of your own definition are operated upon using the standard operators.
However, the list you linked to is not complete. In Python 3.5, they added special methods for the # operator. Here's the rather terse listing in the Python operator module docs and here are the docs on operator overloading.
operator.matmul(a, b)
operator.__matmul__(a, b)
Return a # b.
New in version 3.5.
I hadn't seen that operator personally, so I did a little more research. It's intended specifically for matrix multiplication. But, I was able to use it for other purposes, though I would argue against doing so as a matter of style:
In [1]: class RichGuyEmailAddress(str):
...: def __matmul__(self, domain_name):
...: return f'{self}#{domain_name}'
...:
In [2]: my_email = RichGuyEmailAddress('billg') # 'microsoft.com'
In [3]: print(my_email)
billg#microsoft.com
So, no, you can't overload any random character, but you can overload the # operator.
I would like to define my own operator. Does python support such a thing?
While technically you cannot define new operators in Python, this clever hack works around this limitation. It allows you to define infix operators like this:
# simple multiplication
x=Infix(lambda x,y: x*y)
print 2 |x| 4
# => 8
# class checking
isa=Infix(lambda x,y: x.__class__==y.__class__)
print [1,2,3] |isa| []
print [1,2,3] <<isa>> []
# => True
No, Python comes with a predefined, yet overridable, set of operators.
No, you can't create new operators. However, if you are just evaluating expressions, you could process the string yourself and calculate the results of the new operators.
Sage provides this functionality, essentially using the "clever hack" described by #Ayman Hourieh, but incorporated into a module as a decorator to give a cleaner appearance and additional functionality – you can choose the operator to overload and therefore the order of evaluation.
from sage.misc.decorators import infix_operator
#infix_operator('multiply')
def dot(a,b):
return a.dot_product(b)
u=vector([1,2,3])
v=vector([5,4,3])
print(u *dot* v)
# => 22
#infix_operator('or')
def plus(x,y):
return x*y
print(2 |plus| 4)
# => 6
See the Sage documentation and this enhancement tracking ticket for more information.
Python 3.5 introduces the symbol # for an extra operator.
PEP465 introduced this new operator for matrix multiplication, to simplify the notation of many numerical code. The operator will not be implemented for all types, but just for arrays-like-objects.
You can support the operator for your classes/objects by implementing __matmul__().
The PEP leaves space for a different usage of the operator for non-arrays-like objects.
Of course you can implement with # any sort of operation different from matrix multiplication also for arrays-like objects, but the user experience will be affected, because everybody will expect your data type to behave in a different way.
If you intend to apply the operation on a particular class of objects, you could just override the operator that matches your function the closest... for instance, overriding __eq__() will override the == operator to return whatever you want. This works for almost all the operators.
I would like to define my own operator. Does python support such a thing?
While technically you cannot define new operators in Python, this clever hack works around this limitation. It allows you to define infix operators like this:
# simple multiplication
x=Infix(lambda x,y: x*y)
print 2 |x| 4
# => 8
# class checking
isa=Infix(lambda x,y: x.__class__==y.__class__)
print [1,2,3] |isa| []
print [1,2,3] <<isa>> []
# => True
No, Python comes with a predefined, yet overridable, set of operators.
No, you can't create new operators. However, if you are just evaluating expressions, you could process the string yourself and calculate the results of the new operators.
Sage provides this functionality, essentially using the "clever hack" described by #Ayman Hourieh, but incorporated into a module as a decorator to give a cleaner appearance and additional functionality – you can choose the operator to overload and therefore the order of evaluation.
from sage.misc.decorators import infix_operator
#infix_operator('multiply')
def dot(a,b):
return a.dot_product(b)
u=vector([1,2,3])
v=vector([5,4,3])
print(u *dot* v)
# => 22
#infix_operator('or')
def plus(x,y):
return x*y
print(2 |plus| 4)
# => 6
See the Sage documentation and this enhancement tracking ticket for more information.
Python 3.5 introduces the symbol # for an extra operator.
PEP465 introduced this new operator for matrix multiplication, to simplify the notation of many numerical code. The operator will not be implemented for all types, but just for arrays-like-objects.
You can support the operator for your classes/objects by implementing __matmul__().
The PEP leaves space for a different usage of the operator for non-arrays-like objects.
Of course you can implement with # any sort of operation different from matrix multiplication also for arrays-like objects, but the user experience will be affected, because everybody will expect your data type to behave in a different way.
If you intend to apply the operation on a particular class of objects, you could just override the operator that matches your function the closest... for instance, overriding __eq__() will override the == operator to return whatever you want. This works for almost all the operators.
From what I remember from my C++ class, the professor said that operator overloading is cool, but since it takes relatively a lot of thought and code to cover all end-cases (e.g. when overloading + you probably also want to overload ++ and +=, and also make sure to handle end cases like adding an object to itself etc.), you should only consider it in those cases where this feature will have a major impact on your code, like overloading the operators for the matrix class in a math application.
Does the same apply to python? Would you recommend overriding operator behavior in python? And what rules of thumb can you give me?
Operator overloading is mostly useful when you're making a new class that falls into an existing "Abstract Base Class" (ABC) -- indeed, many of the ABCs in standard library module collections rely on the presence of certain special methods (and special methods, one with names starting and ending with double underscores AKA "dunders", are exactly the way you perform operator overloading in Python). This provides good starting guidance.
For example, a Container class must override special method __contains__, i.e., the membership check operator item in container (as in, if item in container: -- don't confuse with the for statement, for item in container:, which relies on __iter__!-).
Similarly, a Hashable must override __hash__, a Sized must override __len__, a Sequence or a Mapping must override __getitem__, and so forth. (Moreover, the ABCs can provide your class with mixin functionality -- e.g., both Sequence and Mapping can provide __contains__ on the basis of your supplied __getitem__ override, and thereby automatically make your class a Container).
Beyond the collections, you'll want to override special methods (i.e. provide for operator overloading) mostly if your new class "is a number". Other special cases exist, but resist the temptation of overloading operators "just for coolness", with no semantic connection to the "normal" meanings, as C++'s streams do for << and >> and Python strings (in Python 2.*, fortunately not in 3.* any more;-) do for % -- when such operators do not any more mean "bit-shifting" or "division remainder", you're just engendering confusion. A language's standard library can get away with it (though it shouldn't;-), but unless your library gets as widespread as the language's standard one, the confusion will hurt!-)
I've written software with significant amounts of overloading, and lately I regret that policy. I would say this:
Only overload operators if it's the natural, expected thing to do and doesn't have any side effects.
So if you make a new RomanNumeral class, it makes sense to overload addition and subtraction etc. But don't overload it unless it's natural: it makes no sense to define addition and subtraction for a Car or a Vehicle object.
Another rule of thumb: don't overload ==. It makes it very hard (though not impossible) to actually test if two objects are the same. I made this mistake and paid for it for a long time.
As for when to overload +=, ++ etc, I'd actually say: only overload additional operators if you have a lot of demand for that functionality. It's easier to have one way to do something than five. Sure, it means sometimes you'll have to write x = x + 1 instead of x += 1, but more code is ok if it's clearer.
In general, like with many 'fancy' features, it's easy to think that you want something when you don't really, implement a bunch of stuff, not notice the side effects, and then figure it out later. Err on the conservative side.
EDIT: I wanted to add an explanatory note about overloading ==, because it seems various commenters misunderstand this, and it's caught me out. Yes, is exists, but it's a different operation. Say I have an object x, which is either from my custom class, or is an integer. I want to see if x is the number 500. But if you set x = 500, then later test x is 500, you will get False, due to the way Python caches numbers. With 50, it would return True. But you can't use is, because you might want x == 500 to return True if x is an instance of your class. Confusing? Definitely. But this is the kind of detail you need to understand to successfully overload operators.
Here is an example that uses the bitwise or operation to simulate a unix pipeline. This is intended as a counter example to most of the rules of thumb.
I just found Lumberjack which uses this syntax in real code
class pipely(object):
def __init__(self, *args, **kw):
self._args = args
self.__dict__.update(kw)
def __ror__(self, other):
return ( self.map(x) for x in other if self.filter(x) )
def map(self, x):
return x
def filter(self, x):
return True
class sieve(pipely):
def filter(self, x):
n = self._args[0]
return x==n or x%n
class strify(pipely):
def map(self, x):
return str(x)
class startswith(pipely):
def filter(self, x):
n=str(self._args[0])
if x.startswith(n):
return x
print"*"*80
for i in xrange(2,100) | sieve(2) | sieve(3) | sieve(5) | sieve(7) | strify() | startswith(5):
print i
print"*"*80
for i in xrange(2,100) | sieve(2) | sieve(3) | sieve(5) | sieve(7) | pipely(map=str) | startswith(5):
print i
print"*"*80
for i in xrange(2,100) | sieve(2) | sieve(3) | sieve(5) | sieve(7) | pipely(map=str) | pipely(filter=lambda x: x.startswith('5')):
print i
Python's overloading is "safer" in general than C++'s -- for example, the assignment operator can't be overloaded, and += has a sensible default implementation.
In some ways, though, overloading in Python is still as "broken" as in C++. Programmers should restrain the desire to "re-use" an operator for unrelated purposes, such as C++ re-using the bitshifts to perform string formatting and parsing. Don't overload an operator with different semantics from your implementation just to get prettier syntax.
Modern Python style strongly discourages "rogue" overloading, but many aspects of the language and standard library retain poorly-named operators for backwards compatibility. For example:
%: modulus and string formatting
+: addition and sequence concatenation
*: multiplication and sequence repetition
So, rule of thumb? If your operator implementation will surprise people, don't do it.
Let's say you have a small calculator program that takes numbers and an operator to perform on those numbers as input, then prints out the result of applying the specified operation. So if you input "4 + 5" it will print out "9". Simple, right? Well what I want to be able to write is something this:
a, op, b = raw_input().split()
print somehowInvokeOperator(op, a, b)
The problem is that "somehowInvokeOperator()" part. Is there anyway to do this without resorting to either (a) eval() or (b) some type of dictionary mapping keys like "+" and "-" to functions that perform the appropriate operation? getattr() doesn't appear to work for this. I don't really need this code for anything, I'm just curious to see if this can be solved in Python as elegantly as it can in other dynamic languages.
Basically no, you will at least need to have a dictionary or function to map operator characters to their implementations. It's actually a little more complicated than that, since not all operators take the form a [op] b, so in general you'd need to do a bit of parsing; see https://docs.python.org/library/operator.html for the full list of correspondences, and for the functions you'll probably want to use for the operator implementations.
If you're only trying to implement the binary arithmetic operators like + - * / % ** then a dictionary should be good enough.
If you really wanted to do this, you would need the standard operator module. See also Emulating numeric types. And, yes, a dictionary full of functions would be a perfectly sound dynamic way to make this happen.
import operator
operations = {'+' : operator.add}
result = operations[op](a, b)
Warning: this is not pythonic at all!! (goes against every rule of the Zen of Python!)
Here's a magical, one-liner dictionary:
ops = eval( '{%s}'%','.join([('\''+op + '\' : lambda a,b: a ' + op + ' b') for op in '+-*/%']) )
That defines your dictionary .. which you can use
ops['+'](10,4) #returns 14
the basic idea is mapping each operator to a lambda function:
{ '+' : lambda a,b: a + b }