I would like to define my own operator. Does python support such a thing?
While technically you cannot define new operators in Python, this clever hack works around this limitation. It allows you to define infix operators like this:
# simple multiplication
x=Infix(lambda x,y: x*y)
print 2 |x| 4
# => 8
# class checking
isa=Infix(lambda x,y: x.__class__==y.__class__)
print [1,2,3] |isa| []
print [1,2,3] <<isa>> []
# => True
No, Python comes with a predefined, yet overridable, set of operators.
No, you can't create new operators. However, if you are just evaluating expressions, you could process the string yourself and calculate the results of the new operators.
Sage provides this functionality, essentially using the "clever hack" described by #Ayman Hourieh, but incorporated into a module as a decorator to give a cleaner appearance and additional functionality – you can choose the operator to overload and therefore the order of evaluation.
from sage.misc.decorators import infix_operator
#infix_operator('multiply')
def dot(a,b):
return a.dot_product(b)
u=vector([1,2,3])
v=vector([5,4,3])
print(u *dot* v)
# => 22
#infix_operator('or')
def plus(x,y):
return x*y
print(2 |plus| 4)
# => 6
See the Sage documentation and this enhancement tracking ticket for more information.
Python 3.5 introduces the symbol # for an extra operator.
PEP465 introduced this new operator for matrix multiplication, to simplify the notation of many numerical code. The operator will not be implemented for all types, but just for arrays-like-objects.
You can support the operator for your classes/objects by implementing __matmul__().
The PEP leaves space for a different usage of the operator for non-arrays-like objects.
Of course you can implement with # any sort of operation different from matrix multiplication also for arrays-like objects, but the user experience will be affected, because everybody will expect your data type to behave in a different way.
If you intend to apply the operation on a particular class of objects, you could just override the operator that matches your function the closest... for instance, overriding __eq__() will override the == operator to return whatever you want. This works for almost all the operators.
Related
Is it possible to overload arbitrary operators in Python? Or is one restricted to the list of operators which have associated magic methods as listed here: https://www.python-course.eu/python3_magic_methods.php ?
I'm asking because I noticed that Numpy uses the # operator to perform matrix multiplication e.g. C=A#B where A,B are Numpy arrays, and I was wondering how they did it.
Edit: The # operator is not in the list I linked to.
Could someone point me to the Numpy source where this is done?
In Python, you cannot create new operators, no. By defining those "magic" functions, you can affect what happens when objects of your own definition are operated upon using the standard operators.
However, the list you linked to is not complete. In Python 3.5, they added special methods for the # operator. Here's the rather terse listing in the Python operator module docs and here are the docs on operator overloading.
operator.matmul(a, b)
operator.__matmul__(a, b)
Return a # b.
New in version 3.5.
I hadn't seen that operator personally, so I did a little more research. It's intended specifically for matrix multiplication. But, I was able to use it for other purposes, though I would argue against doing so as a matter of style:
In [1]: class RichGuyEmailAddress(str):
...: def __matmul__(self, domain_name):
...: return f'{self}#{domain_name}'
...:
In [2]: my_email = RichGuyEmailAddress('billg') # 'microsoft.com'
In [3]: print(my_email)
billg#microsoft.com
So, no, you can't overload any random character, but you can overload the # operator.
I would like to define my own operator. Does python support such a thing?
While technically you cannot define new operators in Python, this clever hack works around this limitation. It allows you to define infix operators like this:
# simple multiplication
x=Infix(lambda x,y: x*y)
print 2 |x| 4
# => 8
# class checking
isa=Infix(lambda x,y: x.__class__==y.__class__)
print [1,2,3] |isa| []
print [1,2,3] <<isa>> []
# => True
No, Python comes with a predefined, yet overridable, set of operators.
No, you can't create new operators. However, if you are just evaluating expressions, you could process the string yourself and calculate the results of the new operators.
Sage provides this functionality, essentially using the "clever hack" described by #Ayman Hourieh, but incorporated into a module as a decorator to give a cleaner appearance and additional functionality – you can choose the operator to overload and therefore the order of evaluation.
from sage.misc.decorators import infix_operator
#infix_operator('multiply')
def dot(a,b):
return a.dot_product(b)
u=vector([1,2,3])
v=vector([5,4,3])
print(u *dot* v)
# => 22
#infix_operator('or')
def plus(x,y):
return x*y
print(2 |plus| 4)
# => 6
See the Sage documentation and this enhancement tracking ticket for more information.
Python 3.5 introduces the symbol # for an extra operator.
PEP465 introduced this new operator for matrix multiplication, to simplify the notation of many numerical code. The operator will not be implemented for all types, but just for arrays-like-objects.
You can support the operator for your classes/objects by implementing __matmul__().
The PEP leaves space for a different usage of the operator for non-arrays-like objects.
Of course you can implement with # any sort of operation different from matrix multiplication also for arrays-like objects, but the user experience will be affected, because everybody will expect your data type to behave in a different way.
If you intend to apply the operation on a particular class of objects, you could just override the operator that matches your function the closest... for instance, overriding __eq__() will override the == operator to return whatever you want. This works for almost all the operators.
I would like to define my own operator. Does python support such a thing?
While technically you cannot define new operators in Python, this clever hack works around this limitation. It allows you to define infix operators like this:
# simple multiplication
x=Infix(lambda x,y: x*y)
print 2 |x| 4
# => 8
# class checking
isa=Infix(lambda x,y: x.__class__==y.__class__)
print [1,2,3] |isa| []
print [1,2,3] <<isa>> []
# => True
No, Python comes with a predefined, yet overridable, set of operators.
No, you can't create new operators. However, if you are just evaluating expressions, you could process the string yourself and calculate the results of the new operators.
Sage provides this functionality, essentially using the "clever hack" described by #Ayman Hourieh, but incorporated into a module as a decorator to give a cleaner appearance and additional functionality – you can choose the operator to overload and therefore the order of evaluation.
from sage.misc.decorators import infix_operator
#infix_operator('multiply')
def dot(a,b):
return a.dot_product(b)
u=vector([1,2,3])
v=vector([5,4,3])
print(u *dot* v)
# => 22
#infix_operator('or')
def plus(x,y):
return x*y
print(2 |plus| 4)
# => 6
See the Sage documentation and this enhancement tracking ticket for more information.
Python 3.5 introduces the symbol # for an extra operator.
PEP465 introduced this new operator for matrix multiplication, to simplify the notation of many numerical code. The operator will not be implemented for all types, but just for arrays-like-objects.
You can support the operator for your classes/objects by implementing __matmul__().
The PEP leaves space for a different usage of the operator for non-arrays-like objects.
Of course you can implement with # any sort of operation different from matrix multiplication also for arrays-like objects, but the user experience will be affected, because everybody will expect your data type to behave in a different way.
If you intend to apply the operation on a particular class of objects, you could just override the operator that matches your function the closest... for instance, overriding __eq__() will override the == operator to return whatever you want. This works for almost all the operators.
If I have a value, and a list of additional terms I want multiplied to the value:
n = 10
terms = [1,2,3,4]
Is it possible to use a list comprehension to do something like this:
n *= (term for term in terms) #not working...
Or is the only way:
n *= reduce(lambda x,y: x*y, terms)
This is on Python 2.6.2. Thanks!
reduce is the best way to do this IMO, but you don't have to use a lambda; instead, you can use the * operator directly:
import operator
n *= reduce(operator.mul, terms)
n is now 240. See the docs for the operator module for more info.
Reduce is not the only way. You can also write it as a simple loop:
for term in terms:
n *= term
I think this is much more clear than using reduce, especially when you consider that many Python programmers have never seen reduce and the name does little to convey to people who see it for the first time what it actually does.
Pythonic does not mean write everything as comprehensions or always use a functional style if possible. Python is a multi-paradigm language and writing simple imperative code when appropriate is Pythonic.
Guido van Rossum also doesn't want reduce in Python:
So now reduce(). This is actually the one I've always hated most, because, apart from a few examples involving + or *, almost every time I see a reduce() call with a non-trivial function argument, I need to grab pen and paper to diagram what's actually being fed into that function before I understand what the reduce() is supposed to do. So in my mind, the applicability of reduce() is pretty much limited to associative operators, and in all other cases it's better to write out the accumulation loop explicitly.
There aren't a whole lot of associative operators. (Those are operators X for which (a X b) X c equals a X (b X c).) I think it's just about limited to +, *, &, |, ^, and shortcut and/or. We already have sum(); I'd happily trade reduce() for product(), so that takes care of the two most common uses. [...]
In Python 3 reduce has been moved to the functools module.
Yet another way:
import operator
n = reduce(operator.mul, terms, n)
From what I remember from my C++ class, the professor said that operator overloading is cool, but since it takes relatively a lot of thought and code to cover all end-cases (e.g. when overloading + you probably also want to overload ++ and +=, and also make sure to handle end cases like adding an object to itself etc.), you should only consider it in those cases where this feature will have a major impact on your code, like overloading the operators for the matrix class in a math application.
Does the same apply to python? Would you recommend overriding operator behavior in python? And what rules of thumb can you give me?
Operator overloading is mostly useful when you're making a new class that falls into an existing "Abstract Base Class" (ABC) -- indeed, many of the ABCs in standard library module collections rely on the presence of certain special methods (and special methods, one with names starting and ending with double underscores AKA "dunders", are exactly the way you perform operator overloading in Python). This provides good starting guidance.
For example, a Container class must override special method __contains__, i.e., the membership check operator item in container (as in, if item in container: -- don't confuse with the for statement, for item in container:, which relies on __iter__!-).
Similarly, a Hashable must override __hash__, a Sized must override __len__, a Sequence or a Mapping must override __getitem__, and so forth. (Moreover, the ABCs can provide your class with mixin functionality -- e.g., both Sequence and Mapping can provide __contains__ on the basis of your supplied __getitem__ override, and thereby automatically make your class a Container).
Beyond the collections, you'll want to override special methods (i.e. provide for operator overloading) mostly if your new class "is a number". Other special cases exist, but resist the temptation of overloading operators "just for coolness", with no semantic connection to the "normal" meanings, as C++'s streams do for << and >> and Python strings (in Python 2.*, fortunately not in 3.* any more;-) do for % -- when such operators do not any more mean "bit-shifting" or "division remainder", you're just engendering confusion. A language's standard library can get away with it (though it shouldn't;-), but unless your library gets as widespread as the language's standard one, the confusion will hurt!-)
I've written software with significant amounts of overloading, and lately I regret that policy. I would say this:
Only overload operators if it's the natural, expected thing to do and doesn't have any side effects.
So if you make a new RomanNumeral class, it makes sense to overload addition and subtraction etc. But don't overload it unless it's natural: it makes no sense to define addition and subtraction for a Car or a Vehicle object.
Another rule of thumb: don't overload ==. It makes it very hard (though not impossible) to actually test if two objects are the same. I made this mistake and paid for it for a long time.
As for when to overload +=, ++ etc, I'd actually say: only overload additional operators if you have a lot of demand for that functionality. It's easier to have one way to do something than five. Sure, it means sometimes you'll have to write x = x + 1 instead of x += 1, but more code is ok if it's clearer.
In general, like with many 'fancy' features, it's easy to think that you want something when you don't really, implement a bunch of stuff, not notice the side effects, and then figure it out later. Err on the conservative side.
EDIT: I wanted to add an explanatory note about overloading ==, because it seems various commenters misunderstand this, and it's caught me out. Yes, is exists, but it's a different operation. Say I have an object x, which is either from my custom class, or is an integer. I want to see if x is the number 500. But if you set x = 500, then later test x is 500, you will get False, due to the way Python caches numbers. With 50, it would return True. But you can't use is, because you might want x == 500 to return True if x is an instance of your class. Confusing? Definitely. But this is the kind of detail you need to understand to successfully overload operators.
Here is an example that uses the bitwise or operation to simulate a unix pipeline. This is intended as a counter example to most of the rules of thumb.
I just found Lumberjack which uses this syntax in real code
class pipely(object):
def __init__(self, *args, **kw):
self._args = args
self.__dict__.update(kw)
def __ror__(self, other):
return ( self.map(x) for x in other if self.filter(x) )
def map(self, x):
return x
def filter(self, x):
return True
class sieve(pipely):
def filter(self, x):
n = self._args[0]
return x==n or x%n
class strify(pipely):
def map(self, x):
return str(x)
class startswith(pipely):
def filter(self, x):
n=str(self._args[0])
if x.startswith(n):
return x
print"*"*80
for i in xrange(2,100) | sieve(2) | sieve(3) | sieve(5) | sieve(7) | strify() | startswith(5):
print i
print"*"*80
for i in xrange(2,100) | sieve(2) | sieve(3) | sieve(5) | sieve(7) | pipely(map=str) | startswith(5):
print i
print"*"*80
for i in xrange(2,100) | sieve(2) | sieve(3) | sieve(5) | sieve(7) | pipely(map=str) | pipely(filter=lambda x: x.startswith('5')):
print i
Python's overloading is "safer" in general than C++'s -- for example, the assignment operator can't be overloaded, and += has a sensible default implementation.
In some ways, though, overloading in Python is still as "broken" as in C++. Programmers should restrain the desire to "re-use" an operator for unrelated purposes, such as C++ re-using the bitshifts to perform string formatting and parsing. Don't overload an operator with different semantics from your implementation just to get prettier syntax.
Modern Python style strongly discourages "rogue" overloading, but many aspects of the language and standard library retain poorly-named operators for backwards compatibility. For example:
%: modulus and string formatting
+: addition and sequence concatenation
*: multiplication and sequence repetition
So, rule of thumb? If your operator implementation will surprise people, don't do it.