Is it possible to overload arbitrary operators in Python? Or is one restricted to the list of operators which have associated magic methods as listed here: https://www.python-course.eu/python3_magic_methods.php ?
I'm asking because I noticed that Numpy uses the # operator to perform matrix multiplication e.g. C=A#B where A,B are Numpy arrays, and I was wondering how they did it.
Edit: The # operator is not in the list I linked to.
Could someone point me to the Numpy source where this is done?
In Python, you cannot create new operators, no. By defining those "magic" functions, you can affect what happens when objects of your own definition are operated upon using the standard operators.
However, the list you linked to is not complete. In Python 3.5, they added special methods for the # operator. Here's the rather terse listing in the Python operator module docs and here are the docs on operator overloading.
operator.matmul(a, b)
operator.__matmul__(a, b)
Return a # b.
New in version 3.5.
I hadn't seen that operator personally, so I did a little more research. It's intended specifically for matrix multiplication. But, I was able to use it for other purposes, though I would argue against doing so as a matter of style:
In [1]: class RichGuyEmailAddress(str):
...: def __matmul__(self, domain_name):
...: return f'{self}#{domain_name}'
...:
In [2]: my_email = RichGuyEmailAddress('billg') # 'microsoft.com'
In [3]: print(my_email)
billg#microsoft.com
So, no, you can't overload any random character, but you can overload the # operator.
Related
Absolute newbie to programming and to python. I am still going through my first set of tutorials and merely just learning about in-place assignment operators, so newer than new. I am using python 3.8.
I know that the operator *= calls the imul method to change/update the assigned value of the named variable in-place during an iterative operation. What character combination do I use to call the mul method so the script does not change the original assigned value of the variable but I can use the temporary resulting value as an object for an intermediate result that will not be reused later? Or is the mul operation never actually used and that is why I can not find a keystroke combination for it?
I don't have an example I need help with. This is purely theoretical as I try to educate myself with syntax. Just wondering how to invoke the non-in-place versions of these operators as I have only seen reference to the standard in-place operators like +=, -=, *=, #=, /=, //=, %=, etc.
Sorry if this is too basic a topic for this forum. If these sorts of simple newbie questions are inappropriate here, can you suggest a more appropriate forum for me to go to until I become accomplished? I am sure I will have more questions along these lines and do not want to intrude on the community.
It's just *, see:
class SomeClass:
origin = 10
def __mul__(self, other):
return self.origin * other
cls = SomeClass()
print(cls * 10)
# 100
This question already has answers here:
How do I use numpy.where()? What should I pass, and what does the result mean? [closed]
(2 answers)
Closed 4 years ago.
I have already found the source for the numpy.ma.where() function but it seems to be calling the numpy.where() function and to better understand it I would like to take a look if possible.
Most Python functions are written in the Python language, but some functions are written in something more native (often the C language).
Regular Python functions ("pure Python")
There are a few techniques you can use to ask Python itself where a function is defined. Probably the most portable uses the inspect module:
>>> import numpy
>>> import inspect
>>> inspect.isbuiltin(numpy.ma.where)
False
>>> inspect.getsourcefile(numpy.ma.where)
'.../numpy/core/multiarray.py'
But this won't work with a native ("built-in") function:
>>> import numpy
>>> import inspect
>>> inspect.isbuiltin(numpy.where)
True
>>> inspect.getsourcefile(numpy.where)
TypeError: <built-in function where> is not a module, class, method, function, traceback, frame, or code object
Native ("built-in") functions
Unfortunately, Python doesn't provide a record of source files for built-in functions. You can find out which module provides the function:
>>> import numpy as np
>>> np.where
<built-in function where>
>>> np.where.__module__
'numpy.core.multiarray'
Python won't help you find the native (C) source code for that module, but in this case it's reasonable to look in the numpy project for C source that has similar names. I found the following file:
numpy/core/src/multiarray/multiarraymodule.c
And in that file, I found a list of definitions (PyMethodDef) including:
{"where",
(PyCFunction)array_where,
METH_VARARGS, NULL},
This suggests that the C function array_where is the one that Python sees as "where".
The array_where function is defined in the same file, and it mostly delegates to the PyArray_Where function.
In short
NumPy's np.where function is written in C, not Python. A good place to look is PyArray_Where.
First there are 2 distinct versions of where, one that takes just the condition, the other that takes 3 arrays.
The simpler one is most commonly used, and is just another name for np.nonzero. This scans through the condition array twice. Once with np.count_nonzero to determine how many nonzero entries there are, which allows it to allocate the return arrays. The second step is to fill in the coordinates of all nonzero entries. The key is that it returns a tuple of arrays, one array for each dimension of condition.
The condition, x, y version takes three arrays, which it broadcasts against each other. The return array has the common broadcasted shape, with elements chosen from x and y as explained in the answers to your previous question, How exactly does numpy.where() select the elements in this example?
You do realize that most of this code is c or cython, with a significant about of preprocessing. It is hard to read, even for experienced users. It is easier to run a variety of test cases and get a feel for what is happening that way.
A couple things to watch out for. np.where is a python function, and python evaluates each input fully before passing them to it. This is conditional assignment, not conditional evaluation function.
And unless you pass 3 arrays that match in shape, or scalar x and y, you'll need a good understanding of broadcasting.
You can find the code in numpy.core.multiarray
C:\Users\<name>\AppData\Local\Programs\Python\Python37-32\Lib\site-packages\numpy\core\multiarray.py is where I found it.
I would like to define my own operator. Does python support such a thing?
While technically you cannot define new operators in Python, this clever hack works around this limitation. It allows you to define infix operators like this:
# simple multiplication
x=Infix(lambda x,y: x*y)
print 2 |x| 4
# => 8
# class checking
isa=Infix(lambda x,y: x.__class__==y.__class__)
print [1,2,3] |isa| []
print [1,2,3] <<isa>> []
# => True
No, Python comes with a predefined, yet overridable, set of operators.
No, you can't create new operators. However, if you are just evaluating expressions, you could process the string yourself and calculate the results of the new operators.
Sage provides this functionality, essentially using the "clever hack" described by #Ayman Hourieh, but incorporated into a module as a decorator to give a cleaner appearance and additional functionality – you can choose the operator to overload and therefore the order of evaluation.
from sage.misc.decorators import infix_operator
#infix_operator('multiply')
def dot(a,b):
return a.dot_product(b)
u=vector([1,2,3])
v=vector([5,4,3])
print(u *dot* v)
# => 22
#infix_operator('or')
def plus(x,y):
return x*y
print(2 |plus| 4)
# => 6
See the Sage documentation and this enhancement tracking ticket for more information.
Python 3.5 introduces the symbol # for an extra operator.
PEP465 introduced this new operator for matrix multiplication, to simplify the notation of many numerical code. The operator will not be implemented for all types, but just for arrays-like-objects.
You can support the operator for your classes/objects by implementing __matmul__().
The PEP leaves space for a different usage of the operator for non-arrays-like objects.
Of course you can implement with # any sort of operation different from matrix multiplication also for arrays-like objects, but the user experience will be affected, because everybody will expect your data type to behave in a different way.
If you intend to apply the operation on a particular class of objects, you could just override the operator that matches your function the closest... for instance, overriding __eq__() will override the == operator to return whatever you want. This works for almost all the operators.
I would like to define my own operator. Does python support such a thing?
While technically you cannot define new operators in Python, this clever hack works around this limitation. It allows you to define infix operators like this:
# simple multiplication
x=Infix(lambda x,y: x*y)
print 2 |x| 4
# => 8
# class checking
isa=Infix(lambda x,y: x.__class__==y.__class__)
print [1,2,3] |isa| []
print [1,2,3] <<isa>> []
# => True
No, Python comes with a predefined, yet overridable, set of operators.
No, you can't create new operators. However, if you are just evaluating expressions, you could process the string yourself and calculate the results of the new operators.
Sage provides this functionality, essentially using the "clever hack" described by #Ayman Hourieh, but incorporated into a module as a decorator to give a cleaner appearance and additional functionality – you can choose the operator to overload and therefore the order of evaluation.
from sage.misc.decorators import infix_operator
#infix_operator('multiply')
def dot(a,b):
return a.dot_product(b)
u=vector([1,2,3])
v=vector([5,4,3])
print(u *dot* v)
# => 22
#infix_operator('or')
def plus(x,y):
return x*y
print(2 |plus| 4)
# => 6
See the Sage documentation and this enhancement tracking ticket for more information.
Python 3.5 introduces the symbol # for an extra operator.
PEP465 introduced this new operator for matrix multiplication, to simplify the notation of many numerical code. The operator will not be implemented for all types, but just for arrays-like-objects.
You can support the operator for your classes/objects by implementing __matmul__().
The PEP leaves space for a different usage of the operator for non-arrays-like objects.
Of course you can implement with # any sort of operation different from matrix multiplication also for arrays-like objects, but the user experience will be affected, because everybody will expect your data type to behave in a different way.
If you intend to apply the operation on a particular class of objects, you could just override the operator that matches your function the closest... for instance, overriding __eq__() will override the == operator to return whatever you want. This works for almost all the operators.
I would like to define my own operator. Does python support such a thing?
While technically you cannot define new operators in Python, this clever hack works around this limitation. It allows you to define infix operators like this:
# simple multiplication
x=Infix(lambda x,y: x*y)
print 2 |x| 4
# => 8
# class checking
isa=Infix(lambda x,y: x.__class__==y.__class__)
print [1,2,3] |isa| []
print [1,2,3] <<isa>> []
# => True
No, Python comes with a predefined, yet overridable, set of operators.
No, you can't create new operators. However, if you are just evaluating expressions, you could process the string yourself and calculate the results of the new operators.
Sage provides this functionality, essentially using the "clever hack" described by #Ayman Hourieh, but incorporated into a module as a decorator to give a cleaner appearance and additional functionality – you can choose the operator to overload and therefore the order of evaluation.
from sage.misc.decorators import infix_operator
#infix_operator('multiply')
def dot(a,b):
return a.dot_product(b)
u=vector([1,2,3])
v=vector([5,4,3])
print(u *dot* v)
# => 22
#infix_operator('or')
def plus(x,y):
return x*y
print(2 |plus| 4)
# => 6
See the Sage documentation and this enhancement tracking ticket for more information.
Python 3.5 introduces the symbol # for an extra operator.
PEP465 introduced this new operator for matrix multiplication, to simplify the notation of many numerical code. The operator will not be implemented for all types, but just for arrays-like-objects.
You can support the operator for your classes/objects by implementing __matmul__().
The PEP leaves space for a different usage of the operator for non-arrays-like objects.
Of course you can implement with # any sort of operation different from matrix multiplication also for arrays-like objects, but the user experience will be affected, because everybody will expect your data type to behave in a different way.
If you intend to apply the operation on a particular class of objects, you could just override the operator that matches your function the closest... for instance, overriding __eq__() will override the == operator to return whatever you want. This works for almost all the operators.