I know np.exp2(x) exists that calculates 2^x where x is a numpy array, however, I am looking for a method that does K^x where K is any arbitrary number.
Is there any elegant way of doing it rather than stretching out K to the shape of x and doing a piecewise exponent?
Just use the standard Python exponentiation operator **:
K**x
For example, if you have:
x = np.array([1,2,3])
K = 3
print(K**x)
The output is:
[ 3 9 27]
Notes
For Python classes, the behavior of the binary ** operator is implemented via the __pow__, __rpow__, and __ipow__ magic methods (the reality for np.ndarray is slightly more complicated since it's implemented in the C layer, but that's not actually important here). For Numpy arrays, these magic methods in turn appear to call numpy.power, so you can expect that ** will have the same behavior as documented for numpy.power. In particular,
Note that an integer type raised to a negative integer power will raise a ValueError.
With numpy you can just use numpy.power
arr = numpy.array([1,2,3])
print(numpy.power(3,arr)) # Outputs [ 3 9 27]
Related
In numpy if we want to raise a matrix A to power N (but raise it as defined in mathematics, in linear algebra in particular), then it seems we need to use this function
numpy.linalg.matrix_power
Isn't there a simpler way? Some Python symbol/operator?
E.g. I was expecting A**N to do this but it doesn't.
Seems that A**N is raising each element to power N, and not the whole matrix to power N (in the usual math sense). So A**N is some strange element-wise raising to power N.
By matrix I mean of course a two-dimensional ndarray.
In [4]: x=np.arange(4).reshape(2,2)
For this square array:
In [6]: np.linalg.matrix_power(x,3)
Out[6]:
array([[ 6, 11],
[22, 39]])
In [7]: x#x#x
Out[7]:
array([[ 6, 11],
[22, 39]])
matrix_power is written in python so you can easily read it. It essentially does a sequence of dot products, with some refinements to reduce the steps.
For np.matrix subclass, ** does the same thing:
In [8]: mx=np.matrix(x)
In [9]: mx**3
Out[9]:
matrix([[ 6, 11],
[22, 39]])
** is translated by the interpreter to a __pow__ call. For this class that just amounts to a matrix_power call:
In [10]: mx.__pow__??
Signature: mx.__pow__(other)
Docstring: Return pow(self, value, mod).
Source:
def __pow__(self, other):
return matrix_power(self, other)
File: c:\users\paul\anaconda3\lib\site-packages\numpy\matrixlib\defmatrix.py
Type: method
But for ndarray this method is a compiled one:
In [3]: x.__pow__??
Signature: x.__pow__(value, mod=None, /)
Call signature: x.__pow__(*args, **kwargs)
Type: method-wrapper
String form: <method-wrapper '__pow__' of numpy.ndarray object at 0x0000022A8B2B5ED0>
Docstring: Return pow(self, value, mod).
numpy does not alter python syntax. It has not added any operators. The # operator was added to python several years ago, largely as a convenience for packages like numpy. But it had to added to the interpreter's syntax first.
Note that matrix_power works for a
a : (..., M, M) array_like
Matrix to be "powered".
That means it has to have at least 2 dimensions, and the trailing two must be equal size. So even that extends the normal linear algebra definition (which is limited to 2d).
numpy isn't just a linear algebra package. It's meant to be a general purpose array tool. Linear algebra is just a subset of math that can be performed with multidimensional collections of numbers (and other objects).
numpy.linalg.matrix_power is the best way as far as I know. You could use dot or * in a loop, but that would just be more code, and probably less efficient.
It is apparent that NumPy has an upper bound for its integers. But my question is, is there a way to store the elements in NumPy arrays, like by keeping the values and the magnitudes separate? Wouldn't that technically allow storing of larger numbers than what the int64 limit allows?
for example you can store arbitrary precision integers in numpy array using dtype = object and perform addition, multiplication, element-wise multiplication, subtraction and integer division but not operations, which lead to float results, for example np.exp(x) wont work.
x = np.ones((10,10),dtype=object)
x *= 2**100
x *= x
print(x)
if you want truly arbitrary precision arithmetic matrix classes I would implement on my own with proper operator overload with help of mpmath
In python 3.5, the # operator was introduced for matrix multiplication, following PEP465. This is implemented e.g. in numpy as the matmul operator.
However, as proposed by the PEP, the numpy operator throws an exception when called with a scalar operand:
>>> import numpy as np
>>> np.array([[1,2],[3,4]]) # np.array([[1,2],[3,4]]) # works
array([[ 7, 10],
[15, 22]])
>>> 1 # 2 # doesn't work
Traceback (most recent call last):
File "<input>", line 1, in <module>
TypeError: unsupported operand type(s) for #: 'int' and 'int'
This is a real turnoff for me, since I'm implementing numerical signal processing algorithms that should work for both scalars and matrices. The equations for both cases are mathematically exactly equivalent, which is no surprise, since "1-D x 1-D matrix multiplication" is equivalent to scalar multiplication. The current state however forces me to write duplicate code in order to handle both cases correctly.
So, given that the current state is not satisfactory, is there any reasonable way I can make the # operator work for scalars? I thought about adding a custom __matmul__(self, other) method to scalar data types, but this seems like a lot of hassle considering the number of involved internal data types. Could I change the implementation of the __matmul__ method for numpy array data types to not throw an exception for 1x1 array operands?
And, on a sidenote, which is the rationale behind this design decision? Off the top of my head, I cannot think of any compelling reasons not to implement that operator for scalars as well.
As ajcr suggested, you can work around this issue by forcing some minimal dimensionality on objects being multiplied. There are two reasonable options: atleast_1d and atleast_2d which have different results in regard to the type being returned by #: a scalar versus a 1-by-1 2D array.
x = 3
y = 5
z = np.atleast_1d(x) # np.atleast_1d(y) # returns 15
z = np.atleast_2d(x) # np.atleast_2d(y) # returns array([[15]])
However:
Using atleast_2d will lead to an error if x and y are 1D-arrays that would otherwise be multiplied normally
Using atleast_1d will result in the product that is either a scalar or a matrix, and you don't know which.
Both of these are more verbose than np.dot(x, y) which would handle all of those cases.
Also, the atleast_1d version suffers from the same flaw that would also be shared by having scalar # scalar = scalar: you don't know what can be done with the output. Will z.T or z.shape throw an error? These work for 1-by-1 matrices but not for scalars. In the setting of Python, one simply cannot ignore the distinction between scalars and 1-by-1 arrays without also giving up all the methods and properties that the latter have.
Basically I have an array that may vary between any two numbers, and I want to preserve the distribution while constraining it to the [0,1] space. The function to do this is very very simple. I usually write it as:
def to01(array):
array -= array.min()
array /= array.max()
return array
Of course it can and should be more complex to account for tons of situations, such as all the values being the same (divide by zero) and float vs. integer division (use np.subtract and np.divide instead of operators). But this is the most basic.
The problem is that I do this very frequently across stuff in my project, and it seems like a fairly standard mathematical operation. Is there a built in function that does this in NumPy?
Don't know if there's a builtin for that (probably not, it's not really a difficult thing to do as is). You can use vectorize to apply a function to all the elements of the array:
def to01(array):
a = array.min()
# ignore the Runtime Warning
with numpy.errstate(divide='ignore'):
b = 1. /(array.max() - array.min())
if not(numpy.isfinite(b)):
b = 0
return numpy.vectorize(lambda x: b * (x - a))(array)
Is this documented anywhere? Why such a drastic difference?
# Python 3.2
# numpy 1.6.2 using Intel's Math Kernel Library
>>> import numpy as np
>>> x = np.float64(-0.2)
>>> x ** 0.8
__main__:1: RuntimeWarning: invalid value encountered in double_scalars
nan
>>> x = -0.2 # note: `np.float` is same built-in `float`
>>> x ** 0.8
(-0.2232449487530631+0.16219694943147778j)
This is especially confusing since according to this, np.float64 and built-in float are identical except for __repr__.
I can see how the warning from np may be useful in some cases (especially since it can be disabled or enabled in np.seterr); but the problem is that the return value is nan rather than the complex value provided by the built-in. Therefore, this breaks code when you start using numpy for some of the calculations, and don't convert its return values to built-in float explicitly.
numpy.float may or may not be float, but complex numbers are not float at all:
In [1]: type((-0.2)**0.8)
Out[1]: builtins.complex
So there's no float result of the operation, hence nan.
If you don't want to do an explicit conversion to float (which is recommended), do the numpy calculation in complex numbers:
In [3]: np.complex(-0.2)**0.8
Out[3]: (-0.2232449487530631+0.16219694943147778j)
The behaviour of returning a complex number from a float operation is certainly not usual, and was only introduced with Python 3 (like the float division of integers with the / operator). In Python 2.7 you get the following:
In [1]: (-0.2)**0.8
ValueError: negative number cannot be raised to a fractional power
On a scalar, if instead of np.float64 you use np.float, you'll get the same float type as Python uses. (And you'll either get the above error in 2.7 or the complex number in 3.x.)
For arrays, all the numpy operators return the same type of array, and most ufuncs do not support casting from float > complex (e.g., check np.<ufunc>.type).
If what you want is a consistent operation on scalars, use np.float
If you are interested in array operations, you'll have to cast the array as complex: x = x.astype('complex')