If I have a MxN numpy array denoted arr, I wish to index over all elements and adjust the values like so
for m in range(arr.shape[0]):
for n in range(arr.shape[1]):
arr[m, n] += x**2 * np.cos(m) * np.sin(n)
Where x is a random float.
Is there a way to broadcast this over the entire array without needing to loop? Thus, speeding up the run time.
You are just adding zeros, because sin(2*pi*k) = 0 for integer k.
However, if you want to vectorize this, the function np.meshgrid could help you.
Check the following example, where I removed the 2 pi in the trigonometric functions to add something unequal zero.
x = 2
arr = np.arange(12, dtype=float).reshape(4, 3)
n, m = np.meshgrid(np.arange(arr.shape[1]), np.arange(arr.shape[0]), sparse=True)
arr += x**2 * np.cos(m) * np.sin(n)
arr
Edit: use the sparse argument to reduce memory consumption.
You can use nested generators of two-dimensional arrays:
import numpy as np
from random import random
x = random()
n, m = 10,20
arr = [[x**2 * np.cos(2*np.pi*j) * np.sin(2*np.pi*i) for j in range(m)] for i in range(n)]
In [156]: arr = np.ones((2, 3))
Replace the range with arange:
In [157]: m, n = np.arange(arr.shape[0]), np.arange(arr.shape[1])
And change the first array to (2,1) shape. A (2,1) array broadcasts with a (3,) to produce a (2,3) result.
In [158]: A = 0.23**2 * np.cos(m[:, None]) * np.sin(n)
In [159]: A
Out[159]:
array([[0. , 0.04451382, 0.04810183],
[0. , 0.02405092, 0.02598953]])
In [160]: arr + A
Out[160]:
array([[1. , 1.04451382, 1.04810183],
[1. , 1.02405092, 1.02598953]])
The meshgrid suggested in the accepted answer does the same thing:
In [161]: np.meshgrid(m, n, sparse=True, indexing="ij")
Out[161]:
[array([[0],
[1]]),
array([[0, 1, 2]])]
This broadcasting may be clearer with:
In [162]: m, n
Out[162]: (array([0, 1]), array([0, 1, 2]))
In [163]: m[:, None] * 10 + n
Out[163]:
array([[ 0, 1, 2],
[10, 11, 12]])
Related
I pretend to remove slices from the third dimension of a 4d numpy array if it's contains only zeros.
I have a 4d numpy array of dimensions [256,256,336,6] and I need to delete the slices in the third dimension that only contains zeros. So the result would have a shape like this , e.g. [256,256,300,6] if 36 slices are fully zeros. I have tried multiple approaches including for loops, np.delete and all(), any() functions without success.
You need to reduce on all axes but the one you are interested in.
An example using np.any() where there are all-zero subarrays along the axis 1 (at position 0 and 2):
import numpy as np
a=np.ones((2, 3, 2, 3))
a[:, 0, :, :] = a[:, 2, :, :] =0
mask = np.any(a, axis=(0, 2, 3))
new_a = a[:, mask, :, :]
print(new_a.shape)
# (2, 1, 2, 3)
print(new_a)
# [[[[1. 1. 1.]
# [1. 1. 1.]]]
#
#
# [[[1. 1. 1.]
# [1. 1. 1.]]]]
The same code parametrized and refactored as a function:
def remove_all_zeros(arr: np.ndarray, axis: int) -> np.ndarray:
red_axes = tuple(i for i in range(arr.ndim) if i != axis)
mask = np.any(arr, axis=red_axes)
slicing = tuple(slice(None) if i != axis else mask for i in range(arr.ndim))
return arr[slicing]
a = np.ones((2, 3, 2, 3))
a[:, 0, :, :] = a[:, 2, :, :] = 0
new_a = remove_all_zeros(a, 1)
print(new_a.shape)
# (2, 1, 2, 3)
print(new_a)
# [[[[1. 1. 1.]
# [1. 1. 1.]]]
#
#
# [[[1. 1. 1.]
# [1. 1. 1.]]]]
I'm not an afficionado with numpy, but does this do what you want?
I take the following small example matrix with 4 dimensions all full of 1s and then I set some slices to zero:
import numpy as np
a=np.ones((4,4,5,2))
The shape of a is:
>>> a.shape
(4, 4, 5, 2)
I will artificially set some of the slices in dimension 3 to zero:
a[:,:,0,:]=0
a[:,:,3,:]=0
I can find the indices of the slices with not all zeros by calculating sums (not very efficient, perhaps!)
indices = [i for i in range(a.shape[2]) if a[:,:,i,:].sum() != 0]
>>> indices
[1, 2, 4]
So, in your general case you could do this:
indices = [i for i in range(a.shape[2]) if a[:,:,i,:].sum() != 0]
a_new = a[:, :, indices, :].copy()
Then the shape of a_new is:
>>> anew.shape
(4, 4, 3, 2)
I am trying to create permutations of size 4 from a group of real numbers. After that, I'd like to know the position of the first element in a permutation after I sort it. Here is what I have tried so far. What's the best way to do this?
import numpy as np
from itertools import chain, permutations
N_PLAYERS = 4
N_STATES = 60
np.random.seed(0)
state_space = np.linspace(0.0, 1.0, num=N_STATES, retstep=True)[0].tolist()
perms = permutations(state_space, N_PLAYERS)
perms_arr = np.fromiter(chain(*perms),dtype=np.float16)
def loc(row):
return np.where(np.argsort(row) == 0)[0].tolist()[0]
locs = np.apply_along_axis(loc, 0, perms)
In [153]: N_PLAYERS = 4
...: N_STATES = 60
...: np.random.seed(0)
...: state_space = np.linspace(0.0, 1.0, num=N_STATES, retstep=True)[0].tolist()
...: perms = itertools.permutations(state_space, N_PLAYERS)
In [154]: alist = list(perms)
In [155]: len(alist)
Out[155]: 11703240
Simply making a list from the permuations produces a list of lists, with all sublists of length N_PLAYERS.
Making an array from that with chain flattens it:
In [156]: perms = itertools.permutations(state_space, N_PLAYERS)
In [158]: perms_arr = np.fromiter(itertools.chain(*perms),dtype=np.float16)
In [159]: perms_arr.shape
Out[159]: (46812960,)
In [160]: alist[0]
Which could be reshaped to (11703240,4).
Using apply on that 1d array doesn't work (or make sense):
In [170]: perms_arr.shape
Out[170]: (46812960,)
In [171]: locs = np.apply_along_axis(loc, 0, perms_arr)
In [172]: locs.shape
Out[172]: ()
Reshape to 4 columns:
In [173]: locs = np.apply_along_axis(loc, 0, perms_arr.reshape(-1,4))
In [174]: locs.shape
Out[174]: (4,)
In [175]: locs
Out[175]: array([ 0, 195054, 578037, 769366])
This applies loc to each column, returning one value for each. But loc has a row variable. Is that supposed to be significant?
I could switch the axis; this takes much longer, and al
In [176]: locs = np.apply_along_axis(loc, 1, perms_arr.reshape(-1,4))
In [177]: locs.shape
Out[177]: (11703240,)
list comprehension
This iteration does the same thing as your apply_along_axis, and I expect is faster (though I haven't timed it - it's too slow).
In [188]: locs1 = np.array([loc(row) for row in perms_arr.reshape(-1,4)])
In [189]: np.allclose(locs, locs1)
Out[189]: True
whole array sort
But argsort takes an axis, so I can sort all rows at once (instead of iterating):
In [185]: np.nonzero(np.argsort(perms_arr.reshape(-1,4), axis=1)==0)
Out[185]:
(array([ 0, 1, 2, ..., 11703237, 11703238, 11703239]),
array([0, 0, 0, ..., 3, 3, 3]))
In [186]: np.allclose(_[1],locs)
Out[186]: True
Or going the other direction: - cf with Out[175]
In [187]: np.nonzero(np.argsort(perms_arr.reshape(-1,4), axis=0)==0)
Out[187]: (array([ 0, 195054, 578037, 769366]), array([0, 1, 2, 3]))
I want to calculate the following:
but I have no idea how to do this in python, I do not want to implement this manually but use a predefined function for this, something from numpy for example.
But numpy seems to ignore that x.T should be transposed.
Code:
import numpy as np
x = np.array([1, 5])
print(np.dot(x, x.T)) # = 26, This is not the matrix it should be!
While your vectors are defined as 1-d arrays, you can use np.outer:
np.outer(x, x.T)
> array([[ 1, 5],
> [ 5, 25]])
Alternatively, you could also define your vectors as matrices and use normal matrix multiplication:
x = np.array([[1], [5]])
x # x.T
> array([[ 1, 5],
> [ 5, 25]])
You can do:
x = np.array([[1], [5]])
print(np.dot(x, x.T))
Your original x is of shape (2,), while you need a shape of (2,1). Another way is reshaping your x:
x = np.array([1, 5]).reshape(-1,1)
print(np.dot(x, x.T))
.reshape(-1,1) reshapes your array to have 1 column and implicitely takes care of number of rows.
output:
[[ 1 5]
[ 5 25]]
np.matmul(x[:, np.newaxis], [x])
Is there a quick simple way to multiply multiple columns from a numpy matrix? I'm using the code I show bellow but I was wondering if numpy offers a direct method.
x = np.array([[1,2,3,4],[5,6,7,8],[9,10,11,12]])
temp = np.ones(3)
for ind in [0,3]:
temp *= x[:,ind]
print(temp)
array([ 4., 40., 108.])
Using numpy indexing and numpy.prod. idx can be any number of columns from your array:
>>> idx = [0, 3]
>>> np.prod(x[:, idx], axis=1)
array([ 4, 40, 108])
Also equivalent:
x[:, idx].prod(1)
You can multiply the columns since numpy multiplication is element-wise:
x[:, 0] * x[:, 3]
returns
array([ 4, 40, 108])
I would like to append my numpy array in a loop. In the begining my numpy array is empty.
x = np.array([])
I would like to append x with 3 element long array in order to get Mx3 matrix, but my array is appending in one dimension... What's wrong?
In [166]: x = np.array([])
In [167]: a
Out[167]: array([248, 249, 250])
In [168]: x = np.append(x,a, axis=0)
In [169]: x
Out[169]: array([ 248., 249., 250.])
In [170]: x = np.append(x,a, axis=0)
In [171]: x
Out[171]: array([ 248., 249., 250., 248., 249., 250.])
Use vstack:
In [51]: x = np.array([])
In [52]: a= np.array([248, 249, 250])
In [53]: x = np.append(x,a, axis=0)
In [54]: np.vstack((x,a))
Out[54]:
array([[ 248., 249., 250.],
[ 248., 249., 250.]])
Not sure what way you are using this but I doubt you need to use np.append(x,a, axis=0) at all. Just set x=a then vstack.
What's wrong is that your initial x is one-dimensional. See:
z = np.array([])
z.shape
# (0,)
np.ndim(z)
# 1
So if you np.append to x you will always end up with a one-dimensional array, i.e. a vector. Note that in Numpy one-dimensional arrays are row-vectors.
To use np.append you could start with a 2D array like so. Also, the array you append must have the same number of dimensions as the array you append to.
z = np.array([]).reshape((0,3))
a = np.array(248, 249, 250)
a2d = a.reshape(1, 3)
# a2d = np.atleast_2d(a)
# a2d = a[None, :]
# a2d = a[np.newaxis, :]
z = np.append(z, a2d, axis=0)