I have an ndarray of shape, say
my_array = np.zeros((Na, Nb, Nc))
and I have Nb bunches of data of shape (Na, Nc)
I would like to fill "my_array" with my bunches of data as follows:
for b, bit in zip(range(Nb), my_bits):
my_array[:, b, :] = bit
but I would like to do this for any arbitray axis for an arbitrary shape data set. How would I do this?
** EDIT **
Someone pointed out an ambiguity in my question. Below is an example code of what I know how to do, leading to what I don't know.
What I know:
my_bits = [np.ones((Na, Nc)) for j in range(Nb)]
my_array = np.zeros((Na, Nb, Nc))
for b, bit in zip(range(Nb), my_bits):
my_array[:, b, :] = bit
Generically we could consider
my_array = np.zeros((N1, N2, ..., Nj, ..., Nmax))
bit = np.ones((N1, N2, ..., Nj-1, Nj+1, ..., Nmax))
my_bits = [bit]*Nj
From there I would like to fill my_array with my_bits along axis Nj. How would I do this? I am looking at the nditer function, but I am not sure what to do with it.
There is a problem with dimensions here.
If I understood it correctly, my_array[:, 0, :] is same dimension as my_bits. So the assignment should be done like this:
import numpy as np
Na = 2
Nb = 2
Nc = 2
my_array = np.zeros((Na, Nb, Nc))
my_bits = np.ones((Na, Nc))
print('my_array before', my_array)
for b in range(Nb):
my_array[:, b, :] = my_bits
print('my_array after', my_array)
I think you might want numpy.transpose
list_of_2d_arrays = [ make_2d_array() for i in range(Nb) ] # each has shape=(Na,Nc)
tensor3d = np.array(list_of_2d_arrays) # shape = (Nb,Na,Nc)
# now shift the dimensions around
X = np.transpose(tensor3d,(1,0,2)) # has shape (Na,Nb,Nc)
Using transpose to move the iteration axis to a known position, front or last, is done in some numpy functions.
Another approach is to construct a slice object, and iterate that.
In [46]: arr = np.zeros((2,3,4),int)
In [47]: idx = np.full((arr.ndim,), slice(None))
In [48]: n=2
In [49]: for i in range(arr.shape[n]):
...: idx[n] = i
...: arr[tuple(idx)] = i+1
...:
In [50]: arr
Out[50]:
array([[[1, 2, 3, 4],
[1, 2, 3, 4],
[1, 2, 3, 4]],
[[1, 2, 3, 4],
[1, 2, 3, 4],
[1, 2, 3, 4]]])
for another axis
In [56]: arr = np.zeros((2,3,4),int)
In [57]: idx = np.full((arr.ndim,), slice(None))
In [58]: n=1
In [59]: for i in range(arr.shape[n]):
...: idx[n] = i
...: arr[tuple(idx)] = i+1
...:
In [60]: arr
Out[60]:
array([[[1, 1, 1, 1],
[2, 2, 2, 2],
[3, 3, 3, 3]],
[[1, 1, 1, 1],
[2, 2, 2, 2],
[3, 3, 3, 3]]])
In this case idx varies as:
[slice(None, None, None) 0 slice(None, None, None)]
[slice(None, None, None) 1 slice(None, None, None)]
[slice(None, None, None) 2 slice(None, None, None)]
The key is that an indexing expression has a tuple equivalent, which can be constructed programatically:
In [61]: np.s_[:,3,:]
Out[61]: (slice(None, None, None), 3, slice(None, None, None))
Related
What is the most elegant way to access an n dimensional array with an (n-1) dimensional array along a given dimension as in the dummy example
a = np.random.random_sample((3,4,4))
b = np.random.random_sample((3,4,4))
idx = np.argmax(a, axis=0)
How can I access now with idx a to get the maxima in a as if I had used a.max(axis=0)? or how to retrieve the values specified by idx in b?
I thought about using np.meshgrid but I think it is an overkill. Note that the dimension axis can be any usefull axis (0,1,2) and is not known in advance. Is there an elegant way to do this?
Make use of advanced-indexing -
m,n = a.shape[1:]
I,J = np.ogrid[:m,:n]
a_max_values = a[idx, I, J]
b_max_values = b[idx, I, J]
For the general case:
def argmax_to_max(arr, argmax, axis):
"""argmax_to_max(arr, arr.argmax(axis), axis) == arr.max(axis)"""
new_shape = list(arr.shape)
del new_shape[axis]
grid = np.ogrid[tuple(map(slice, new_shape))]
grid.insert(axis, argmax)
return arr[tuple(grid)]
Quite a bit more awkward than such a natural operation should be, unfortunately.
For indexing a n dim array with a (n-1) dim array, we could simplify it a bit to give us the grid of indices for all axes, like so -
def all_idx(idx, axis):
grid = np.ogrid[tuple(map(slice, idx.shape))]
grid.insert(axis, idx)
return tuple(grid)
Hence, use it to index into input arrays -
axis = 0
a_max_values = a[all_idx(idx, axis=axis)]
b_max_values = b[all_idx(idx, axis=axis)]
using indexing in numpy https://docs.scipy.org/doc/numpy-1.10.1/reference/arrays.indexing.html#advanced-indexing
a = np.array([[1, 2], [3, 4], [5, 6]])
a
> a: array([[1, 2],
[3, 4],
[5, 6]])
idx = a.argmax(axis=1)
idx
> array([1, 0, 0], dtype=int64)
since you want all rows but only columns with idx indexes you can use [0, 1, 2] or np.arange(a.shape[0]) for the row indexes
rows = np.arange(a.shape[0])
a[rows, idx]
>array([3, 2, 1])
which is the same as a.max(axis=1)
a.max(axis=1)
>array([3, 2, 1])
if you have 3 dimensions you add the indexes of the 3rd dimension as well:
index2 = np.arange(a.shape[2])
a[rows, idx, index2]
I suggest the following:
a = np.array([[1, 3], [2, -2], [1, -1]])
a
>array([[ 1, 3],
[ 2, -2],
[ 1, -1]])
idx = a.argmax(axis=1)
idx
> array([1, 0, 0], dtype=int64)
np.take_along_axis(a, idx[:, None], axis=1).squeeze()
>array([3, 2, 1])
a.max(axis=1)
>array([3, 2, 1])
What is the most elegant way to access an n dimensional array with an (n-1) dimensional array along a given dimension as in the dummy example
a = np.random.random_sample((3,4,4))
b = np.random.random_sample((3,4,4))
idx = np.argmax(a, axis=0)
How can I access now with idx a to get the maxima in a as if I had used a.max(axis=0)? or how to retrieve the values specified by idx in b?
I thought about using np.meshgrid but I think it is an overkill. Note that the dimension axis can be any usefull axis (0,1,2) and is not known in advance. Is there an elegant way to do this?
Make use of advanced-indexing -
m,n = a.shape[1:]
I,J = np.ogrid[:m,:n]
a_max_values = a[idx, I, J]
b_max_values = b[idx, I, J]
For the general case:
def argmax_to_max(arr, argmax, axis):
"""argmax_to_max(arr, arr.argmax(axis), axis) == arr.max(axis)"""
new_shape = list(arr.shape)
del new_shape[axis]
grid = np.ogrid[tuple(map(slice, new_shape))]
grid.insert(axis, argmax)
return arr[tuple(grid)]
Quite a bit more awkward than such a natural operation should be, unfortunately.
For indexing a n dim array with a (n-1) dim array, we could simplify it a bit to give us the grid of indices for all axes, like so -
def all_idx(idx, axis):
grid = np.ogrid[tuple(map(slice, idx.shape))]
grid.insert(axis, idx)
return tuple(grid)
Hence, use it to index into input arrays -
axis = 0
a_max_values = a[all_idx(idx, axis=axis)]
b_max_values = b[all_idx(idx, axis=axis)]
using indexing in numpy https://docs.scipy.org/doc/numpy-1.10.1/reference/arrays.indexing.html#advanced-indexing
a = np.array([[1, 2], [3, 4], [5, 6]])
a
> a: array([[1, 2],
[3, 4],
[5, 6]])
idx = a.argmax(axis=1)
idx
> array([1, 0, 0], dtype=int64)
since you want all rows but only columns with idx indexes you can use [0, 1, 2] or np.arange(a.shape[0]) for the row indexes
rows = np.arange(a.shape[0])
a[rows, idx]
>array([3, 2, 1])
which is the same as a.max(axis=1)
a.max(axis=1)
>array([3, 2, 1])
if you have 3 dimensions you add the indexes of the 3rd dimension as well:
index2 = np.arange(a.shape[2])
a[rows, idx, index2]
I suggest the following:
a = np.array([[1, 3], [2, -2], [1, -1]])
a
>array([[ 1, 3],
[ 2, -2],
[ 1, -1]])
idx = a.argmax(axis=1)
idx
> array([1, 0, 0], dtype=int64)
np.take_along_axis(a, idx[:, None], axis=1).squeeze()
>array([3, 2, 1])
a.max(axis=1)
>array([3, 2, 1])
What is the most elegant way to access an n dimensional array with an (n-1) dimensional array along a given dimension as in the dummy example
a = np.random.random_sample((3,4,4))
b = np.random.random_sample((3,4,4))
idx = np.argmax(a, axis=0)
How can I access now with idx a to get the maxima in a as if I had used a.max(axis=0)? or how to retrieve the values specified by idx in b?
I thought about using np.meshgrid but I think it is an overkill. Note that the dimension axis can be any usefull axis (0,1,2) and is not known in advance. Is there an elegant way to do this?
Make use of advanced-indexing -
m,n = a.shape[1:]
I,J = np.ogrid[:m,:n]
a_max_values = a[idx, I, J]
b_max_values = b[idx, I, J]
For the general case:
def argmax_to_max(arr, argmax, axis):
"""argmax_to_max(arr, arr.argmax(axis), axis) == arr.max(axis)"""
new_shape = list(arr.shape)
del new_shape[axis]
grid = np.ogrid[tuple(map(slice, new_shape))]
grid.insert(axis, argmax)
return arr[tuple(grid)]
Quite a bit more awkward than such a natural operation should be, unfortunately.
For indexing a n dim array with a (n-1) dim array, we could simplify it a bit to give us the grid of indices for all axes, like so -
def all_idx(idx, axis):
grid = np.ogrid[tuple(map(slice, idx.shape))]
grid.insert(axis, idx)
return tuple(grid)
Hence, use it to index into input arrays -
axis = 0
a_max_values = a[all_idx(idx, axis=axis)]
b_max_values = b[all_idx(idx, axis=axis)]
using indexing in numpy https://docs.scipy.org/doc/numpy-1.10.1/reference/arrays.indexing.html#advanced-indexing
a = np.array([[1, 2], [3, 4], [5, 6]])
a
> a: array([[1, 2],
[3, 4],
[5, 6]])
idx = a.argmax(axis=1)
idx
> array([1, 0, 0], dtype=int64)
since you want all rows but only columns with idx indexes you can use [0, 1, 2] or np.arange(a.shape[0]) for the row indexes
rows = np.arange(a.shape[0])
a[rows, idx]
>array([3, 2, 1])
which is the same as a.max(axis=1)
a.max(axis=1)
>array([3, 2, 1])
if you have 3 dimensions you add the indexes of the 3rd dimension as well:
index2 = np.arange(a.shape[2])
a[rows, idx, index2]
I suggest the following:
a = np.array([[1, 3], [2, -2], [1, -1]])
a
>array([[ 1, 3],
[ 2, -2],
[ 1, -1]])
idx = a.argmax(axis=1)
idx
> array([1, 0, 0], dtype=int64)
np.take_along_axis(a, idx[:, None], axis=1).squeeze()
>array([3, 2, 1])
a.max(axis=1)
>array([3, 2, 1])
How can I append the last row of an array to itself ?
something like:
x= np.array([(1,2,3,4,5)])
x= np.append(x, x[0], 1)
Also, Could you explain why this way of working with vectors yields an error?
for i in range(3):
x.append(0)
x
[0, 0, 0]
x= np.append(x, x[0],0)
Which way of working with vectors would be best ? I am trying to work with 2D vectors as being a matrix, keeping in mind i would like to do some future matrix calculations like multiplication etc.
In [3]: x=np.array([(1,2,3,4,5)])
In [4]: x
Out[4]: array([[1, 2, 3, 4, 5]])
In [5]: x=np.append(x,x[0],1)
...
ValueError: all the input arrays must have same number of dimensions
x is (1,5), x[0] is (5,) - one is 2d, the other 1d.
In [11]: x=np.vstack([x,x[0]])
In [12]: x
Out[12]:
array([[1, 2, 3, 4, 5],
[1, 2, 3, 4, 5]])
this works because vstack changes the x[0] to 2d, e.g. (1,5), so it can concatenate it with x.
In [16]: x=np.concatenate([x, np.atleast_2d(x[-1,:])])
In [17]: x.shape
Out[17]: (3, 5)
We can use concatenate (or append) by first expanding x[-1,:] to 2d.
But in general repeated concatenation is a slow way of building an array.
For a list, repeated append like this works. But it does not work for arrays. For one thing, an array does not have an append method. And np.append function returns a new array. It does not change x in place.
In [19]: z=[]
In [20]: for i in range(3):
...: z.append(0)
...:
In [21]: z
Out[21]: [0, 0, 0]
Repeated append to a list is fine. Repeated append to an array is slow.
In [25]: z=[]
In [26]: for i in range(3):
...: z.append(list(range(i,i+4)))
In [27]: z
Out[27]: [[0, 1, 2, 3], [1, 2, 3, 4], [2, 3, 4, 5]]
In [28]: np.array(z)
Out[28]:
array([[0, 1, 2, 3],
[1, 2, 3, 4],
[2, 3, 4, 5]])
>>> np.append(x,x[-1:],0)
array([[1, 2, 3, 4, 5],
[1, 2, 3, 4, 5]])
How about this:
np.append(arr=x, values=x[-1,None], axis=0)
#array([[1, 2, 3, 4, 5],
# [1, 2, 3, 4, 5]])
I have a 2D NumPy array and I hope to expand its size on both dimensions by copying the bottom row and right column.
For example, from 2x2:
[[0,1],
[2,3]]
to 4x4:
[[0,1,1,1],
[2,3,3,3],
[2,3,3,3],
[2,3,3,3]]
What's the best way to do it?
Thanks.
Here, the hstack and vstack functions can come in handy. For example,
In [16]: p = array(([0,1], [2,3]))
In [20]: vstack((p, p[-1], p[-1]))
Out[20]:
array([[0, 1],
[2, 3],
[2, 3],
[2, 3]])
And remembering that p.T is the transpose:
So now you can do something like the following:
In [16]: p = array(([0,1], [2,3]))
In [22]: p = vstack((p, p[-1], p[-1]))
In [25]: p = vstack((p.T, p.T[-1], p.T[-1])).T
In [26]: p
Out[26]:
array([[0, 1, 1, 1],
[2, 3, 3, 3],
[2, 3, 3, 3],
[2, 3, 3, 3]])
So the 2 lines of code should do it...
Make an empty array and copy whatever rows, columns you want into it.
def expand(a, new_shape):
x, y = a.shape
r = np.empty(new_shape, a.dtype)
r[:x, :y] = a
r[x:, :y] = a[-1:, :]
r[:x, y:] = a[:, -1:]
r[x:, y:] = a[-1, -1]
return r