This seems like a really simple question but I can't find a good answer anywhere. How might I multiply (in place) select columns (perhaps selected by a list) by a scalar using numpy?
E.g. Multiply columns 0 and 2 by 4
In: arr=([(1,2,3,5,6,7), (4,5,6,2,5,3), (7,8,9,2,5,9)])
Out: arr=([(4,2,12,5,6,7), (16,5,24,2,5,3), (28,8,36,2,5,9)])
Currently I am doing this in multiple steps but I feel like there must be a better way especially if the list gets larger. Current way:
arr['f0'] *= 4
arr['f2'] *= 4
You can use array slicing as follows for this -
In [10]: arr=([(1,2,3,5,6,7), (4,5,6,2,5,3), (7,8,9,2,5,9)])
In [11]: narr = np.array(arr)
In [13]: narr[:,(0,2)] = narr[:,(0,2)]*4
In [14]: narr
Out[14]:
array([[ 4, 2, 12, 5, 6, 7],
[16, 5, 24, 2, 5, 3],
[28, 8, 36, 2, 5, 9]])
An alternate solution is to create a class that inherits from np.ndarray and add a method to it to make the in-place mutation more intuitive.
Code:
import numpy as np
class A(np.ndarray):
def __new__(cls, a):
arr = np.asarray(a)
arr = arr.view(cls)
return arr
def mutate(self, col, k):
self[:,col] *= k
return self
a = A([(1,2,3,5,6,7), (4,5,6,2,5,3), (7,8,9,2,5,9)])
print a
print '---------------------'
a.mutate(0, 4)
a.mutate(2, 4)
print a
Result:
[[1 2 3 5 6 7]
[4 5 6 2 5 3]
[7 8 9 2 5 9]]
---------------------
[[ 4 2 12 5 6 7]
[16 5 24 2 5 3]
[28 8 36 2 5 9]]
You can use the following along with array slicing
arr=([(1,2,3,5,6,7), (4,5,6,2,5,3), (7,8,9,2,5,9)])
array = np.array(arr)
array[:,(0,2)]*=4
array
Out[10]:
array([[ 4, 2, 12, 5, 6, 7],
[16, 5, 24, 2, 5, 3],
[28, 8, 36, 2, 5, 9]])
Related
I had to make a matrix using numpy.array method. How can I now update every third element of my matrix? I have made a for loop for the problem but that is not the optimal solution. Is there a way to avoid loops? For example if I have this matrix:
matrix = np.array([[1,2,3,4],
[5,6,7,8],
[4,7,6,9]])
is there a way to add 1 to every third element and get this matrix:
[[2,2,3,5],[5,6,8,8],[4,8,6,9]]
Solution:
matrix = np.ascontiguousarray(matrix)
matrix.ravel()[::3] += 1
Why does the ascontiguousarray is needed? Because matrix may not be c-contiguous (for example matrix may have fortran-order - column major). It that case ravel returns a copy instead of a view so a simple inplace operation matrix.ravel()[::3] += 1 will not work as expected.
Example 1
import numpy as np
arr = np.array([
[1, 2, 3, 4],
[5, 6, 7, 8],
[4, 7, 6, 9]])
arr.ravel()[::3] += 1
print(arr)
Works as expected:
[[2 2 3 5]
[5 6 8 8]
[4 8 6 9]]
Example 2
But with fortran-order
import numpy as np
arr = np.array([
[1, 2, 3, 4],
[5, 6, 7, 8],
[4, 7, 6, 9]])
arr = np.asfortranarray(arr)
arr.ravel()[::3] += 1
print(arr)
produces:
[[1 2 3 4]
[5 6 7 8]
[4 7 6 9]]
Example 3
Will work as expected in both cases
import numpy as np
arr = np.array([
[1, 2, 3, 4],
[5, 6, 7, 8],
[4, 7, 6, 9]])
# arr = np.asfortranarray(arr)
arr = np.ascontiguousarray(arr)
arr.ravel()[::3] += 1
print(arr)
I am trying to extract a sub-array using logical indexes as,
a = np.array([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, 16]])
a
Out[45]:
array([[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12],
[13, 14, 15, 16]])
b = np.array([False, True, False, True])
a[b, b]
Out[49]: array([ 6, 16])
python evaluates the logical indexes in b per element of a. However in matlab you can do something like
>> a = [1 2 3 4; 5 6 7 8; 9 10 11 12; 13 14 15 16]
a =
1 2 3 4
5 6 7 8
9 10 11 12
13 14 15 16
>> b = [2 4]
b =
2 4
>> a(b, b)
ans =
6 8
14 16
how can I achieve the same result in python without doing,
c = a[:, b]
c[b,:]
Out[51]:
array([[ 6, 8],
[14, 16]])
Numpy supports logical indexing, though it is a little different than what you are familiar in MATLAB. To get the results you want you can do the following:
a[b][:,b] # first brackets isolates the rows, second brackets isolate the columns
Out[27]:
array([[ 6, 8],
[14, 16]])
The more "numpy" method will be understood after you will understand what happend in your case.
b = np.array([False, True, False, True]) is similar to b=np.array([1,3]) and will be easier for me to explain. When writing a[[1,3],[1,3]] what happens is that numpy crates a (2,1) shape array, and places a[1,1] in the [0] location and a[3,3] in the second location. To create an output of shape (2,2), the indexing must have the same dimensionality. Therefore, the following will get your result:
a[[[1,1],[3,3]],[[1,3],[1,3]]]
Out[28]:
array([[ 6, 8],
[14, 16]])
Explanation:
The indexing arrays are:
temp_rows = np.array([[1,1],
[3,3]])
temp_cols = np.array([[1,3],
[1,3])
both arrays have dimensions of (2,2) and therefore, numpy will create an output of shape (2,2). Then, it places a[1,1] in location [0,0], a[1,3] in [0,1], a[3,1] in location [1,0] and a[3,3] in location [1,1]. This can be expanded to any shape but for your purposes, you wanted a shape of (2,2)
After figuring this out, you can make things even simpler by utilizing the fact you if you insert a (2,1) array in the 1st dimension and a (1,2) array in the 2nd dimension, numpy will perform the broadcasting, similar to the MATLAB operation. This means that by using:
temp_rows = np.array([[1],[3]])
temp_cols = np.array([1,3])
you can do:
a[[[1],[3]], [1,3])
Out[29]:
array([[ 6, 8],
[14, 16]])
You could use np.ix_ here.
a[np.ix_(b, b)]
# array([[ 6, 8],
# [14, 16]])
Output returned by np.ix_
>>> np.ix_(b, b)
(array([[1],
[3]]),
array([[1, 3]]))
You could make use of a outer product of the b vector. The new dimesion you can obtain from the number of True values using a sum.
import numpy as np
a = np.array([[1, 2, 3, 4], [5, 6, 7, 8], [9, 10, 11, 12], [13, 14, 15, 16]])
b = np.array([False, True, False, True])
#
M = np.outer(b, b)
new_dim = b.sum()
new_shape = (new_dim, new_dim)
selection = a[M].reshape(new_shape)
The result looks like
[[ 6 8]
[14 16]]
I'm running into this problem over and over again, and can't seem to find a clean solution for this. So I'm trying to index an array with another array. I have a 2d numpy array. And a 1d numpy array with the same length as the 1st dimension of the 2d array I'm trying to index and the elements represent the indices of the columns I try to extract:
import numpy as np
A = np.random.rand((5,3))
B = np.asarray([2,1,2,0,1])
The behaviour that I want is extracting for all rows the corresponding column in array B. This could be done by
C = A[np.arange(A.shape[0]),B]
But I can imagine that there is a better way to get this behaviour. Using a : as indexing the first row gives the wrong behaviour.
If there is a cleaner way of doing this that would be great. I'm really used to this array indexing from Matlab, but maybe there is no equivalent in numpy. Using boolean indices is of course an option, but that also requires converting arrays all the time.
I think what you look after is np.choose(B,A.T) :
In [125]: A
Out[125]:
array([[ 0, 1, 2],
[ 3, 4, 5],
[ 6, 7, 8],
[ 9, 10, 11],
[12, 13, 14]])
In [126]: B = np.asarray([2,1,2,0,1])
In [127]: np.choose(B,A.T)
Out[127]: array([ 2, 4, 8, 9, 13])
In [60]: A = np.arange(1,16).reshape(5,3)
In [61]: B = np.array([2,1,2,0,1])
In [62]: C = A[np.arange(A.shape[0]),B]
In [63]: C
Out[63]: array([ 3, 5, 9, 10, 14])
In Octave
>> A = reshape(1:15, 3,5).';
>> B = [3,2,3,1,2];
>> A
A =
1 2 3
4 5 6
7 8 9
10 11 12
13 14 15
>> A(:,B)
ans =
3 2 3 1 2
6 5 6 4 5
9 8 9 7 8
12 11 12 10 11
15 14 15 13 14
This is the same as numpy:
In [65]: A[:,B]
Out[65]:
array([[ 3, 2, 3, 1, 2],
[ 6, 5, 6, 4, 5],
[ 9, 8, 9, 7, 8],
[12, 11, 12, 10, 11],
[15, 14, 15, 13, 14]])
You imply that there's something clean for indexing one item from each column in MATLAB/Octave, but I missing it. I used to work a lot in that language, but I've gotten out of practice.
sub2ind does the job:
>> sub2ind([3,5],B, 1:5)
ans =
3 5 9 10 14
>> A.'(:)(sub2ind([3,5],B,1:5))
ans =
3
5
9
10
14
(The 'F' vs 'C' ordering is complicating my comparison)
numpy has a similar ravel_multi_index:
In [69]: np.ravel_multi_index((np.arange(5),B),(5,3))
Out[69]: array([ 2, 4, 8, 9, 13], dtype=int32)
In [71]: A.flat[_]
Out[71]: array([ 3, 5, 9, 10, 14])
I want to concatenate numpy matrices that have different shapes in order to get an array with dimension=3.
example :
A= [[2 1 3 4]
[2 4 0 6]
[9 5 7 4]]
B= [[7 2 8 4]
[8 6 8 6]]
and result what I need should be like that:
C=[[[2 1 3 4]
[2 4 0 6]
[9 5 7 4]]
[[7 2 8 4]
[8 6 8 6]]]
Thanks for help
If I understand your question correctly, a 3dim numpy array is probably not the way to represent your data, because there's no definitive shape.
A 3dim numpy array should have a shape of the form N1 x N2 x N3, whereas in your case each "2dim row" has a different shape.
Alternatives would be to keep your data in lists (or a list of arrays), or to use masked arrays, if that happens to be reasonable in you case.
You can only convert to a 3D np.ndarray in a useful manner if A.shape == B.shape. In that case all you need to do is e.g. C = np.array([A, B]).
import numpy as np
A = np.array([[2, 1, 3, 4],
[9, 5, 7, 4]])
B = np.array([[7, 2, 8, 4],
[8, 6, 8, 6]])
C = np.array([A, B])
print C
Because A and B have different sizes (# of rows), the best you can do make an array of shape (2,) and dtype object. Or at least that's what a simple construction gives you:
In [9]: np.array([A,B])
Out[9]:
array([array([[2, 1, 3, 4],
[2, 4, 0, 6],
[9, 5, 7, 4]]),
array([[7, 2, 8, 4],
[8, 6, 8, 6]])], dtype=object)
But constructing an array like this doesn't help much. Just use the list [A,B].
np.vstack([A,B]) produces a (5,4) array.
np.array([A[:2,:],B]) gives a (2,2,4) array. Or you could pad B so they are both (3,4).
So one way or other you need to redefine your problem.
I'm doing image processing for object detection using python. I need to divide my image into all possible blocks. For example given this toy image:
x = np.arange(25)
x = x.reshape((5, 5))
[[ 0 1 2 3 4]
[ 5 6 7 8 9]
[10 11 12 13 14]
[15 16 17 18 19]
[20 21 22 23 24]]
I want to retrieve all possible blocks of a given size, for example the 2x2 blocks are:
[[0 1]
[5 6]]
[[1 2]
[6 7]]
.. and so on. How can I do this?
The scikit image extract_patches_2d does that
>>> from sklearn.feature_extraction import image
>>> one_image = np.arange(16).reshape((4, 4))
>>> one_image
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11],
[12, 13, 14, 15]])
>>> patches = image.extract_patches_2d(one_image, (2, 2))
>>> print(patches.shape)
(9, 2, 2)
>>> patches[0]
array([[0, 1],
[4, 5]])
>>> patches[1]
array([[1, 2],
[5, 6]])
>>> patches[8]
array([[10, 11],
[14, 15]])
You can use something like this:
def rolling_window(arr, window):
"""Very basic multi dimensional rolling window. window should be the shape of
of the desired subarrays. Window is either a scalar or a tuple of same size
as `arr.shape`.
"""
shape = np.array(arr.shape*2)
strides = np.array(arr.strides*2)
window = np.asarray(window)
shape[arr.ndim:] = window # new dimensions size
shape[:arr.ndim] -= window - 1
if np.any(shape < 1):
raise ValueError('window size is too large')
return np.lib.stride_tricks.as_strided(arr, shape=shape, strides=strides)
# Now:
slices = rolling_window(arr, 2)
# Slices will be 4-d not 3-d as you wanted. You can reshape
# but it may need to copy (not if you have done no slicing, etc. with the array):
slices = slices.reshape(-1,slices.shape[2:])
Simple code with a double loop and slice:
>>> a = np.arange(12).reshape(3,4)
>>> print(a)
[[ 0 1 2 3]
[ 4 5 6 7]
[ 8 9 10 11]]
>>> r = 2
>>> n_rows, n_cols = a.shape
>>> for row in range(n_rows - r + 1):
... for col in range(n_cols - r + 1):
... print(a[row:row + r, col:col + r])
...
[[0 1]
[4 5]]
[[1 2]
[5 6]]
[[2 3]
[6 7]]
[[4 5]
[8 9]]
[[ 5 6]
[ 9 10]]
[[ 6 7]
[10 11]]