Is there a quick way to rescale a matrix by simply adding adjacent pixels?
So for a X=N*M matrix you get a Y=(N/n) *(N/m) where n * m is the area I should add the pixel in.
I've been doing that manually (via script) but I think there has to be somewhere a way to do it.
for i in range(0, X.shape[0]/n):
for j in range(0, X.shape[1]/m):
Y[i, j] = np.sum(X[i*n:i*n+n, j*m:j*m+m])
E.G.
X = [[0 1 2 3]
[2 3 4 5]
[3 4 6 8]
[2 3 4 5]]
Y = [[ 6 14]
[12 23]]
A pure numpy way would be to reshape the matrix into more axes and sum over the appropiate axes.
Y = X.reshape(X.shape[0]/n, n, X.shape[1]/m, m).sum((1, 3))
You can vectorize your code by using the functions skimage.utils.view_as_blocks and numpy.sum:
import numpy as np
from skimage.util import view_as_blocks
N, M = 4, 4
n, m = 2, 2
X = np.random.randint(size=(N, M), low=0, high=10)
blocks = view_as_blocks(X, (N//n, M//n))
Y = np.sum(blocks, axis=(-1, -2))
Nand M have to be integer multiples of n and m, respectively. Otherwise you'll get the following error:
ValueError: block_shape is not compatible with arr_in
Sample run
In [74]: X
Out[74]:
array([[5, 6, 3, 7],
[5, 0, 3, 0],
[6, 1, 8, 0],
[4, 0, 7, 2]])
In [75]: Y
Out[75]:
array([[16, 13],
[11, 17]])
Related
For a project I need to be able to get, from a vector with shape (k, m), the indexes of the N greatest values of each row greater than a fixed threshold.
For example, if k=3, m=5, N=3 and the threshold is 5 and the vector is :
[[3 2 6 7 0],
[4 1 6 4 0],
[7 10 6 9 8]]
I should get the result (or the flattened version, I don't care) :
[[2, 3],
[2],
[1, 3, 4]]
The indexes don't have to be sorted.
My code is currently :
indexes = []
for row, inds in enumerate(np.argsort(results, axis=1)[:, -N:]):
for index in inds:
if results[row, index] > threshold:
indexes.append(index)
but I feel like I am not using Numpy to its full capacity.
Does anybody know a better and more elegant solution ?
How about this method:
import numpy as np
arr = np.array(
[[3, 2, 6, 7, 0],
[4, 1, 6, 4, 0],
[7, 10, 6, 9, 8]]
)
t = 5
n = 3
sorted_idxs = arr.argsort(1)[:, -n:]
sorted_arr = np.sort(arr, 1)[:, -n:]
item_nums = np.cumsum((sorted_arr > t).sum(1))
masked_idxs = sorted_idxs[sorted_arr > t]
idx_lists = np.split(masked_idxs, item_nums)
output:
[array([2, 3]), array([2]), array([4, 3, 1])]
Problem
I have a 2D NumPy array, arr, and for each row, I would like to reverse a section of the array. Crucially, for each row, the start and stop indices must be unique. I can achieve this using the following.
import numpy as np
arr = np.repeat(np.arange(10)[np.newaxis, :], 3, axis=0)
reverse = np.sort(np.random.choice(arr.shape[1], [arr.shape[0], 2], False))
# arr
# array([[0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
# [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
# [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]])
# reverse
# array([[1, 7],
# [8, 9],
# [4, 6]])
Reverse each row between the start, stop indices in `reverse.
for idx, (i, j) in enumerate(reverse):
arr[idx, i:j+1] = arr[idx, i:j+1][::-1]
# arr
# array([[0, 7, 6, 5, 4, 3, 2, 1, 8, 9],
# [0, 1, 2, 3, 4, 5, 6, 7, 9, 8],
# [0, 1, 2, 3, 6, 5, 4, 7, 8, 9]])
Question
Is this possible using basic slicing and indexing? I tried to use the output of reverse to form multiple slice objects, but was unsuccessful.
Update
A simple comparison of the original method vs answer. For my data, the solution is only required to deal with 2D matrices with shape (50, 100).
import numpy as np
def reverse_one(arr, n):
temp = np.repeat(arr.copy(), n, axis=0)
reverse = np.sort(np.random.choice(temp.shape[1], [n, 2], False))
for idx, (i, j) in enumerate(reverse):
temp[idx, i:j+1] = temp[idx, i:j+1][::-1]
return temp
def reverse_two(arr, n):
temp = np.repeat(arr.copy(), n, axis=0)
reverse = np.sort(np.random.choice(temp.shape[1], [n, 2], False))
rev = np.ravel_multi_index((np.arange(n)[:, np.newaxis], reverse), temp.shape)
rev[:, 1] += 1
idx = np.arange(temp.size).reshape(temp.shape)
s = np.searchsorted(rev.ravel(), idx, 'right')
m = (s % 2 == 1)
g = rev[s[m] // 2]
idx[m] = g[:, 0] - (idx[m] - g[:, 1]) - 1
return temp.take(idx)
m = 100
arr = np.arange(m)[np.newaxis, :]
print("reverse_one:")
%timeit reverse_one(arr, m//2)
print("=" * 40)
print("reverse_two:")
%timeit reverse_two(arr, m//2)
Running the following code in a Jupyter Notebook gives the following results.
reverse_one:
1000 loops, best of 5: 202 µs per loop
========================================
reverse_two:
1000 loops, best of 5: 363 µs per loop
This was kinda tricky but I figured out one way to do it. Advanced indexing is expensive though so you'd have to see whether it is really faster or not depending on the data that you have.
import numpy as np
np.random.seed(0)
arr = np.repeat(np.arange(10)[np.newaxis, :], 3, axis=0)
reverse = np.sort(np.random.choice(arr.shape[1], [arr.shape[0], 2], False))
print(arr)
# [[0 1 2 3 4 5 6 7 8 9]
# [0 1 2 3 4 5 6 7 8 9]
# [0 1 2 3 4 5 6 7 8 9]]
print(reverse)
# [[2 8]
# [4 9]
# [1 6]]
# Get "flat" indices of the bounds
rev = np.ravel_multi_index((np.arange(arr.shape[0])[:, np.newaxis], reverse), arr.shape)
# Add one to the second bound (so it is first index after the slice)
rev[:, 1] += 1
# Make array of flat indices for the data
idx = np.arange(arr.size).reshape(arr.shape)
# Find the position of flat indices with respect to bounds
s = np.searchsorted(rev.ravel(), idx, 'right')
# For each "i" within a slice, "s[i]" is odd
m = (s % 2 == 1)
# Replace indices within slices with their reversed ones
g = rev[s[m] // 2]
idx[m] = g[:, 0] - (idx[m] - g[:, 1]) - 1
# Apply indices to array
res = arr.take(idx)
print(res)
# [[0 1 8 7 6 5 4 3 2 9]
# [0 1 2 3 9 8 7 6 5 4]
# [0 6 5 4 3 2 1 7 8 9]]
I want to rotate an array but not like a basic matrix rotation. If I have a 3x3 array, I want each of the cells to turn around the central cell.
Here is a 3x3 array:
import numpy as np
tab = np.array([[1,2,3],[4,5,6],[7,8,9]])
[[1 2 3]
[4 5 6]
[7 8 9]]
I want for instance to rotate it by 45 degrees:
[[2 3 6]
[1 5 9]
[4 7 8]]
I can't use scipy.ndimage.interpolation.rotate(tab,45) because it applies a basic matrix rotation and this is not what I want.
import numpy as np
from scipy import ndimage
tab = np.array([[1,2,3],[4,5,6],[7,8,9]])
ndimage.interpolation.rotate(tab,45)
[[0 0 0 0]
[0 2 6 0]
[0 4 8 0]
[0 0 0 0]]
Does anybody know how this can be achieved?
Your almost had it just use this:
ndimage.interpolation.rotate(tab,45,reshape=False,mode='nearest')
the thing is you need to force the method to not reshape your matrix and also use nearest number as fill in the blank instead of zeros.
The problem with zeros is that some numbers (the ones in the corners) become out of boundaries when you rotate the matrix so you need to "predict" those missing corners by nearest like precised here
output:
[[2 3 6]
[1 5 9]
[4 7 8]]
rotate it again it gives:
[[3 6 9]
[2 5 8]
[1 4 7]]
etc..
If you only want to use it on 3x3 arrays, a simple solution would be to list the indexes in a clockwise/anticlockwise order (eg: [(0, 0), (0, 1), (0, 2), (1, 2), (2, 2), (2, 1), (2, 0), (1, 0)]), then for the rotations you just shift the values according to the list indexes (by degrees/45° places).
You can do this by creating a pair of index arrays (in which a rotation can be seen visually):
i = np.array([
[0, 0, 1],
[0, 1, 2],
[1, 2, 2]
])
j = np.array([
[1, 2, 2],
[0, 1, 2],
[0, 0, 1]
])
Which then works as desired:
>>> tab = np.array([[1,2,3],[4,5,6],[7,8,9]])
>>> tab[i,j]
array([[2, 3, 6],
[1, 5, 9],
[4, 7, 8]])
This is essentially a vectorized implementation of Balázs Kovacsics' solution, which should be faster than the ndimage solution
To rotate twice, you can use either
>>> tab[i,j][i,j]
array([[3, 6, 9],
[2, 5, 8],
[1, 4, 7]])
>>> i2 = i[i,j]
>>> j2 = j[i,j]
>>> tab[i2,j2]
array([[3, 6, 9],
[2, 5, 8],
[1, 4, 7]])
I have the following to calculate the difference of a matrix, i.e. the i-th element - the (i-1) element.
How can I (easily) calculate the difference for each element horizontally and vertically? With a transpose?
inputarr = np.arange(12)
inputarr.shape = (3,4)
inputarr+=1
#shift one position
newarr = list()
for x in inputarr:
newarr.append(np.hstack((np.array([0]),x[:-1])))
z = np.array(newarr)
print inputarr
print 'first differences'
print inputarr-z
Output
[[ 1 2 3 4]
[ 5 6 7 8]
[ 9 10 11 12]]
first differences
[[1 1 1 1]
[5 1 1 1]
[9 1 1 1]]
Check out numpy.diff.
From the documentation:
Calculate the n-th order discrete difference along given axis.
The first order difference is given by out[n] = a[n+1] - a[n] along
the given axis, higher order differences are calculated by using diff
recursively.
An example:
>>> import numpy as np
>>> a = np.arange(12).reshape((3,4))
>>> a
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11]])
>>> np.diff(a,axis = 1) # row-wise
array([[1, 1, 1],
[1, 1, 1],
[1, 1, 1]])
>>> np.diff(a, axis = 0) # column-wise
array([[4, 4, 4, 4],
[4, 4, 4, 4]])
I'm doing image processing for object detection using python. I need to divide my image into all possible blocks. For example given this toy image:
x = np.arange(25)
x = x.reshape((5, 5))
[[ 0 1 2 3 4]
[ 5 6 7 8 9]
[10 11 12 13 14]
[15 16 17 18 19]
[20 21 22 23 24]]
I want to retrieve all possible blocks of a given size, for example the 2x2 blocks are:
[[0 1]
[5 6]]
[[1 2]
[6 7]]
.. and so on. How can I do this?
The scikit image extract_patches_2d does that
>>> from sklearn.feature_extraction import image
>>> one_image = np.arange(16).reshape((4, 4))
>>> one_image
array([[ 0, 1, 2, 3],
[ 4, 5, 6, 7],
[ 8, 9, 10, 11],
[12, 13, 14, 15]])
>>> patches = image.extract_patches_2d(one_image, (2, 2))
>>> print(patches.shape)
(9, 2, 2)
>>> patches[0]
array([[0, 1],
[4, 5]])
>>> patches[1]
array([[1, 2],
[5, 6]])
>>> patches[8]
array([[10, 11],
[14, 15]])
You can use something like this:
def rolling_window(arr, window):
"""Very basic multi dimensional rolling window. window should be the shape of
of the desired subarrays. Window is either a scalar or a tuple of same size
as `arr.shape`.
"""
shape = np.array(arr.shape*2)
strides = np.array(arr.strides*2)
window = np.asarray(window)
shape[arr.ndim:] = window # new dimensions size
shape[:arr.ndim] -= window - 1
if np.any(shape < 1):
raise ValueError('window size is too large')
return np.lib.stride_tricks.as_strided(arr, shape=shape, strides=strides)
# Now:
slices = rolling_window(arr, 2)
# Slices will be 4-d not 3-d as you wanted. You can reshape
# but it may need to copy (not if you have done no slicing, etc. with the array):
slices = slices.reshape(-1,slices.shape[2:])
Simple code with a double loop and slice:
>>> a = np.arange(12).reshape(3,4)
>>> print(a)
[[ 0 1 2 3]
[ 4 5 6 7]
[ 8 9 10 11]]
>>> r = 2
>>> n_rows, n_cols = a.shape
>>> for row in range(n_rows - r + 1):
... for col in range(n_cols - r + 1):
... print(a[row:row + r, col:col + r])
...
[[0 1]
[4 5]]
[[1 2]
[5 6]]
[[2 3]
[6 7]]
[[4 5]
[8 9]]
[[ 5 6]
[ 9 10]]
[[ 6 7]
[10 11]]