Numpy: efficiently expand a matrix from submatrix - python

Given a square array D of dimension(4x4):
array([[ 1, 2, 5, 6],
[ 3, 4, 7, 8],
[ 9, 10, 13, 14],
[11, 12, 15, 16]])
how can you, for each sub-matrix d of size (2x2) create another set of matrices D*, such that
D* = [...,
[[ 1, 2, 1, 2],
[ 3, 4, 3, 4],
[ 1, 2, 1, 2],
[ 3, 4, 3, 4]]
]
I then wish to build another square array out of D*, D**, such that:
D** = [[ 1, 2, 1, 2, 5, 6, 5, 6],
[ 3, 4, 3, 4, 7, 8, 7, 8],
[ 1, 2, 1, 2, 5, 6, 5, 6],
[ 3, 4, 3, 4, 7, 8, 7, 8],
[ 9, 10, 9, 10, 13, 14, 13, 14],
[ 11, 12, 11, 12, 15, 16, 15, 16],
[ 9, 10, 9, 10, 13, 14, 13, 14],
[ 11, 12, 11, 12, 15, 16, 15, 16]]]
My actual starting matrix D's dimension is 184x184, so I found that for-loops were too slow to achieve this. Is this too computationally intensive for numpy? Or is there a way to achieve this elegantly an efficiently?
Here is an example of the foor-loop pseudo-code:
segments = [(0,0), (2,2), (0,2), (2, 0)]
for seg in segments:
actual_seg = D[seg[0]:seg[0]+2, seg[1]:seg[1]+2]
D*.append(numpy.kron(numpy.ones((2, 2), dtype=int), actual_seg))


Given D and looking to expand each such (2x2) submatrix, one approach using a combination of np.tile and np.repeat would be -
m,n = D.shape
out = np.repeat(np.tile(D.reshape(m//2,2,n//2,2),2),2,axis=0).reshape(2*m,2*n)
Sample run -
In [116]: D
Out[116]:
array([[ 1, 2, 5, 6],
[ 3, 4, 7, 8],
[ 9, 10, 13, 14],
[11, 12, 15, 16]])
In [117]: m,n = D.shape
In [118]: np.repeat(np.tile(D.reshape(m//2,2,n//2,2),2),2,axis=0).reshape(2*m,2*n)
Out[118]:
array([[ 1, 2, 1, 2, 5, 6, 5, 6],
[ 3, 4, 3, 4, 7, 8, 7, 8],
[ 1, 2, 1, 2, 5, 6, 5, 6],
[ 3, 4, 3, 4, 7, 8, 7, 8],
[ 9, 10, 9, 10, 13, 14, 13, 14],
[11, 12, 11, 12, 15, 16, 15, 16],
[ 9, 10, 9, 10, 13, 14, 13, 14],
[11, 12, 11, 12, 15, 16, 15, 16]])


from __future__ import print_function
import numpy as np
def fn(m_):
rows, cols = m_.shape
assert rows%2==0 and cols%2==0
return np.reshape(
np.transpose(
np.tile(
np.transpose(
np.reshape(
m_,(rows//2,2, cols//2,2)
),(0,2,1,3)),(2,2)), (0,2,1,3)
),
(rows*2,cols*2)
)
m = np.array([[ 1, 2, 5, 6],
[ 3, 4, 7, 8],
[ 9, 10, 13, 14],
[11, 12, 15, 16]])
print(fn(m))
output:
[[ 1 2 1 2 5 6 5 6]
[ 3 4 3 4 7 8 7 8]
[ 1 2 1 2 5 6 5 6]
[ 3 4 3 4 7 8 7 8]
[ 9 10 9 10 13 14 13 14]
[11 12 11 12 15 16 15 16]
[ 9 10 9 10 13 14 13 14]
[11 12 11 12 15 16 15 16]]

Related

How to make a 2d numpy array from an empty numpy array by adding 1d numpy arrays?

So I'm trying to start an empty numpy array with a = np.array([]), but when i append other numpy arrays (like [1, 2, 3, 4, 5, 6, 7, 8] and [9, 10, 11, 12, 13, 14, 15, 16] to this array, then the result im basically getting is
[1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16].
But what i want as result is: [[1, 2, 3, 4, 5, 6, 7, 8], [9, 10, 11, 12, 13, 14, 15, 16]]

IIUC you want to keep adding lists to your np.array. In that case, you can use something like np.vstack to "append" the new lists to the array.
a = np.array([[1, 2, 3],[4, 5, 6]])
np.vstack([a, [7, 8, 9]])
>>> array([[1, 2, 3],
[4, 5, 6],
[7, 8, 9]])

You can also use np.c_[], especially if a and b are already 1D arrays (but it also works with lists):
a = [1, 2, 3, 4, 5, 6, 7, 8]
b = [9, 10, 11, 12, 13, 14, 15, 16]
>>> np.c_[a, b]
array([[ 1, 9],
[ 2, 10],
[ 3, 11],
[ 4, 12],
[ 5, 13],
[ 6, 14],
[ 7, 15],
[ 8, 16]])
It also works "multiple times":
>>> np.c_[np.c_[a, b], a, b]
array([[ 1, 9, 1, 9],
[ 2, 10, 2, 10],
[ 3, 11, 3, 11],
[ 4, 12, 4, 12],
[ 5, 13, 5, 13],
[ 6, 14, 6, 14],
[ 7, 15, 7, 15],
[ 8, 16, 8, 16]])

Python: how to select contiguity neighbors of matrix values?

I have a matrix like the following:
A = array([[12, 6, 14, 8, 4, 1],
[18, 13, 8, 10, 9, 19],
[ 8, 15, 6, 5, 6, 18],
[ 3, 0, 2, 14, 13, 12],
[ 4, 4, 5, 19, 0, 14],
[16, 8, 7, 7, 11, 0],
[ 3, 11, 2, 19, 11, 5],
[ 4, 2, 1, 9, 12, 12]])
For each cell I want to select the values in a radius of k=2 closest cells.
For instance if I select the A[3,4] I would like a submatrix like the following
array([[18, 13, 8, 10, 9],
[ 8, 15, 6, 5, 6],
[ 3, 0, 2, 14, 13],
[ 4, 4, 5, 19, 0],
[16, 8, 7, 7, 11]])
I defined the following function
def queen_neighbourhood(Adj, in_row, in_col, k):
j=k
k+=1
neighbourhood = Adj[in_row-j:in_row+k, in_col-j:in_col+k]
return neighbourhood
such as queen_neighbourhood(A, 3, 2, 2) returns
array([[18, 13, 8, 10, 9],
[ 8, 15, 6, 5, 6],
[ 3, 0, 2, 14, 13],
[ 4, 4, 5, 19, 0],
[16, 8, 7, 7, 11]])
However it does not work in borders.
For instance, for the cell [0,0] I would like to have
array([[12, 6, 14],
[18, 13, 8],
[ 8, 15, 16])
but it returns queen_neighbourhood(A, 0, 0, 2)
array([], shape=(0, 0), dtype=int64)

You could avoid negative indices:
neighbourhood = Adj[max(in_row-j, 0) : in_row+k,
max(in_col-j, 0) : in_col+k]

Adding to the previous answer; taking into consideration the extreme values
def queen_neighbourhood(Adj, in_row, in_col, k):
j=k
k+=1
neighbourhood = Adj[max(in_row-j, 0) : min(in_row+k,Adj.shape[0]),
max(in_col-j, 0) : min(in_col+k,Adj.shape[1])]
return(neighbourhood)

You can use numpy roll to ensure you are always dealing with the middle value,
import numpy as np
def queen_neighbourhood(Adj, in_row, in_col, k):
j=k
k+=1
midrow = int(Adj.shape[0]/2.)+1
midcol = int(Adj.shape[1]/2.)+1
Ashift = np.roll(Adj,(in_row-midrow,in_col-midcol),(0,1))
neighbourhood = Ashift[1:k+1, 1:k+1]
return neighbourhood
A = np.array([[18, 13, 8, 10, 9],
[ 8, 15, 6, 5, 6],
[ 3, 0, 2, 14, 13],
[ 4, 4, 5, 19, 0],
[16, 8, 7, 7, 11]])
print(A)
An = queen_neighbourhood(A, 0, 0, 2)
print(An)
which gives,
[[11 16 8]
[ 9 18 13]
[ 6 8 15]]

Reshaping Sequence Numpy Pandas python

I have tried this:
>>> import pandas as pd
>>> import numpy as np
>>> df = pd.read_csv("test.csv")
>>> df
input1 input2 input3 input4 input5 input6 input7 input8 output
0 1 2 3 4 5 6 7 8 1
1 2 3 4 5 6 7 8 9 0
2 3 4 5 6 7 8 9 10 -1
3 4 5 6 7 8 9 10 11 -1
4 5 6 7 8 9 10 11 12 1
5 6 7 8 9 10 11 12 13 0
6 7 8 9 10 11 12 13 14 1
>>> seq_len=3
>>> data = []
>>> data_raw = df.values
>>> for index in range(len(data_raw) - seq_len + 1):
... data.append(data_raw[index: index + seq_len])
...
>>> data
[array([[ 1, 2, 3, 4, 5, 6, 7, 8, 1],
[ 2, 3, 4, 5, 6, 7, 8, 9, 0],
[ 3, 4, 5, 6, 7, 8, 9, 10, -1]], dtype=int64), array([[ 2, 3, 4, 5, 6, 7, 8, 9, 0],
[ 3, 4, 5, 6, 7, 8, 9, 10, -1],
[ 4, 5, 6, 7, 8, 9, 10, 11, -1]], dtype=int64), array([[ 3, 4, 5, 6, 7, 8, 9, 10, -1],
[ 4, 5, 6, 7, 8, 9, 10, 11, -1],
[ 5, 6, 7, 8, 9, 10, 11, 12, 1]], dtype=int64), array([[ 4, 5, 6, 7, 8, 9, 10, 11, -1],
[ 5, 6, 7, 8, 9, 10, 11, 12, 1],
[ 6, 7, 8, 9, 10, 11, 12, 13, 0]], dtype=int64), array([[ 5, 6, 7, 8, 9, 10, 11, 12, 1],
[ 6, 7, 8, 9, 10, 11, 12, 13, 0],
[ 7, 8, 9, 10, 11, 12, 13, 14, 1]], dtype=int64)]
>>> data = np.asarray(data)
>>> data.shape
(5, 3, 9)
>>> data_reshape = data.reshape(5,9,3)
>>> data_reshape
array([[[ 1, 2, 3],
[ 4, 5, 6],
[ 7, 8, 1],
[ 2, 3, 4],
[ 5, 6, 7],
[ 8, 9, 0],
[ 3, 4, 5],
[ 6, 7, 8],
[ 9, 10, -1]],
[[ 2, 3, 4],
[ 5, 6, 7],
[ 8, 9, 0],
[ 3, 4, 5],
[ 6, 7, 8],
[ 9, 10, -1],
[ 4, 5, 6],
[ 7, 8, 9],
[10, 11, -1]],
[[ 3, 4, 5],
[ 6, 7, 8],
[ 9, 10, -1],
[ 4, 5, 6],
[ 7, 8, 9],
[10, 11, -1],
[ 5, 6, 7],
[ 8, 9, 10],
[11, 12, 1]],
[[ 4, 5, 6],
[ 7, 8, 9],
[10, 11, -1],
[ 5, 6, 7],
[ 8, 9, 10],
[11, 12, 1],
[ 6, 7, 8],
[ 9, 10, 11],
[12, 13, 0]],
[[ 5, 6, 7],
[ 8, 9, 10],
[11, 12, 1],
[ 6, 7, 8],
[ 9, 10, 11],
[12, 13, 0],
[ 7, 8, 9],
[10, 11, 12],
[13, 14, 1]]], dtype=int64)
I was willing to have the series as:
array([[[1,2,3],
[2,3,4],
[3,4,5],
[4,5,6],
[5,6,7],
[6,7,8],
[7,8,9],
[8,9,10],
[1,0,-1]],
[[2,3,4],
[3,4,5],
[4,5,6],
[5,6,7],
[6,7,8],
[7,8,9],
[8,9,10],
[9,10,11],
[0,-1,-1]],
[[3,4,5],
[4,5,6],
[5,6,7],
[6,7,8],
[7,8,9],
[8,9,10],
[9,10,11],
[10,11,12],
[-1,-1,1]],
[[4,5,6],
[5,6,7],
[6,7,8],
[7,8,9],
[8,9,10],
[9,10,11],
[10,11,12],
[11,12,13],
[-1,1,0]],
[[5,6,7],
[6,7,8],
[7,8,9],
[8,9,10],
[9,10,11],
[10,11,12],
[11,12,13],
[12,13,14],
[1,0,1]]], dtype=int64)
Kindly, help me achieve this.

I have tried the data you have supplied in the question. I understood what you wanted to have. See the following:
>>> import pandas as pd
>>> import numpy as np
>>> df = pd.read_csv("test.csv")
>>> df
input1 input2 input3 input4 input5 input6 input7 input8 output
0 1 2 3 4 5 6 7 8 1
1 2 3 4 5 6 7 8 9 0
2 3 4 5 6 7 8 9 10 -1
3 4 5 6 7 8 9 10 11 -1
4 5 6 7 8 9 10 11 12 1
5 6 7 8 9 10 11 12 13 0
6 7 8 9 10 11 12 13 14 1
>>> seq_len=3
>>> data = []
>>> data_raw = df.values
>>> for index in range(len(data_raw) - seq_len + 1):
... data.append(data_raw[index: index + seq_len].T)
...
>>> data
[array([[ 1, 2, 3],
[ 2, 3, 4],
[ 3, 4, 5],
[ 4, 5, 6],
[ 5, 6, 7],
[ 6, 7, 8],
[ 7, 8, 9],
[ 8, 9, 10],
[ 1, 0, -1]], dtype=int64), array([[ 2, 3, 4],
[ 3, 4, 5],
[ 4, 5, 6],
[ 5, 6, 7],
[ 6, 7, 8],
[ 7, 8, 9],
[ 8, 9, 10],
[ 9, 10, 11],
[ 0, -1, -1]], dtype=int64), array([[ 3, 4, 5],
[ 4, 5, 6],
[ 5, 6, 7],
[ 6, 7, 8],
[ 7, 8, 9],
[ 8, 9, 10],
[ 9, 10, 11],
[10, 11, 12],
[-1, -1, 1]], dtype=int64), array([[ 4, 5, 6],
[ 5, 6, 7],
[ 6, 7, 8],
[ 7, 8, 9],
[ 8, 9, 10],
[ 9, 10, 11],
[10, 11, 12],
[11, 12, 13],
[-1, 1, 0]], dtype=int64), array([[ 5, 6, 7],
[ 6, 7, 8],
[ 7, 8, 9],
[ 8, 9, 10],
[ 9, 10, 11],
[10, 11, 12],
[11, 12, 13],
[12, 13, 14],
[ 1, 0, 1]], dtype=int64)]
>>> data = np.asarray(data)
>>> data.shape
(5, 9, 3)
Hope this is what you wanted to achieve. :)

Numpy index, get bands of width 2

I am wondering if there is a way it index/slice a numpy array, such that one can get every other band of 2 elements. In other words, given:
test = np.array([[1,2,3,4,5,6,7,8],[9,10,11,12,13,14,15,16]])
I would like to get the array:
[[1, 2, 5, 6],
[9, 10, 13, 14]]
Thoughts on how this can be accomplished with slicing/indexing?

Not that difficult with a few smart reshapes :)
test.reshape((4, 4))[:, :2].reshape((2, 4))

Given:
>>> test
array([[ 1, 2, 3, 4, 5, 6, 7, 8],
[ 9, 10, 11, 12, 13, 14, 15, 16]])
You can do:
>>> test.reshape(-1,2)[::2].reshape(-1,4)
array([[ 1, 2, 5, 6],
[ 9, 10, 13, 14]])
Which works even for different shapes of initial arrays:
>>> test2
array([ 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16])
>>> test2.reshape(-1,2)[::2].reshape(-1,4)
array([[ 1, 2, 5, 6],
[ 9, 10, 13, 14]])
>>> test3
array([[ 1, 2, 3, 4],
[ 5, 6, 7, 8],
[ 9, 10, 11, 12],
[13, 14, 15, 16]])
>>> test3.reshape(-1,2)[::2].reshape(-1,4)
array([[ 1, 2, 5, 6],
[ 9, 10, 13, 14]])
How it works:
1. Reshape into two columns by however many rows:
>>> test.reshape(-1,2)
array([[ 1, 2],
[ 3, 4],
[ 5, 6],
[ 7, 8],
[ 9, 10],
[11, 12],
[13, 14],
[15, 16]])
2. Stride the array by stepping every second element
>>> test.reshape(-1,2)[::2]
array([[ 1, 2],
[ 5, 6],
[ 9, 10],
[13, 14]])
3. Set the shape you want of 4 columns, however many rows:
>>> test.reshape(-1,2)[::2].reshape(-1,4)
array([[ 1, 2, 5, 6],
[ 9, 10, 13, 14]])

How to invert the values of a two-dimensional matrix by using slicing in NumPy?

To reverse the last row is the first, we can write:
import numpy as np
a = np.arange(20)
a = a.reshape(4,5)
c = a[::-1,:]
print c
c:
[[15 16 17 18 19]
[10 11 12 13 14]
[ 5 6 7 8 9]
[ 0 1 2 3 4]]
But how does the slicing reverse use the last column last line be the first before?
I got just a line this way. But how do I arrive until the beginning of the matrix with this statement?
a[-1, -1::-1]
a:
[19 18 17 16 15]

You can reverse both the rows and columns of the 2D array by using the slice ::-1 in each axis:
>>> a[::-1, ::-1]
array([[19, 18, 17, 16, 15],
[14, 13, 12, 11, 10],
[ 9, 8, 7, 6, 5],
[ 4, 3, 2, 1, 0]])

A couple of other ways:
Reverse before making it 2d:
In [928]: np.arange(20)[::-1].reshape(4,5)
Out[928]:
array([[19, 18, 17, 16, 15],
[14, 13, 12, 11, 10],
[ 9, 8, 7, 6, 5],
[ 4, 3, 2, 1, 0]])
Reverse the values and copy them back in with flat.
In [929]: a=np.arange(20).reshape(4,5)
In [930]: a.flat[::-1]
Out[930]:
array([19, 18, 17, 16, 15, 14, 13, 12, 11, 10, 9, 8, 7, 6, 5, 4, 3,
2, 1, 0])
In [931]: a.flat[:]=a.flat[::-1]
In [932]: a
Out[932]:
array([[19, 18, 17, 16, 15],
[14, 13, 12, 11, 10],
[ 9, 8, 7, 6, 5],
[ 4, 3, 2, 1, 0]])

Categories