We Keep Coding

Python, Numpy: How to change two vectors at once

I'm trying to change two vectors of a NumPy matrix at once, but I'm losing one vector components:
import numpy as np
data = np.array([[1, 2, 3, 4],[1, 2, 3, 4],[1, 2, 3, 4],[1, 2, 3, 4]])
last = data[:, -1]
print(last)
data[:, 1:] = data[:, :-1]
data[:, 0] = last
print(data)
Gives this result:
[4 4 4 4]
[[3 1 2 3]
[3 1 2 3]
[3 1 2 3]
[3 1 2 3]]
But I want to maintain the 4s in the first column. Is there any form to accomplish that?

Use:
>>> np.roll(data, 1, 1)
array([[4, 1, 2, 3],
[4, 1, 2, 3],
[4, 1, 2, 3],
[4, 1, 2, 3]])

Split up numpy array

I've been looking all over but I'm not really sure how to even describe what it is I want. Essentially I need to turn
np.array(
[[0,0, 1,1, 2,2],
[0,0, 1,1, 2,2],
[3,3, 4,4, 5,5],
[3,3, 4,4, 5,5]]
)
into
np.array(
[[[0,1,2,3,4,5], [0,1,2,3,4,5]],
[[0,1,2,3,4,5], [0,1,2,3,4,5]]
)
I think I can accomplish that using np.reshape and maybe some other stuff but if I try reshape with arguments (2,2,6) I get back
[[[0 0 1 1 2 2]
[0 0 1 1 2 2]]
[[3 3 4 4 5 5]
[3 3 4 4 5 5]]]
which is not quite what I want.

Make your array with a couple of repeats:
In [208]: arr = np.arange(0,6).reshape(2,3)
In [209]: arr
Out[209]:
array([[0, 1, 2],
[3, 4, 5]])
In [210]: arr = arr.repeat(2,0).repeat(2,1)
In [211]: arr
Out[211]:
array([[0, 0, 1, 1, 2, 2],
[0, 0, 1, 1, 2, 2],
[3, 3, 4, 4, 5, 5],
[3, 3, 4, 4, 5, 5]])
Now break it into blocks which we can transpose:
In [215]: arr1 = arr.reshape(2,2,3,2)
In [216]: arr1
Out[216]:
array([[[[0, 0],
[1, 1],
[2, 2]],
[[0, 0],
[1, 1],
[2, 2]]],
[[[3, 3],
[4, 4],
[5, 5]],
[[3, 3],
[4, 4],
[5, 5]]]])
In [217]: arr1.shape
Out[217]: (2, 2, 3, 2)
In [218]: arr1.transpose(1,0,2,3)
Out[218]:
array([[[[0, 0],
[1, 1],
[2, 2]],
[[3, 3],
[4, 4],
[5, 5]]],
[[[0, 0],
[1, 1],
[2, 2]],
[[3, 3],
[4, 4],
[5, 5]]]])
Let's consolidate the middle 2 axes:
In [220]: arr1.transpose(1,0,2,3).reshape(2,6,2)
Out[220]:
array([[[0, 0],
[1, 1],
[2, 2],
[3, 3],
[4, 4],
[5, 5]],
[[0, 0],
[1, 1],
[2, 2],
[3, 3],
[4, 4],
[5, 5]]])
Almost there; just need another transpose:
In [221]: arr1.transpose(1,0,2,3).reshape(2,6,2).transpose(0,2,1)
Out[221]:
array([[[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5]],
[[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5]]])
The basic idea is to reshape the array into blocks, do a transpose, and reshape again. Here I needed another transpose, but if I choose the right one to start with I might not have needed that.
I don't know of a systematic way of doing this; there may be one, but so far I've just used a bit of trial and error when answering this kind of question. Everyone wants a different final arrangement.

This should work:
>>> import numpy as np
>>> A = np.array(
... [[0,0, 1,1, 2,2],
... [0,0, 1,1, 2,2],
... [3,3, 4,4, 5,5],
... [3,3, 4,4, 5,5]]
... )
>>> B = a[::2,::2].flatten()
>>> B
array([0, 1, 2, 3, 4, 5])
>>> C = np.tile(b, (2,2,1))
>>> C
array([[[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5]],
[[0, 1, 2, 3, 4, 5],
[0, 1, 2, 3, 4, 5]]])
We can generalize this for a given n * m matrix, that contain blocks sized n/y * m/x of identical values (so there are y rows and x columns of blocks)
def transform(A, y, x):
dy = A.shape[0]/y
dx = A.shape[1]/x
B = A[::dy, ::dx].flatten()
return np.tile(B, (y,x,1))

I think this is what you are looking for:
import numpy
b=numpy.array([[0,0,1,1,2,2],[0,0,1,1,2,2],[3,3,4,4,5,5],[3,3,4,4,5,5]])
c1=(b[::2,::2].flatten(),b[::2,1::2].flatten())
c2=(b[1::2,::2].flatten(),b[1::2,1::2].flatten())
c=numpy.vstack((c1,c2)).reshape((2,2,6))
print(c)
which outputs:
[[[0 1 2 3 4 5]
[0 1 2 3 4 5]]
[[0 1 2 3 4 5]
[0 1 2 3 4 5]]]
and for general size target array and general size input array this is the algorithm with an example of 3*3 input array:
import numpy
b=numpy.array([[0,0,1,1,2,2],[0,0,1,1,2,2],[0,0,1,1,2,2],[3,3,4,4,5,5],[3,3,4,4,5,5],[3,3,4,4,5,5]])
(m,n)=b.shape
C=b[::int(m/2),::2].flatten(),b[::int(m/2),1::2].flatten()
for i in range(1,int(m/2)):
C=numpy.vstack((C,(b[i::int(m/2),::2].flatten(),b[i::int(m/2),1::2].flatten())))
print(C)
which outputs:
[[0 1 2 3 4 5]
[0 1 2 3 4 5]
[0 1 2 3 4 5]
[0 1 2 3 4 5]
[0 1 2 3 4 5]
[0 1 2 3 4 5]]

How to get the difference between two sparse matrices?

I have two sparse matrix, and want to get the difference between the two sparse matrix.
import pandas as pd
import numpy as np
from scipy.sparse import csr_matrix,find
row = np.array([0, 0, 1, 2, 2, 2])
col = np.array([0, 2, 2, 0, 1, 2])
data = np.array([1, 2, 3, 4, 5, 6])
sparse_matrix = csr_matrix((data, (row, col)), shape=(3, 3))
print(sparse_matrix.toarray())
row2= np.array([0, 1, 1, 0, 1])
col2= np.array([0, 0, 1, 2, 2])
data2= np.array([1, 4, 5, 2, 6])
sparse_matrix2 =csr_matrix((data2,(row2, col2)),shape=(2,3))
print(sparse_matrix2.toarray())
Output:
[[1 0 2]
[0 0 3]
[4 5 6]]
[[1 0 2]
[4 5 6]]
Expected:
I want to get a sparse matrix, and it's dense matrix is shown as follow.
[[0 0 3]]

numpy matrix. move all 0's to the end of each row

Given a matrix in python numpy which has for some of its rows, leading zeros. I need to shift all zeros to the end of the line.
E.g.
0 2 3 4
0 0 1 5
2 3 1 1
should be transformed to
2 3 4 0
1 5 0 0
2 3 1 1
Is there any nice way to do this in python numpy?

To fix for leading zeros rows -
def fix_leading_zeros(a):
mask = a!=0
flipped_mask = mask[:,::-1]
a[flipped_mask] = a[mask]
a[~flipped_mask] = 0
return a
To push all zeros rows to the back -
def push_all_zeros_back(a):
# Based on http://stackoverflow.com/a/42859463/3293881
valid_mask = a!=0
flipped_mask = valid_mask.sum(1,keepdims=1) > np.arange(a.shape[1]-1,-1,-1)
flipped_mask = flipped_mask[:,::-1]
a[flipped_mask] = a[valid_mask]
a[~flipped_mask] = 0
return a
Sample runs -
In [220]: a
Out[220]:
array([[0, 2, 3, 4],
[0, 0, 1, 5],
[2, 3, 1, 1]])
In [221]: fix_leading_zero_rows(a)
Out[221]:
array([[2, 3, 4, 0],
[1, 5, 0, 0],
[2, 3, 1, 1]])
In [266]: a
Out[266]:
array([[0, 2, 3, 4, 0],
[0, 0, 1, 5, 6],
[2, 3, 0, 1, 0]])
In [267]: push_all_zeros_back(a)
Out[267]:
array([[2, 3, 4, 0, 0],
[1, 5, 6, 0, 0],
[2, 3, 1, 0, 0]])

leading zeros, simple loop
ar = np.array([[0, 2, 3, 4],
[0, 0, 1, 5],
[2, 3, 1, 1]])
for i in range(ar.shape[0]):
for j in range(ar.shape[1]): # prevent infinite loop if row all zero
if ar[i,0] == 0:
ar[i]=np.roll(ar[i], -1)
ar
Out[31]:
array([[2, 3, 4, 0],
[1, 5, 0, 0],
[2, 3, 1, 1]])

Applying a mask to an multidimensional array

I want to do this in a proper way:
data = np.array(data)
data =[
[1, 1, 2, 1],
[0, 1, 3, 2],
[0, 2, 3, 2],
[2, 4, 3, 1],
[0, 2, 1, 4],
[3, 1, 4, 1]]
this should become (delete the lines that start with 0):
[1, 1, 2, 1]
[2, 4, 3, 1]
[3, 1, 4, 1]
So far I did it like this:
lines = []
for i in range(0, len(data[0])):
if data[0,i] != 0:
lines.append(data[:,i])
lines = np.array(lines)
Then I found this fine method:
mask = 1 <= data[0,:]
and now I want to apply that mask to that array. This Mask reads: [True, False, False, True, False, True]. How do I do that?

Why not just:
[ar for ar in data if ar[0] != 0]
This assumes that arrays are not empty.

I presume you have a numpy array based on the data[0,:] and data[0,i] you have in your question and you mean data[:, 0] :
import numpy as np
data = np.array([
[1, 1, 2, 1],
[0, 1, 3, 2],
[0, 2, 3, 2],
[2, 4, 3, 1],
[0, 2, 1, 4],
[3, 1, 4, 1]])
data = data[data[:,0] != 0]
print(data)
Output:
[[1 1 2 1]
[2 4 3 1]
[3 1 4 1]]
data[0,:] is the first row [1 1 2 1] not the first column

Using List comprehension
In [56]: [elem for elem in data if elem[0] !=0]
Out[56]: [[1, 1, 2, 1], [2, 4, 3, 1], [3, 1, 4, 1]]