This is likely a repost but I'm not sure what wording to use for the title.
I'm trying to subtract the values of arrays inside arrays by reshaping them to create a larger array.
xn = np.array([[1,2,3],[4,5,6]])
yn = np.array(([1,2,3,4,5], [6,7,8,9,10]])
xn.shape
Out[42]: (2, 3)
yn.shape
Out[43]: (2, 5)
The functionality I want is:
yn.reshape(2,-1,1) - xn
This throws a value error, but the below works just fine when I remove the first dimension as a factor:
yn.reshape(2,-1,1)[0] - xn[0]
Out[44]:
array([[ 0, -1, -2],
[ 1, 0, -1],
[ 2, 1, 0],
[ 3, 2, 1],
[ 4, 3, 2]])
Which would be the first output I would expect because xn and yn both have a first dimension of 2.
Is there a proper way to do this with the desired broadcasting?
Desired output:
array([[[ 0, -1, -2],
[ 1, 0, -1],
[ 2, 1, 0],
[ 3, 2, 1],
[ 4, 3, 2]],
[[2, 1, 0],
[3, 2, 1],
[4, 3, 2],
[5, 4, 3],
[6, 5, 4]]])
>>> x
array([[1, 2, 3],
[4, 5, 6]])
>>> y
array([[ 1, 2, 3, 4, 5],
[ 6, 7, 8, 9, 10]])
>>> z = y.reshape(2,-1,1)
Add another axis to x:
>>> z-x[:,None,:]
array([[[ 0, -1, -2],
[ 1, 0, -1],
[ 2, 1, 0],
[ 3, 2, 1],
[ 4, 3, 2]],
[[ 2, 1, 0],
[ 3, 2, 1],
[ 4, 3, 2],
[ 5, 4, 3],
[ 6, 5, 4]]])
>>>
Or just:
>>> y[...,None] - x[:,None,:]
array([[[ 0, -1, -2],
[ 1, 0, -1],
[ 2, 1, 0],
[ 3, 2, 1],
[ 4, 3, 2]],
[[ 2, 1, 0],
[ 3, 2, 1],
[ 4, 3, 2],
[ 5, 4, 3],
[ 6, 5, 4]]])
From broadcasting rules, to be able to broadcast the shapes must be equal or one of them needs to be equal to 1 (starting from trailing dimensions and moving forward). So swapping two last dimensions of xn will allow you to broadcast (after adding another dimension to xn):
yn.reshape(2, -1, 1) - xn.reshape(2, -1, 1).swapaxes(-1, -2)
array([[[ 0, -1, -2],
[ 1, 0, -1],
[ 2, 1, 0],
[ 3, 2, 1],
[ 4, 3, 2]],
[[ 2, 1, 0],
[ 3, 2, 1],
[ 4, 3, 2],
[ 5, 4, 3],
[ 6, 5, 4]]])
The shape of yn.reshape(2, -1, 1) is (2, 5, 1) and the shape of xn.reshape(2, -1, 1).swapaxes(-1, -2) is (2, 1, 3). Now you can broadcast because the dimensions are equal or one of them is equal one by element-wise comparison starting from trailing dimensions.
Related
I have a numpy ndarray as follows"
x = np.array([[1, 2, 3], [4, 5, 6],[4, 1, 6],[1, 5, 11],[4,3, 4]], np.int32)
i set the value to zero if index with same col, row
rows = x.shape[0]
cols = x.shape[1]
for k in range(0, rows):
for y in range(0, cols):
if k==y:
x[k,y]=0
expected output:
array([[ 0, 2, 3],
[ 4, 0, 6],
[ 4, 1, 0],
[ 1, 5, 11],
[ 4, 3, 4]])
Is there any simple (pythonic way to achieve the same results)? my actual matrix is very large...
Use np.fill_diagonal:
x = np.array([[1, 2, 3], [4, 5, 6],[4, 1, 6],[1, 5, 11],[4,3, 4]], np.int32)
np.fill_diagonal(x,0)
print(x)
array([[ 0, 2, 3],
[ 4, 0, 6],
[ 4, 1, 0],
[ 1, 5, 11],
[ 4, 3, 4]], dtype=int32)
Let's say I have the following array
import numpy as np
matrix = np.array([
[[1, 2, 3, 4], [0, 1], [2, 3, 4, 5]],
[[1, 2, 3], [4], [0, 1], [2, 0], [0, 0]],
[[2, 2], [3, 4, 0], [1, 1, 0, 0], [0]],
[[6, 3, 3, 4, 0], [4, 2, 3, 4, 5]],
[[1, 2, 3, 2], [0, 1, 2], [3, 4, 5]]])
As you can see, it's a staggered array. What I want to do is to sum the elements in a way so that the output is:
[11, 11, 15, 18, 0, 8, 9, 9, 12, 15]
I want to sum the elements in the "columns" of the matrix, but I don't know how to do it.
As mentioned by juanpa.arrivillaga in the comments, you don't have a multi-dimensional array, you have a 1-D array of lists of lists. You need to flatten the inner lists first :
>>> np.array([[z for y in x for z in y] for x in matrix])
array([[1, 2, 3, 4, 0, 1, 2, 3, 4, 5],
[1, 2, 3, 4, 0, 1, 2, 0, 0, 0],
[2, 2, 3, 4, 0, 1, 1, 0, 0, 0],
[6, 3, 3, 4, 0, 4, 2, 3, 4, 5],
[1, 2, 3, 2, 0, 1, 2, 3, 4, 5]])
It should be much easier to solve your problem now. This matrix has a shape of (5,10), and supports T for transposition and np.sum() for summing rows or columns.
You didn't write any code, so I won't solve the problem completely, but with this matrix, you're one step away from:
array([11, 11, 15, 18, 0, 8, 9, 9, 12, 15])
This is my current matrix:
[[0, 1, 2, 4],
[0, 3, 1, 3],
[0, 2, 3, 2],
[0, 2, 4, 1],
[0, 4, 1, 2],
[0, 3, 2, 2],
[1, 2, 2, 2]]
I want to transpose it and get this as output:
[[0, 0, 0, 0, 1],
[2, 2, 4, 3, 2],
[3, 4, 1, 2, 2],
[2, 1, 2, 2, 2]]
I used inverse = np.swapaxes(ate,0,7) but I am not sure what will be my axis2 value be. Here the axis2 is 7.
I think what you're looking for is np.transpose()
You can use np.swapaxes, however this swaps "dimensions", so for a matrix that's either 0 or 1 because you have two dimensions:
>>> np.swapaxes(arr, 0, 1) # assuming your matrix is called arr
array([[0, 0, 0, 0, 0, 0, 1],
[1, 3, 2, 2, 4, 3, 2],
[2, 1, 3, 4, 1, 2, 2],
[4, 3, 2, 1, 2, 2, 2]])
To get your desired output you'd need to remove the first two columns before the swapaxes:
>>> np.swapaxes(arr[2:], 0, 1)
array([[0, 0, 0, 0, 1],
[2, 2, 4, 3, 2],
[3, 4, 1, 2, 2],
[2, 1, 2, 2, 2]])
However generally you should use np.transpose or .T if you want to transpose the matrix/array:
>>> arr[2:].T
array([[0, 0, 0, 0, 1],
[2, 2, 4, 3, 2],
[3, 4, 1, 2, 2],
[2, 1, 2, 2, 2]])
If i have an numpy array like:
x= [[3, 3], [2, 2]]
I want to add an element -1 to the end of each the rows to be like this:
x= [[3, 3, -1], [2, 2, -1]]
any simple way to do that ?
A simple way would be with np.insert -
np.insert(x,x.shape[1],-1,axis=1)
We can also use np.column_stack -
np.column_stack((x,[-1]*x.shape[0]))
Sample run -
In [161]: x
Out[161]:
array([[0, 8, 7, 0, 1],
[0, 1, 8, 6, 8],
[3, 4, 7, 0, 2]])
In [162]: np.insert(x,x.shape[1],-1,axis=1)
Out[162]:
array([[ 0, 8, 7, 0, 1, -1],
[ 0, 1, 8, 6, 8, -1],
[ 3, 4, 7, 0, 2, -1]])
In [163]: np.column_stack((x,[-1]*x.shape[0]))
Out[163]:
array([[ 0, 8, 7, 0, 1, -1],
[ 0, 1, 8, 6, 8, -1],
[ 3, 4, 7, 0, 2, -1]])
I have an interesting puzzle. Suppose you have a numpy 2D array, in which each line corresponds to a measurement event and each column corresponds to different measured variable. One additional column in this array specifies the date at which the measurement was taken. The lines are sorted according to the time stamp. There are several (or many) measurements on each day. The goal is to identify the lines that correspond to a new day and subtract the respective values from the subsequent lines in that day.
I approach this problem by a loop that loops over the days, creating a boolean vector that selects the proper lines and then subtracting the first selected line. This approach works, but feels non-elegant. Are there better ways to do this?
Just a small example. The lines below define a matrix in which the first colum
is the day and the remaining two are the measured values
before = array([[ 1, 1, 2],
[ 1, 3, 4],
[ 1, 5, 6],
[ 2, 7, 8],
[ 3, 9, 10],
[ 3, 11, 12],
[ 3, 13, 14]])
at the end of the process I expect to see the following array:
array([[1, 0, 0],
[1, 2, 2],
[1, 4, 4],
[2, 0, 0],
[3, 0, 0],
[3, 2, 2],
[3, 4, 4]])
PS Please help me finding a better and more informative title for this post. I'm out of ideas
numpy.searchsorted is a convenient function for this:
In : before
Out:
array([[ 1, 1, 2],
[ 1, 3, 4],
[ 1, 5, 6],
[ 2, 7, 8],
[ 3, 9, 10],
[ 3, 11, 12],
[ 3, 13, 14]])
In : diff = before[before[:,0].searchsorted(x[:,0])]
In : diff[:,0] = 0
In : before - diff
Out:
array([[1, 0, 0],
[1, 2, 2],
[1, 4, 4],
[2, 0, 0],
[3, 0, 0],
[3, 2, 2],
[3, 4, 4]])
Longer explanation
If you take the first column, and search for itself you get the minimum indices for those particular values:
In : before
Out:
array([[ 1, 1, 2],
[ 1, 3, 4],
[ 1, 5, 6],
[ 2, 7, 8],
[ 3, 9, 10],
[ 3, 11, 12],
[ 3, 13, 14]])
In : before[:,0].searchsorted(x[:,0])
Out: array([0, 0, 0, 3, 4, 4, 4])
You can then use this to construct the matrix that you will subtract by indexing:
In : diff = before[before[:,0].searchsorted(x[:,0])]
In : diff
Out:
array([[ 1, 1, 2],
[ 1, 1, 2],
[ 1, 1, 2],
[ 2, 7, 8],
[ 3, 9, 10],
[ 3, 9, 10],
[ 3, 9, 10]])
You need to make the first column 0 so that they won't be subtracted.
In : diff[:,0] = 0
In : diff
Out:
array([[ 0, 1, 2],
[ 0, 1, 2],
[ 0, 1, 2],
[ 0, 7, 8],
[ 0, 9, 10],
[ 0, 9, 10],
[ 0, 9, 10]])
Finally, subtract two matrices to get the desired output:
In : before - diff
Out:
array([[1, 0, 0],
[1, 2, 2],
[1, 4, 4],
[2, 0, 0],
[3, 0, 0],
[3, 2, 2],
[3, 4, 4]])