Remove all columns matching a value in Numpy - python

Let's suppose I have a matrix with a number of binary values:
matrix([[1., 1., 1., 0., 0.],
[0., 0., 1., 1., 1.],
[0., 0., 0., 1., 0.],
[0., 0., 0., 0., 1.]])
Using np.sum(M, 0) produces:
matrix([[1., 1., 2., 2., 2.]])
How do I remove all of the columns from the matrix that have only the value of 1?


Easier to have an array here:
M = M.A
Now using simple slicing:
M[:, np.sum(M, 0)!=1]
array([[1., 0., 0.],
[1., 1., 1.],
[0., 1., 0.],
[0., 0., 1.]])


You can convert the matrix to array. Then find the index with values 1 and then use those indexes to delete the values. For example you can do the following.
import numpy as np
M = np.matrix([[1, 1, 1, 0, 0], [0, 0, 1, 1, 1], [0, 0, 0, 1, 0], [0, 0, 0, 0, 1]])
M = np.sum(M, 0)
# conversion to array
array = np.squeeze(np.asarray(M))
index_of_elements_with_value_1 = [i for i, val in enumerate(array) if val == 1]
array = np.delete(array, index_of_elements_with_value_1)
print(array)

Related

Add width to a numpy 1d "signal array "

I have a numpy int 1D array. Which looks like this:
[0,0,0,0,0,1,0,0,0,0,2,0,0,0,0,5,0,0,0,1,0,0,0,0,0,0,0,0,0,0]
Basically, it's an array of mostly zeros with some signals that are ints [1,2,3,4,5,...] and the signals always have a "width" of 1, meaning they are surrounded by 0s.
I want to add "width" to each signal so instead of taking only 1 space in the array it would take width space in the array.
So, in this example with the width of 3, I would get
[0,0,0,0,1,1,1,0,0,2,2,2,0,0,5,5,5,0,1,1,1,0,0,0,0,0,0,0,0,0]
The length of the array stays the same, the width can be 3,5,7, but nothing too outrageous.
What would be the fastest way to do this? I feel like there probably is an easy way to do this, but not sure how to correctly call this operation.

Convolution might be what you're looking for?
>>> import numpy as np
>>> width = 3
>>> a = np.array([0,0,0,0,0,1,0,0,0,0,2,0,0,0,0,5,0,0,0,1,0,0,0,0,0,0,0,0,0,0])
>>> np.convolve(a, np.ones(width))
array([0., 0., 0., 0., 0., 1., 1., 1., 0., 0., 2., 2., 2., 0., 0., 5., 5.,
5., 0., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])
This does not preserve the length of the array though. If you want to preserve the length, you should use the 'same' mode as such:
>>> np.convolve(a, np.ones(width), mode='same')
array([0., 0., 0., 0., 1., 1., 1., 0., 0., 2., 2., 2., 0., 0., 5., 5., 5.,
0., 1., 1., 1., 0., 0., 0., 0., 0., 0., 0., 0., 0.])
If this is not fast enough, I suggest you take a look at scipy.signal.fftconvolve.

I know it's not the perfect solution but here it is:
I made a duplicate of the intial list and created a width range so when I find a number diffrent than 0 I replace the surrounding zeros with the appropriate number
arr = [0,0,0,0,0,1,0,0,0,0,2,0,0,0,0,5,0,0,0,1,0,0,0,0,0,0,0,0,0,0]
arr1 = [0,0,0,0,0,1,0,0,0,0,2,0,0,0,0,5,0,0,0,1,0,0,0,0,0,0,0,0,0,0]
width = 3
width_range = [i for i in range(width//(-2)+1,width//(2)+1)]
print('width_range: ',width_range)
for idx,elem in enumerate(arr):
if elem !=0:
for i in width_range:
arr1[idx+i]=elem
print(arr1)
Output:
width_range: [-1, 0, 1]
[0, 0, 0, 0, 1, 1, 1, 0, 0, 2, 2, 2, 0, 0, 5, 5, 5, 0, 1, 1, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0]
PS: This example only works with 3 and 5 if you want to test it with 7 you need to add zeros between your signals.

numpy fill ndarray with values of a 1D array

I want to fill Ndarray x with values from array b along dimension i without using a for loop. This snippet of code is what I'm currently using but it's not that fast. Is there a better way?
for i in range(len(b)):
x[...,i,:,:] = b[i]
Edit 1: It's almost what I'm looking for but for higher dimensions it doesn't seem to work. x has a dimension of 8 and it's important that the shape of the Ndarray remains the same. Any more ideas?
import numpy as np
x = np.ones((2,3,4))
b = np.arange(3)
for i in range(len(b)):
x[:,i,:] = b[i]
x
Out[5]:
array([[[0., 0., 0., 0.],
[1., 1., 1., 1.],
[2., 2., 2., 2.]],
[[0., 0., 0., 0.],
[1., 1., 1., 1.],
[2., 2., 2., 2.]]])
y = np.tile(b,(4,1,2)).T
y
Out[7]:
array([[[0, 0, 0, 0]],
[[1, 1, 1, 1]],
[[2, 2, 2, 2]],
[[0, 0, 0, 0]],
[[1, 1, 1, 1]],
[[2, 2, 2, 2]]])
Edit 2: This seems to do the job
z[...] = b.reshape(1,-1,1)
z
Out[20]:
array([[[0., 0., 0., 0.],
[1., 1., 1., 1.],
[2., 2., 2., 2.]],
[[0., 0., 0., 0.],
[1., 1., 1., 1.],
[2., 2., 2., 2.]]])

There is a faster way. You can reshape b to add new dimensions and get the advantages of numpy broadcasting rules:
x[...,:,:,:] = b.reshape(-1,1,1)
Here I am assuming that b is a vector.
Another equivalent way to create new dimensions is as the following code indicates:
x[...,:,:,:] = b[:, np.newaxis, np.newaxis]

Depending on the shape of your destination array you can do something like this
>>> import numpy as np
>>> x = np.ones((4,8))
>>> x
array([[1., 1., 1., 1., 1., 1., 1., 1.],
[1., 1., 1., 1., 1., 1., 1., 1.],
[1., 1., 1., 1., 1., 1., 1., 1.],
[1., 1., 1., 1., 1., 1., 1., 1.]])
>>> b = np.arange(4)
>>> b
array([0, 1, 2, 3])
>>> x[:,1] = b
>>> x
array([[1., 0., 1., 1., 1., 1., 1., 1.],
[1., 1., 1., 1., 1., 1., 1., 1.],
[1., 2., 1., 1., 1., 1., 1., 1.],
[1., 3., 1., 1., 1., 1., 1., 1.]])
In this example we assigned b to column 1 of the 2D array x
If instead you are trying to repeat b a certain number of times you can use np.tile
>>> x = np.tile(b, (8,1)).T
>>> x
array([[0, 0, 0, 0, 0, 0, 0, 0],
[1, 1, 1, 1, 1, 1, 1, 1],
[2, 2, 2, 2, 2, 2, 2, 2],
[3, 3, 3, 3, 3, 3, 3, 3]])

just fill specific entrys of numpy array -> generating sparse matrix

I want to code a really big matrix with the following structure:
a = np.array([[1, 1, 1, 0, 0 ,0, 0, 0, 0],
[0, 0, 0, 1, 1, 1, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 1, 1, 1]])
Dimension of this array is (3,9) so basically the 1's depend on the dimension.
In my first row the first 3 entrys should be 1's, in my second row the entrys 3-5 should be 1's,
and so on...
How do I code this?

You can use the kronecker tensor product:
a = np.kron(np.eye(3),np.ones((1,3)))
# array([[1., 1., 1., 0., 0., 0., 0., 0., 0.],
# [0., 0., 0., 1., 1., 1., 0., 0., 0.],
# [0., 0., 0., 0., 0., 0., 1., 1., 1.]])

Smoothing one-hot encoded matrix rows

Assuming that I have the following matrix consisting of one-hot encoded rows:
X = np.array([[0., 0., 0., 1., 0.], [1., 0., 0., 0., 0.], [0., 0., 1., 0., 0.]])
What I aim to do is smooth/expand the one-hot encoding in a way such that I will obtain the following output:
Y = np.array([[0., 0., 1., 1., 1.], [1., 1., 0., 0., 0.], [0., 1., 1., 1., 0.]])
assuming that I want to smooth/expand 1 element to the left or the right of the one-hot element. Thank you for the help!

We can use convolution -
In [22]: from scipy.signal import convolve2d
In [23]: convolve2d(X,np.ones((1,3)),'same')
Out[23]:
array([[0., 0., 1., 1., 1.],
[1., 1., 0., 0., 0.],
[0., 1., 1., 1., 0.]])
With binary-dilation to be more memory-efficient -
In [43]: from scipy.ndimage.morphology import binary_dilation
In [46]: binary_dilation(X,np.ones((1,3), dtype=bool)).view('i1')
Out[46]:
array([[0, 0, 1, 1, 1],
[1, 1, 0, 0, 0],
[0, 1, 1, 1, 0]], dtype=int8)
Or since we only 0s and 1s, uniform filter would also work and additionally we can use it along a generic axis (axis=1 in our case) and should be better on perf. -
In [47]: from scipy.ndimage import uniform_filter1d
In [50]: (uniform_filter1d(X,size=3,axis=1)>0).view('i1')
Out[50]:
array([[0, 0, 1, 1, 1],
[1, 1, 0, 0, 0],
[0, 1, 1, 1, 0]], dtype=int8)

You could convolve X with an array of ones:
from scipy.signal import convolve2d
convolve2d(X, np.ones((1,3)), mode='same')
array([[0., 0., 1., 1., 1.],
[1., 1., 0., 0., 0.],
[0., 1., 1., 1., 0.]])

Solution based on standard np.convolve:
import numpy as np
np.array([np.convolve(x, np.array([1,1,1]), mode='same') for x in X])
Iterate rows using list comprehension to convolve, then convert back to np.array

Is there a way to do Pytorch element wise equality treating each dimension as an element?

I have two tensors and I want to check for equality treating an array in one dimension as the element
I have 2 tensors
lo = torch.Tensor(([1., 1., 0.],
[0., 1., 1.],
[0., 0., 0.],
[1., 1., 1.]))
lo = torch.Tensor(([1., 1., 0.],
[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]))
I've tried using
torch.eq(lee, lo)
which returns a tensor like
tensor([[1, 1, 1],
[1, 0, 0],
[1, 1, 1],
[0, 0, 0]], dtype=torch.uint8)
Is there a way to have the output become
tensor([1, 0, 1, 0])
as the only complete element that matches is the first?
edit:
I've come up with this solution
lee = lee.tolist()
lo = lo.tolist()
out = []
for i, j in enumerate(lee):
if j == lo[i]:
out.append(1)
else:
out.append(0)
and out will be [1, 0, 1, 0]
But is there an easier way?

You can simply use torch.all(tensor, dim).
code:
l1 = torch.Tensor(([1., 1., 0.],
[0., 1., 1.],
[0., 0., 0.],
[1., 1., 1.]))
l2 = torch.Tensor(([1., 1., 0.],
[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]))
print(torch.eq(l1, l2))
print(torch.all(torch.eq(l1, l2), dim=0)) # equivalent to dim = -2
print(torch.all(torch.eq(l1, l2), dim=1)) # equivalent to dim = -1
output:
tensor([[1, 1, 1],
[1, 0, 0],
[1, 1, 1],
[0, 0, 0]], dtype=torch.uint8)
tensor([0, 0, 0], dtype=torch.uint8)
tensor([1, 0, 1, 0], dtype=torch.uint8) # your desired output

Or take torch.eq(lee, lo) and row must summ to its len , means all 1 must be there
import torch
lo = torch.Tensor(([1., 1., 0.],
[0., 1., 1.],
[0., 0., 0.],
[1., 1., 1.]))
l1 = torch.Tensor(([1., 1., 0.],
[0., 0., 0.],
[0., 0., 0.],
[0., 0., 0.]))
teq = torch.eq(l1, lo)
print(teq)
tsm = teq.sum(-1)
print(tsm == 3)
tsm is tensor([3, 1, 3, 0])
printout returns [1, 0, 1, 0]

Categories