Unpad all-zero rows from the end of an array - python

I have this line:
x = np.zeros((10, 2), int)
and a for loop that fills this for the first 6 elements.
Is there a way to remove the remaining 0 elements?


If you want to remove the last all zero rows:
x = x[:(np.where(x.any(axis=1))[0]).max()+1]
example:
x:
[[1 2]
[0 0]
[1 2]
[0 0]]
output:
[[1 2]
[0 0]
[1 2]]


Try this:
x = x[:6]
Note that this solution will create 6 element long view to original array. Modifying view will modify original array.

Related

How to create list of array combinations lexographically in numpy?

I have this array and I want to return unique array combinations. I tried meshgrid but it creates duplicates and inverse array values
>> import numpy as np
>> array = np.array([0,1,2,3])
>> combinations = np.array(np.meshgrid(array, array)).T.reshape(-1,2)
>> print(combinations)
[[0 0]
[0 1]
[0 2]
[0 3]
[1 0]
[1 1]
[1 2]
[1 3]
[2 0]
[2 1]
[2 2]
[2 3]
[3 0]
[3 1]
[3 2]
[3 3]]
What I want to exclude are the repeating arrays: [0,0] [1,1] [2,2] [3,3] and the inverse arrays when [2,3] is returned exclude [3,2] in the output.
Take a look at this combination calculator, this is the output that I like but how can I create it in NumPy?

you could use combinations from itertools
import numpy as np
from itertools import combinations
array = np.array([0,1,2,3])
combs = np.array(list(combinations(arr, 2)))

All combinations of a numpy 2d array filled with 0s and 1s

Given K, I need to have all the possibile combinations of K x 2 numpy matrices so that in each matrix there are all 0s except for two 1s in different rows and columns.
Something like this for K = 5:
[[1,0],[0,1],[0,0],[0,0][0,0]]
[[1,0],[0,0],[0,1],[0,0][0,0]]
[[1,0],[0,0],[0,0],[0,1][0,0]]
[[1,0],[0,0],[0,0],[0,0][0,1]]
[[0,0],[1,0],[0,1],[0,0][0,0]]
[[0,0],[1,0],[0,0],[0,1][0,0]]
... and so on
So the resulting array should be a K x 2 x (K*(K-1)/2).
I want to avoid loops since it's not an efficient way when K is big enough (in my specific case K = 300)

I can't think of an elegant solution but here's a not-so-elegant pure numpy one:
import numpy as np
def combination_matrices(K):
# get combination indices
i, j = np.indices((K, K))
comb_indices = np.transpose((i < j).nonzero()) # (num_combs, 2) array where ones are
num_combs = comb_indices.shape[0] # K*(K-1)/2
# create a matrix of the desired shape, first axis enumerates combinations
matrices = np.zeros((num_combs, K, 2), dtype=int)
# broadcasting assignment of ones
comb_range, col_index = np.ogrid[:num_combs, :2]
matrices[comb_range, comb_indices, col_index] = 1
return matrices
This first uses the indices of a (K, K)-shaped array to find the index pairs for every combination (these are indices that encode the upper triangle of the array, excluding the diagonal). Then we use a bit tricky broadcasting assignment (heavy fancy indexing) to set each corresponding element of the pre-allocated output array to 1.
Note that I put the K*(K-1)/2-sized axis first, because this makes the most sense in numpy with C-contiguous memory layout. This way when you take the matrix for combination index 3, arr[3, ...] will be a contiguous chunk of memory of shape (K, 2) that's fast to work with in vectorised operations.
The output for K = 4:
[[[1 0]
[0 1]
[0 0]
[0 0]]
[[1 0]
[0 0]
[0 1]
[0 0]]
[[1 0]
[0 0]
[0 0]
[0 1]]
[[0 0]
[1 0]
[0 1]
[0 0]]
[[0 0]
[1 0]
[0 0]
[0 1]]
[[0 0]
[0 0]
[1 0]
[0 1]]]

This is an oddly specific question, but an interesting problem, I'd love to know what the context is?
You are looking for all permutations of a multiset , python's itertools doesn't currently support this. So simplest solution is to use the multiset tools of the sympy library.
The following code took about ~2.5 minutes to run on my machine, so is fairly fast for a single thread. You're looking at 179700 unique permutations for K=300.
(I took inspiration from https://stackoverflow.com/a/40289807/10739860)
from collections import Counter
from math import factorial, prod
import numpy as np
from sympy.utilities.iterables import multiset_permutations
from tqdm import tqdm
def No_multiset_permutations(multiset: list) -> int:
"""Calculates the No. possible permutations given a multiset.
See: https://en.wikipedia.org/wiki/Permutation#Permutations_of_multisets
:param multiset: List representing a multiset.
"""
value_counts = Counter(multiset).values()
denominator = prod([factorial(val) for val in value_counts])
return int(factorial(len(multiset)) / denominator)
def multiset_Kx2_permutations(K: int) -> np.ndarray:
"""This will generate all possible unique Kx2 permutations of an array
withsize K where two values are 1 and the rest are 0.
:param K: The size of the array.
"""
# Construct number multiset, e.g. K=5 gives [1, 1, 0, 0, 0, 0, 0, 0, 0, 0]
numbers = [1, 1] + [0] * (K - 1) * 2
# Use sympy's multiset_permutations to get a multiset permutation generator
generator = multiset_permutations(numbers)
# Calculate the No. possible permutations
number_of_perms = No_multiset_permutations(numbers)
# Get all permutations, bonus progress bar is included :)
unique_perms = [next(generator) for _ in tqdm(range(number_of_perms))]
# Reshape each permutation to Kx2
unique_perms = np.array(unique_perms, dtype=np.int8)
return unique_perms.reshape(-1, K, 2)
if __name__ == "__main__":
solution = multiset_Kx2_permutations(300)

Another possibility (with rearranged axes for clearer output):
from itertools import combinations
import numpy as np
k = 4
x = list(combinations(range(k), 2))
out = np.zeros((n := len(x), k, 2), dtype=int)
out[np.c_[:n], x, [0, 1]] = 1
print(out)
It gives:
[[[1 0]
[0 1]
[0 0]
[0 0]]
[[1 0]
[0 0]
[0 1]
[0 0]]
[[1 0]
[0 0]
[0 0]
[0 1]]
[[0 0]
[1 0]
[0 1]
[0 0]]
[[0 0]
[1 0]
[0 0]
[0 1]]
[[0 0]
[0 0]
[1 0]
[0 1]]]

How to make a list of 2D 3x3 array and save the ndarray with numpy save?

So let's say I have some 3x3 matrix I get with a calculation I am doing, let's say
np.array([[1,2,3],[4,5,6],[7,8,9]])
np.array([[0,0,0],[0,0,0],[0,0,0]])
I want to add this onto some matrix A and be able to access them so that if I do
> A[0]
> [[1,2,3],[4,5,6],[7,8,9]]
> A[1]
> [[0,0,0],[0,0,0],[0,0,0]]
and keep adding on bunch of these 2D array and save them with np.save('A', A) for fast access later. I kind of saw it is possible with appending to a list but I can't save a list with np.save for fast efficient access. How can I create a empty ndarray I can add matrix onto and save it all as .npy?

You can convert list to array it is the same:
A = list()
A.append(x)
A.append(y)
X = np.array(A)
np.save('X', X)

Originally use this:
A = np.stack((a,b))
[[[1 2 3]
[4 5 6]
[7 8 9]]
[[0 0 0]
[0 0 0]
[0 0 0]]]
And once you have formed A, if you want to add another array c to A use:
A = np.vstack((A,[c]))
output for c=a:
[[[1 2 3]
[4 5 6]
[7 8 9]]
[[0 0 0]
[0 0 0]
[0 0 0]]
[[1 2 3]
[4 5 6]
[7 8 9]]]

Remove row from a numpy array according to some condition and concatenate result

I have a list of numpy arrays and I wanted to remove a row according to some condition.
Lets suppose I have the following list of numpy arrays and I want to delete the rows which contain an item that is > 8.
test = [np.array([[2,2,4],[10,3,5],[1,2,4,],[1,2,4]]),
np.array([[1,2,3],[1,3,5],[6,3,1],[9,1,2]])]
for i in test:
z = np.argwhere(i>8)
print(z)#[[1 0]] and [[3 0]]
a1 = np.delete(i,z,axis=0)
print(a1)
This for loop skips the numpy array of index[0]. How can Ifix this?
Returns:
[[1 2 4]
[1 2 4]]
[[1 3 5]
[6 3 1]]
Desirable Return:
[[2,2,4]
[1 2 4]
[1 2 4]]
[[1,2,3]
[1 3 5]
[6 3 1]]

From your example, you want to remove row with index 1 from the first array,
and row with index 3 from the second array.
So use those indices when executing np.delete:
a1 = np.delete(i, z[0][0], axis=0)

np.argwhere will return both indices, but we're only interested in the rows:
np.argwhere(i > 8)[:, 0]
But really, we're only interested in unique rows, so we can take care of that too:
np.unique(np.argwhere(i > 8)[:, 0])
Altogether we get:
test = [np.array([[2,2,4],[10,3,5],[1,2,4,],[1,2,4]]),np.array([[1,2,3],[1,3,5],[6,3,1],[9,1,2]])]
for i in test:
z = np.unique(np.argwhere(i>8)[:, 0])
a1 = np.delete(i,z,axis=0)
print(a1)
#[[2 2 4]
# [1 2 4]
# [1 2 4]]
#[[1 2 3]
# [1 3 5]
# [6 3 1]]

python: Find subsets coordinate is in

I have a set of coordinates and try to find those subsets the coordinate is in.
import numpy as np
a=np.array([[[0,1,1],[1,1,1]],[[0,1,1],[2,1,1]],[[3,3,3],[2,2,2]]])
If I try things like:
print(np.argwhere(a==[[0,1,1]]))
print(a[[0,1,1]])
print(np.isin([0,1,1],a))
I get:
[[0 0 0]
[0 0 1]
[0 0 2]
[0 1 1]
[0 1 2]
[1 0 0]
[1 0 1]
[1 0 2]
[1 1 1]
[1 1 2]]
[[[0 1 1]
[1 1 1]]
[[0 1 1]
[2 1 1]]
[[0 1 1]
[2 1 1]]]
[ True True True]
But I expect something like:
[true,true,false]
EDIT
The best case would be If I get an array where only all other coordinates which are members of the founded subsets are in like:
out = [[1,1,1],[2,1,1]]

Use all(-1) to assert the array equal at the last axis and then any(1) to check if such condition exists at the second axis:
(a == [0,1,1]).all(-1).any(1)
# array([ True, True, False], dtype=bool)
On the update:
mask = (a == [0,1,1]).all(-1)
a[mask.any(1)[:,None] & ~mask]
#array([[1, 1, 1],
# [2, 1, 1]])

I got the results you're looking for by doing this:
[[0,1,1] in b for b in a]
I'll try to figure out why isin didnt work.

Categories