I am trying to implement a quick tool to find the combinations of a set of numbers (0...k), for j lots of this array, where the sum across the row is equal to k-j, and k>=j
For instance, for k=3, and j=2, I have all the following combinations:
[[0 0]
[0 1]
[0 2]
[1 0]
[1 1]
[1 2]
[2 0]
[2 1]
[2 2]]
There are only two elements that sum up to k-j=1:
[[0 1]
[1 0]]
My current implementation is slow as I approach high numbers of k and j:
from itertools import product
from numpy import
combs = np.array(list(product(np.arange(0, k), repeat=j)))
combs = combs[np.sum(combs, axis=1)==k-j]
print(combs)
Can anyone please suggest a more efficient algorithm than I have at the moment?
For one, you don't have to consider the range up to k, but only up to k-j. Further, you could use itertools.combinations (if you are only interested in the set and not the order), as follows:
combs = np.array(list(combinations(range(k-j+1), j)))
combs = combs[np.sum(combs, axis=1)==k-j]
Related
I have this array and I want to return unique array combinations. I tried meshgrid but it creates duplicates and inverse array values
>> import numpy as np
>> array = np.array([0,1,2,3])
>> combinations = np.array(np.meshgrid(array, array)).T.reshape(-1,2)
>> print(combinations)
[[0 0]
[0 1]
[0 2]
[0 3]
[1 0]
[1 1]
[1 2]
[1 3]
[2 0]
[2 1]
[2 2]
[2 3]
[3 0]
[3 1]
[3 2]
[3 3]]
What I want to exclude are the repeating arrays: [0,0] [1,1] [2,2] [3,3] and the inverse arrays when [2,3] is returned exclude [3,2] in the output.
Take a look at this combination calculator, this is the output that I like but how can I create it in NumPy?
you could use combinations from itertools
import numpy as np
from itertools import combinations
array = np.array([0,1,2,3])
combs = np.array(list(combinations(arr, 2)))
Given K, I need to have all the possibile combinations of K x 2 numpy matrices so that in each matrix there are all 0s except for two 1s in different rows and columns.
Something like this for K = 5:
[[1,0],[0,1],[0,0],[0,0][0,0]]
[[1,0],[0,0],[0,1],[0,0][0,0]]
[[1,0],[0,0],[0,0],[0,1][0,0]]
[[1,0],[0,0],[0,0],[0,0][0,1]]
[[0,0],[1,0],[0,1],[0,0][0,0]]
[[0,0],[1,0],[0,0],[0,1][0,0]]
... and so on
So the resulting array should be a K x 2 x (K*(K-1)/2).
I want to avoid loops since it's not an efficient way when K is big enough (in my specific case K = 300)
I can't think of an elegant solution but here's a not-so-elegant pure numpy one:
import numpy as np
def combination_matrices(K):
# get combination indices
i, j = np.indices((K, K))
comb_indices = np.transpose((i < j).nonzero()) # (num_combs, 2) array where ones are
num_combs = comb_indices.shape[0] # K*(K-1)/2
# create a matrix of the desired shape, first axis enumerates combinations
matrices = np.zeros((num_combs, K, 2), dtype=int)
# broadcasting assignment of ones
comb_range, col_index = np.ogrid[:num_combs, :2]
matrices[comb_range, comb_indices, col_index] = 1
return matrices
This first uses the indices of a (K, K)-shaped array to find the index pairs for every combination (these are indices that encode the upper triangle of the array, excluding the diagonal). Then we use a bit tricky broadcasting assignment (heavy fancy indexing) to set each corresponding element of the pre-allocated output array to 1.
Note that I put the K*(K-1)/2-sized axis first, because this makes the most sense in numpy with C-contiguous memory layout. This way when you take the matrix for combination index 3, arr[3, ...] will be a contiguous chunk of memory of shape (K, 2) that's fast to work with in vectorised operations.
The output for K = 4:
[[[1 0]
[0 1]
[0 0]
[0 0]]
[[1 0]
[0 0]
[0 1]
[0 0]]
[[1 0]
[0 0]
[0 0]
[0 1]]
[[0 0]
[1 0]
[0 1]
[0 0]]
[[0 0]
[1 0]
[0 0]
[0 1]]
[[0 0]
[0 0]
[1 0]
[0 1]]]
This is an oddly specific question, but an interesting problem, I'd love to know what the context is?
You are looking for all permutations of a multiset , python's itertools doesn't currently support this. So simplest solution is to use the multiset tools of the sympy library.
The following code took about ~2.5 minutes to run on my machine, so is fairly fast for a single thread. You're looking at 179700 unique permutations for K=300.
(I took inspiration from https://stackoverflow.com/a/40289807/10739860)
from collections import Counter
from math import factorial, prod
import numpy as np
from sympy.utilities.iterables import multiset_permutations
from tqdm import tqdm
def No_multiset_permutations(multiset: list) -> int:
"""Calculates the No. possible permutations given a multiset.
See: https://en.wikipedia.org/wiki/Permutation#Permutations_of_multisets
:param multiset: List representing a multiset.
"""
value_counts = Counter(multiset).values()
denominator = prod([factorial(val) for val in value_counts])
return int(factorial(len(multiset)) / denominator)
def multiset_Kx2_permutations(K: int) -> np.ndarray:
"""This will generate all possible unique Kx2 permutations of an array
withsize K where two values are 1 and the rest are 0.
:param K: The size of the array.
"""
# Construct number multiset, e.g. K=5 gives [1, 1, 0, 0, 0, 0, 0, 0, 0, 0]
numbers = [1, 1] + [0] * (K - 1) * 2
# Use sympy's multiset_permutations to get a multiset permutation generator
generator = multiset_permutations(numbers)
# Calculate the No. possible permutations
number_of_perms = No_multiset_permutations(numbers)
# Get all permutations, bonus progress bar is included :)
unique_perms = [next(generator) for _ in tqdm(range(number_of_perms))]
# Reshape each permutation to Kx2
unique_perms = np.array(unique_perms, dtype=np.int8)
return unique_perms.reshape(-1, K, 2)
if __name__ == "__main__":
solution = multiset_Kx2_permutations(300)
Another possibility (with rearranged axes for clearer output):
from itertools import combinations
import numpy as np
k = 4
x = list(combinations(range(k), 2))
out = np.zeros((n := len(x), k, 2), dtype=int)
out[np.c_[:n], x, [0, 1]] = 1
print(out)
It gives:
[[[1 0]
[0 1]
[0 0]
[0 0]]
[[1 0]
[0 0]
[0 1]
[0 0]]
[[1 0]
[0 0]
[0 0]
[0 1]]
[[0 0]
[1 0]
[0 1]
[0 0]]
[[0 0]
[1 0]
[0 0]
[0 1]]
[[0 0]
[0 0]
[1 0]
[0 1]]]
I have a 4x2 and a 2x2 matrices. I would like to loop each combination of rows (vectors of dimension 2) through a function foo using vectorize.
Here are the matrices:
X = np.array([[1, 0], [2, 0], [3, 0], [4,0]])
Y = np.array([[1, 0], [2, 0]])
Here's how I'm trying to run it:
def foo(x, y):
print("inputs:", x, y)
return x[0] * y[0]
bar = np.vectorize(foo, signature="???")
output = bar(X, Y)
print(output)
I'm looking for the following output. bar would return a 4x2 matrice:
inputs: [1,0] [1,0]
inputs: [1,0] [2,0]
inputs: [2,0] [1,0]
inputs: [2,0] [2,0]
inputs: [3,0] [1,0]
inputs: [3,0] [2,0]
inputs: [4,0] [1,0]
inputs: [4,0] [2,0]
[[1,2], [2,4], [3,6], [4,8]]
I've tried various combinations of signature, but I'm just not grokking how to use it given the output I'm looking for.
NB: I am aware vectorize just uses Python for loops under the hood and offers no performance benefit. I just want to understand how to use it.
The basic use of vectorize broadcasts the inputs against each other, and passes scalar tuples to your function. A (4,2) can't broadcast with a (2,2). signature is an addition that should make it possible to pass "rows" of your arrays. It's even slower, and I haven't see it used much (or recommended it).
In [536]: bar = np.vectorize(foo, signature="(n),(n)->()")
In [533]: bar(X,Y[0,:])
inputs: [1 0] [1 0]
inputs: [2 0] [1 0]
inputs: [3 0] [1 0]
inputs: [4 0] [1 0]
Out[533]: array([1, 2, 3, 4])
In [537]: bar(X[:,None],Y[None])
inputs: [1 0] [1 0]
inputs: [1 0] [2 0]
inputs: [2 0] [1 0]
inputs: [2 0] [2 0]
inputs: [3 0] [1 0]
inputs: [3 0] [2 0]
inputs: [4 0] [1 0]
inputs: [4 0] [2 0]
Out[537]:
array([[1, 2],
[2, 4],
[3, 6],
[4, 8]])
So this gives bar a (4,1,2) and (1,2,2); which broadcast as (4,2,2). Or with this signature it's broadcasting a (4,1) with 1,2) => (4,2). It's the signature that determines how the last dimensions match.
It may in some cases be convenient, but I wouldn't recommend devoting too much time to understanding vectorize.
So let's say I have some 3x3 matrix I get with a calculation I am doing, let's say
np.array([[1,2,3],[4,5,6],[7,8,9]])
np.array([[0,0,0],[0,0,0],[0,0,0]])
I want to add this onto some matrix A and be able to access them so that if I do
> A[0]
> [[1,2,3],[4,5,6],[7,8,9]]
> A[1]
> [[0,0,0],[0,0,0],[0,0,0]]
and keep adding on bunch of these 2D array and save them with np.save('A', A) for fast access later. I kind of saw it is possible with appending to a list but I can't save a list with np.save for fast efficient access. How can I create a empty ndarray I can add matrix onto and save it all as .npy?
You can convert list to array it is the same:
A = list()
A.append(x)
A.append(y)
X = np.array(A)
np.save('X', X)
Originally use this:
A = np.stack((a,b))
[[[1 2 3]
[4 5 6]
[7 8 9]]
[[0 0 0]
[0 0 0]
[0 0 0]]]
And once you have formed A, if you want to add another array c to A use:
A = np.vstack((A,[c]))
output for c=a:
[[[1 2 3]
[4 5 6]
[7 8 9]]
[[0 0 0]
[0 0 0]
[0 0 0]]
[[1 2 3]
[4 5 6]
[7 8 9]]]
I have this line:
x = np.zeros((10, 2), int)
and a for loop that fills this for the first 6 elements.
Is there a way to remove the remaining 0 elements?
If you want to remove the last all zero rows:
x = x[:(np.where(x.any(axis=1))[0]).max()+1]
example:
x:
[[1 2]
[0 0]
[1 2]
[0 0]]
output:
[[1 2]
[0 0]
[1 2]]
Try this:
x = x[:6]
Note that this solution will create 6 element long view to original array. Modifying view will modify original array.