I have two numpy arrays of unequal length. I would like to compare the two arrays for mismatch at index.
for example in these numpy arrays. The number of mismatches are
import numpy as np
a = np.array([0, 1, 0, 1, 1])
b = np.array([1, 1, 0, 0, 1, 1])
expected output : 3 ( three mismatches at index 0,3 and 5)
As per my comment above.
Assuming we don't know which of the arrays is the longer one:
def foo(a, b):
# Get equal length arrays
c = a[: min([a.shape[0], b.shape[0]])]
d = b[: min([a.shape[0], b.shape[0]])]
# now compare equal arrays
mis = np.equal(c, d)
# Add difference in shapes of arrays as mismatches
mis = np.concatenate((mis, np.full(abs(a.shape[0] - b.shape[0]), False)))
return np.where(~mis)[0].shape[0]
>>> a = np.array([0, 1, 0, 1, 1])
b = np.array([1, 1, 0, 0, 1, 1])
>>> x = foo(a, b)
>>> x
Out: 3
EDIT: Oops... forgot to add a bitwise not to the return statement.
this is what I got . built on #pavel and #hpaulj answers.
def comp(a, b):
# Get equal length arrays
c = a[: min([a.shape[0], b.shape[0]])]
d = b[: min([a.shape[0], b.shape[0]])]
# now compare equal arrays
mis = np.sum(c != d)
mis = mis + abs(a.shape[0]-b.shape[0])
return mis
a = np.array([0, 1, 0, 1, 1, 0, 0, 1])
b = np.array([1, 1, 1, 0, 1, 1])
x = comp(a,b)
x
Related
Basically, I have three arrays that I multiply with values from 0 to 2, expanding the number of rows to the number of products (the values to be multiplied are the same for each array). From there, I want to calculate the product of every combination of rows from all three arrays. So I have three arrays
A = np.array([1, 2, 3])
B = np.array([1, 2, 3])
C = np.array([1, 2, 3])
and I'm trying to reduce the operation given below
search_range = np.linspace(0, 2, 11)
results = np.array([[0, 0, 0]])
for i in search_range:
for j in search_range:
for k in search_range:
sm = i*A + j*B + k*C
results = np.append(results, [sm], axis=0)
What I tried doing:
A = np.array([[1, 2, 3]])
B = np.array([[1, 2, 3]])
C = np.array([[1, 2, 3]])
n = 11
scale = np.linspace(0, 2, n).reshape(-1, 1)
A = np.repeat(A, n, axis=0) * scale
B = np.repeat(B, n, axis=0) * scale
C = np.repeat(C, n, axis=0) * scale
results = np.array([[0, 0, 0]])
for i in range(n):
A_i = A[i]
for j in range(n):
B_j = B[j]
C_k = C
sm = A_i + B_j + C_k
results = np.append(results, sm, axis=0)
which only removes the last for loop. How do I reduce the other for loops?
You can get the same result like this:
search_range = np.linspace(0, 2, 11)
search_range = np.array(np.meshgrid(search_range, search_range, search_range))
search_range = search_range.T.reshape(-1, 3)
sm = search_range[:, 0, None]*A + search_range[:, 1, None]*B + search_range[:, 2, None]*C
results = np.concatenate(([[0, 0, 0]], sm))
Instead of using three nested loops to get every combination of elements in the "search_range" array, I used the meshgrid function to convert "search_range" to a 2D array of every possible combination and then instead of i, j and k you can use the 3 items in the arrays in the "search_range".
And finally, as suggested by #Mercury you can use indexing for the new "search_range" array to generate the result. For example search_range[:, 1, None] is an array in shape of (1331, 1), containing singleton arrays of every element at index of 0 in arrays in the "search_range". That concatenate is only there because you wanted the results array to have default value of [[0, 0, 0]], so I appended sm to it; Otherwise, the sm array contains the answer.
I have an array indexs. It's very long (>10k), and each int value is rather small (<100). e.g.
indexs = np.array([1, 4, 3, 0, 0, 1, 2, 0]) # int index array
indexs_max = 4 # already known
Now I want to count occurrence of each index value (e.g. 0 for 3 times, 1 for 2 times...), and get counts as np.array([3, 2, 1, 1, 1]). I have tested 4 methods as follows:
UPDATE: _test4 is #Ch3steR's sol:
indexs = np.random.randint(0, 10, (20000,))
indexs_max = 9
def _test1():
counts = np.zeros((indexs_max + 1, ), dtype=np.int32)
for ind in indexs:
counts[ind] += 1
return counts
def _test2():
counts = np.zeros((indexs_max + 1,), dtype=np.int32)
uniq_vals, uniq_cnts = np.unique(indexs, return_counts=True)
counts[uniq_vals] = uniq_cnts
# this is because some value in range may be missing
return counts
def _test3():
therange = np.arange(0, indexs_max + 1)
counts = np.sum(indexs[None] == therange[:, None], axis=1)
return counts
def _test4():
return np.bincount(indexs, minlength=indexs_max+1)
Run for 500 times, their time usage are respectively 32.499472856521606s, 0.31386804580688477s, 0.14069509506225586s, 0.017721891403198242s. Although _test3 is the fastest, it uses additional big memory.
So I'm asking for any better methods. Thank u :) (#Ch3steR)
UPDATE: np.bincount seems optimal so far.
You can use np.bincount to count the occurrences in an array.
indexs = np.array([1, 4, 3, 0, 0, 1, 2, 0])
np.bincount(indexs)
# array([3, 2, 1, 1, 1])
# 0's 1's 2's 3's 4's count
There's a caveat to it np.bincount(x).size == np.amax(x)+1
Example:
indexs = np.array([5, 10])
np.bincount(indexs)
# array([0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 1])
# 5's 10's count
Here's it would count occurrences of 0 to the max in the array, a workaround can be
c = np.bincount(indexs) # indexs is [5, 10]
c = c[c>0]
# array([1, 1])
# 5's 10's count
If you have no missing values from i.e from 0 to your_max you can use np.bincount.
Another caveat:
From docs:
Count the number of occurrences of each value in an array of non-negative ints.
Originally I had something like this:
a = 1 # Some randomly generated positive integer
b = -1 # Some randomly generated negative integer
c = 0 # Constant 0
i = 0 # Randomly picked from (0, 1, 2)
d = [a, b, c][i]
I would like to vectorise this so that many samples can be generated
So I have three arrays of length N, an index array of length N, and would like to use that index array to pick one of the three arrays
a = np.array([1, 2, 3, 4])
b = np.array([-1, -2, -3, -4])
c = np.array([0, 0, 0, 0])
i = np.array([2, 1, 2, 0])
d = np.array([a, b, c])[i] # Doesn't work
# Would like the result:
d = np.array([0, -2, 0, 4])
d = a * (i == 0) + b * (i == 1) + c * (i == 2) works, but surely there is a way that looks more like the unvectorised code
Make a 2-d array from the three arrays then use Integer indexing
>>> e = np.vstack([a,b,c])
>>> i = np.array([2, 1, 2, 0])
>>> e[(i,np.arange(i.shape[0]))]
array([ 0, -2, 0, 4])
>>>
Notice that your answer is on the diagonal of
np.array([a, b, c])[i]
so you can go:
np.array([a, b, c])[i].diagonal()
I have two boolean arrays a and b. The number of True elements in a is equal to the length of the array b, like this:
import numpy as np
a = np.array([0, 1, 0, 1, 0, 0, 0, 1], dtype='bool')
b = np.array([1,1,0], dtype='bool')
I know that I can use np.where(a)[0] to find the indices of True elements in a:
idx = np.where(a)[0]
And I have idx:
array([1, 3, 7])
Now according to b
array([1, 1, 0])
I want to keep the first two True values in a to be True, and flip the last True value to False. That is to say, to flip the value of a[7] to 0 and keep the rest of values in a:
res = np.array([0, 1, 0, 1, 0, 0, 0, 0])
How to do it in a python way? Suppose I have a long array of a and a relative short b. The False values in b are not necessarily to be the last one, could happen anywhere and multiple times, so b could also be
b = np.array([0,1,0], dtype='bool')
Just use b to select the indices that needs to be set to False.
a[idx[~b]] = False
Use negative indexing with np.where:
a[np.where(a)[0][-1]]=0
Your array as integer values:
array([0, 1, 0, 1, 0, 0, 0, 0])
Imagine a matrix A having one column with a lot of inequality/equality operators (≥, = ≤) and a vector b, where the number of rows in A is equal the number of elements in b. Then one row, in my setting would be computed by, e.g
dot(A[0, 1:], x) ≥ b[0]
where x is some vector, column A[,0] represents all operators and we'd know that for row 0 we were suppose to calculate using ≥ operator (e.i. A[0,0] == "≥" is true). Now, is there a way for dynamically calculate all rows in following so far imaginary way
dot(A[, 1:], x) A[, 0] b
My hope was for a dynamic evaluation of each row where we evaluate which operator is used for each row.
Example, let
A = [
[">=", -2, 1, 1],
[">=", 0, 1, 0],
["==", 0, 1, 1]
]
b = [0, 1, 1]
and x be some given vector, e.g. x = [1,1,0] we wish to compute as following
A[,1:] x A[,0] b
dot([-2, 1, 1], [1, 1, 0]) >= 0
dot([0, 1, 0], [1, 1, 0]) >= 1
dot([0, 1, 1], [1, 1, 0]) == 1
The output would be [False, True, True]
If I understand correctly, this is a way to do that operation:
import numpy as np
# Input data
a = [
[">=", -2, 1, 1],
[">=", 0, 1, 0],
["==", 0, 1, 1]
]
b = np.array([0, 1, 1])
x = np.array([1, 1, 0])
# Split in comparison and data
a0 = np.array([lst[0] for lst in a])
a1 = np.array([lst[1:] for lst in a])
# Compute dot product
c = a1 # x
# Compute comparisons
leq = c <= b
eq = c == b
geq = c >= b
# Find comparison index for each row
cmps = np.array(["<=", "==", ">="]) # This array is lex sorted
cmp_idx = np.searchsorted(cmps, a0)
# Select the right result for each row
result = np.choose(cmp_idx, [leq, eq, geq])
# Convert to numeric type if preferred
result = result.astype(np.int32)
print(result)
# [0 1 1]