I have list, A, containing 8 values. I like to make combinations first, then add two more points in list B to each combination. Here is my code:
def combination(arr, r):
return list(itertools.combinations(arr, r))
A = [[0.0, 0.0, 0.0], [0.0, 0.5, 0.0], [0.5, 0.0, 0.5], [0.5, 0.5, 0.5], [0.0, 0.25, 0.0], [0.0, 0.7499999819999985, 0.0], [0.5, 0.25, 0.5], [0.5, 0.7499999819999985, 0.5]]
B = [[0.4950293947410103, 0.5021785267638279, 0.4935703740156043], [1, 1, 1]]
n = 1 #can be change
com = combination(A, n)
for item in com:
item.extend(B)
print(item)
But I got an error:
AttributeError: 'tuple' object has no attribute 'extend'
Expected results:
[[0.0, 0.0, 0.0], [0.4950293947410103, 0.5021785267638279, 0.4935703740156043], [1, 1, 1]]
[[0.0, 0.5, 0.0], [0.4950293947410103, 0.5021785267638279, 0.4935703740156043], [1, 1, 1]]
[[0.5, 0.0, 0.5], [0.4950293947410103, 0.5021785267638279, 0.4935703740156043], [1, 1, 1]]
[[0.5, 0.5, 0.5], [0.4950293947410103, 0.5021785267638279, 0.4935703740156043], [1, 1, 1]]
[[0.0, 0.25, 0.0], [0.4950293947410103, 0.5021785267638279, 0.4935703740156043], [1, 1, 1]]
[[0.0, 0.7499999819999985, 0.0], [0.4950293947410103, 0.5021785267638279, 0.4935703740156043], [1, 1, 1]]
[[0.5, 0.25, 0.5], [0.4950293947410103, 0.5021785267638279, 0.4935703740156043], [1, 1, 1]]
[[0.5, 0.7499999819999985, 0.5], [0.4950293947410103, 0.5021785267638279, 0.4935703740156043], [1, 1, 1]]
Type tuple (return by combinations) is immutable, you may use a list and populate it with the item and B
com = combination(A, n)
com = [[*item, *B] for item in com]
Or return list of list from your combination
def combination(arr, r):
return [list(c) for c in itertools.combinations(arr, r)]
# ...
for item in com:
item.extend(B)
You can use also map function, to get list of list.
map() function returns a map object(which is an iterator) of the results after applying the given function to each item of a given iterable (in your case list)
instead of:
com = combination(A, n)
use:
com = map(list, combination(A, n))
Related
When we have two 1-D arrays:
import numpy as np
import scipy.stats as stats
a=np.array([0.36619718309859156,
0.32558139534883723,
0.3333333333333333,
0.3333333333333333,
0.2549019607843137,
0.3695652173913043,
0.3157894736842105,
0.3625])
and
b=np.array([0.938456,
0.3239485723,
0.300,
0.8658,
1.254901137,
2.3695,
0.75,
1.3625])
we can perform the t-test by the following:
stats.ttest_ind(a=a, b=b, equal_var=np.amax([np.var(a),np.var(b)])/np.amin([np.var(a),np.var(b)])<4)
However, I would like to compare the columns of A and B where A and B are 2-D arrays:
A=np.array([[0, 0.375, 0.5, 0.3917],
[0, 0.333, 0.4, 0.4285],
[0, 0.27272727, 0.0, 0.2],
[0.0, 0.25, 0.36365, 0.272],
[0, 0.285857, 0.4, 0.25],
[0, 0.416667, 0.33, 0.375],
[0, 0.28, 0.083, 0.41667],
[0, 0.2858, 0.25, 0.41666]])
B=np.array([[0, 0.4, 0.333, 0.142],
[0, 0.33333, 0.4, 0.1111111],
[0, 0.25, 0.285, 0.333333],
[0.0, 0.5, 0.380, 0.333],
[0.0, 0.5, 0.33, 0.375],
[0, 0.25, 0.294, 0.5],
[0.0, 0.5, 0.333, 0.2068965],
[0, 0.5, 0.3846, 0.2]])
ie. I would like to perform t-test on, and compare, the first column of A and the first column of B, then the second column of A and the second column of B, and so on. (I tried specifying the axes but I think I am not sure how to correctly incorporate the equal_var < 4 property in this case.)
You can transpose the data and then go through both at the same time:
def non_paired_t_test(a, b):
return stats.ttest_ind(a=a, b=b, equal_var=np.amax([np.var(a),np.var(b)])/np.amin([np.var(a),np.var(b)])<4)
for a, b in zip(A.transpose(), B.transpose()):
print(non_paired_t_test(a, b))
You can use A[:,i] which denotes all the rows in the i^th column:
for i in range(0,4):
print(stats.ttest_ind(a=A[:,i], b=B[:,i], equal_var=np.amax([np.var(A[:,i]),np.var(B[:,i])])/np.amin([np.var(A[:,i]),np.var(B[:,i])])<4))
I have a pytorch tensor
t = torch.tensor(
[[1.0, 1.5, 0.5, 2.0],
[5.0, 3.0, 4.5, 5.5],
[0.5, 1.0, 3.0, 2.0]]
)
t[:, [-1]] gives me last column value of each row:
tensor([[2.0000],
[5.5000],
[2.0000]])
However, I want to slice values at different columns per row. For example, in t for the 1st, 2nd and 3rd row, I want to slice at 2, -1, 0 index respectively to get the following tensor:
tensor([[0.5],
[5.5],
[0.5]])
How can I do it in torch?
t[[i for i in range(3)], [2, -1, 0]]
The list comprehension creates a list filled with row indexes, then you specify the column index for every row.
you can use the following:
t = torch.tensor(
[[1.0, 1.5, 0.5, 2.0],
[5.0, 3.0, 4.5, 5.5],
[0.5, 1.0, 3.0, 2.0]]
)
t
>tensor([[1.0000, 1.5000, 0.5000, 2.0000],
[5.0000, 3.0000, 4.5000, 5.5000],
[0.5000, 1.0000, 3.0000, 2.0000]])
rows = [0, 1, 2]
cols = [2, -1, 0]
t[rows, cols]
>tensor([0.5000, 5.5000, 0.5000])
I have an np.array called arr which is:
arr = np.array([[0.0, 0.0, 0.0], [1 / 3, 1 / 3, 0], [0.0, 0.0, 0.0]])
and I want to write its information to a single-line string called s as:
[[0.0, 0.0, 0.0], [1 / 3, 1 / 3, 0], [0.0, 0.0, 0.0]]
For this, I am using this type of conversation(in my code it is in a function):
import re
import numpy as np
arr = np.array([[0.0, 0.0, 0.0], [1 / 3, 1 / 3, 0], [0.0, 0.0, 0.0]])
s = np.array_str(arr, precision=4)
s = re.sub('(\d) +(-|\d)', r'\1,\2', s)
s.replace('^0. $', '0.0')
# s.replace('0. ', '0.0') #gives same result
s.replace('\n', ',')
print(s)
However, the result is:
[[0. 0. 0. ]
[0.3333,0.3333,0. ]
[0. 0. 0. ]]
You need to catch the output of s.replace() and save it as the s variable, or another variable name.
import re
import numpy as np
arr = np.array([[0.0, 0.0, 0.0], [1 / 3, 1 / 3, 0], [0.0, 0.0, 0.0]])
s = np.array_str(arr, precision=4)
s = re.sub('(\d) +(-|\d)', r'\1,\2', s)
s = s.replace('0. ', '0.0') #gives same result
s = s.replace('\n', ',')
print(s)
You could use nested comprehensions on the array and process it using the fraction module:
from fractions import Fraction
s = "["+", ".join("["+", ".join(f"{Fraction(n).limit_denominator(10000)}"
for n in row)+"]"
for row in arr )+"]"
print(s)
[[0, 0, 0], [1/3, 1/3, 0], [0, 0, 0]]
Note that, while this is close to your expected result, there is no way to distinguish the 0.0 from the 0 in the original array because this information is lost when the numbers are converted to floats by numpy. So all 0s will be printed the same way.
For more than 2 dimensions, you could generalize this into a recursive function:
from fractions import Fraction
def arrayToStr(arr):
if isinstance(arr,np.ndarray):
return "["+", ".join(arrayToStr(n) for n in arr)+"]"
if not arr return "0.0"
return f"{Fraction(arr).limit_denominator(10000)}"
This will print zeroes as 0.0 though I cannot fathom why you would want to do that specifically (and only) for zero.
Output:
arr = np.array([[0.0, 0.0, 0.0], [1 / 3, 1 / 3, 0], [0.0, 0.0, 0.0]])
print(arrayToStr(arr))
[[0.0, 0.0, 0.0], [1/3, 1/3, 0.0], [0.0, 0.0, 0.0]]
arr = np.arange(24).reshape((4,3,2))/6
print(arrayToStr(arr))
[[[0.0, 1/6], [1/3, 1/2], [2/3, 5/6]], [[1, 7/6], [4/3, 3/2], [5/3, 11/6]], [[2, 13/6], [7/3, 5/2], [8/3, 17/6]], [[3, 19/6], [10/3, 7/2], [11/3, 23/6]]]
If you don't mind getting the 1/3 values as decimals, you could use the json module which would do the formatting more directly:
import json
s = json.dumps(arr.tolist())
print(s)
[[0.0, 0.0, 0.0], [0.3333333333333333, 0.3333333333333333, 0.0], [0.0, 0.0, 0.0]]
For example, given a predicted probability map, like a
a = np.array([[0.1, 0.2, 0.3, 0.0, 0.0, 0.0],
[0.1, 0.92, 0.3, 0.0, 0.2, 0.1],
[0.1, 0.9, 0.3, 0.0, 0.7, 0.89],
[0.0, 0.0, 0.0, 0.0, 0.4, 0.5],
[0.0, 0.0, 0.0, 0.0, 0.0, 0.0],
[0.0, 0.0, 0.0, 0.0, 0.0, 0.0]])
How can I find two max probability (0.9, 0.9) and coordinates ((1,1), (2,5)) of two connected components in a?
Use np.where or np.argwhere
>>> np.unique(a)[-2:]
array([0.89, 0.92])
>>> np.where(np.isin(a, np.unique(a)[-2:]))
(array([1, 2]), array([1, 5]))
# OR
>>> np.argwhere(np.isin(a, np.unique(a)[-2:]))
array([[1, 1],
[2, 5]])
Here is my answer, but maybe too complicated.
def nms_cls(loc, cls):
"""
Find the max class and prob point in a mask
:param loc: binary prediction with 0 and 1 (h, w)
:param cls: multi-classes prediction with prob (c, h, w)
:return: list of tuple (class, prob, coordinate)
"""
prob = np.max(cls, axis=0) # (H, W)
cls_idx = np.argmax(cls, axis=0)
point_list = []
num_labels, labels, stats, centroids = cv2.connectedComponentsWithStats(loc, connectivity=8, ltype=None)
for i in range(num_labels):
# get the mask of connected component i
label_i = np.copy(labels)
label_i[label_i != i] = 0
label_i[label_i > 0] = 1
prob_mask_i = prob * label_i
# get max prob's coords and class
state_i = {}
state_i['coord'] = np.unravel_index(prob_mask_i.argmax(), prob_mask_i.shape)
state_i['cls'] = cls_idx[state_i['coord'][0], state_i['coord'][1]]
state_i['prob'] = prob[state_i['coord'][0], state_i['coord'][1]]
point_list.append(state_i)
return point_list
I would like to reduce a variable number of elements (or slices) of an array multiple times, and put the result into a new array. Kind of like a masked np.apply_along_axis, but we stay in numpy
For example, to reduce by mean:
to_reduce = np.array([
[0, 1, 1, 0, 0],
[0, 0, 0, 1, 1],
[1, 0, 1, 0, 1],
[1, 1, 1, 1, 0]]).astype(np.bool8)
arr = np.array([
[1.0, 2.0, 3.0],
[1.0, 2.0, 4.0],
[2.0, 2.0, 3.0],
[2.0, 2.0, 4.0],
[1.0, 0.0, 3.0]])
I want:
np.array([
[1.5, 2.0, 3.5],
[1.5, 1.0, 3.5],
[1.33333, 1.33333, 3.0],
[1.5, 2.0, 3.5]])
The slow way would be:
out = np.empty((4, 3))
for j, mask in enumerate(to_reduce):
out[j] = np.mean(arr[mask], axis=0)
Here's one simple and efficient way with matrix-multiplication -
In [56]: to_reduce.dot(arr)/to_reduce.sum(1)[:,None]
Out[56]:
array([[1.5 , 2. , 3.5 ],
[1.5 , 1. , 3.5 ],
[1.33333333, 1.33333333, 3. ],
[1.5 , 2. , 3.5 ]])