I have this array (let's call it my_array) which comes from applying numpy.any:
my_array
array([[ True, True, True, ..., True, True, True],
[ True, True, True, ..., True, True, True],
[ True, True, True, ..., True, True, True],
...,
[ True, True, True, ..., True, True, True],
[ True, True, True, ..., True, True, True],
[ True, True, True, ..., True, True, True]])
Is there a way to convert this array to a ndarray? I mean, how can I see this:
array([[ True, True, True, ..., True, True, True],
[ True, True, True, ..., True, True, True],
[ True, True, True, ..., True, True, True],
...,
[ True, True, True, ..., True, True, True],
[ True, True, True, ..., True, True, True],
[ True, True, True, ..., True, True, True]],dtype=bool)
When you show something like that to the IDE, it uses repr(obj), which creates a string representation of the object. Ideally this also means eval(repr(object)) == object. The reason why you don't see the dtype=bool part is that an array comprised fully of booleans is unambiguously boolean. Likewise, an integer array is by default of dtype int32, so that is also excluded. But if you have an integer of some other type, then it will be shown.
>>> np.array([1, 2])
array([1, 2])
>>> np.array([1, 2]).dtype
dtype('int32')
>>> np.array([1, 2], dtype=np.uint8)
array([1, 2], dtype=uint8)
Your result is already in the output you want. And the docs for numpy.any confirm that:
Returns
-------
any : bool or ndarray
A new boolean or `ndarray` is returned unless `out` is specified,
in which case a reference to `out` is returned.
But if you want to make sure everything is right, you can check type(my_array) and my_array.dtype (assuming the object is a numpy array).
>>> a = np.any([[1, 2], [3, 4]], axis=0)
>>> a
array([ True, True])
>>> type(a)
<class 'numpy.ndarray'>
>>> a.dtype
dtype('bool')
Related
In the below tensor of dimension (5,1,5,5), I want to calculate the number of 5*5(innermost) Boolean matrices where all the values are 'True'.
For example,
tf.constant([
[[[ True, True, True, True, True],
[False, True, False, True, True],
[False, True, False, True, True],
[ True, True, True, True, True],
[ True, True, True, True, True]]],
[[[ True, True, True, True, False],
[False, True, True, True, True],
[False, True, False, True, True],
[ True, True, True, True, True],
[ True, False, True, True, True]]],
[[[ True, True, True, True, True],
[False, True, False, True, True],
[False, True, True, True, True],
[ True, False, True, False, True],
[ True, True, True, True, True]]],
[[[ True, True, True, True, True],
[ True, True, True, True, True],
[ True, True, True, True, True],
[ True, True, True, True, True],
[ True, True, True, True, True]]],
[[[ True, True, True, True, True],
[ True, True, True, True, True],
[ True, True, True, True, True],
[ True, True, True, True, True],
[ True, True, True, True, True]]],
])
In the above tensor, there are total of 5 innermost matrices with dimension 5 by 5. Out of them 3 matrices contain both 'False' and 'True' values whereas the last 2 matrices contain only 'True' values. I need to compute the ratio of total number of matrices with only 'True' values to that of total number of matrices with both 'True' and 'False' values. In this case it would be :
total number of matrices with only 'True' values / Total number of matrices = 2/5
How can I compute this ?
One way, I can think of is to use tf.squeeze() to remove the 1 dimension and then convert it into numpy array and then loop through it. However, this is not the optimized way to do it. I want to do it using tensorflow (with tensor datatype and not numpy array) as this needs to be extremely time efficient.
Further, I also want to compute the ratio of total number of innermost rows where all values are 'True' to that of total number of rows. For example, in our case it would be :
total number of rows with only 'True' values / total number of rows = 16/25.
This too, needs to be using tensor and highly time-efficient.
I think that you need tf.reduce_all .
import tensorflow as tf
temp_tesnor = tf.constant([
[[[ True, True, True, True, True],
[False, True, False, True, True],
[False, True, False, True, True],
[ True, True, True, True, True],
[ True, True, True, True, True]]],
[[[ True, True, True, True, False],
[False, True, True, True, True],
[False, True, False, True, True],
[ True, True, True, True, True],
[ True, False, True, True, True]]],
[[[ True, True, True, True, True],
[False, True, False, True, True],
[False, True, True, True, True],
[ True, False, True, False, True],
[ True, True, True, True, True]]],
[[[ True, True, True, True, True],
[ True, True, True, True, True],
[ True, True, True, True, True],
[ True, True, True, True, True],
[ True, True, True, True, True]]],
[[[ True, True, True, True, True],
[ True, True, True, True, True],
[ True, True, True, True, True],
[ True, True, True, True, True],
[ True, True, True, True, True]]],
])
temp_tesnor.shape # TensorShape([5, 1, 5, 5])
Then reduce the tensor where the dimension is 1.
temp_tesnor = tf.squeeze(temp_tesnor, axis = [1])
temp_tesnor.shape # TensorShape([5, 5, 5])
Result for query1
result1 = 0
for i in temp_tesnor:
if tf.reduce_all(i):
result1 += 1
print(result1) # prints 2 . Divide here with your first dimension. here it is 5.
Result for query2
result2 = 0
for inner_tensor in temp_tesnor:
for i in inner_tensor:
if tf.reduce_all(i):
result2 += 1
print(result2) # prints 16 . divide here by multiplication of dimension
#of last two axis . here it is 5*5 = 25
This is all done in tensorflow.
A = np.array([5,1,5,8])
B = np.array([2,5])
I want to compare the A array to each element of B. In other words I'm lookin for a function which do the following computations :
A>2
A>5
(array([ True, False, True, True]), array([False, False, False, True]))
Not particularly fancy but a list comprehension will work:
[A > b for b in B]
[array([ True, False, True, True], dtype=bool),
array([False, False, False, True], dtype=bool)]
You can also use np.greater(), which requires the dimension-adding trick that Brenlla uses in the comments:
np.greater(A, B[:,np.newaxis])
array([[ True, False, True, True],
[False, False, False, True]], dtype=bool)
In numpy you can use the allclose(X, Y) function to check element-wise for approximate equality between two arrays. Moreover, with an expression like X==5 you can check element-wise equality between an array and a scalar.
Is there a function that combines both functionalities?? That is, can compare an array and a scalar for approximate element-wise equality??
The term array or array-like in the numpy documentation mostly indicates that the input is converted to an array with np.asarray(in_arg) or np.asanyarray(in_arg). So if you input a scalar it will be converted to an scalar array:
>>> import numpy as np
>>> np.asarray(5) # or np.asanyarray
array(5)
And the functions np.allclose or np.isclose just do element-wise comparison no matter if the second argument is a scalar array, an array with the same shape or an array that correctly broadcasts to the first array:
>>> import numpy as np
>>> arr = np.array([1,2,1,0,1.00001,0.9999999])
>>> np.allclose(arr, 1)
False
>>> np.isclose(arr, 1)
array([ True, False, True, False, True, True], dtype=bool)
>>> np.isclose(arr, np.ones((10, 6)))
array([[ True, False, True, False, True, True],
[ True, False, True, False, True, True],
[ True, False, True, False, True, True],
[ True, False, True, False, True, True],
[ True, False, True, False, True, True],
[ True, False, True, False, True, True],
[ True, False, True, False, True, True],
[ True, False, True, False, True, True],
[ True, False, True, False, True, True],
[ True, False, True, False, True, True]], dtype=bool)
So no need to find another function that explicitly handles scalars, these functions already correctly work with scalars.
I want to compare the elements of two 3D numpy arrays of different lengths. The goal is, to find overlapping elements in the two arrays.
All functions I found so far, rely on the two arrays being of the same lengths.
Is there an efficient way to do compare the 2D-elements (for loops will be very inefficient, since each array has tens of thousands of elements)?
Here a few ways of comparing 2 1d arrays:
In [325]: n=np.arange(0,10)
In [326]: m=np.arange(3,9)
In [327]: np.in1d(n,m)
Out[327]: array([False, False, False, True, True, True, True, True, True, False], dtype=bool)
In [328]: np.in1d(m,n)
Out[328]: array([ True, True, True, True, True, True], dtype=bool)
In [329]: n[:,None]==m[None,:]
Out[329]:
array([[False, False, False, False, False, False],
[False, False, False, False, False, False],
[False, False, False, False, False, False],
[ True, False, False, False, False, False],
[False, True, False, False, False, False],
[False, False, True, False, False, False],
[False, False, False, True, False, False],
[False, False, False, False, True, False],
[False, False, False, False, False, True],
[False, False, False, False, False, False]], dtype=bool)
and farenorths suggestion
In [330]: np.intersect1d(n,m)
Out[330]: array([3, 4, 5, 6, 7, 8])
In [331]: np.where(np.in1d(n,m))
Out[331]: (array([3, 4, 5, 6, 7, 8], dtype=int64),)
Is intersect1d what you want? For example, if your arrays are a and b, you could simply do:
duplicates = np.intersect1d(a, b)
I'm trying to dynamically construct a 2-D matrix with numpy based on the values of an array, like this:
In [113]: A = np.zeros((5,5),dtype=bool)
In [114]: A
Out[114]: array([[False, False, False, False, False],
[False, False, False, False, False],
[False, False, False, False, False],
[False, False, False, False, False],
[False, False, False, False, False]], dtype=bool)
In [116]: B = np.array([0,1,3,0,2])
In [117]: B
Out[117]: array([0, 1, 3, 0, 2])
Now, I'd like to use the values of B to assign the first n values of each row to A to True. For this A and B, the correct output would be:
In [118]: A
Out[118]: array([[False, False, False, False, False],
[ True, False, False, False, False],
[ True, True, True, False, False],
[False, False, False, False, False],
[ True, True, False, False, False]], dtype=bool)
The length of B will always equal the number of rows of A, and the the values of B will always be less than or equal to the number of columns of A. The size of A and the values of B are constantly changing, so I need to build these on the fly.
I'm certain that this has a simple(-ish) solution in numpy, but I've spent the last hour banging my head against variations of repeat, tile, and anything else I can think of. Can anyone help me out before I give myself a concussion? :)
EDIT: I'm going to need to do this a lot, so speed will be an issue. The only version that I can come up with for now is something like:
np.vstack([ [True]*x + [False]*(500-x) for x in B ])
but I expect that this will be slow due to the for loop (I would time it if I had anything to compare it to).
How about:
>>> A = np.zeros((5, 7),dtype=bool)
>>> B = np.array([0,1,3,0,2])
>>> (np.arange(len(A[0])) < B[:,None])
array([[False, False, False, False, False, False, False],
[ True, False, False, False, False, False, False],
[ True, True, True, False, False, False, False],
[False, False, False, False, False, False, False],
[ True, True, False, False, False, False, False]], dtype=bool)
(I changed the shape from (5,5) because I was getting confused about which axis was which, and I wanted to make sure I was using the right one.)
[Simplified from (np.arange(len(A[0]))[:,None] < B).T -- if we expand B and not A, there's no need for the transpose.]