Related
I have a numpy array like this:
array([[ 3., 2., 3., ..., 0., 0., 0.],
[ 3., 2., -4., ..., 0., 0., 0.],
[ 3., -4., 1., ..., 0., 0., 0.],
...,
[-1., -2., 4., ..., 0., 0., 0.],
[ 4., -2., -2., ..., 0., 0., 0.],
[-2., 2., 4., ..., 0., 0., 0.]], dtype=float32)
what I want to do is removing all the rows that do not sum to zero and remove them, while also saving such rows indexes/positions in order to eliminate them to another array.
I'm trying the following:
for i in range(len(arr1)):
count=0
for j in arr1[i]:
count+=j
if count != 0:
arr_1 = np.delete(arr1,i,axis=0)
arr_2 = np.delete(arr2,i,axis=0)
the resulting arr_1 and arr_2 still contain rows that do not sum to zero. What am I doing wrong?
You can compute sum then keep row that have sum == 0 like below:
a=np.array([
[ 3., 2., 3., 0., 0., 0.],
[ 3., 2., -4., 0., 0., 0.],
[ 3., -4., 1., 0., 0., 0.]])
b = a.sum(axis=1)
# array([8., 1., 0.])
print(a[b==0])
Output:
array([[ 3., -4., 1., 0., 0., 0.]])
Just use sum(axis=1):
mask = a.sum(axis=1) != 0
do_sum_to_0 = a[~mask]
dont_sum_to_0 = a[mask]
I'm still playing around with tensorflow and been trying to use the gather_nd op, but the return value is not in the shape/format I want...
Input Tensor: - shape: (2, 7, 4)
array([[[ 0., 0., 1., 2.],
[ 0., 0., 2., 2.],
[ 0., 0., 3., 3.],
[ 0., 0., 4., 3.],
[ 0., 0., 5., 4.],
[ 0., 0., 6., 4.],
[ 0., 0., 7., 5.]],
[[ 1., 1., 0., 2.],
[ 1., 2., 0., 2.],
[ 1., 3., 0., 3.],
[ 1., 4., 0., 3.],
[ 1., 5., 0., 4.],
[ 1., 6., 0., 5.],
[ 1., 7., 0., 5.]]], dtype=float32)
Indices returned by tf.where op: - shape: (3, 2)
array([[0, 0],
[0, 1],
[1, 0]])
tf.gather results: (shape = [3, 4])
array([[ 0., 0., 1., 2.],
[ 0., 0., 2., 2.],
[ 1., 1., 0., 2.]], dtype=float32)
desired results: = (2, sparse, 4)
array([[[ 0., 0., 1., 2.],
[ 0., 0., 2., 2.]],
[[ 1., 1., 0., 2.]]], dtype=float32)
What's the best way to achieve this, keeping in mind that tf.where = dynamic shapes and no guarantees of shape consistency across the 2nd dimension (axis=1)?
NB: Ignore this question - See my answer
I think its a Tensorflow version problem. In my version (1.2.1), I get the exact desired output from your inputs. However, I also tried the following code according to the older version.
import tensorflow as tf
indices = [[[0, 0, 0], [0, 0, 1], [0, 0, 2], [0, 0, 3]],
[[0, 1, 0], [0, 1, 1], [0, 1, 2], [0, 1, 3]],
[[1, 0, 0], [1, 0, 1], [1, 0, 2], [1, 0, 3]]]
params = [[[ 0., 0., 1., 2.],
[ 0., 0., 2., 2.],
[ 0., 0., 3., 3.],
[ 0., 0., 4., 3.],
[ 0., 0., 5., 4.],
[ 0., 0., 6., 4.],
[ 0., 0., 7., 5.]],
[[ 1., 1., 0., 2.],
[ 1., 2., 0., 2.],
[ 1., 3., 0., 3.],
[ 1., 4., 0., 3.],
[ 1., 5., 0., 4.],
[ 1., 6., 0., 5.],
[ 1., 7., 0., 5.]]]
output = tf.gather_nd(params, indices)
with tf.Session()as sess:
print (sess.run(output))
Hope this helps.
I realized the idiocy of my question.
# of tuples when 1st dim is 0 != # of tuples when 1st dim is 1
I'm not sure that what I'm asking is feasible ...
I am using theano.clip to limit values of my numpy array. For e.g.
array = np.array([[ 1., -1., -3., 1., 1.],
[ 3., -4., -5., 0., -1.],
[ 8., -3., -7., -3., -3.],
[ 8., 2., -2., -3., -3.],
[ 7., 0., 0., 1., 0.]])
max_val = np.array([2.0]).astype('float32')
T.clip(array, -max_val, max_val).eval()
Output:
array([[ 1., -1., -2., 1., 1.],
[ 2., -2., -2., 0., -1.],
[ 2., -2., -2., -2., -2.],
[ 2., 2., -2., -2., -2.],
[ 2., 0., 0., 1., 0.]])
I want to calculate how many values were clipped after the clipping operation. Is it possible?
Here's one approach with np.count_nonzero on a mask of values beyond the limits computed with comparison against the min and max limits -
np.count_nonzero((array < -max_val) | (array > max_val))
np.count_nonzero is meant for performance, as it operates on a mask/boolean array to get the total count pretty efficiently.
Alternatively, a shorter version using absolute values as the min and max limits as in this case they are just negative and positive values of the same limiting number -
np.count_nonzero(np.abs(array) > max_val)
Sample run -
In [267]: array
Out[267]:
array([[ 1., -1., -3., 1., 1.],
[ 3., -4., -5., 0., -1.],
[ 8., -3., -7., -3., -3.],
[ 8., 2., -2., -3., -3.],
[ 7., 0., 0., 1., 0.]])
In [268]: max_val = np.array([2.0]).astype('float32')
In [269]: np.count_nonzero((array < -max_val) | (array > max_val))
Out[269]: 13
In [270]: np.count_nonzero(np.abs(array) > max_val)
Out[270]: 13
If your array's name is a, you can do
np.logical_or(a >= 1, a <= -1).sum()
You won't count elements twice, since - max_val < max_val. However, this requires two passes on a.
Let's say I have a matrix
x=array([[ 0., 0., 0.],
[ 0., 0., 1.],
[ 0., 1., 0.],
[ 0., 1., 1.],
[ 1., 0., 0.],
[ 1., 0., 1.],
[ 1., 1., 0.],
[ 1., 1., 1.]])
I want to get
array([[ 0., 0., 0.],
[ 0., 0., 2.],
[ 0., 3., 0.],
[ 0., 4., 4.],
[ 5., 0., 0.],
[ 6., 0., 6.],
[ 7., 7., 0.],
[ 8., 8., 8.]])
How to write the one-line expression between x and range(1,9)? And what is the code for the same operation for columns?
x * np.arange(1, 9).reshape(-1, 1)
or
x * arange(1, 9)[:, np.newaxis]
Both forms make a column vector out of arange(1, 9), which broadcasts nicely along the y axis of x.
"The same operation for columns" is just the transpose of the above, i.e. skip the reshape operation:
x * arange(1, 4)
In the package scipy there is the function to define a binary structure (such as a taxicab (2,1) or a chessboard (2,2)).
import numpy
from scipy import ndimage
a = numpy.zeros((6,6), dtype=numpy.int)
a[1:5, 1:5] = 1;a[3,3] = 0 ; a[2,2] = 2
s = ndimage.generate_binary_structure(2,2) # Binary structure
#.... Calculate Sum of
result_array = numpy.zeros_like(a)
What i want is to iterate over all cells of this array with the given structure s. Then i want to append a function to the current cell value indexed in a empty array (example function sum), which uses the values of all cells in the binary structure.
For example:
array([[0, 0, 0, 0, 0, 0],
[0, 1, 1, 1, 1, 0],
[0, 1, 2, 1, 1, 0],
[0, 1, 1, 0, 1, 0],
[0, 1, 1, 1, 1, 0],
[0, 0, 0, 0, 0, 0]])
# The array a. The value in cell 1,2 is currently one. Given the structure s and an example function such as sum the value in the resulting array (result_array) becomes 7 (or 6 if the current cell value is excluded).
Someone got an idea?
For the particular case of sums, you could use ndimage.convolve:
In [42]: import numpy as np
In [43]: a = np.zeros((6,6), dtype=np.int)
a[1:5, 1:5] = 1;
a[3,3] = 0;
a[2,2] = 2
In [48]: s = ndimage.generate_binary_structure(2,2) # Binary structure
In [49]: ndimage.convolve(a,s)
Out[49]:
array([[1, 2, 3, 3, 2, 1],
[2, 5, 7, 7, 4, 2],
[3, 7, 9, 9, 5, 3],
[3, 7, 9, 9, 5, 3],
[2, 4, 5, 5, 3, 2],
[1, 2, 3, 3, 2, 1]])
For the particular case of products, you could use the fact that log(a*b) = log(a)+log(b) to convert the problem back to one involving sums. For example, if we wanted to "product-convolve" b:
b = a[1:-1, 1:-1]
print(b)
# [[1 1 1 1]
# [1 2 1 1]
# [1 1 0 1]
# [1 1 1 1]]
we could compute:
print(np.exp(ndimage.convolve(np.log(b), s, mode = 'constant')))
# [[ 2. 2. 2. 1.]
# [ 2. 0. 0. 0.]
# [ 2. 0. 0. 0.]
# [ 1. 0. 0. 0.]]
The situation becomes more complicated if b includes negative values:
b[0,1] = -1
print(b)
# [[ 1 -1 1 1]
# [ 1 2 1 1]
# [ 1 1 0 1]
# [ 1 1 1 1]]
but not impossible:
logb = np.log(b.astype('complex'))
real, imag = logb.real, logb.imag
print(np.real_if_close(
np.exp(
sum(j * ndimage.convolve(x, s, mode = 'constant')
for x,j in zip((real, imag),(1,1j))))))
# [[-2. -2. -2. 1.]
# [-2. -0. -0. 0.]
# [ 2. 0. 0. 0.]
# [ 1. 0. 0. 0.]]
It's easier if you use a 2-deep wall of zeroes:
In [11]: a0
Out[11]:
array([[ 0., 0., 0., 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0., 0., 0., 0.],
[ 0., 0., 1., 1., 1., 1., 0., 0.],
[ 0., 0., 1., 2., 1., 1., 0., 0.],
[ 0., 0., 1., 1., 0., 1., 0., 0.],
[ 0., 0., 1., 1., 1., 1., 0., 0.],
[ 0., 0., 0., 0., 0., 0., 0., 0.],
[ 0., 0., 0., 0., 0., 0., 0., 0.]])
In [12]: b0 = zeros_like(a0)
In [13]: for i in range(1,len(a0)-1):
....: for j in range(1,len(a0)-1):
....: b0[i,j] = sum(a0[i-1:i+2, j-1:j+2] * s)
This enables you to multiply the two sub-matrices together and sum, as desired. (You could also do something more elaborate here...)
In [14]: b0
Out[14]:
array([[ 0., 0., 0., 0., 0., 0., 0., 0.],
[ 0., 1., 2., 3., 3., 2., 1., 0.],
[ 0., 2., 5., 7., 7., 4., 2., 0.],
[ 0., 3., 7., 9., 9., 5., 3., 0.],
[ 0., 3., 7., 9., 9., 5., 3., 0.],
[ 0., 2., 4., 5., 5., 3., 2., 0.],
[ 0., 1., 2., 3., 3., 2., 1., 0.],
[ 0., 0., 0., 0., 0., 0., 0., 0.]])
In [15]: b0[1:len(b0)-1, 1:len(b0)-1]
Out[15]:
array([[ 1., 2., 3., 3., 2., 1.],
[ 2., 5., 7., 7., 4., 2.],
[ 3., 7., 9., 9., 5., 3.],
[ 3., 7., 9., 9., 5., 3.],
[ 2., 4., 5., 5., 3., 2.],
[ 1., 2., 3., 3., 2., 1.]])