Related
I want to find the missing elements in I[:,1] and insert 0 for these elements. For instance, the missing elements in I[:,1] are 3,6. I want to insert 0 corresponding to these elements as shown in the expected output.
import numpy as np
I=np.array([[ 0.24414794669159817 , 1. ],
[ 0.2795127725932865 , 2. ],
[ 0.2630129055948728 , 4. ],
[ 0.2518744176621288 , 5. ],
[ 0.0000000000000051625370645, 7. ]])
The expected output is:
array([[ 0.24414794669159817 , 1. ],
[ 0.2795127725932865 , 2. ],
[ 0.0 , 3. ]
[ 0.2630129055948728 , 4. ],
[ 0.2518744176621288 , 5. ],
[ 0.0 , 6. ]
[ 0.0000000000000051625370645, 7. ]])
Try this out
Using numpy
mx = int(np.max(I[:, 1])) # find max length to construct new array
I2 = np.stack([np.zeros(mx), np.arange(1, mx + 1)], axis=1) # new array
indices = I[:,1].astype(int) # take column as index
I2[indices-1] = I # assign prev values to new array
Using For loop:
I1 = np.copy(I)
prev = 1
for i in I[:, 1]:
if abs(prev - i) >= 1:
I1 = np.insert(I1, int(i), [[0.0, float(i - 1)]], axis=0)
prev = i
print(I1)
You do not need the loop if you use np.isin:
# Create new matrix with zeros in first column.
idx_min = min(I[:,1])
idx_max = max(I[:,1])
idxs = np.arange(idx_min, idx_max+1)
vals = np.zeros_like(idxs)
I_new = np.stack([vals, idxs], axis=1)
# replace zeros with existing data
idx_data = np.isin(I_new[:,1], I[:,1])
I_new[idx_data, 0] = I[:,0]
I have come across some code (which may answer this question of mine). Here is the code (from Vivek Maskara's solution to my issue):
import cv2 as cv
import numpy as np
def read(image_path, label):
image = cv.imread(image_path)
image = cv.cvtColor(image, cv.COLOR_BGR2RGB)
image_h, image_w = image.shape[0:2]
image = cv.resize(image, (448, 448))
image = image / 255.
label_matrix = np.zeros([7, 7, 30])
for l in label:
l = l.split(',')
l = np.array(l, dtype=np.int)
xmin = l[0]
ymin = l[1]
xmax = l[2]
ymax = l[3]
cls = l[4]
x = (xmin + xmax) / 2 / image_w
y = (ymin + ymax) / 2 / image_h
w = (xmax - xmin) / image_w
h = (ymax - ymin) / image_h
loc = [7 * x, 7 * y]
loc_i = int(loc[1])
loc_j = int(loc[0])
y = loc[1] - loc_i
x = loc[0] - loc_j
if label_matrix[loc_i, loc_j, 24] == 0:
label_matrix[loc_i, loc_j, cls] = 1
label_matrix[loc_i, loc_j, 20:24] = [x, y, w, h]
label_matrix[loc_i, loc_j, 24] = 1 # response
return image, label_matrix
Would it be possible for you to explain how this part of the code works and what it specifically does:
if label_matrix[loc_i, loc_j, 24] == 0:
label_matrix[loc_i, loc_j, cls] = 1
label_matrix[loc_i, loc_j, 20:24] = [x, y, w, h]
label_matrix[loc_i, loc_j, 24] = 1 # response
I will first create and explain a simplified example, and then explain the part you pointed.
First, we create the ndarray named label_matrix:
import numpy as np
label_matrix = np.ones([2, 3, 4])
print(label_matrix)
This code means that you wil get an array containing 2 arrays, each of these 2 arrays will contain 3 arrays, and each of these 3 arrays will contain 4 elements.
And because we used np.ones, all these elements will have a value of 1.
So, printing label_matrix wil output this:
[[[1. 1. 1. 1.]
[1. 1. 1. 1.]
[1. 1. 1. 1.]]
[[1. 1. 1. 1.]
[1. 1. 1. 1.]
[1. 1. 1. 1.]]]
Now, we will change the values of first 4 elements of the first array contained by the first array of label_matrix.
To acces the first array of label_matrix, we do: label_matrix[0]
To access the first array contained by the first array of label_matrix we do: label_matrix[0, 0]
To access the first element of the first array contained by the first array of label_matrix we do: label_matrix[0, 0, 0]
To access the second element of the first array contained by the first array of label_matrix we do: label_matrix[0, 0, 1]
etc.
So, now, we will change the values of first 4 elements of the first array contained by the first array of label_matrix:
label_matrix[0, 0, 0] = 100
label_matrix[0, 0, 1] = 200
label_matrix[0, 0, 2] = 300
label_matrix[0, 0, 2] = 400
Output of label_matrix:
[[[100. 200. 300. 400.]
[ 1. 1. 1. 1.]
[ 1. 1. 1. 1.]]
[[ 1. 1. 1. 1.]
[ 1. 1. 1. 1.]
[ 1. 1. 1. 1.]]]
But we could have written it like this, instead of wrting 4 lines of codes:
label_matrix[0, 0, 0:4] = [100,200,300,400]
Writing label_matrix[0, 0, 0:4] means:
in the first array contained by the first array of label_matrix, select the 4 first elements (from index 0 to 4 (4 being not included))
So now you know the meaning of each line.
I'll explain the part of code you pointed:
if label_matrix[loc_i, loc_j, 24] == 0::
Test if the element at index 24 (the 23th element) has value 0
if yes, then:
label_matrix[loc_i, loc_j, cls] = 1:
assign the value 1 to the element at index cls. (If the variable named cls has value 4, it will assigne the value 1 to the element at index 4 of the first array contained by the first array of label_matrix)
label_matrix[loc_i, loc_j, 20:24] = [x, y, w, h]:
Say "x==100", "y==200", "w==300" and "h==400". So, in the first array contained by the first array of label_matrix, assign value 100 to the elemnt at index 20, value 200 to the elemnt at index 21, 300 at index 22 and 400 to index 23
label_matrix[loc_i, loc_j, 24] = 1:
in the first array contained by the first array of label_matrix, assign value 1 to the element at index 24
I am trying to do some math with my matrix, i can write it down but i am not sure how to code it. This involves getting a column of row marginal values, then making a new matrix that has all non-zero row values replaced with the marginals, after that I would like to divide the sum of non zero new values to be the column marginals.
I can get to the row marginals but I cant seem to think of a way to repopulate.
example of what i want
import numpy as np
matrix = np.matrix([[1,3,0],[0,1,2],[1,0,4]])
matrix([[1, 3, 0],
[0, 1, 2],
[1, 0, 4]])
marginals = ((matrix != 0).sum(1) / matrix.sum(1))
matrix([[0.5 ],
[0.66666667],
[0.4 ]])
What I want done next is a filling of the matrix based on the non zero locations of the first.
matrix([[0.5, 0.5, 0],
[0, 0.667, 0.667],
[0.4, 0, 0.4]])
Final wanted result is the new matrix column sum divided by the number of non zero occurrences in that column.
matrix([[(0.5+0.4)/2, (0.5+0.667)/2, (0.667+0.4)/2]])
To get the final matrix we can use matrix-multiplication for efficiency -
In [84]: mask = matrix!=0
In [100]: (mask.T*marginals).T/mask.sum(0)
Out[100]: matrix([[0.45 , 0.58333334, 0.53333334]])
Or simpler -
In [110]: (marginals.T*mask)/mask.sum(0)
Out[110]: matrix([[0.45 , 0.58333334, 0.53333334]])
If you need that intermediate filled output too, use np.multiply for broadcasted elementwise multiplication -
In [88]: np.multiply(mask,marginals)
Out[88]:
matrix([[0.5 , 0.5 , 0. ],
[0. , 0.66666667, 0.66666667],
[0.4 , 0. , 0.4 ]])
I am using python-3.x, and I am trying to generate an initial population that contains random real numbers between 0 and 1 where these numbers should be one of the following:
0, 0.33333, 0.666667 or 1
That means the difference between these numbers is 0.33333 (1/3). I tried to modify this code in many ways but their no luck
import numpy as np
import random
from random import randint
from itertools import product
pop_size = 7
i_length = 2
i_min = 0
i_max = 1
level = 2
step = ((1/((2**level)-1))*(i_max-i_min))
def individual(length, min, max):
return [ randint(min,max) for x in range(length) ]
def population(count, length, min, max):
return [ individual(length, min, max) for x in range(count) ]
population = population(pop_size, i_length, i_min, i_max)
##count: the number of individuals in the population
##length: the number of values per individual
##min: the minimum possible value in an individual's list of values
##max: the maximum possible value in an individual's list of values
##this code was taken from :https://lethain.com/genetic-algorithms-cool-name-damn-simple/
I did this lines which works very well for me:
population2 = np.array(list(product(np.linspace(i_min, i_max, 2**level), repeat=2)))
population3 = [j for j in product(np.arange(i_min, i_max+step, step), repeat=2)]
but the problem it will list all the possible values which are not what I want. I want random numbers where the population size will be given
the result I want to see is smailar to (numpy array or list):
population = [[0, 1],
[0, 0.3333],
[0.3333, 1],
[1, 0.6667],
[0.3333, 0.6667],
[0.6667, 0],
[0.3333, 0.3333]]
keep in mind the:
level = 2
where I can calculat the the step value:
step = ((1/((2**level)-1))*(i_max-i_min))
for example, if I changed the level = 2 to level = 3 then it is no more using the 0.3333 it will change to 0.1428 1/7) which I will get different values.
Any advice would be much appreciated
>>> np.random.choice([0, 1/3., 2/3., 1], size=(7,2), replace=True)
array([[0. , 0.33333333],
[0.33333333, 0.66666667],
[0. , 0. ],
[0.66666667, 0. ],
[0.33333333, 0.33333333],
[1. , 1. ],
[0.33333333, 0.33333333]])
>>> i_min = 0
>>> i_max = 1
>>> level = 3
>>> np.random.choice(np.linspace(i_min, i_max, 2**level), size=(7,2), replace=True)
array([[0.28571429, 0.14285714],
[0.85714286, 0.57142857],
[0.71428571, 0.42857143],
[0.71428571, 1. ],
[0.14285714, 0.85714286],
[0. , 0. ],
[1. , 0. ]])
Without numpy:
from random import randint
def get_population(num, repeats, fraction):
return [[randint(0, fraction)/fraction for x in range(num)] for i in range(repeats)]
print(get_population(2, 7, 3))
Output is:
[[0.3333333333333333, 0.0],
[0.3333333333333333, 1.0],
[1.0, 0.3333333333333333],
[0.3333333333333333, 0.0],
[0.0, 0.3333333333333333],
[0.3333333333333333, 0.6666666666666666],
[1.0, 1.0]]
Fraction 7:
print(get_population(2, 7, 7))
Output is:
[[0.8571428571428571, 0.7142857142857143],
[0.7142857142857143, 0.14285714285714285],
[0.0, 0.7142857142857143],
[0.42857142857142855, 0.5714285714285714],
[0.42857142857142855, 0.7142857142857143],
[1.0, 0.5714285714285714],
[1.0, 1.0]]
I'm trying to write an operation in tensorflow that propagates only the top k values of each feature map.
example:
k=1, input size is [batch_size, x, y, channels] let's say it's [1,2,2,3]
The output should be the same size and if k=1 than each x,y plane will have only one nonzero.
example in numpy:
input = [[[[6.4 1.4 1.3] [2.1 6.5 4.8]][[2.3 9.2 2.8][7.9 5.1 0.6]]]]]
output should be:
[[[[6.4 0. 0.] [0. 6.5 0.]] [[0. 9.2 0.] [7.9 0. 0.]]]]
in order to do this in tensorflow I would like to use nn.top_k followed by scatter_nd.
the problem is that top_k returns the indices of the requested elements very diffrently from how scatter_nd needs it.
top_k returns array of indices[[[[0],[1]], [[1],[0]]]] in shape (1,2,2,1)
scatter_nd needs it as a list of all coordinates per value like this:
[[0, 0, 0, 0], [0, 0, 1, 1], [0, 1, 0, 1], [0, 1, 1, 0]]
Does anyone know about a way to convert between them? or even maybe diffrent approch entirly for this operation?
tf.nn.top_k() only returns the top k values in the last dimension. So you have to add back all the other dimensions. Easiest with tf.where(). Code (tested):
import tensorflow as tf
inp = tf.constant( [ [ [ [6.4, 1.4, 1.3], [2.1, 6.5, 4.8] ], [ [2.3, 9.2, 2.8], [7.9, 5.1, 10.6] ] ] ] )
t, idx = tf.nn.top_k( inp, k = 2 )
idx_one_hot = tf.one_hot( idx, depth = 3 )
idx_red = tf.reduce_sum( idx_one_hot, axis = -2 )
idx2 = tf.where( tf.not_equal( idx_red, 0 ) )
with tf.Session() as sess:
print( sess.run( idx2 ) )
Outputs (note I've changed the last number in your example to have an index of 2 as well, only 0s and 1s looked a bit misleading as if it were a boolean tensor):
[[0 0 0 0]
[0 0 0 1]
[0 0 1 1]
[0 0 1 2]
[0 1 0 1]
[0 1 0 2]
[0 1 1 0]
[0 1 1 2]]
Please note, this loses the order of indices in the last dimension as reported by top_k, it changes it to increasing order in the indices themselves.