Related
I have several lists that can only contain the following values: 0, 0.5, 1, 1.5
I want to efficiently convert each of these lists into probability mass functions. So if a list is as follows: [0.5, 0.5, 1, 1.5], the PMF will look like this: [0, 0.5, 0.25, 0.25].
I need to do this many times (and with very large lists), so avoiding looping will be optimal, if at all possible. What's the most efficient way to make this happen?
Edit: Here's my current system. This feels like a really inefficient/unelegant way to do it:
def get_distribution(samplemodes1):
n, bin_edges = np.histogram(samplemodes1, bins = 9)
totalcount = np.sum(n)
bin_probability = n / totalcount
bins_per_point = np.fmin(np.digitize(samplemodes1, bin_edges), len(bin_edges)-1)
probability_perpoint = [bin_probability[bins_per_point[i]-1] for i in range(len(samplemodes1))]
counts = Counter(samplemodes1)
total = sum(counts.values())
probability_mass = {k:v/total for k,v in counts.items()}
#print(probability_mass)
key_values = {}
if(0 in probability_mass):
key_values[0] = probability_mass.get(0)
else:
key_values[0] = 0
if(0.5 in probability_mass):
key_values[0.5] = probability_mass.get(0.5)
else:
key_values[0.5] = 0
if(1 in probability_mass):
key_values[1] = probability_mass.get(1)
else:
key_values[1] = 0
if(1.5 in probability_mass):
key_values[1.5] = probability_mass.get(1.5)
else:
key_values[1.5] = 0
distribution = list(key_values.values())
return distribution
Here are some solution for you to benchmark:
Using collections.Counter
from collections import Counter
bins = [0, 0.5, 1, 1.5]
a = [0.5, 0.5, 1.0, 0.5, 1.0, 1.5, 0.5]
denom = len(a)
counts = Counter(a)
pmf = [counts[bin]/denom for bin in Bins]
NumPy based solution
import numpy as np
bins = [0, 0.5, 1, 1.5]
a = np.array([0.5, 0.5, 1.0, 0.5, 1.0, 1.5, 0.5])
denom = len(a)
pmf = [(a == bin).sum()/denom for bin in bins]
but you can probably do better by using np.bincount() instead.
Further reading on this idea: https://thispointer.com/count-occurrences-of-a-value-in-numpy-array-in-python/
I'm trying to find the fundamental matrix between two images. The points of correspondence in my images are given as follows -
pts1_list =
[
[224.95256042, 321.64755249],
[280.72879028, 296.15835571],
[302.34194946, 364.82437134],
[434.68283081, 402.86990356],
[244.64321899, 308.50286865],
[488.62979126, 216.26953125],
[214.77470398, 430.75869751],
[299.20846558, 312.07217407],
[266.94125366, 119.36679077],
[384.41549683, 442.05865479],
[475.28448486, 254.28138733]
]
pts2_list =
[
[253.88285828, 335.00772095],
[304.884552, 308.89205933],
[325.33914185, 375.91308594],
[455.15515137, 411.18075562],
[271.48794556, 322.07028198],
[515.11816406, 221.74610901],
[245.31390381, 441.54830933],
[321.74771118, 324.31417847],
[289.86627197, 137.46456909],
[403.3711853, 451.08905029],
[496.16610718, 261.36074829]
]
I have found a code that does what I'm looking for, but it looks like it works only for 3D points.
I've linked the reference code links here and here, but fundamentally, the code snippets that I am looking at are -
def compute_fundamental(x1, x2):
'''Computes the fundamental matrix from corresponding points x1, x2 using
the 8 point algorithm.'''
n = x1.shape[1]
if x2.shape[1] != n:
raise ValueError('Number of points do not match.')
# Normalization is done in compute_fundamental_normalized().
A = numpy.zeros((n, 9))
for i in range(n):
A[i] = [x1[0, i] * x2[0, i], x1[0, i] * x2[1, i], x1[0, i] * x2[2, i],
x1[1, i] * x2[0, i], x1[1, i] * x2[1, i], x1[1, i] * x2[2, i],
x1[2, i] * x2[0, i], x1[2, i] * x2[1, i], x1[2, i] * x2[2, i],
]
# Solve A*f = 0 using least squares.
U, S, V = numpy.linalg.svd(A)
F = V[-1].reshape(3, 3)
# Constrain F to rank 2 by zeroing out last singular value.
U, S, V = numpy.linalg.svd(F)
S[2] = 0
F = numpy.dot(U, numpy.dot(numpy.diag(S), V))
return F / F[2, 2]
and
def setUp(self):
points = array([
[-1.1, -1.1, -1.1], [ 1.4, -1.4, -1.4], [-1.5, 1.5, -1], [ 1, 1.8, -1],
[-1.2, -1.2, 1.2], [ 1.3, -1.3, 1.3], [-1.6, 1.6, 1], [ 1, 1.7, 1],
])
points = homography.make_homog(points.T)
P = hstack((eye(3), array([[0], [0], [0]])))
cam = camera.Camera(P)
self.x = cam.project(points)
r = [0.05, 0.1, 0.15]
rot = camera.rotation_matrix(r)
cam.P = dot(cam.P, rot)
cam.P[:, 3] = array([1, 0, 0])
self.x2 = cam.project(points)
def testComputeFundamental(self):
E = sfm.compute_fundamental(self.x2[:, :8], self.x[:, :8])
In this code, the parameters that are being passed are 3 dimensional whereas my requirement is only a two-coordinate frame. I would like to know how to modify this code and how the A matrix should be calculated in my case. Thank you.
F, _ = cv2.findFundamentalMat(pts1_list, pts2_list)
So I have a 2D tf.float32 Tensor of xyz coords and a 1D tf.int32 Tensor of segment_ids.
I want to subtract every point from the mean of the corresponding segment_id.
Please check the code below:
x_index = tf.constant([1, 1, 2, 2])
y_index = tf.constant([1, 1, 3, 4])
points = tf.constant([[0.1, 0.1, 0.1],
[0.11, 0.11, 0.11],
[0.2, 0.3, 0.1],
[0.2, 0.4, 0.1]])
points_x_y_indices = tf.transpose(tf.stack([x_index, y_index]))
uniques, idx = tf.raw_ops.UniqueV2(x=points_x_y_indices, axis=[0], out_idx=tf.dtypes.int32)
n_pillars = int(tf.reduce_max(idx))+1
x_means = tf.math.unsorted_segment_mean(points[:, 0], idx, n_pillars)
y_means = tf.math.unsorted_segment_mean(points[:, 1], idx, n_pillars)
z_means = tf.math.unsorted_segment_mean(points[:, 2], idx, n_pillars)
Now, I have the means over every segment_id in x_means, y_means and z_means. How can I subtract those values from original points tensor?? of course without looping as I am trying to avoid tf.py_func
Thanks!
I figured it out, you can use
full_x_means = tf.gather(x_means, idx)
full_y_means = tf.gather(y_means, idx)
full_z_means = tf.gather(z_means, idx)
Then
pillar_points_xc = points[:, 0] - full_x_means
pillar_points_yc = points[:, 1] - full_y_means
pillar_points_zc = points[:, 2] - full_z_means
Problem:
I'm new to neural networks topic and today wanted to learn how to make my neural network to learn.
I'm trying to do an exercise found in the internet.
The sum of errors for all series should be:
1.501535 but I am getting 7.394650000000001, so I thought that my weights are not updating. And that's exactly the issue, but I have no idea how to update the weights correctly.
Code:
import numpy as np
def calculate(input_numbers: list[float], weights: np.array, iterations: int, alpha: float, goal: list[float]):
num_rows, num_cols = weights.shape
if not len(input_numbers) == num_cols:
print("Wrong matrix")
return 0
error = 0
prediction = np.dot(input_numbers, np.transpose(weights))
delta = prediction - goal
weights_delta = np.outer(delta, input_numbers)
weights = weights - (weights_delta * alpha)
error = error + (pow(prediction - goal, 2))
print("\nXXXXXXXXXXXXXXXX SUMMARY XXXXXXXXXXXXXXXXX")
print("delta :" + str(delta))
print("weights_delta :" + str(weights_delta))
print("Weights : " + str(weights))
print("error : " + str(error))
return np.sum(error)
input_1 = [8.5, 0.65, 1.2]
goal_1 = [0.1, 1, 0.1]
input_2 = [9.5, 0.8, 1.3]
goal_2 = [0, 1, 0]
input_3 = [9.9, 0.8, 0.5]
goal_3 = [0, 0, 0.1]
input_4 = [9.0, 0.9, 1.0]
goal_4 = [0.1, 1, 0.2]
weights_matrix = np.array([[0.1, 0.1, -0.3], [0.1, 0.2, 0.0], [0.0, 1.3, 0.1]])
for x in range(50):
print("\nITERATION: ", x)
error_sum = calculate(input_1, weights_matrix, 1, 0.01, goal_1)
error_sum = error_sum + calculate(input_2, weights_matrix, 1, 0.01, goal_2)
error_sum = error_sum + calculate(input_3, weights_matrix, 1, 0.01, goal_3)
error_sum = error_sum + calculate(input_4, weights_matrix, 1, 0.01, goal_4)
print("TOTAL ERROR: ", error_sum)
Would someone be so kind to guide me on where and how I should update the weights? I have tried returning weights in calculate(), but results were totally wrong, so I guess it should be done in a different way.
Ok, so, I don't know why this is a thing, probably something to do with value assignment. Like, in the first case, it defines a new variable weights, and in the second line it changes the array you've passed.
Anyway, change
weights = weights - (weights_delta * alpha)
to
weights -= (weights_delta * alpha)
I have encoded my images(masks) with dimensions (img_width x img_height x 1) with OneHotEncoder in this way:
import numpy as np
def OneHotEncoding(im,n_classes):
one_hot = np.zeros((im.shape[0], im.shape[1], n_classes),dtype=np.uint8)
for i, unique_value in enumerate(np.unique(im)):
one_hot[:, :, i][im == unique_value] = 1
return one_hot
After doing some data manipulation with deep learning, softmax activation function will result in probabilities instead of 0 and 1 values, so in my Decoder I wanted to implement the following approach:
Threshold the output to obtain 0 or 1 only.
Multiply each channel with weight equal to the channel index.
take the max between labels along channels axis.
import numpy as np
arr = np.array([
[[0.1,0.2,0,5],[0.2,0.4,0.7],[0.3,0.5,0.8]],
[[0.3,0.6,0 ],[0.4,0.9,0.1],[0 ,0 ,0.2]],
[[0.7,0.1,0.1],[0,6,0.1,0.1],[0.6,0.6,0.3]],
[[0.6,0.2,0.3],[0.4,0.5,0.3],[0.1,0.2,0.7]]
])
# print(arr.dtype,arr.shape)
def oneHotDecoder(img):
# Thresholding
img[img<0.5]=0
img[img>=0.5]=1
# weigts of the labels
img = [i*img[:,:,i] for i in range(img.shape[2])]
# take the max label
img = np.amax(img,axis=2)
print(img.shape)
return img
arr2 = oneHotDecoder(arr)
print(arr2)
My questions is:
How to git rid of the error:
line 15, in oneHotDecoder
img[img<0.5]=0 TypeError: '<' not supported between instances of 'list' and 'float'
Is there any other issues in my implementaion that you suggest to improve?
Thanks in advance.
You have typos with commas and dots with some of your items (e.g. your first list should be [0.1, 0.2, 0.5] instead of [0.1, 0.2, 0, 5]).
The fixed list is:
l = [
[[0.1,0.2,0.5],[0.2,0.4,0.7],[0.3,0.5,0.8]],
[[0.3,0.6,0 ],[0.4,0.9,0.1],[0 ,0 ,0.2]],
[[0.7,0.1,0.1],[0.6,0.1,0.1],[0.6,0.6,0.3]],
[[0.6,0.2,0.3],[0.4,0.5,0.3],[0.1,0.2,0.7]]
]
Then you could do:
np.array(l) # np.dstack(l) would work as well
Which would yield:
array([[[0.1, 0.2, 0.5],
[0.2, 0.4, 0.7],
[0.3, 0.5, 0.8]],
[[0.3, 0.6, 0. ],
[0.4, 0.9, 0.1],
[0. , 0. , 0.2]],
[[0.7, 0.1, 0.1],
[0.6, 0.1, 0.1],
[0.6, 0.6, 0.3]],
[[0.6, 0.2, 0.3],
[0.4, 0.5, 0.3],
[0.1, 0.2, 0.7]]])