Related
I have found a neural network for semantic segmentation purpose. The network works just fine, I feed my training, validation and test data and I get the output (segmented parts in different colors). Until here, all is OK. I am using Keras with Tensorflow 1.7.0, GPU enabled. Python version is 3.5
What I want to achieve though is to get access to the pixel groups (segments) so that I can get their boundaries' image coordinates, i.e. an array of points which forms the boundary of the segment X shown in green in the prediction image.
How to do that? Obviously I cannot put the entire code here but here is a snippet which I should modify to achieve what I would like to:
I have the following in my evaluate function:
def evaluate(model_file):
net = load_model(model_file, custom_objects={'iou_metric': create_iou_metric(1 + len(PART_NAMES)),
'acc_metric': create_accuracy_metric(1 + len(PART_NAMES), output_mode='pixelwise_mean')})
img_size = net.input_shape[1]
image_filename = lambda fp: fp + '.jpg'
d_test_x = TensorResize((img_size, img_size))(ImageSource(TEST_DATA, image_filename=image_filename))
d_test_x = PixelwiseSubstract([103.93, 116.78, 123.68], use_lane_names=['X'])(d_test_x)
d_test_pred = Predict(net)(d_test_x)
d_test_pred.metadata['properties'] = ['background'] + PART_NAMES
d_x, d_y = process_data(VALIDATION_DATA, img_size)
d_x = PixelwiseSubstract([103.93, 116.78, 123.68], use_lane_names=['X'])(d_x)
d_y = AddBackgroundMap(use_lane_names=['Y'])(d_y)
d_train = Join()([d_x, d_y])
print('losses:', net.evaluate_generator(d_train.batch_array_tuple_generator(batch_size=3), 3))
# the tensor which needs to be modified
pred_y = Predict(net)(d_x)
Visualize(('slices', 'labels'))(Join()([d_test_x, d_test_pred]))
Visualize(('slices', 'labels', 'labels'))(Join()([d_x, pred_y, d_y]))
As for the Predict function, here is the snippet:
Alternatively, I've found that by using the following, one can get access to the tensor:
# for sample_img, in d_x.batch_array_tuple_generator(batch_size=3, n_samples=5):
# aa = net.predict(sample_img)
# indexes = np.argmax(aa,axis=3)
# print(indexes)
# import pdb
# pdb.set_trace()
But I have no idea how this works, I've never used pdb, therefore no idea.
In case if anyone wants to also see the training function, here it is:
def train(model_name='refine_res', k=3, recompute=False, img_size=224,
epochs=10, train_decoder_only=False, augmentation_boost=2, learning_rate=0.001,
opt='rmsprop'):
print("Traning on: " + str(PART_NAMES))
print("In Total: " + str(1 + len(PART_NAMES)) + " parts.")
metrics = [create_iou_metric(1 + len(PART_NAMES)),
create_accuracy_metric(1 + len(PART_NAMES), output_mode='pixelwise_mean')]
if model_name == 'dummy':
net = build_dummy((224, 224, 3), 1 + len(PART_NAMES)) # 1+ because background class
elif model_name == 'refine_res':
net = build_resnet50_upconv_refine((img_size, img_size, 3), 1 + len(PART_NAMES), k=k, optimizer=opt, learning_rate=learning_rate, softmax_top=True,
objective_function=categorical_crossentropy,
metrics=metrics, train_full=not train_decoder_only)
elif model_name == 'vgg_upconv':
net = build_vgg_upconv((img_size, img_size, 3), 1 + len(PART_NAMES), k=k, optimizer=opt, learning_rate=learning_rate, softmax_top=True,
objective_function=categorical_crossentropy,metrics=metrics, train_full=not train_decoder_only)
else:
net = load_model(model_name)
d_x, d_y = process_data(TRAINING_DATA, img_size, recompute=recompute, ignore_cache=False)
d = Join()([d_x, d_y])
# create more samples by rotating top view images and translating
images_to_be_rotated = {}
factor = 5
for root, dirs, files in os.walk(TRAINING_DATA, topdown=False):
for name in dirs:
format = str(name + '/' + name) # construct the format of foldername/foldername
images_to_be_rotated.update({format: factor})
d_aug = ImageAugmentation(factor_per_filepath_prefix=images_to_be_rotated, rotation_variance=90, recalc_base_seed=True)(d)
d_aug = ImageAugmentation(factor=3 * augmentation_boost, color_interval=0.03, shift_interval=0.1, contrast=0.4, recalc_base_seed=True, use_lane_names=['X'])(d_aug)
d_aug = ImageAugmentation(factor=2, rotation_variance=20, recalc_base_seed=True)(d_aug)
d_aug = ImageAugmentation(factor=7 * augmentation_boost, rotation_variance=10, translation=35, mirror=True, recalc_base_seed=True)(d_aug)
# apply augmentation on the images of the training dataset only
d_aug = AddBackgroundMap(use_lane_names=['Y'])(d_aug)
d_aug.metadata['properties'] = ['background'] + PART_NAMES
# substract mean and shuffle
d_aug = Shuffle()(d_aug)
d_aug, d_val = RandomSplit(0.8)(d_aug)
d_aug = PixelwiseSubstract([103.93, 116.78, 123.68], use_lane_names=['X'])(d_aug)
d_val = PixelwiseSubstract([103.93, 116.78, 123.68], use_lane_names=['X'])(d_val)
# Visualize()(d_aug)
d_aug.configure()
d_val.configure()
print('training size:', d_aug.size())
batch_size = 4
callbacks = []
#callbacks += [EarlyStopping(patience=10)]
callbacks += [ModelCheckpoint(filepath="trained_models/"+model_name + '.hdf5', monitor='val_iou_metric', mode='max',
verbose=1, save_best_only=True)]
callbacks += [CSVLogger('logs/'+model_name + '.csv')]
history = History()
callbacks += [history]
# sess = K.get_session()
# sess.run(tf.initialize_local_variables())
net.fit_generator(d_aug.batch_array_tuple_generator(batch_size=batch_size, shuffle_samples=True), steps_per_epoch=d_aug.size() // batch_size,
validation_data=d_val.batch_array_tuple_generator(batch_size=batch_size), validation_steps=d_val.size() // batch_size,
callbacks=callbacks, epochs=epochs)
return {k: (max(history.history[k]), min(history.history[k])) for k in history.history.keys()}
for segmentation tasks, considering that your batch is one image, each pixel in the image is assigned a probability to belong to a class. Suppose you have 5 classes, and the image has 784 pixels(28x28) , you will get from the net.predict an array of shape (784,5) each pixel among 784 is assigned 5 probabilities values to belong to those classes. when you do np.argmax(aa,axis=3) you get the index of the highests probabilities for each pixel that would of shape (784,1) you can then reshape it to 28x28 indexes.reshape(28,28) and you get the mask of your predictions.
Reducing the problem to a 7x7 dimension and 4 classes(0-3) that looks like
array([[2, 1, 0, 1, 2, 3, 1],
[3, 1, 1, 0, 3, 0, 0],
[3, 3, 2, 2, 0, 3, 1],
[1, 1, 0, 3, 1, 3, 1],
[0, 0, 0, 3, 3, 1, 0],
[1, 2, 3, 0, 1, 2, 3],
[0, 2, 1, 1, 0, 1, 3]])
you want to extract the indexes where the model predicted 1
segment_1=np.where(indexes==1)
since its 2 dimension array, segment_1 will be 2x7 array,where the first array is the row indexes, and second array will be column value.
(array([0, 0, 0, 1, 1, 2, 3, 3, 3, 3, 4, 5, 5, 6, 6, 6]), array([1, 3, 6, 1, 2, 6, 0, 1, 4, 6, 5, 0, 4, 2, 3, 5]))
looking at first number in the first and second array,0 and 1 point to where the located in indexes
You can extract its value like
indexes[segment_1]
array([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1])
and then proceed with second class you want to get ,lets say 2
segment_2=np.where(image==2)
segment_2
(array([0, 0, 2, 2, 5, 5, 6]), array([0, 4, 2, 3, 1, 5, 1]))
and if you want to get each classes itsself.
you can create a copy of indexes for each class,4 copies in total class_1=indexes and set to zero any value that is not equal to 1. class_1[class_1!=1]=0 and get something like this
array([[0, 1, 0, 1, 0, 0, 1],
[0, 1, 1, 0, 0, 0, 0],
[0, 0, 0, 0, 0, 0, 1],
[1, 1, 0, 0, 1, 0, 1],
[0, 0, 0, 0, 0, 1, 0],
[1, 0, 0, 0, 1, 0, 0],
[0, 0, 1, 1, 0, 1, 0]])
for the eye, you may think that there are countour but from this example, you can tell that there is no clear contour of each segment. The only way i could think of,is to loop the image in rows and record where the value change and do the same in columns.
I am not entired sure if this would be ideal situation.
I hope i covered some part of your question.
PDB is just a debugging package that allows you execute your code step by step
I have this code below, I already optimised the algorithm to make it as fast as possible but it is still too slow. So a was thinking about using multiprocessing (I have no expierience with this kind of stuff), but I tried some things with pool and threading but either it was slower than before or didn't work. So is was wondering how I should do this so that it works and is faster. And if there are other options than multithreading to make kind of code this faster.
def calc(indices, data):
matrix = [[0] * len(indices) for i in range(len(indices))]
for i_a, i_b in list(itertools.combinations(indices, 2)):
a_res, b_res = algorithm(data[i_a], data[i_b])
matrix[i_b][i_a] = a_res
matrix[i_a][i_b] = b_res
return matrix
def algorithm(a,b):
# Verry slow and complex
Building upon Simon's answer, here is an example applying a multiprocessing pool to a version of your problem. Your mileage will vary depending on how many cores you have on your machine but I hope that this will be a helpful demonstration of how you could structure a solution to your problem:
import itertools
import numpy as np
import multiprocessing as mp
import time
def calc_mp(indices, data):
# construct pool
pool = mp.Pool(mp.cpu_count())
# we are going to populate the matrix; organize all the inputs; then map them
matrix = [[0] * len(indices) for i in range(len(indices))]
args = [(data[i_a], data[i_b]) for i_a, i_b in list(itertools.combinations(indices, 2))]
results = pool.starmap(algorithm, args)
# unpack the results into the matrix
for i_tuple, result in zip([(i_a, i_b) for i_a, i_b in list(itertools.combinations(indices, 2))], results):
# unpack
i_a, i_b = i_tuple
a_res, b_res = result
# set it in the matrix
matrix[i_b][i_a] = a_res
matrix[i_a][i_b] = b_res
return matrix
def calc_single(indices, data):
# do the simple single process version
matrix = [[0] * len(indices) for i in range(len(indices))]
for i_a, i_b in list(itertools.combinations(indices, 2)):
a_res, b_res = algorithm(data[i_a], data[i_b])
matrix[i_b][i_a] = a_res
matrix[i_a][i_b] = b_res
return matrix
def algorithm(a,b):
# Very slow and complex
time.sleep(2)
return a + b, a - b
if __name__ == "__main__":
# generate test data;
indices = range(5)
data = range(len(indices))
# test single
time_start = time.time()
print(calc_single(indices, data))
print("Took {}".format(time.time() - time_start))
# mp
time_start = time.time()
print(calc_mp(indices, data))
print("Took {}".format(time.time() - time_start))
The results, with 8 cores, are
[[0, -1, -2, -3, -4], [1, 0, -1, -2, -3], [2, 3, 0, -1, -2], [3, 4, 5, 0, -1], [4, 5, 6, 7, 0]]
Took 20.02155065536499
[[0, -1, -2, -3, -4], [1, 0, -1, -2, -3], [2, 3, 0, -1, -2], [3, 4, 5, 0, -1], [4, 5, 6, 7, 0]]
Took 4.073369264602661
Your best bet in Multiprocessing. You will need to partition your data into chunks and pass each chunk to a process. Threading won't help you in Python because all Python processes run on a single cpu thread. It's still useful for some use cases, such as where you have several activities going on some of which might block, but not for parallel workloads.
I need to have a fitness proportionate selection approach to a GA, however my population cant loose the structure (order), in this case while generating the probabilities, I believe the individuals get the wrong weights, the program is:
population=[[[0, 0, 1, 0, 1, 1, 0, 1, 1, 1, 1, 0, 0, 0, 0, 1], [6], [0]],
[[0, 0, 1, 1, 1, 0, 0, 1, 1, 0, 1, 1, 0, 0, 0, 1], [4], [1]],
[[0, 1, 1, 0, 1, 1, 0, 0, 1, 1, 1, 0, 0, 1, 0, 0], [6], [2]],
[[1, 0, 0, 1, 1, 1, 0, 0, 1, 1, 0, 1, 1, 0, 0, 0], [4], [3]]]
popultion_d={'0,0,1,0,1,1,0,1,1,1,1,0,0,0,0,1': 6,
'0,0,1,1,1,0,0,1,1,0,1,1,0,0,0,1': 4,
'0,1,1,0,1,1,0,0,1,1,1,0,0,1,0,0': 6,
'1,0,0,1,1,1,0,0,1,1,0,1,1,0,0,0': 4}
def ProbabilityList(population_d):
fitness = population_d.values()
total_fit = (sum(fitness))
relative_fitness = [f/total_fit for f in fitness]
probabilities = [sum(relative_fitness[:i+1]) for i in range(len(relative_fitness))]
return (probabilities)
def FitnessProportionateSelection(population, probabilities, number):
chosen = []
for n in range(number):
r = random.random()
for (i, individual) in enumerate(population):
if r <= probabilities[i]:
chosen.append(list(individual))
break
return chosen
number=2
The population element is: [[individual],[fitness],[counter]]
The probabilities function output is: [0.42857142857142855, 0.5714285714285714, 0.8571428571428571, 1.0]
What I notice here is that the previous weight is summed up to the next one, not necessarily being in crescent order, so a think a higher weight is given to the cromosome with a lowest fitness.
I dont want to order it because I need to index the lists by position later, so I think I will have wrong matches.
Anyone knows a possible solution, package or different approach to perform a weighted the selection in this case?
p.s: I know the dictionary may be redundant here, but I had several other problems using the list itself.
Edit: I tried to use random.choices() as you can see below (using relative fitness):
def FitnessChoices(population, probabilities, number):
return random.choices(population, probabilities, number)
But I get this error: TypeError: choices() takes from 2 to 3 positional arguments but 4 were given
Thank you!
Using random.choices is certainly a good idea. You just need to understand the function call. You have to specify, whether your probabilities are marginal or cumulated. So you could use either
import random
def ProbabilityList(population_d):
fitness = population_d.values()
total_fit = sum(fitness)
relative_fitness = [f/total_fit for f in fitness]
return relative_fitness
def FitnessChoices(population, relative_fitness, number):
return random.choices(population, weights = relative_fitness, k = number)
or
import random
def ProbabilityList(population_d):
fitness = population_d.values()
total_fit = sum(fitness)
relative_fitness = [f/total_fit for f in fitness]
cum_probs = [sum(relative_fitness[:i+1]) for i in range(len(relative_fitness))]
return cum_probs
def FitnessChoices(population, cum_probs, number):
return random.choices(population, cum_weights = cum_probs, k = number)
I'd recommend you to have a look at the differences between keyword and positional arguments in python.
I'm trying to visualize a pair of two lists, represented by lines_x and lines_y which are meant to be plugged into the coordinates argument of either the plot function in Axes or in Lines2D.
Right now, I'm getting this result, which has extra lines compared to the result I am trying to get.
What I'm currently getting:
Previously, I tried using a loop to plot the lines one by one, and that worked for a while. However, after a few runs, it no longer worked.
Could someone please suggest a way for me to achieve the following result on my window?
The plot I want to achieve:
from pylab import *
import matplotlib
from matplotlib.backends.backend_tkagg import FigureCanvasTkAgg
from matplotlib.figure import Figure
import matplotlib.pylab as plt
import matplotlib.pyplot as pltplot
import matplotlib.lines
from matplotlib.collections import LineCollection
matplotlib.use ("gTkAgg")
import numpy as np
import tkinter as tk
from tkinter import Tk
from tkinter import *
class Window (Frame):
lines_x = [-2, -2, -1, -1, 0, 0, 1, 1, 2, 2, 0, 1, 1, 2, -2, 2, -2, -1, -1, 0]
lines_y = [0, 1, 1, 2, -2, 2, -2, -1, -1, 0, -2, -2, -1, -1, 0, 0, 1, 1, 2, 2]
def __init__(self, parent = None):
Frame.__init__(self,parent)
parent.title("Shape Grammar Interpreter")
self.top=Frame()
self.top.grid()
self.top.update_idletasks
self.menu()
self.makeWidgets()
def makeWidgets(self):
self.f = Figure(figsize = (6,6), dpi = 100)
self.a = self.f.add_subplot(111)
#self.a.plot(self.lines_x, self.lines_y, linewidth = 4.0, picker=5)
line = Line2D(self.lines_x, self.lines_y)
self.a.add_line(line)
for i in range(len(self.lines_x)):
self.a.plot(self.lines_x[i:i+1], self.lines_y[i:i+1], linewidth = 4.0)
#self.a.plot(lines_x, lines_y, linewidth = 4.0, color = "blue")
self.a.margins(y=0.5)
self.a.margins(x=0.5)
#self.a.axes.get_xaxis().set_visible(False)
#self.a.axes.get_yaxis().set_visible(False)
# a tk.DrawingArea
self.canvas = FigureCanvasTkAgg(self.f, master=self.top)
#to show window
self.canvas.get_tk_widget().pack(side=tk.TOP, fill=tk.BOTH, expand=1)
def menu(self):
menubar = Menu (root)
#to close window
menubar.add_command(label="Exit", command=self.quit_window)
root.config(menu=menubar)
def quit_window(self):
root.quit()
root.destroy()
if __name__ == "__main__":
root = Tk()
my_gui = Window(root)
root.mainloop()
It makes sense if you annotate the order in which the line segments are drawn. For example (only plotting the first 10 points, otherwise it becomes a bit of a mess):
import matplotlib.pylab as pl
lines_x = [-2, -2, -1, -1, 0, 0, 1, 1, 2, 2, 0, 1, 1, 2, -2, 2, -2, -1, -1, 0]
lines_y = [0, 1, 1, 2, -2, 2, -2, -1, -1, 0, -2, -2, -1, -1, 0, 0, 1, 1, 2, 2]
n = 10
pl.figure()
pl.plot(lines_x[:n], lines_y[:n])
# Number the coordinates to indicate their order:
for i in range(len(lines_x[:n])):
pl.text(lines_x[i], lines_y[i], '{}'.format(i))
pl.xlim(-3,3)
pl.ylim(-3,3)
Results in:
If I increase n, it becomes a larger mess since a number of x,y coordinates are duplicates. So:
Make sure that there are no duplicate coordinates
Make sure that the coordinates are ordered correctly.
Try these sequences instead:
lines_x = [-2, -2, -1, -1, 0, 0, 1, 1, 2, 2, -2]
lines_y = [ 0, 1, 1, 2, 2, -2, -2, -1, -1, 0, 0]
Worked for me:
Also I should note that I used simply
In [1]: import matplotlib.pyplot as plt
In [2]: plt.plot(lines_x,lines_y)
So I believe Lines has nothing to do with it.
I don't want the answer, but I'm having trouble keeping track of the nodes. Meaning, say I have nodes 0, 1, 2, 3,4, 5, 6, 7, where 0 is start, and 7 is goal, I made an adjacency matrix like so:
[
[0, 3, 0, 0, 4, 0, 0, 0],
[3, 0, 0, 0, 5, 0, 8, 0],
[0, 0, 0, 4, 0, 5, 0, 0],
[0, 0, 4, 0, 0, 0, 0, 14],
[4, 5, 0, 0, 0, 2, 0, 0],
[0, 0, 5, 0, 2, 0, 4, 0],
[0, 8, 0, 5, 0, 4, 0, 0],
[0, 0, 0, 14, 0, 0, 0, 0]
]
if it's a 0, there is no link between the nodes, otherwise, if it's greater than 1, then the number is the weight of the edge between those nodes.
I'm having trouble identifying what the actual node would be, versus a path.
I can find the goal, but I wouldn't know how to show the path to the goal, and what the total weight would be?
EDIT:
Here is what I am trying to achieve (this will not work, but this is the general idea):
def dfs(graph, start, goal):
stack = []
visited = []
stack.append(graph[start])
visited.append(start)
while (len(stack) != 0):
current_node = stack.pop()
if current_node not in visited:
visited.append(current_node)
if current_node = goal:
return path
else:
for nodes in current_node:
if nodes not in visited:
stack.append(nodes)
if the edges were unweighed this would be easier, but I'm basically adding all neighbors of the current node as long as I haven't visited it to the stack, until I find the goal node, and then I want to return the path. but in this case I know it's broken because 1) I'm not sure how to check if it's the goal node, since I'm only storing the nodes neighbors, and 2) not checking for the full path.
Maintain a path variable to store the vertex as u encounter them. When u found end vertex, the path variable will have the path.
Find the pseudo code for reference. Pardon any minor mistake in the code
DFS (vertex start, vertex end, Graph G, list path):
if(start==end):
return TRUE
for vertex in adjacent(start):
if vertex not in path: # has not been traversed
path = path + [vertex]
a = DFS(vertex, end, G, path)
if a==TRUE: # end vertex was found
return TRUE
path.delete(vertex) # delete the vertex added,as its not in the path from start to end
Acc. to your code, when u found the goal vertex, the visited stack is contains the element in the path.
I hope it helped.