Related
This is my code:
class solution:
lot = [[1,0,0], [1,0,0], [1,9,1]]
def move(lot):
if lot is None:
return -1
if (len(lot) <= 0 or len(lot[0]) <= 0):
return -1
q = []
visited = [len(lot)], [len(lot[0])]
direction= [(0,1), (0,-1), (1,0), (-1,0)]
q.append((0,0))
result = 0
while (len(q) > 0):
size = len(q)
for i in range(size):
node= q.pop(0)
x = node[0]
y = node[1]
visited[x][y]= True
if(lot[x][y] == 9):
return result
for dir in direction:
nx = x+ dir[0]
ny = y + dir[1]
r= len(lot[nx])
if (nx < 0 or nx >= len(lot) or ny < 0 or ny > r or lot[nx][ny] == 0 or visited[nx][ny] == True):
continue
q.append((nx,ny))
result += result
return result
print(move(lot))
It generates the following error:
File "prog.py", line 29, in move
if (nx < 0 or nx >= len(lot) or ny < 0 or ny > r or lot[nx][ny] == 0 or visited[nx][ny] == True):
IndexError: tuple index out of range
Your error is because of your visited variable. You're trying to access visited[nx][ny] which doesn't exist.
Separately, your DFS algorithm is incorrect. See here for how to implement DFS correctly.
Your function should be like this:
def move(lot):
if lot is None:
return -1
if (len(lot) <= 0 or len(lot[0]) <= 0):
return -1
q = []
visited = set()
q.append((0,0))
result = 0
while len(q)>0:
x,y = q.pop(0)
if(lot[x][y]==9):
return result
if (x,y) in visited:
continue
visited.add((x,y))
for d in [(0,1), (0,-1), (1,0), (-1,0)]:
nx = x + d[0]
ny = y + d[1]
if (nx<0 or nx>=len(lot) or ny<0 or ny>len(lot[0]) or lot[nx][ny]==0 or (nx,ny) in visited):
continue
q.append((nx,ny))
result += 1
return result
just a small labyrinth solver. try it to on python but dont even understand why it doesnt print anything. Im just started with python and trying to actually translate my teachers code from cpp just for interest.
need to reach to the destination after visiting each cell
m = [
[2,0,0,0,0,0],
[0,0,0,0,0,0],
[0,0,0,0,0,0],
[0,0,0,0,0,0],
[0,0,0,2,0,0],
[0,0,0,0,0,0]
]
STOP = 34
r = 0
c = 0
def init( r_: int = 0, c_: int = 0) -> None:
r = 0
c = 0
def step(r,c,level):
if level==STOP-1:
if r == 3 and c == 5:
for i in range(0,level):
print("("+init(int(r),int(c))+"," + init(int(r),int(c)) + ") - ")
print("(3,5)\n")
return True
else:
return False
m[r][c]=1
if c > 0 and m[r][c - 1] == 0:
if step(r, c - 1, level + 1):
return True
if c < 5 and m[r][c + 1] == 0:
if (step(r, c + 1, level + 1)):
return True
if r > 0 and m[r - 1][c] == 0:
if (step(r - 1, c, level + 1)):
return True
if r < 5 and m[r + 1][c] == 0:
if (step(r + 1, c, level + 1)):
return True
m[r][c] = 0
return False
def main(argc, argv):
step(0, 1, 0)
Python doesn't automatically call main like c does, nor does it require a main function. Your code can just live in the top level (e.g. print('hello world') is a valid, complete program)
Change this line:
def main(argc, argv):
to this:
if __name__ == '__main__':
got it
import matplotlib.pyplot as plt
S, F, W, X = 1, 2, 0, None
FIELD = [[X,S,W,W,W,W],
[W,W,W,W,W,W],
[W,W,W,W,W,W],
[W,W,W,W,W,F],
[W,W,W,X,W,W],
[W,W,W,W,W,W]]
ROWS = len(FIELD)
COLS = len(FIELD[0])
START_POS = [(i, line.index(S)) for i, line in enumerate(FIELD) if S in line][0]
FINISH_POS = [(i, line.index(F)) for i, line in enumerate(FIELD) if F in line][0]
STEPS = sum(line.count(W) for line in FIELD) + 1
print("Start:", START_POS, "\nFinish:", FINISH_POS, "\nTotal steps:", STEPS)
def do_step(level, path, points):
global FIELD
r, c = points[-1]
if r<0 or c<0 or r>=ROWS or c>=COLS or FIELD[r][c] == X:
return
if level == STEPS and (r, c) == FINISH_POS:
print("Found path:", path)
yy, xx = zip(*points)
plt.plot(xx, yy, "-ok")
plt.gca().invert_yaxis()
plt.show()
exit()
current = FIELD[r][c]
FIELD[r][c] = X
for dr, dc, dir in ((-1,0,'^'), (0,1,'>'), (1,0,'v'), (0,-1,'<')):
do_step(level+1, path+dir, points+[(r+dr,c+dc)])
FIELD[r][c] = current
do_step(0, "", [START_POS])
print("No path's found.")
I am making a n-puzzle solver for my Artificial Intelligence class using A* in Python. The problem I have with my solution is , that the solve_puzzle() is not working as it should. While searching through the nodes it gets just deeper and deeper, but it never ends, the depth goes into really high numbers (depth of some child node is 5k for example).
I think it gets stuck in some loop and it keeps just looping through the same nodes (That is just what I think, I am not sure about that). I don't really know, I tried 3 different A* solutions, but the result is the same - the search loop never ends and never reaches the Goal.
I can solve very primitive puzzle like
start -> 0, 1, 2, 3, 4, 5, 6, 7, 8 goal -> 1, 2, 0, 3, 4, 5, 6, 7, 8 (just two moves to left are made)
but for just a little more complex puzzle like
start = 0, 1, 2, 3, 4, 5, 6, 7, 8
goal = 1, 2, 5, 0, 3, 4, 6, 7, 8 it is a never ending search.
I move the tile with number, not the blank tile.
Below is the whole code:
'''
class Node():
def __init__(self, n_size, m_size, current_state, goal_state, choosen_heuristic, parent):
self.n_size = n_size
self.m_size = m_size
self.dimension = self.n_size * self.m_size
self.current_state = current_state
self.goal_state = goal_state
self.choosen_heuristic = choosen_heuristic
self.parent = parent
self.child = [None, None, None, None]
self.last_operator = None
self.heuristic_cost = 0
self.depth = 0
self.priority = self.depth + self.heuristic_cost
def check_if_goal_is_reached(self):
if (self.heuristic_cost == 0):
print("GOAL IS REACHED!")
exit(1) #for now
#return
def get_blank_tile_position(self):
blank_position = 0
for i in range(self.dimension):
if (self.current_state[i] == 0):
blank_position = i
return blank_position
def misplaced_tiles_heuristic(self):
misplaced_sum = 0
for i in range(self.dimension):
if self.current_state[i] != self.goal_state[i]:
misplaced_sum += 1
#print("Count of misplaced tiless in this node is : ", misplaced_sum)
self.heuristic_cost = "misplaced"
self.heuristic_cost = misplaced_sum
self.check_if_goal_is_reached()
return misplaced_sum
def manhattan_distance(self):
distance_sum = 0
for i in range(self.dimension):
current_x = self.current_state[i] % self.n_size
goal_x = self.goal_state[i] % self.n_size
current_y = self.current_state[i] // self.m_size
goal_y = self.goal_state[i] // self.m_size
distance_sum += abs(current_x - goal_x) + abs(current_y - goal_y)
#print("Sum of Manhattan distance for this node is : ", distance_sum)
self.heuristic_cost = "manhattan"
#print("Hĺbka tohto uzla : ", self.depth)
self.check_if_goal_is_reached()
return distance_sum
def generate_children(self, choosen_heuristic):
possible_directions = []
current_node_blank_position = self.get_blank_tile_position()
# UP - I move a tile with number on it, not the blank tile
if current_node_blank_position < (self.dimension - self.n_size):
self.child[0] = Node(self.n_size, self.m_size, self.current_state, self.goal_state, self.choosen_heuristic, self.current_state)
self.child[0] = copy.deepcopy(self)
self.child[0].parent = self.current_state
self.child[0].last_operator = "UP"
self.child[0].depth = self.depth + 1
new_blank_position = current_node_blank_position + self.m_size
temp = self.child[0].current_state[current_node_blank_position]
self.child[0].current_state[current_node_blank_position] = self.child[0].current_state[new_blank_position]
self.child[0].current_state[new_blank_position] = temp
if choosen_heuristic == "misplaced":
self.child[0].misplaced_tiles_heuristic()
if choosen_heuristic == "manhattan":
self.child[0].manhattan_distance()
possible_directions.append("UP")
print("Depth of this node is : : ", self.child[0].depth)
else:
self.child[0] = None
# DOWN - I move a tile with number on it, not the blank tile
if current_node_blank_position > (self.n_size - 1):
self.child[1] = Node(self.n_size, self.m_size, self.current_state, self.goal_state, self.choosen_heuristic, self.current_state)
self.child[1] = copy.deepcopy(self)
self.child[1].parent = self.current_state
self.child[1].last_operator = "DOWN"
self.child[1].depth = self.depth + 1
new_blank_position = current_node_blank_position - self.m_size
temp = self.child[1].current_state[current_node_blank_position]
self.child[1].current_state[current_node_blank_position] = self.child[1].current_state[new_blank_position]
self.child[1].current_state[new_blank_position] = temp
if choosen_heuristic == "misplaced":
self.child[1].misplaced_tiles_heuristic()
if choosen_heuristic == "manhattan":
self.child[1].manhattan_distance()
possible_directions.append("DOWN")
#print("Depth of this node is : : ", self.child[1].depth)
else:
self.child[1] = None
# RIGHT - I move a tile with number on it, not the blank tile
if (current_node_blank_position + self.n_size) % self.m_size:
self.child[2] = Node(self.n_size, self.m_size, self.current_state, self.goal_state, self.choosen_heuristic, self.current_state)
self.child[2] = copy.deepcopy(self)
self.child[2].parent = self.current_state
self.child[2].last_operator = "RIGHT"
self.child[2].depth = self.depth + 1
new_blank_position = current_node_blank_position - 1
temp = self.child[2].current_state[current_node_blank_position]
self.child[2].current_state[current_node_blank_position] = self.child[2].current_state[new_blank_position]
self.child[2].current_state[new_blank_position] = temp
if choosen_heuristic == "misplaced":
self.child[2].misplaced_tiles_heuristic()
if choosen_heuristic == "manhattan":
self.child[2].manhattan_distance()
possible_directions.append("RIGHT")
#print("Depth of this node is : : ", self.child[2].depth)
else:
self.child[2] = None
# LEFT - I move a tile with number on it, not the blank tile
if (current_node_blank_position + 1) % self.m_size:
self.child[3] = Node(self.n_size, self.m_size, self.current_state, self.goal_state, self.choosen_heuristic, self.current_state)
self.child[3] = copy.deepcopy(self)
self.child[3].parent = self.current_state
self.child[3].last_operator = "LEFT"
self.child[3].depth = self.depth + 1
new_blank_position = current_node_blank_position + 1
temp = self.child[3].current_state[current_node_blank_position]
self.child[3].current_state[current_node_blank_position] = self.child[3].current_state[new_blank_position]
self.child[3].current_state[new_blank_position] = temp
if choosen_heuristic == "misplaced":
self.child[3].misplaced_tiles_heuristic()
if choosen_heuristic == "manhattan":
self.child[3].manhattan_distance()
possible_directions.append("LEFT")
#print("Depth of this node is : ", self.child[3].depth)
else:
self.child[3] = None
#print("From this node (the parent node) I can move to -> ", possible_directions)
def get_node_priority(node):
return node.priority
def solve_puzzle(n_size, m_size, current_state, goal_state, choosen_heuristic):
open_list = []
closed_list = []
init_node_parent = None
init_node = Node(n_size, m_size, current_state, goal_state, choosen_heuristic, init_node_parent)
open_list.append(init_node)
while len(open_list) != 0:
if (len(open_list) == 0):
print("Fail - solution not found !")
else:
node = open_list.pop(0)
if (node.parent != None):
node.check_if_goal_is_reached()
node.generate_children(choosen_heuristic)
closed_list.append(node)
temp_list = []
for i in range(4):
if node.child[i] != None:
temp_list.insert(0, node.child[i])
sorted_temp_list = sorted(temp_list, key=get_node_priority, reverse=True)
for x in range(len(sorted_temp_list)):
if sorted_temp_list[x] != None:
open_list.insert(0, sorted_temp_list[x])
def print_current_node(current_node):
puzzle = ""
if current_node is None:
puzzle = ""
print(puzzle)
else:
puzzle = ""
count_lines = 0
for i in range(current_node.dimension):
puzzle += (str(current_node.current_state[i]) + " ")
count_lines += 1
if (count_lines % current_node.m_size == 0):
puzzle += "\n"
print(puzzle)
def main():
###################
# Static Data #
###################
# static for now, later I will let user to choose
n_size = 3
m_size = 3
current_state = [0, 1, 2, 3, 4, 5, 6, 7, 8]
goal_state = [8, 0, 6, 5, 4, 7, 2, 3, 1]
choosen_heuristic = "manhattan"
start_time = time.process_time()
solve_puzzle(n_size, m_size, current_state, goal_state, choosen_heuristic)
end_time = time.process_time()
search_time = end_time - start_time
print("Solved in : ", search_time, "seconds.")
main()
'''
Make sure you test with a solvable puzzle.
Here are some solvable tests:
goal: {1,2,3,4,5,6,7,8,0}
6 steps: {1,3,5,4,0,2,7,8,6}
18 steps: {1,4,0,5,2,8,7,6,3}
26 steps: {2,1,7,5,0,8,3,4,6}
27 steps: {8,5,3,4,7,0,6,1,2}
28 steps: {0,6,7,3,8,5,4,2,1}
30 steps: {5,7,0,4,6,8,1,2,3}
31 steps: {8,6,7,2,5,4,3,0,1} (highest number of steps possible for 3x3 )
I have converted the code given at this link into a python version. The code is supposed to calculate the correct value of maximum value to be filled in knapsack of weight W. I have attached the code below:
#http://www.geeksforgeeks.org/branch-and-bound-set-2-implementation-of-01-knapsack/
from queue import Queue
class Node:
def __init__(self):
self.level = None
self.profit = None
self.bound = None
self.weight = None
def __str__(self):
return "Level: %s Profit: %s Bound: %s Weight: %s" % (self.level, self.profit, self.bound, self.weight)
def bound(node, n, W, items):
if(node.weight >= W):
return 0
profit_bound = int(node.profit)
j = node.level + 1
totweight = int(node.weight)
while ((j < n) and (totweight + items[j].weight) <= W):
totweight += items[j].weight
profit_bound += items[j].value
j += 1
if(j < n):
profit_bound += (W - totweight) * items[j].value / float(items[j].weight)
return profit_bound
Q = Queue()
def KnapSackBranchNBound(weight, items, total_items):
items = sorted(items, key=lambda x: x.value/float(x.weight), reverse=True)
u = Node()
v = Node()
u.level = -1
u.profit = 0
u.weight = 0
Q.put(u)
maxProfit = 0;
while not Q.empty():
u = Q.get()
if u.level == -1:
v.level = 0
if u.level == total_items - 1:
continue
v.level = u.level + 1
v.weight = u.weight + items[v.level].weight
v.profit = u.profit + items[v.level].value
if (v.weight <= weight and v.profit > maxProfit):
maxProfit = v.profit;
v.bound = bound(v, total_items, weight, items)
if (v.bound > maxProfit):
Q.put(v)
v.weight = u.weight
v.profit = u.profit
v.bound = bound(v, total_items, weight, items)
if (v.bound > maxProfit):
# print items[v.level]
Q.put(v)
return maxProfit
if __name__ == "__main__":
from collections import namedtuple
Item = namedtuple("Item", ['index', 'value', 'weight'])
input_data = open("test.data").read()
lines = input_data.split('\n')
firstLine = lines[0].split()
item_count = int(firstLine[0])
capacity = int(firstLine[1])
print "running from main"
items = []
for i in range(1, item_count+1):
line = lines[i]
parts = line.split()
items.append(Item(i-1, int(parts[0]), float(parts[1])))
kbb = KnapSackBranchNBound(capacity, items, item_count)
print kbb
The program is supposed to calculate value of 235 for following items inside file test.data:
5 10
40 2
50 3.14
100 1.98
95 5
30 3
The first line shows number of items and knapsack weight. Lines below first line shows the value and weight of those items. Items are made using a namedtuple and sorted according to value/weight. For this problem I am getting 135 instead of 235. What am I doing wrong here?
EDIT:
I have solved the problem of finding correct items based on branch and bound. If needed, one can check it here
The problem is that you're inserting multiple references to the same Node() object into your queue. The fix is to initialize two new v objects in each iteration of the while-loop as follows:
while not Q.empty():
u = Q.get()
v = Node() # Added line
if u.level == -1:
v.level = 0
if u.level == total_items - 1:
continue
v.level = u.level + 1
v.weight = u.weight + items[v.level].weight
v.profit = u.profit + items[v.level].value
if (v.weight <= weight and v.profit > maxProfit):
maxProfit = v.profit;
v.bound = bound(v, total_items, weight, items)
if (v.bound > maxProfit):
Q.put(v)
v = Node() # Added line
v.level = u.level + 1 # Added line
v.weight = u.weight
v.profit = u.profit
v.bound = bound(v, total_items, weight, items)
if (v.bound > maxProfit):
# print(items[v.level])
Q.put(v)
Without these reinitializations, you're modifying the v object that you already inserted into the queue.
This is different from C++ where the Node objects are values that are implicitly copied into the queue to avoid aliasing problems such as these.
I'm currently trying to code a non linear SVM for handwritten digits recognition using the MNIST data base.
I chose to use the SMO algorithm (based on Platt's paper and other books), but I have some trouble implementing it.
When I run the code over the training set, the bias goes higher and higher, sometimes until "Inf" value, leading the SVM to "classify" every example in the same class.
Here is my code:
import numpy
import gzip
import struct
import matplotlib
from sklearn import datasets
from copy import copy
class SVM:
def __init__(self, constant, data_set, label_set):
self._N = len(data_set)
if self._N != len(label_set):
raise Exception("Data size and label size don't match.")
self._C = constant
self._epsilon = 0.001
self._tol = 0.001
self._data = [numpy.ndarray.flatten((1/255)*elt) for elt in data_set]
self._dimension = len(self._data[0])
self._label = label_set
self._alphas = numpy.zeros((1, self._N))
self._b = 0
self._errors = numpy.ndarray((2, 0))
def kernel(self, x1, x2):
x1 = x1.reshape(1,self._dimension)
result = numpy.power(numpy.dot(x1, x2), 3)
return result
def evaluate(self, x):
result = 0
i = 0
while i < self._N:
result += self._alphas[0, i]*self._label[i]*self.kernel(x, self._data[i])
i += 1
result += self._b
return result
def update(self, i1, i2, E2):
i1 = int(i1)
i2 = int(i2)
if i1 == i2:
return 0
y1 = self._label[i1]
y2 = self._label[i2]
alpha1 = self._alphas[0, i1]
alpha2 = self._alphas[0, i2]
#If alpha1 is non-bound, its error is in the cache.
#So we check its position to extract its error.
#Else, we compute it.
if alpha1 > 0 and alpha1 < self._C :
position = 0
for i, elt in enumerate(self._errors[0, :]):
if elt == i1:
position = i
E1 = self._errors[1, position]
else:
E1 = self.evaluate(self._data[i1]) - y1
s = y1*y2
H = L = 0
if y1 != y2:
L = max(0, alpha2 - alpha1)
H = min(self._C, self._C + alpha2 - alpha1)
else:
L = max(0, alpha2 + alpha1 - self._C)
H = min(self._C, alpha2 + alpha1)
if H == L:
return 0
K11 = self.kernel(self._data[i1], self._data[i1])
K12 = self.kernel(self._data[i1], self._data[i2])
K22 = self.kernel(self._data[i2], self._data[i2])
eta = K11 + K22 - 2*K12
if eta > 0:
alpha2_new = alpha2 + (y2*(E1 - E2)/eta)
if alpha2_new < L:
alpha2_new = L
elif alpha2_new > H:
alpha2_new = H
else:
f1 = y1*(E1 + self._b) - alpha1*K11 - s*alpha2*K12
f2 = y2*(E2 + self._b) - alpha2*K22 - s*alpha1*K12
L1 = alpha1 + s*(alpha2 - L)
H1 = alpha1 + s*(alpha2 - H)
FuncL = L1*f1 + L*f2 + (1/2)*numpy.square(L1)*K11 + (1/2)*numpy.square(L)*K22 + s*L1*L*K12
FuncH = H1*f1 + H*f2 + (1/2)*numpy.square(H1)*K11 + (1/2)*numpy.square(H)*K22 + s*H1*H*K12
if FuncL < FuncH - self._epsilon:
alpha2_new = L
elif FuncL > FuncH + self._epsilon:
alpha2_new = H
else:
alpha2_new = alpha2
if numpy.abs(alpha2_new - alpha2) < self._epsilon*(alpha2_new+alpha2+ self._epsilon):
return 0
alpha1_new = alpha1 + s*(alpha2 - alpha2_new)
#Update of the threshold.
b1 = E1 + y1*(alpha1_new - alpha1)*K11 + y2*(alpha2_new - alpha2)*K12 + self._b
b2 = E2 + y1*(alpha1_new - alpha1)*K12 + y2*(alpha2_new - alpha2)*K22 + self._b
if L < alpha1_new < H:
b_new = b1
elif L < alpha2_new < H:
b_new = b2
else:
b_new = (b1+b2)/2
#Update the cache error
#If alpha2 was bound and its new value is non-bound, we add its index and its error to the cache.
#If alpha2 was unbound and its new value is bound, we delete it from the cache.
if (alpha2 == 0 or alpha2 == self._C) and (alpha2_new > 0 and alpha2_new < self._C):
vector_alpha2_new = numpy.array([i2, E2])
vector_alpha2_new = vector_alpha2_new.reshape((2, 1))
self._errors = numpy.concatenate((self._errors, vector_alpha2_new), 1)
if (alpha2 > 0 and alpha2 < self._C) and (alpha2_new == 0 or alpha2_new == self._C):
l = 0
position = 0
while l < len(self._errors[0, :]):
if self._errors[0, l] == i2:
position = l
l += 1
self._errors = numpy.delete(self._errors, position, 1)
#We do the exact same thing with alpha1.
if (alpha1 == 0 or alpha1 == self._C) and (alpha1_new > 0 and alpha1_new < self._C):
vector_alpha1_new = numpy.array([i1, E1])
vector_alpha1_new = vector_alpha1_new.reshape((2, 1))
self._errors = numpy.concatenate((self._errors, vector_alpha1_new), 1)
if (alpha1 > 0 and alpha1 < self._C) and (alpha1_new == 0 or alpha1_new == self._C):
l = 0
position = 0
while l < len(self._errors[0, :]):
if self._errors[0, l] == i1:
position = l
l += 1
self._errors = numpy.delete(self._errors, position, 1)
#Then we update the error for each non bound point using the new values for alpha1 and alpha2.
for i,error in enumerate(self._errors[1, :]):
self._errors[1, i] = error + (alpha2_new - alpha2)*y2*self.kernel(self._data[i2], self._data[int(self._errors[0, i])]) + (alpha1_new - alpha1)*y1*self.kernel(self._data[i1], self._data[int(self._errors[0, i])]) - self._b + b_new
#Storing the new values of alpha1 and alpha2:
self._alphas[0, i1] = alpha1_new
self._alphas[0, i2] = alpha2_new
self._b = b_new
print(self._errors)
return 1
def examineExample(self, i2):
i2 = int(i2)
y2 = self._label[i2]
alpha2 = self._alphas[0, i2]
if alpha2 > 0 and alpha2 < self._C:
position = 0
for i, elt in enumerate(self._errors[0, :]):
if elt == i2:
position = i
E2 = self._errors[1, position]
else:
E2 = self.evaluate(self._data[i2]) - y2
r2 = E2*y2
if (r2< -self._tol and alpha2 < self._C) or (r2 > self._tol and alpha2 > 0):
n = numpy.shape(self._errors)[1]
if n > 1:
i1 = 0
if E2 > 0:
min = self._errors[1, 0]
position = 0
for l, elt in enumerate(self._errors[1, :]):
if elt < min:
min = elt
position = l
i1 = self._errors[0, position]
else:
max = self._errors[1, 0]
position = 0
for l, elt in enumerate(self._errors[1, :]):
if elt > max:
max = elt
position = l
i1 = self._errors[0, position]
if self.update(i1, i2, E2):
return 1
#loop over all non bound examples starting at a random point.
list_index = [i for i in range(n)]
numpy.random.shuffle(list_index)
for i in list_index:
i1 = self._errors[0, i]
if self.update(i1, i2, E2):
return 1
#Loop over all the training examples, starting at a random point.
list_bound = [i for i in range(self._N) if not numpy.any(self._errors[0, :] == i)]
numpy.random.shuffle(list_bound)
for i in list_bound:
i1 = i
if self.update(i1, i2, E2):
return 1
return 0
def SMO(self):
numChanged = 0
examineAll = 1
cpt = 1
while(numChanged > 0 or examineAll):
numChanged = 0
if examineAll == 1:
for i in range(self._N):
numChanged += self.examineExample(i)
else:
for i in self._errors[0, :]:
numChanged += self.examineExample(i)
if examineAll == 1:
examineAll = 0
elif numChanged == 0:
examineAll = 1
cpt += 1
def load_training_data(a, b):
train = gzip.open("train-images-idx3-ubyte.gz", "rb")
labels = gzip.open("train-labels-idx1-ubyte.gz", "rb")
train.read(4)
labels.read(4)
number_images = train.read(4)
number_images = struct.unpack(">I", number_images)[0]
rows = train.read(4)
rows = struct.unpack(">I", rows)[0]
cols = train.read(4)
cols = struct.unpack(">I", cols)[0]
number_labels = labels.read(4)
number_labels = struct.unpack(">I", number_labels)[0]
image_list = []
label_list = []
if number_images != number_labels:
raise Exception("The number of labels doesn't match with the number of images")
else:
for l in range(number_labels):
if l % 1000 == 0:
print("l:{}".format(l))
mat = numpy.zeros((rows, cols), dtype = numpy.uint8)
for i in range(rows):
for j in range(cols):
pixel = train.read(1)
pixel = struct.unpack(">B", pixel)[0]
mat[i][j] = pixel
image_list += [mat]
lab = labels.read(1)
lab = struct.unpack(">B", lab)[0]
label_list += [lab]
train.close()
labels.close()
i = 0
index_a = []
index_b = []
while i < number_labels:
if label_list[i] == a:
index_a += [i]
elif label_list[i] == b:
index_b += [i]
i += 1
image_list = [m for i,m in enumerate(image_list) if (i in index_a) | (i in index_b)]
mean = (a+b)/2
label_list = [ numpy.sign(m - mean) for l,m in enumerate(label_list) if l in index_a+index_b]
return ([image_list, label_list])
def load_test_data():
test = gzip.open("t10k-images-idx3-ubyte.gz", "rb")
labels = gzip.open("t10k-labels-idx1-ubyte.gz", "rb")
test.read(4)
labels.read(4)
number_images = test.read(4)
number_images = struct.unpack(">I", number_images)[0]
rows = test.read(4)
rows = struct.unpack(">I", rows)[0]
cols = test.read(4)
cols = struct.unpack(">I", cols)[0]
number_labels = labels.read(4)
number_labels = struct.unpack(">I", number_labels)[0]
image_list = []
label_list = []
if number_images != number_labels:
raise Exception("The number of labels doesn't match with the number of images")
else:
for l in range(number_labels):
if l % 1000 == 0:
print("l:{}".format(l))
mat = numpy.zeros((rows, cols), dtype = numpy.uint8)
for i in range(rows):
for j in range(cols):
pixel = test.read(1)
pixel = struct.unpack(">B", pixel)[0]
mat[i][j] = pixel
image_list += [mat]
lab = labels.read(1)
lab = struct.unpack(">B", lab)[0]
label_list += [lab]
test.close()
labels.close()
return ([image_list, label_list])
data = load_training_data(0, 7)
images_training = data[0]
labels_training = data[1]
svm = SVM(0.1, images_training[0:200], labels_training[0:200])
svm.SMO()
def view(image, label=""):
print("Number : {}".format(label))
pylab.imshow(image, cmap = pylab.cm.gray)
pylab.show()
First, SMO is a fairly complicated algorithm - it is not one easy to debug in this kind of format.
Second, you are starting too high up in your testing. Some advice to help you debug your problems.
1) First, switch to using the linear kernel. Its much easier for you to compute the exact linear solution with another algorithm and compare what you are getting with the exact solution. This way its only the weight vectors and bias term. If you stay in the dual space, you'll have to compare all the coefficients and make sure things stay in the same order.
2) Start with a much simpler 2D problem where you know what the general solution should look like. You can then visualize the solution, and watch as it changes at each step - this can be a visual tool to help you find where something goes wrong.
One important thing is you said this:
b1 = E1 + y1*(alpha1_new - alpha1)*K11 + y2*(alpha2_new - alpha2)*K12 + self._b
b2 = E2 + y1*(alpha1_new - alpha1)*K12 + y2*(alpha2_new - alpha2)*K22 + self._b
Basically you're just adding to b every time with this code. Your b's should look more like this:
b1 = smo.b - E1 - y1 * (a1 - alpha1) * smo.K[i1, i1] - y2 * (a2 - alpha2) * smo.K[i1, i2]
b2 = smo.b - E2 - y1 * (a1 - alpha1) * smo.K[i1, i2] - y2 * (a2 - alpha2) * smo.K[i2, i2]
This version is not perfect, but I recommend checking apex51's version on Github for pointers:
SVM-and-sequential-minimal-optimization
The mathematical basis in the notes are very strong (despite some minor discrepancies with Platt's paper) and the code is not perfect, but a good direction for you. I would also suggest looking at other, completed SMOs and trying to tweak that code to math your needs instead of writing from scratch.