Related
I'm working through MIT6.0002 on OpenCourseWare (https://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-0002-introduction-to-computational-thinking-and-data-science-fall-2016/assignments/) and I am stumped on Part B of Problem Set 1. The problem, which is presented as a version of the knapsack problem, is stated as follows:
[The Aucks have found a colony of geese that lay golden eggs of various weights] They want to carry as few eggs as possible on their trip as they don’t have a lot of space
on their ships. They have taken detailed notes on the weights of all the eggs that geese can lay
in a given flock and how much weight their ships can hold.
Implement a dynamic programming algorithm to find the minimum number of eggs needed to
make a given weight for a certain ship in dp_make_weight. The result should be an integer
representing the minimum number of eggs from the given flock of geese needed to make the
given weight. Your algorithm does not need to return what the weight of the eggs are, just the
minimum number of eggs.
Assumptions:
All the eggs weights are unique between different geese, but a given goose will always lay the same size egg
The Aucks can wait around for the geese to lay as many eggs as they need [ie there is an infinite supply of each size of egg].
There are always eggs of size 1 available
The problem also states that the solution must use dynamic programming. I have written a solution (in Python) which I think finds the optimal solution, but it does not use dynamic programming, and I fail to understand how dynamic programming is applicable. It was also suggested that the solution should use recursion.
Can anybody explain to me what the advantage is of using memoization in this case, and what I would gain by implementing a recursive solution?
(Apologies if my question is too vague or if the solution is too obvious for words; I'm a relative beginner to programming, and to this site).
My code:
#================================
# Part B: Golden Eggs
#================================
# Problem 1
def dp_make_weight(egg_weights, target_weight, memo = {}):
"""
Find number of eggs to bring back, using the smallest number of eggs. Assumes there is
an infinite supply of eggs of each weight, and there is always a egg of value 1.
Parameters:
egg_weights - tuple of integers, available egg weights sorted from smallest to largest value (1 = d1 < d2 < ... < dk)
target_weight - int, amount of weight we want to find eggs to fit
memo - dictionary, OPTIONAL parameter for memoization (you may not need to use this parameter depending on your implementation)
Returns: int, smallest number of eggs needed to make target weight
"""
egg_weights = sorted(egg_weights, reverse=True)
eggs = 0
while target_weight != 0:
while egg_weights[0] <= target_weight:
target_weight -= egg_weights[0]
eggs += 1
del egg_weights[0]
return eggs
# EXAMPLE TESTING CODE, feel free to add more if you'd like
if __name__ == '__main__':
egg_weights = (1, 5, 10, 25)
n = 99
print("Egg weights = (1, 5, 10, 25)")
print("n = 99")
print("Expected ouput: 9 (3 * 25 + 2 * 10 + 4 * 1 = 99)")
print("Actual output:", dp_make_weight(egg_weights, n))
print()
The problem here is a classic DP situation where greediness can sometimes give optimal solutions, but sometimes not.
The situation in this problem is similar to the classic DP problem coin change where we wish to find the fewest number of different valued coins to make change given a target value. The denominations available in some countries such as the USA (which uses coins valued 1, 5, 10, 25, 50, 100) are such that it's optimal to greedily choose the largest coin until the value drops below it, then move on to the next coin. But with other denomination sets like 1, 3, 4, greedily choosing the largest value repeatedly can produce sub-optimal results.
Similarly, your solution works fine for certain egg weights but fails on others. If we choose our egg weights to be 1, 6, 9 and give a target weight of 14, the algorithm chooses 9 immediately and is then unable to make progress on 6. At that point, it slurps a bunch of 1s and ultimately thinks 6 is the minimal solution. But that's clearly wrong: if we intelligently ignore the 9 and pick two 6s first, then we can hit the desired weight with only 4 eggs.
This shows that we have to consider the fact that at any decision point, taking any of our denominations might ultimately lead us to a globally optimal solution. But we have no way of knowing in the moment. So, we try every denomination at every step. This is very conducive to recursion and could be written like this:
def dp_make_weight(egg_weights, target_weight):
least_taken = float("inf")
if target_weight == 0:
return 0
elif target_weight > 0:
for weight in egg_weights:
sub_result = dp_make_weight(egg_weights, target_weight - weight)
least_taken = min(least_taken, sub_result)
return least_taken + 1
if __name__ == "__main__":
print(dp_make_weight((1, 6, 9), 14))
For each call, we have 3 possibilities:
Base case target_weight < 0: return something to indicate no solution possible (I used infinity for convenience).
Base case target_weight == 0: we found a candidate solution. Return 0 to indicate no step was taken here and give the caller a base value to increment.
Recursive case target_weight > 0: try taking every available egg_weight by subtracting it from the total and recursively exploring the path rooted at the new state. After exploring every possible outcome from the current state, pick the one that took the least number of steps to reach the target. Add 1 to count the current step's egg taken and return.
So far, we've seen that a greedy solution is incorrect and how to fix it but haven't motivated dynamic programming or memoization. DP and memoization are purely optimization concepts, so you can add them after you've found a correct solution and need to speed it up. Time complexity of the above solution is exponential: for every call, we have to spawn len(egg_weights) recursive calls.
There are many resources explaining DP and memoization and I'm sure your course covers it, but in brief, our recursive solution shown above re-computes the same results over and over by taking different recursive paths that ultimately lead to the same values being given for target_weight. If we keep a memo (dictionary) that stores the results of every call in memory, then whenever we re-encounter a call, we can look up its result instead of re-computing it from scratch.
def dp_make_weight(egg_weights, target_weight, memo={}):
least_taken = float("inf")
if target_weight == 0:
return 0
elif target_weight in memo:
return memo[target_weight]
elif target_weight > 0:
for weight in egg_weights:
sub_result = dp_make_weight(egg_weights, target_weight - weight)
least_taken = min(least_taken, sub_result)
memo[target_weight] = least_taken + 1
return least_taken + 1
if __name__ == "__main__":
print(dp_make_weight((1, 6, 9, 12, 13, 15), 724)) # => 49
Since we're using Python, the "Pythonic" way to do it is probably to decorate the function. In fact, there's a builtin memoizer called lru_cache, so going back to our original function without any memoization, we can add memoization (caching) with two lines of code:
from functools import lru_cache
#lru_cache
def dp_make_weight(egg_weights, target_weight):
# ... same code as the top example ...
Memoizing with a decorator has the downside of increasing the size of the call stack proportional to the wrapper's size so it can increase the likelihood of blowing the stack. That's one motivation for writing DP algorithms iteratively, bottom-up (that is, start with the solution base cases and build up a table of these small solutions until you're able to build the global solution), which might be a good exercise for this problem if you're looking for another angle on it.
I'm learning about genetic algorithms and in order to better understand the concepts I tried to build genetic algorithm from scratch using python without using any external module (just the standard library and a little bit of numpy)
The goal is to find a target string, so if I give it the string hello and define 26 chars + a space, there are 26^5 possibilities which is huge. Thus the need to use a GA to solve this probem.
I defined the following functions:
Generate population : we generate the population given size n and a target we generate n string having len(target) of random chars, we return the population as a list of str
Compute a fitness score: if the char at position i is equal to the char at position i of target we increment the score, here's the code:
def fitness(indiv,target):
score = 0
#print(indiv," vs ",target)
for idx,char in enumerate(list(target)):
if char == indiv[idx]:
score += 1
else:
score = 0
return score
Select parents, crossing between parents and generating a new population of children
Here are the function responsible for that:
from numpy.random import choice
def crossover(p1,p2):
# we define a crossover between p1 and p2 (single point cross over)
point = random.choice([i for i in range (len(target))])
#print("Parents:",p1,p2)
# C1 and C2 are the new children, before the cross over point they are equalt to their prantes, after that we swap
c = [p1[i] for i in range(point)]
#print("Crossover point: ",point)
for i in range(point,len(p1)):
c.append(p2[i])
#print("Offsprings:", c1," and ", c2)
c = "".join(c)
# we mutate c too
c = mutate(c)
return c
def mutate(ind):
point = random.choice([i for i in range (len(target))])
new_ind = list(ind)
new_ind[point] = random.choice(letters)
return "".join(new_ind)
def select_parent(new_pop,fit_scores):
totale = sum(fit_scores)
probs = [score/totale for score in fit_scores]
parent = choice(new_pop,1,p=probs)[0]
return parent
I'm selecting parents by computing the probabilities of each individual (individual score/ total score of population), then using a weighted random choice function to select a parent (this is a numpy function).
For the crossover, I'm generating a child c and a random splitting point, all chars before this random point are the first parent chars, and all chars after the splitting point are chars from the parent.
besides that I defined a function called should_stop which check whether we found the target, and print_best which gets the best individuals out of a population (highest fitness score).
Then I created a find function that use all the functions defined above:
def find(size,target,pop):
scores = [fitness(ind,target) for ind in pop]
#print("len of scores is ", len(scores))
#good_indiv = select_individuals(pop,scores)
#print("Length of good indivs is", len(good_indiv))
new_pop = []
# corssover good individuals
for ind in pop:
pa = select_parent(pop,scores)
pb = select_parent(pop,scores)
#print(pa,pb)
child = crossover(pa,pb)
#print(type(child))
new_pop.append(child)
best = print_best(new_pop,scores)
print("********** The best individual is: ", best, " ********")
return (new_pop,best)
n = 200
target = "hello"
popu = generate_pop(n,target)
#find(n,target,popu)
for i in range(1000):
print(len(popu))
data = find(n,target,popu)
popu = data[0]
print("iteration number is ", i)
if data[1] == target:
break
The Problem The problem is that it's taking too many iterations than it shoud be to generate hello (more than 200 iterations most of the time), while in this example, it only takes few iterations: https://jbezerra.github.io/The-Shakespeare-and-Monkey-Problem/index.html
Sure the problem is not coded in the same way, I used python and a procedural way to code things but the logic is the same. So what I'm doing wrong ?
You mutate 100% of the time. You select 'suitable' parents which are likely to produce a fit offspring, but then you apply a mutation that's more likely than not to "throw it off". The example link your provided behaves the same way if you increase mutation rate to 100%.
The purpose of mutation is to "nudge" the search in a different direction if you appear to be stuck in a local optimum, applying it all the time turns this from an evolutionary algorithm to something much closer to random search.
The idea of genetic algorithms supports that best ones survive and create new generations
First off all you should keep best ones in the every generation for the next generation (for example best 40% of every generation keep living on the next generatio) and you should breed those 40 percent with each other and mutate only limited number of individual in every generation those numbers should be low like lower than 5% of the individuals mutates i believe this will reduce the number of generations
I would suggest define your strings in a dictionary and give a number to them
then analyse this arrays
example
my dictionary is
I : 1
eat : 23
want : 12
to : 2
so I want to eat
convert to [ 1 , 12, 2, 23]
so the randomness is reduce by a factor.
here the words are inferred from dictionary
so the only variable is the order and which words appear in your string.
re-write you algorithm with a dictionary
your algo run time will improve by a factor.
with regards
teja
I know Python isn't the best idea to be writing any kind of software of this nature. My reasoning is to use this type of algorithm for a Raspberry Pi 3 in it's decision making (still unsure how that will go), and the libraries and APIs that I'll be using (Adafruit motor HATs, Google services, OpenCV, various sensors, etc) all play nicely for importing in Python, not to mention I'm just more comfortable in this environment for the rPi specifically. Already I've cursed it as object oriented such as Java or C++ just makes more sense to me, but Id rather deal with its inefficiencies and focus on the bigger picture of integration for the rPi.
I won't explain the code here, as it's pretty well documented in the comment sections throughout the script. My questions is as stated above; can this be considered basically a genetic algorithm? If not, what must it have to be a basic AI or genetic code? Am I on the right track for this type of problem solving? I know usually there are weighted variables and functions to promote "survival of the fittest", but that can be popped in as needed, I think.
I've read up quite a bit of forums and articles about this topic. I didn't want to copy someone else's code that I barely understand and start using it as a base for a larger project of mine; I want to know exactly how it works so I'm not confused as to why something isn't working out along the way. So, I just tried to comprehend the basic idea of how it works, and write how I interpreted it. Please remember I'd like to stay in Python for this. I know rPi's have multiple environments for C++, Java, etc, but as stated before, most hardware components I'm using have only Python APIs for implementation. if I'm wrong, explain at the algorithmic level, not just with a block of code (again, I really want to understand the process). Also, please don't nitpick code conventions unless it's pertinent to my problem, everyone has a style and this is just a sketch up for now. Here it is, and thanks for reading!
# Created by X3r0, 7/3/2016
# Basic genetic algorithm utilizing a two dimensional array system.
# the 'DNA' is the larger array, and the 'gene' is a smaller array as an element
# of the DNA. There exists no weighted algorithms, or statistical tracking to
# make the program more efficient yet; it is straightforwardly random and solves
# its problem randomly. At this stage, only the base element is iterated over.
# Basic Idea:
# 1) User inputs constraints onto array
# 2) Gene population is created at random given user constraints
# 3) DNA is created with randomized genes ( will never randomize after )
# a) Target DNA is created with loop control variable as data (basically just for some target structure)
# 4) CheckDNA() starts with base gene from DNA, and will recurse until gene matches the target gene
# a) Randomly select two genes from DNA
# b) Create a candidate gene by splicing both parent genes together
# c) Check candidate gene against the target gene
# d) If there exists a match in gene elements, a child gene is created and inserted into DNA
# e) If the child gene in DNA is not equal to target gene, recurse until it is
import random
DNAsize = 32
geneSize = 5
geneDiversity = 9
geneSplit = 4
numRecursions = 0
DNA = []
targetDNA = []
def init():
global DNAsize, geneSize, geneDiversity, geneSplit, DNA
print("This is a very basic form of genetic software. Input variable constraints below. "
"Good starting points are: DNA strand size (array size): 32, gene size (sub array size: 5, gene diversity (randomized 0 - x): 5"
"gene split (where to split gene array for splicing): 2")
DNAsize = int(input('Enter DNA strand size: '))
geneSize = int(input('Enter gene size: '))
geneDiversity = int(input('Enter gene diversity: '))
geneSplit = int(input('Enter gene split: '))
# initializes the gene population, and kicks off
# checkDNA recursion
initPop()
checkDNA(DNA[0])
def initPop():
# builds an array of smaller arrays
# given DNAsize
for x in range(DNAsize):
buildDNA()
# builds the goal array with a recurring
# numerical pattern, in this case just the loop
# control variable
buildTargetDNA(x)
def buildDNA():
newGene = []
# builds a smaller array (gene) using a given geneSize
# and randomized with vaules 0 - [given geneDiversity]
for x in range(geneSize):
newGene.append(random.randint(0,geneDiversity))
# append the built array to the larger array
DNA.append(newGene)
def buildTargetDNA(x):
# builds the target array, iterating with x as a loop
# control from the call in init()
newGene = []
for y in range(geneSize):
newGene.append(x)
targetDNA.append(newGene)
def checkDNA(childGene):
global numRecursions
numRecursions = numRecursions+1
gene = DNA[0]
targetGene = targetDNA[0]
parentGeneA = DNA[random.randint(0,DNAsize-1)] # randomly selects an array (gene) from larger array (DNA)
parentGeneB = DNA[random.randint(0,DNAsize-1)]
pos = random.randint(geneSplit-1,geneSplit+1) # randomly selects a position to split gene for splicing
candidateGene = parentGeneA[:pos] + parentGeneB[pos:] # spliced gene given split from parentA and parentB
print("DNA Splice Position: " + str(pos))
print("Element A: " + str(parentGeneA))
print("Element B: " + str(parentGeneB))
print("Candidate Element: " + str(candidateGene))
print("Target DNA: " + str(targetDNA))
print("Old DNA: " + str(DNA))
# iterates over the candidate gene and compares each element to the target gene
# if the candidate gene element hits a target gene element, the resulting child
# gene is created
for x in range(geneSize):
#if candidateGene[x] != targetGene[x]:
#print("false ")
if candidateGene[x] == targetGene[x]:
#print("true ")
childGene.pop(x)
childGene.insert(x, candidateGene[x])
# if the child gene isn't quite equal to the target, and recursion hasn't reached
# a max (apparently 900), the child gene is inserted into the DNA. Recursion occurs
# until the child gene equals the target gene, or max recursuion depth is exceeded
if childGene != targetGene and numRecursions < 900:
DNA.pop(0)
DNA.insert(0, childGene)
print("New DNA: " + str(DNA))
print(numRecursions)
checkDNA(childGene)
init()
print("Final DNA: " + str(DNA))
print("Number of generations (recursions): " + str(numRecursions))
I'm working with evolutionary computation right now so I hope my answer will be helpful for you, personally, I work with java, mostly because is one of my main languages, and for the portability, because I tested in linux, windows and mac. In my case I work with permutation encoding, but if you are still learning how GA works, I strongly recommend binary encoding. This is what I called my InitialPopulation. I try to describe my program's workflow:
1-. Set my main variables
This are PopulationSize, IndividualSize, MutationRate, CrossoverRate. Also you need to create an objective function and decide the crossover method you use. For this example lets say that my PopulationSize is equals to 50, the IndividualSize is 4, MutationRate is 0.04%, CrossoverRate is 90% and the crossover method will be roulette wheel.
My objective function only what to check if my Individuals are capable to represent the number 15 in binary, so the best individual must be 1111.
2-. Initialize my Population
For this I create 50 individuals (50 is given by my PopulationSize) with random genes.
3-. Loop starts
For each Individuals in Population you need to:
Evaluate fitness according to the objective function. If an Individual is represented by the next characters: 00100 this mean that his fitness is 1. As you can see this is a simple fitness function. You can create your own while you are learning, like fitness = 1/numberOfOnes. Also you need to assign the sum of all the fitness to a variable called populationFitness, this will be useful in the next step.
Select the best individuals. For this task there's a lot of methods you can use, but we will use the roulette wheel method as we say before. In this method, You assign a value to every individual inside your population. This value is given by the next formula: (fitness/populationFitness) * 100. So, if your population fitness is 10, and a certain individual fitness is 3, this mean that this individual has a 30% chance to be selected to make a crossover with another individual. Also, if another individual have a 4 in his fitness, his value will be 40%.
Apply crossover. Once you have the "best" individuals of your population, you need to create a new population. This new population is formed by others individuals of the previous population. For each individual you create a random number from 0 to 1. If this numbers is in the range of 0.9 (since our crossoverRate = 90%), this individual can reproduce, so you select another individual. Each new individual has this 2 parents who inherit his genes. For example:
Lets say that parentA = 1001 and parentB = 0111. We need to create a new individual with this genes. There's a lot of methods to do this, uniform crossover, single point crossover, two point crossover, etc. We will use the single point crossover. In this method we choose a random point between the first gene and the last gene. Then, we create a new individual according to the first genes of parentA and the last genes of parentB. In a visual form:
parentA = 1001
parentB = 0111
crossoverPoint = 2
newIndividual = 1011
As you can see, the new individual share his parents genes.
Once you have a new population with new individuals, you apply the mutation. In this case, for each individual in the new population generate a random number between 0 and 1. If this number is in the range of 0.04 (since our mutationRate = 0.04), you apply the mutation in a random gene. In binary encoding the mutation is just change the 1's for 0's or viceversa. In a visual form:
individual = 1011
randomPoint = 3
mutatedIndividual = 1010
Get the best individual
If this individual has reached the solution stop. Else, repeat the loop
End
As you can see, my english is not very good, but I hope you understand the basic idea of a genetic algorithm. If you are truly interested in learning this, you can check the following links:
http://www.obitko.com/tutorials/genetic-algorithms/
This link explains in a clearer way the basics of a genetic algorithm
http://natureofcode.com/book/chapter-9-the-evolution-of-code/
This book also explain what a GA is, but also provide some code in Processing, basically java. But I think you can understand.
Also I would recommend the following books:
An Introduction to Genetic Algorithms - Melanie Mitchell
Evolutionary algorithms in theory and practice - Thomas Bäck
Introduction to genetic algorithms - S. N. Sivanandam
If you have no money, you can easily find all this books in PDF.
Also, you can always search for articles in scholar.google.com
Almost all are free to download.
Just to add a bit to Alberto's great answer, you need to watch out for two issues as your solution evolves.
The first one is Over-fitting. This basically means that your solution is complex enough to "learn" all samples, but it is not applicable outside the training set. To avoid this, your need to make sure that the "amount" of information in your training set is a lot larger than the amount of information that can fit in your solution.
The second problem is Plateaus. There are cases where you would arrive at certain mediocre solutions that are nonetheless, good enough to "outcompete" any emerging solution, so your progress stalls (one way to see this is, if you see your fitness "stuck" at a certain, less than optimal number). One method for dealing with this is Extinctions: You could track the rate of improvement of your optimal solution, and if the improvement has been 0 for the last N generations, you just Nuke your population. (That is, delete your population and the list of optimal individuals and start over). Randomness will make it so that eventually the Solutions will surpass the Plateau.
Another thing to keep in mind, is that the default Random class is really bad at Randomness. I have had solutions improve dramatically by simply using something like the Mesernne Twister Random generator or a Hardware Entropy Generator.
I hope this helps. Good luck.
I'm trying to implement a simple Markov Chain Monte Carlo in Python 2.7, using numpy. The goal is to find the solution to the "Knapsack Problem," where given a set of m objects of value vi and weight wi, and a bag with holding capacity b, you find the greatest value of objects that can be fit into your bag, and what those objects are. I started coding in the Summer, and my knowledge is extremely lopsided, so I apologize if I'm missing something obvious, I'm self-taught and have been jumping all over the place.
The code for the system is as follows (I split it into parts in an attempt to figure out what's going wrong).
import numpy as np
import random
def flip_z(sackcontents):
##This picks a random object, and changes whether it's been selected or not.
pick=random.randint(0,len(sackcontents)-1)
clone_z=sackcontents
np.put(clone_z,pick,1-clone_z[pick])
return clone_z
def checkcompliance(checkedz,checkedweights,checkedsack):
##This checks to see whether a given configuration is overweight
weightVector=np.dot(checkedz,checkedweights)
weightSum=np.sum(weightVector)
if (weightSum > checkedsack):
return False
else:
return True
def getrandomz(length):
##I use this to get a random starting configuration.
##It's not really important, but it does remove the burden of choice.
z=np.array([])
for i in xrange(length):
if random.random() > 0.5:
z=np.append(z,1)
else:
z=np.append(z,0)
return z
def checkvalue(checkedz,checkedvalue):
##This checks how valuable a given configuration is.
wealthVector= np.dot(checkedz,checkedvalue)
wealthsum= np.sum(wealthVector)
return wealthsum
def McKnapsack(weights, values, iterations,sack):
z_start=getrandomz(len(weights))
z=z_start
moneyrecord=0.
zrecord=np.array(["error if you see me"])
failures=0.
for x in xrange(iterations):
current_z= np.array ([])
current_z=flip_z(z)
current_clone=current_z
if (checkcompliance(current_clone,weights,sack))==True:
z=current_clone
if checkvalue(current_z,values)>moneyrecord:
moneyrecord=checkvalue(current_clone,values)
zrecord= current_clone
else:failures+=1
print "The winning knapsack configuration is %s" %(zrecord)
print "The highest value of objects contained is %s" %(moneyrecord)
testvalues1=np.array([3,8,6])
testweights1= np.array([1,2,1])
McKnapsack(testweights1,testvalues1,60,2.)
What should happen is the following: With a maximum carrying capacity of 2, it should randomly switch between the different potential bag carrying configurations, of which there are 2^3=8 with the test weights and values I've fed it, with each one or zero in the z representing having or not having a given item. It should discard the options with too much weight, while keeping track of the ones with the highest value and acceptable weight. The correct answer would be to see 1,0,1 as the configuration, with 9 as the maximized value. I get nine for value every time when I use even moderately high amounts of iterations, but the configurations seem completely random, and somehow break the weight rule. I've double-checked my "checkcompliance" function with a lot of test arrays, and it seems to work. How are these faulty, overweight configurations getting past my if statements and into my zrecord ?
The trick is that z (and therefore also current_z and also zrecord) end up always referring to the exact same object in memory. flip_z modifies this object in-place via np.put.
Once you find a new combination that increases your moneyrecord, you set a reference to it -- but then in the subsequent iteration you go ahead and change the data at that same reference.
In other words, lines like
current_clone=current_z
zrecord= current_clone
do not copy, they only make yet another alias to the same data in memory.
One way to fix this is to explicitly copy that combination once you find it's a winner:
if checkvalue(current_z, values) > moneyrecord:
moneyrecord = checkvalue(current_clone, values)
zrecord = current_clone.copy()
I have looked at some common tools like Heapy to measure how much memory is being utilized by each traversal technique but I don't know if they are giving me the right results. Here is some code to give the context.
The code simply measures the number of unique nodes in a graph. Two traversal techniques provided viz. count_bfs and count_dfs
import sys
from guppy import hpy
class Graph:
def __init__(self, key):
self.key = key #unique id for a vertex
self.connections = []
self.visited = False
def count_bfs(start):
parents = [start]
children = []
count = 0
while parents:
for ind in parents:
if not ind.visited:
count += 1
ind.visited = True
for child in ind.connections:
children.append(child)
parents = children
children = []
return count
def count_dfs(start):
if not start.visited:
start.visited = True
else:
return 0
n = 1
for connection in start.connections:
n += count_dfs(connection)
return n
def construct(file, s=1):
"""Constructs a Graph using the adjacency matrix given in the file
:param file: path to the file with the matrix
:param s: starting node key. Defaults to 1
:return start vertex of the graph
"""
d = {}
f = open(file,'rU')
size = int(f.readline())
for x in xrange(1,size+1):
d[x] = Graph(x)
start = d[s]
for i in xrange(0,size):
l = map(lambda x: int(x), f.readline().split())
node = l[0]
for child in l[1:]:
d[node].connections.append(d[child])
return start
if __name__ == "__main__":
s = construct(sys.argv[1])
#h = hpy()
print(count_bfs(s))
#print h.heap()
s = construct(sys.argv[1])
#h = hpy()
print(count_dfs(s))
#print h.heap()
I want to know by what factor is the total memory utilization different in the two traversal techniques viz. count_dfs and count_bfs? One might have the intuition that dfs may be expensive as a new stack is created for every function call. How can the total memory allocations in each traversal technique be measured?
Do the (commented) hpy statements give the desired measure?
Sample file with connections:
4
1 2 3
2 1 3
3 4
4
This being a python question, it may be more important how much stack space is used than how much total memory. Cpython has a low limit of 1000 frames because it shares its call stack with the c call stack, which in turn is limited to the order of one megabyte in most places. For this reason you should almost* always prefer iterative solutions to recursive ones when the recursion depth is unbounded.
* other implementations of python may not have this restriction. The stackless variants of cpython and pypy have this exact property
For your specific problem, I don't know if there's going to be an easy solution. That's because, the peak memory usage of a graph traversal depends on the details of the graph itself.
For a depth-first traversal, the greatest usage will come when the algorithm has gone to the deepest depth. In your example graph, it will traverse 1->2->3->4, and create a stack frame for each level. So while it is at 4 it has allocated the most memory.
For the breadth-first traversal, the memory used will be proportional to the number of nodes at each depth plus the number of child nodes at the next depth. Those values are stored in lists, which are probably more efficient than stack frames. In the example, since the first node is connected to all the others, it happens immediately during the first step [1]->[2,3,4].
I'm sure there are some graphs that will do much better with one search or the other.
For example, imagine a graph that looked like a linked list, with all the vertices in a single long chain. The depth-first traversal will have a very-high peak memory useage, since it will recurse all the way down the chain, allocating a stack frame for each level. The breadth-first traversal will use much less memory, since it will only have a single vertex with a single child to keep track of on each step.
Now, contrast that with a graph that is a depth-2 tree. That is, there's a single root element that is connected to a great many children, none of which are connected to each other. The depth first traversal will not use much memory at any given time, as it will only need to traverse two nodes before it has to back up and try another branch. The depth-first traversal on the other hand will be putting all of the child nodes in memory at once, which for a big tree could be problematic.
Your current profiling code won't find the peak memory usage you want, because it only finds the memory used by objects on the heap at the time you call heap. That's likely to be the the same before and after your traversals. Instead, you'll need to insert profiling code into the traversal functions themselves. I can't find a pre-built package of guppy to try it myself, but I think this untested code will work:
from guppy import hpy
def count_bfs(start):
hp = hpy()
base_mem = hpy.heap().size
max_mem = 0
parents = [start]
children = []
count = 0
while parents:
for ind in parents:
if not ind.visited:
count += 1
ind.visited = True
for child in ind.connections:
children.append(child)
mem = hpy.heap().size - base_mem
if mem > max_mem:
max_mem = mem
parents = children
children = []
return count, max_mem
def count_dfs(start, hp=hpy(), base_mem=None):
if base_mem is None:
base_mem = hp.heap().size
if not start.visited:
start.visited = True
else:
return 0, hp.heap().size - base_mem
n = 1
max_mem = 0
for connection in start.connections:
c, mem = count_dfs(connection, base_mem)
if mem > max_mem:
max_mem = mem
n += c
return n, max_mem
Both traversal functions now return a (count, max-memory-used) tuple. You can try them out on a variety of graphs to see what the differences are.
It's tough to measure exactly how much memory is being used because systems vary in how they implement stack frames. Generally speaking, recursive algorithms use far more memory than iterative algorithms because each stack frame must store the state of its variables whenever a new function call occurs. Consider the difference between dynamic programming solutions and recursive solutions. Runtime is far faster on an iterative implementation of an algorithm than a recursive one.
If you really must know how much memory your code uses, load your software in a debugger such as OllyDbg (http://www.ollydbg.de/) and count the bytes. Happy coding!
Of the two, depth-first uses less memory if most traversals end up hitting most of the graph.
Breadth-first can be better when the target is near the starting node, or when the number of nodes doesn't go up very quickly so the parents/children arrays in your code stay small (e.g. another answer mentioned linked list as worst-case for DFS).
If the graph you're searching is spatial data, or has what's known as an "admissible heuristic," A* is another algorithm that's pretty good: http://en.wikipedia.org/wiki/A_star
However, premature optimization is the root of all evil. Look at the actual data you want to use; if it fits in a reasonable amount of memory, and the search runs in a reasonable time, it doesn't matter which algorithm you use. NB, what's "reasonable" depends on the application you're using it for and the amount of resources on the hardware that will be running it.
For either search order implemented iteratively with the standard data structure describing it (queue for BFS, stack for DFS), I can construct a graph that uses O(n) memory trivially. For BFS, it's an n-star, and for DFS it's an n-chain. I don't believe either of them can be implemented for the general case to do better than that, so that also gives an Omega(n) lower bound on maximum memory usage. So, with efficient implementations of each, it should generally be a wash.
Now, if your input graphs have some characteristics that bias them more toward one of those extremes or the other, that might inform your decision on which to use in practice.