Implementing Levenshtein distance in python - python

I have implemented the algorithm, but now I want to find the edit distance for the string which has the shortest edit distance to the others strings.
Here is the algorithm:
def lev(s1, s2):
return min(lev(a[1:], b[1:])+(a[0] != b[0]), lev(a[1:], b)+1, lev(a, b[1:])+1)

Your "implementation" has several flaws:
(1) It should start with def lev(a, b):, not def lev(s1, s2):. Please get into the good habits of (a) running your code before asking questions about it (b) quoting the code that you've actually run (by copy/paste, not by (error-prone) re-typing).
(2) It has no termination conditions; for any arguments it will eventually end up trying to evaluate lev("", "") which would loop forever were it not for Python implementation limits: RuntimeError: maximum recursion depth exceeded.
You need to insert two lines:
if not a: return len(b)
if not b: return len(a)
to make it work.
(3) The Levenshtein distance is defined recursively. There is no such thing as "the" (one and only) algorithm. Recursive code is rarely seen outside a classroom and then only in a "strawman" capacity.
(4) Naive implementations take time and memory proportional to len(a) * len(b) ... aren't those strings normally a little bit longer than 4 to 8?
(5) Your extremely naive implementation is worse, because it copies slices of its inputs.
You can find working not-very-naive implementations on the web ... google("levenshtein python") ... look for ones which use O(max(len(a), len(b))) additional memory.
What you asked for ("the edit distance for the string who has the shortest edit distance to the others strings.") Doesn't make sense ... "THE string"??? "It takes two to tango" :-)
What you probably want (finding all pairs of strings in a collection which have the minimal distance), or maybe just that minimal distance, is a simple programming exercise. What have you tried?
By the way, finding those pairs by a simplistic algorithm will take O(N ** 2) executions of lev() where N is the number of strings in the collection ... if this is a real-world application, you should look to use proven code rather than try to write it yourself. If this is homework, you should say so.

is this what you're looking for ??
import itertools
import collections
# My Simple implementation of Levenshtein distance
def levenshtein_distance(string1, string2):
"""
>>> levenshtein_distance('AATZ', 'AAAZ')
1
>>> levenshtein_distance('AATZZZ', 'AAAZ')
3
"""
distance = 0
if len(string1) < len(string2):
string1, string2 = string2, string1
for i, v in itertools.izip_longest(string1, string2, fillvalue='-'):
if i != v:
distance += 1
return distance
# Find the string with the shortest edit distance.
list_of_string = ['AATC', 'TAGCGATC', 'ATCGAT']
strings_distances = collections.defaultdict(int)
for strings in itertools.combinations(list_of_string, 2):
strings_distances[strings[0]] += levenshtein_distance(*strings)
strings_distances[strings[1]] += levenshtein_distance(*strings)
shortest = min(strings_distances.iteritems(), key=lambda x: x[1])

Related

Is there a faster way to count non-overlapping occurrences in a string than count()?

Given a minimum length N and a string S of 1's and 0's (e.g. "01000100"), I am trying to return the number of non-overlapping occurrences of a sub-string of length n containing all '0's. For example, given n=2 and the string "01000100", the number of non-overlapping "00"s is 2.
This is what I have done:
def myfunc(S,N):
return S.count('0'*N)
My question: is there a faster way of performing this for very long strings? This is from an online coding practice site and my code passes all but one of the test cases, which fails due to not being able to finish within a time limit. Doing some research it seems I can only find that count() is the fastest method for this.
This might be faster:
>>> s = "01000100"
>>> def my_count( a, n ) :
... parts = a.split('1')
... return sum( len(p)//n for p in parts )
...
>>> my_count(s, 2)
2
>>>
Worst case scenario for count() is O(N^2), the function above is strictly linear O(N). Here's the discussion where O(N^2) number came from: What's the computational cost of count operation on strings Python?
Also, you may always do this manually, without using split(), just loop over the string, reset counter (once saved counter // n somewhere) on 1 and increase counter on 0. This would definitely beat any other approach because strictly O(N).
Finally, for relatively large values of n (n > 10 ?), there might be a sub-linear (or still linear, but with a smaller constant) algorithm, which starts with comparing a[n-1] to 0, and going back to beginning. Chances are, there going to be a 1 somewhere, so we don't have to analyse the beginning of the string if a[n-1] is 1 -- simply because there's no way to fit enough zeros in there. Assuming we have found 1 at position k, the next position to compare would be a[k+n-1], again going back to the beginning of the string.
This way we can effectively skip most of the string during the search.
lenik posted a very good response that worked well. I also found another method faster than count() that I will post here as well. It uses the findall() method from the regex library:
import re
def my_count(a, n):
return len(re.findall('0'*n, a))

Repeated String Match association with Boyer Moore Algorithm

Posting this question after looking around different blogs/SO questions and still not finding an answer
I'm trying to wrap my head around a solution/algorithm used from a leetcode contest.
Here is the question:
Given two strings A and B, find the minimum number of times A has to be repeated such that B is a substring of it. If no such solution, return -1.
For example, with A = "abcd" and B = "cdabcdab".
Return 3, because by repeating A three times (“abcdabcdabcd”), B is a substring of it; and B is not a substring of A repeated two times ("abcdabcd").
I know that rolling hash approach is the preferred way to go but I decided to start with the Boyer Moore approach.
After doing some research I learned that the following code used Boyer Moore Algorithm behind the scenes. Here is the code:
def repeatedStringMatch(self, A, B):
"""
:type A: str
:type B: str
:rtype: int
"""
m = math.ceil(len(B)/len(A))
if B in A * m:
return m
elif B in A * (m + 1):
return m + 1
else:
return -1
Based on my understanding of the algorithm I'm not convinced how the above solution might be using BM approach.
I'm specifically not clear what this line
m = math.ceil(len(B)/len(A))
does and why we have to find m in this fashion. Could anyone help me here?
Thanks in advance
The smaller string must be repeated at least m times for the larger string to be contained within.
The minimum value that m can assume is the smallest integer greater than the ratio of the lengths of two strings (larger / smaller), because, to contain a substring of length l, a string must have at least length l.
Using the example you shared.
A = "abcd"
B = "cdabcdab"
m0 = len(B) / len(A)
# m0 == 2.0
m = math.ceil(m0)
# m = 2
However, "cdabcdab" is not contained in "abcdabcd". But, if we repeat "abcd" 1 more time, we find that "cdabcdab" is then a substring. Repeating "abcd" further doesn't change whether it may be found as a substring or not. So, it is only necessary to check containment for m & (m+1) repetitions.
The python code you shared does not implement any searching algorithm, it just uses whatever searching algorithm is implemented for use with in. The specific algorithm might be implementation dependent, however there is a high likelihood for it to be boyer-moore or a variation of, as it is popular & efficient.
edit:
the algorithm behind B in A appears to be boyer-moore in cpython

DNA alignment -- score is additive or not?

So I have a recursive code that gives the best alignment for 2 DNA strands, but the problem is that it performs very slowly (I need it to be recursive). Then I read on an MIT website that the results are additive, which is great for me, but then I thought about it a little bit and I found out there is a problem:
website: http://ocw.mit.edu/courses/electrical-engineering-and-computer-science/6-096-algorithms-for-computational-biology-spring-2005/lecture-notes/lecture5_newest.pdf
The MIT website says that for a given spilt(i,j):
first_strand(0, i) and second_strand(0,j) alignment
+
first_strand(i, len) and second_strand(j, len) alignment
equals
first_strand and second strand alignment
but:
GTC GTAA
G with GTA alignment is G-- and GTA
TC with A alignment is TC and A-
result = G--TC and GTAA-
real best result = GTC- GTAA
Can anyone explain what they mean on the MIT website? I'm probably getting it all wrong!
I assume you're talking about this link.
If so, read it very carefully hundreds of times ;-) It's "additive" given that you're only considering alignments where the split is fixed at a specific (i, j) pair.
In your supposed counterexample, you started by breaking the initial G off of GTC and the initial GTA off of GTAA. Then G-- is the shortest way to change GTC into G. Fine. Continuing with the same split, you still needed to align the remaining right-hand parts: TC with A. Also fine.
This is no claim that this is the best possible split. There's only the claim that it's the best possible alignment given that you're only considering that specific split.
It's one small step in the dynamic programming approach, which is the part you're missing. It remains to compute the best alignments across all possible splits.
Dynamic programming is tricky at first. You shouldn't expect to learn it from staring at telegraphic slides. Read a real textbook, or search the web for tutorials.
Speeding a recursive version
The comments indicate that the code for this "must" be recursive. Oh well ;-)
Caution: I just threw this together to illustrate a general procedure for speeding suitable recursive functions. It's barely been tested at all.
First an utterly naive recursive version:
def lev(a, b):
if not a:
return len(b)
if not b:
return len(a)
return min(lev(a[:-1], b[:-1]) + (a[-1] != b[-1]),
lev(a[:-1], b) + 1,
lev(a, b[:-1]) + 1)
I'll be using "absd31-km" and "ldk3-1fjm" as arguments in all runs discussed here.
On my box, using Python 3, that simple function returns 7 after about 1.6 seconds. It's horribly slow.
The most obvious problem is the endlessly repeated string slicing. Each : in an index takes time proportional to the current length of the string being sliced. So the first refinement is to pass string indices instead. Since the code always slices off a prefix of a string, we only need to pass the "end of string" indices:
def lev2(a, b):
def inner(j1, j2):
if j1 < 0:
return j2 + 1
if j2 < 0:
return j1 + 1
return min(inner(j1-1, j2-1) + (a[j1] != b[j2]),
inner(j1-1, j2) + 1,
inner(j1, j2-1) + 1)
return inner(len(a)-1, len(b)-1)
Much better! This version returns 7 in "only" about 1.44 seconds. Still horridly slow, but better than the original. It's advantage would increase on longer strings, but who cares ;-)
We're almost done! The important thing to notice now is that the function passes the same arguments many times over the course of a run. We capture those in "a memo" to avoid all the redundant computation:
def lev3(a, b):
memo = {}
def inner(j1, j2):
if j1 < 0:
return j2 + 1
if j2 < 0:
return j1 + 1
args = j1, j2
if args in memo:
return memo[args]
result = min(inner(j1-1, j2-1) + (a[j1] != b[j2]),
inner(j1-1, j2) + 1,
inner(j1, j2-1) + 1)
memo[args] = result
return result
return inner(len(a)-1, len(b)-1)
That version returns 7 in about 0.00026 seconds, over 5000 times faster than lev2 did it.
Now if you've studied the matrix-based algorithms, and squint a little, you'll see that lev3() effectively builds a 2-dimensional matrix mapping index pairs to results in its memo dictionary. They're really the same thing, except that the recursive version builds the matrix in a more convoluted way. On the other hand, the recursive version may be easier to understand and to reason about. Note that the slides you found called the memoization aporoach "top down" and the nested-loop matrix approach "bottom up". Those are nicely descriptive.
You haven't said anything about how your recursive function works, but if it suffers any similar kinds of recursive excess, you should be able to get similar speedups using similar techniques :-)

Generating a list of distinct (distant, by edit distance) words by filtering

I have a long (> 1000 items) list of words, from which I would like to remove words that are "too similar" to other words, until the remaining words are all "significantly different". For example, so that no two words are within an edit distance D.
I do not need a unique solution, and it doesn't have to be exactly optimal, but it should be reasonably quick (in Python) and not discard way too many entries.
How can I achieve this? Thanks.
Edit: to be clear, I can google for a python routine that measures edit distance. The problem is how to do this efficiently, and, perhaps, in some way that finds a "natural" value of D. Maybe by constructing some kind of trie from all words and then pruning?
You can use a bk-tree, and before each item is added check that it is not within distance D of any others (thanks to #DietrichEpp in the comments for this idea.
You can use this recipe for a bk-tree (though any similar recipes are easily modified). Simply make two changes: change the line:
def __init__(self, items, distance, usegc=False):
to
def __init__(self, items, distance, threshold=0, usegc=False):
And change the line
if el not in self.nodes: # do not add duplicates
to
if (el not in self.nodes and
(threshold == None or len(self.find(el, threshold)) == 0)):
This makes sure there are no duplicates when an item is added. Then, the code to remove duplicates from a list is simply:
from Levenshtein import distance
from bktree import BKtree
def remove_duplicates(lst, threshold):
tr = BKtree(iter(lst), distance, threshold)
return tr.nodes.keys()
Note that this relies on the python-Levenshtein package for its distance function, which is much faster than the one provided by bk-tree. python-Levenshtein has C-compiled components, but it's worth the installation.
Finally, I set up a performance test with an increasing number of words (grabbed randomly from /usr/share/dict/words) and graphed the amount of time each took to run:
import random
import time
from Levenshtein import distance
from bktree import BKtree
with open("/usr/share/dict/words") as inf:
word_list = [l[:-1] for l in inf]
def remove_duplicates(lst, threshold):
tr = BKtree(iter(lst), distance, threshold)
return tr.nodes.keys()
def time_remove_duplicates(n, threshold):
"""Test using n words"""
nwords = random.sample(word_list, n)
t = time.time()
newlst = remove_duplicates(nwords, threshold)
return len(newlst), time.time() - t
ns = range(1000, 16000, 2000)
results = [time_remove_duplicates(n, 3) for n in ns]
lengths, timings = zip(*results)
from matplotlib import pyplot as plt
plt.plot(ns, timings)
plt.xlabel("Number of strings")
plt.ylabel("Time (s)")
plt.savefig("number_vs_time.pdf")
Without confirming it mathematically, I don't think it's quadratic, and I think it might actually be n log n, which would make sense if inserting into a bk-tree is a log time operation. Most notably, it runs pretty quickly with under 5000 strings, which hopefully is the OP's goal (and it's reasonable with 15000, which a traditional for loop solution would not be).
Tries will not be helpful, nor will hash maps. They are simply not useful for spatial, high-dimensional problems like this one.
But the real problem here is the ill-specified requirement of "efficient". How fast is "efficient"?
import Levenshtein
def simple(corpus, distance):
words = []
while corpus:
center = corpus[0]
words.append(center)
corpus = [word for word in corpus
if Levenshtein.distance(center, word) >= distance]
return words
I ran this on 10,000 words selected uniformly from the "American English" dictionary I have on my hard drive, looking for sets with a distance of 5, yielding around 2,000 entries.
real 0m2.558s
user 0m2.404s
sys 0m0.012s
So, the question is, "How efficient is efficient enough"? Since you didn't specify your requirements, it's really hard for me to know if this algorithm works for you or not.
The rabbit hole
If you want something faster, here's how I would do it.
Create a VP tree, BK tree, or other suitable spatial index. For each word in the corpus, insert that word into the tree if it has a suitable minimum distance from every word in the index. Spatial indexes are specifically designed to support this kind of query.
At the end, you will have a tree containing nodes with the desired minimum distance.
Your trie thought is definitely and interesting one. This page has a great setup for fast edit distance calculations in a trie and would definitely be efficient if you needed to expand your wordlist to millions rather than a thousand, which is pretty small in the corpora linguistics business.
Good luck, it sounds like a fun representation of the problem!

How to solve the "Mastermind" guessing game?

How would you create an algorithm to solve the following puzzle, "Mastermind"?
Your opponent has chosen four different colours from a set of six (yellow, blue, green, red, orange, purple). You must guess which they have chosen, and in what order. After each guess, your opponent tells you how many (but not which) of the colours you guessed were the right colour in the right place ["blacks"] and how many (but not which) were the right colour but in the wrong place ["whites"]. The game ends when you guess correctly (4 blacks, 0 whites).
For example, if your opponent has chosen (blue, green, orange, red), and you guess (yellow, blue, green, red), you will get one "black" (for the red), and two whites (for the blue and green). You would get the same score for guessing (blue, orange, red, purple).
I'm interested in what algorithm you would choose, and (optionally) how you translate that into code (preferably Python). I'm interested in coded solutions that are:
Clear (easily understood)
Concise
Efficient (fast in making a guess)
Effective (least number of guesses to solve the puzzle)
Flexible (can easily answer questions about the algorithm, e.g. what is its worst case?)
General (can be easily adapted to other types of puzzle than Mastermind)
I'm happy with an algorithm that's very effective but not very efficient (provided it's not just poorly implemented!); however, a very efficient and effective algorithm implemented inflexibly and impenetrably is not of use.
I have my own (detailed) solution in Python which I have posted, but this is by no means the only or best approach, so please post more! I'm not expecting an essay ;)
Key tools: entropy, greediness, branch-and-bound; Python, generators, itertools, decorate-undecorate pattern
In answering this question, I wanted to build up a language of useful functions to explore the problem. I will go through these functions, describing them and their intent. Originally, these had extensive docs, with small embedded unit tests tested using doctest; I can't praise this methodology highly enough as a brilliant way to implement test-driven-development. However, it does not translate well to StackOverflow, so I will not present it this way.
Firstly, I will be needing several standard modules and future imports (I work with Python 2.6).
from __future__ import division # No need to cast to float when dividing
import collections, itertools, math
I will need a scoring function. Originally, this returned a tuple (blacks, whites), but I found output a little clearer if I used a namedtuple:
Pegs = collections.namedtuple('Pegs', 'black white')
def mastermindScore(g1,g2):
matching = len(set(g1) & set(g2))
blacks = sum(1 for v1, v2 in itertools.izip(g1,g2) if v1 == v2)
return Pegs(blacks, matching-blacks)
To make my solution general, I pass in anything specific to the Mastermind problem as keyword arguments. I have therefore made a function that creates these arguments once, and use the **kwargs syntax to pass it around. This also allows me to easily add new attributes if I need them later. Note that I allow guesses to contain repeats, but constrain the opponent to pick distinct colours; to change this, I only need change G below. (If I wanted to allow repeats in the opponent's secret, I would need to change the scoring function as well.)
def mastermind(colours, holes):
return dict(
G = set(itertools.product(colours,repeat=holes)),
V = set(itertools.permutations(colours, holes)),
score = mastermindScore,
endstates = (Pegs(holes, 0),))
def mediumGame():
return mastermind(("Yellow", "Blue", "Green", "Red", "Orange", "Purple"), 4)
Sometimes I will need to partition a set based on the result of applying a function to each element in the set. For instance, the numbers 1..10 can be partitioned into even and odd numbers by the function n % 2 (odds give 1, evens give 0). The following function returns such a partition, implemented as a map from the result of the function call to the set of elements that gave that result (e.g. { 0: evens, 1: odds }).
def partition(S, func, *args, **kwargs):
partition = collections.defaultdict(set)
for v in S: partition[func(v, *args, **kwargs)].add(v)
return partition
I decided to explore a solver that uses a greedy entropic approach. At each step, it calculates the information that could be obtained from each possible guess, and selects the most informative guess. As the numbers of possibilities grow, this will scale badly (quadratically), but let's give it a try! First, I need a method to calculate the entropy (information) of a set of probabilities. This is just -∑p log p. For convenience, however, I will allow input that are not normalized, i.e. do not add up to 1:
def entropy(P):
total = sum(P)
return -sum(p*math.log(p, 2) for p in (v/total for v in P if v))
So how am I going to use this function? Well, for a given set of possibilities, V, and a given guess, g, the information we get from that guess can only come from the scoring function: more specifically, how that scoring function partitions our set of possibilities. We want to make a guess that distinguishes best among the remaining possibilites — divides them into the largest number of small sets — because that means we are much closer to the answer. This is exactly what the entropy function above is putting a number to: a large number of small sets will score higher than a small number of large sets. All we need to do is plumb it in.
def decisionEntropy(V, g, score):
return entropy(collections.Counter(score(gi, g) for gi in V).values())
Of course, at any given step what we will actually have is a set of remaining possibilities, V, and a set of possible guesses we could make, G, and we will need to pick the guess which maximizes the entropy. Additionally, if several guesses have the same entropy, prefer to pick one which could also be a valid solution; this guarantees the approach will terminate. I use the standard python decorate-undecorate pattern together with the built-in max method to do this:
def bestDecision(V, G, score):
return max((decisionEntropy(V, g, score), g in V, g) for g in G)[2]
Now all I need to do is repeatedly call this function until the right result is guessed. I went through a number of implementations of this algorithm until I found one that seemed right. Several of my functions will want to approach this in different ways: some enumerate all possible sequences of decisions (one per guess the opponent may have made), while others are only interested in a single path through the tree (if the opponent has already chosen a secret, and we are just trying to reach the solution). My solution is a "lazy tree", where each part of the tree is a generator that can be evaluated or not, allowing the user to avoid costly calculations they won't need. I also ended up using two more namedtuples, again for clarity of code.
Node = collections.namedtuple('Node', 'decision branches')
Branch = collections.namedtuple('Branch', 'result subtree')
def lazySolutionTree(G, V, score, endstates, **kwargs):
decision = bestDecision(V, G, score)
branches = (Branch(result, None if result in endstates else
lazySolutionTree(G, pV, score=score, endstates=endstates))
for (result, pV) in partition(V, score, decision).iteritems())
yield Node(decision, branches) # Lazy evaluation
The following function evaluates a single path through this tree, based on a supplied scoring function:
def solver(scorer, **kwargs):
lazyTree = lazySolutionTree(**kwargs)
steps = []
while lazyTree is not None:
t = lazyTree.next() # Evaluate node
result = scorer(t.decision)
steps.append((t.decision, result))
subtrees = [b.subtree for b in t.branches if b.result == result]
if len(subtrees) == 0:
raise Exception("No solution possible for given scores")
lazyTree = subtrees[0]
assert(result in endstates)
return steps
This can now be used to build an interactive game of Mastermind where the user scores the computer's guesses. Playing around with this reveals some interesting things. For example, the most informative first guess is of the form (yellow, yellow, blue, green), not (yellow, blue, green, red). Extra information is gained by using exactly half the available colours. This also holds for 6-colour 3-hole Mastermind — (yellow, blue, green) — and 8-colour 5-hole Mastermind — (yellow, yellow, blue, green, red).
But there are still many questions that are not easily answered with an interactive solver. For instance, what is the most number of steps needed by the greedy entropic approach? And how many inputs take this many steps? To make answering these questions easier, I first produce a simple function that turns the lazy tree of above into a set of paths through this tree, i.e. for each possible secret, a list of guesses and scores.
def allSolutions(**kwargs):
def solutions(lazyTree):
return ((((t.decision, b.result),) + solution
for t in lazyTree for b in t.branches
for solution in solutions(b.subtree))
if lazyTree else ((),))
return solutions(lazySolutionTree(**kwargs))
Finding the worst case is a simple matter of finding the longest solution:
def worstCaseSolution(**kwargs):
return max((len(s), s) for s in allSolutions(**kwargs)) [1]
It turns out that this solver will always complete in 5 steps or fewer. Five steps! I know that when I played Mastermind as a child, I often took longer than this. However, since creating this solver and playing around with it, I have greatly improved my technique, and 5 steps is indeed an achievable goal even when you don't have time to calculate the entropically ideal guess at each step ;)
How likely is it that the solver will take 5 steps? Will it ever finish in 1, or 2, steps? To find that out, I created another simple little function that calculates the solution length distribution:
def solutionLengthDistribution(**kwargs):
return collections.Counter(len(s) for s in allSolutions(**kwargs))
For the greedy entropic approach, with repeats allowed: 7 cases take 2 steps; 55 cases take 3 steps; 229 cases take 4 steps; and 69 cases take the maximum of 5 steps.
Of course, there's no guarantee that the greedy entropic approach minimizes the worst-case number of steps. The final part of my general-purpose language is an algorithm that decides whether or not there are any solutions for a given worst-case bound. This will tell us whether greedy entropic is ideal or not. To do this, I adopt a branch-and-bound strategy:
def solutionExists(maxsteps, G, V, score, **kwargs):
if len(V) == 1: return True
partitions = [partition(V, score, g).values() for g in G]
maxSize = max(len(P) for P in partitions) ** (maxsteps - 2)
partitions = (P for P in partitions if max(len(s) for s in P) <= maxSize)
return any(all(solutionExists(maxsteps-1,G,s,score) for l,s in
sorted((-len(s), s) for s in P)) for i,P in
sorted((-entropy(len(s) for s in P), P) for P in partitions))
This is definitely a complex function, so a bit more explanation is in order. The first step is to partition the remaining solutions based on their score after a guess, as before, but this time we don't know what guess we're going to make, so we store all partitions. Now we could just recurse into every one of these, effectively enumerating the entire universe of possible decision trees, but this would take a horrifically long time. Instead I observe that, if at this point there is no partition that divides the remaining solutions into more than n sets, then there can be no such partition at any future step either. If we have k steps left, that means we can distinguish between at most nk-1 solutions before we run out of guesses (on the last step, we must always guess correctly). Thus we can discard any partitions that contain a score mapped to more than this many solutions. This is the next two lines of code.
The final line of code does the recursion, using Python's any and all functions for clarity, and trying the highest-entropy decisions first to hopefully minimize runtime in the positive case. It also recurses into the largest part of the partition first, as this is the most likely to fail quickly if the decision was wrong. Once again, I use the standard decorate-undecorate pattern, this time to wrap Python's sorted function.
def lowerBoundOnWorstCaseSolution(**kwargs):
for steps in itertools.count(1):
if solutionExists(maxsteps=steps, **kwargs):
return steps
By calling solutionExists repeatedly with an increasing number of steps, we get a strict lower bound on the number of steps needed in the worst case for a Mastermind solution: 5 steps. The greedy entropic approach is indeed optimal.
Out of curiosity, I invented another guessing game, which I nicknamed "twoD". In this, you try to guess a pair of numbers; at each step, you get told if your answer is correct, if the numbers you guessed are no less than the corresponding ones in the secret, and if the numbers are no greater.
Comparison = collections.namedtuple('Comparison', 'less greater equal')
def twoDScorer(x, y):
return Comparison(all(r[0] <= r[1] for r in zip(x, y)),
all(r[0] >= r[1] for r in zip(x, y)),
x == y)
def twoD():
G = set(itertools.product(xrange(5), repeat=2))
return dict(G = G, V = G, score = twoDScorer,
endstates = set(Comparison(True, True, True)))
For this game, the greedy entropic approach has a worst case of five steps, but there is a better solution possible with a worst case of four steps, confirming my intuition that myopic greediness is only coincidentally ideal for Mastermind. More importantly, this has shown how flexible my language is: all the same methods work for this new guessing game as did for Mastermind, letting me explore other games with a minimum of extra coding.
What about performance? Obviously, being implemented in Python, this code is not going to be blazingly fast. I've also dropped some possible optimizations in favour of clear code.
One cheap optimization is to observe that, on the first move, most guesses are basically identical: (yellow, blue, green, red) is really no different from (blue, red, green, yellow), or (orange, yellow, red, purple). This greatly reduces the number of guesses we need consider on the first step — otherwise the most costly decision in the game.
However, because of the large runtime growth rate of this problem, I was not able to solve the 8-colour, 5-hole Mastermind problem, even with this optimization. Instead, I ported the algorithms to C++, keeping the general structure the same and employing bitwise operations to boost performance in the critical inner loops, for a speedup of many orders of magnitude. I leave this as an exercise to the reader :)
Addendum, 2018: It turns out the greedy entropic approach is not optimal for the 8-colour, 4-hole Mastermind problem either, with a worst-case length of 7 steps when an algorithm exists that takes at most 6!
I once wrote a "Jotto" solver which is essentially "Master Mind" with words. (We each pick a word and we take turns guessing at each other's word, scoring "right on" (exact) matches and "elsewhere" (correct letter/color, but wrong placement).
The key to solving such a problem is the realization that the scoring function is symmetric.
In other words if score(myguess) == (1,2) then I can use the same score() function to compare my previous guess with any other possibility and eliminate any that don't give exactly the same score.
Let me give an example: The hidden word (target) is "score" ... the current guess is "fools" --- the score is 1,1 (one letter, 'o', is "right on"; another letter, 's', is "elsewhere"). I can eliminate the word "guess" because the `score("guess") (against "fools") returns (1,0) (the final 's' matches, but nothing else does). So the word "guess" is not consistent with "fools" and a score against some unknown word that gave returned a score of (1,1).
So I now can walk through every five letter word (or combination of five colors/letters/digits etc) and eliminate anything that doesn't score 1,1 against "fools." Do that at each iteration and you'll very rapidly converge on the target. (For five letter words I was able to get within 6 tries every time ... and usually only 3 or 4). Of course there's only 6000 or so "words" and you're eliminating close to 95% for each guess.
Note: for the following discussion I'm talking about five letter "combination" rather than four elements of six colors. The same algorithms apply; however, the problem is orders of magnitude smaller for the old "Master Mind" game ... there are only 1296 combinations (6**4) of colored pegs in the classic "Master Mind" program, assuming duplicates are allowed. The line of reasoning that leads to the convergence involves some combinatorics: there are 20 non-winning possible scores for a five element target (n = [(a,b) for a in range(5) for b in range(6) if a+b <= 5] to see all of them if you're curious. We would, therefore, expect that any random valid selection would have a roughly 5% chance of matching our score ... the other 95% won't and therefore will be eliminated for each scored guess. This doesn't account for possible clustering in word patterns but the real world behavior is close enough for words and definitely even closer for "Master Mind" rules. However, with only 6 colors in 4 slots we only have 14 possible non-winning scores so our convergence isn't quite as fast).
For Jotto the two minor challenges are: generating a good world list (awk -f 'length($0)==5' /usr/share/dict/words or similar on a UNIX system) and what to do if the user has picked a word that not in our dictionary (generate every letter combination, 'aaaaa' through 'zzzzz' --- which is 26 ** 5 ... or ~1.1 million). A trivial combination generator in Python takes about 1 minute to generate all those strings ... an optimized one should to far better. (I can also add a requirement that every "word" have at least one vowel ... but this constraint doesn't help much --- 5 vowels * 5 possible locations for that and then multiplied by 26 ** 4 other combinations).
For Master Mind you use the same combination generator ... but with only 4 or 5 "letters" (colors). Every 6-color combination (15,625 of them) can be generated in under a second (using the same combination generator as I used above).
If I was writing this "Jotto" program today, in Python for example, I would "cheat" by having a thread generating all the letter combos in the background while I was still eliminated words from the dictionary (while my opponent was scoring me, guessing, etc). As I generated them I'd also eliminate against all guesses thus far. Thus I would, after I'd eliminated all known words, have a relatively small list of possibilities and against a human player I've "hidden" most of my computation lag by doing it in parallel to their input. (And, if I wrote a web server version of such a program I'd have my web engine talk to a local daemon to ask for sequences consistent with a set of scores. The daemon would keep a locally generated list of all letter combinations and would use a select.select() model to feed possibilities back to each of the running instances of the game --- each would feed my daemon word/score pairs which my daemon would apply as a filter on the possibilities it feeds back to that client).
(By comparison I wrote my version of "Jotto" about 20 years ago on an XT using Borland TurboPascal ... and it could do each elimination iteration --- starting with its compiled in list of a few thousand words --- in well under a second. I build its word list by writing a simple letter combination generator (see below) ... saving the results to a moderately large file, then running my word processor's spell check on that with a macro to delete everything that was "mis-spelled" --- then I used another macro to wrap all the remaining lines in the correct punctuation to make them valid static assignments to my array, which was a #include file to my program. All that let me build a standalone game program that "knew" just about every valid English 5 letter word; the program was a .COM --- less than 50KB if I recall correctly).
For other reasons I've recently written a simple arbitrary combination generator in Python. It's about 35 lines of code and I've posted that to my "trite snippets" wiki on bitbucket.org ... it's not a "generator" in the Python sense ... but a class you can instantiate to an infinite sequence of "numeric" or "symbolic" combination of elements (essentially counting in any positive integer base).
You can find it at: Trite Snippets: Arbitrary Sequence Combination Generator
For the exact match part of our score() function you can just use this:
def score(this, that):
'''Simple "Master Mind" scoring function'''
exact = len([x for x,y in zip(this, that) if x==y])
### Calculating "other" (white pegs) goes here:
### ...
###
return (exact,other)
I think this exemplifies some of the beauty of Python: zip() up the two sequences,
return any that match, and take the length of the results).
Finding the matches in "other" locations is deceptively more complicated. If no repeats were allowed then you could simply use sets to find the intersections.
[In my earlier edit of this message, when I realized how I could use zip() for exact matches, I erroneously thought we could get away with other = len([x for x,y in zip(sorted(x), sorted(y)) if x==y]) - exact ... but it was late and I was tired. As I slept on it I realized that the method was flawed. Bad, Jim! Don't post without adequate testing!* (Tested several cases that happened to work)].
In the past the approach I used was to sort both lists, compare the heads of each: if the heads are equal, increment the count and pop new items from both lists. otherwise pop a new value into the lesser of the two heads and try again. Break as soon as either list is empty.
This does work; but it's fairly verbose. The best I can come up with using that approach is just over a dozen lines of code:
other = 0
x = sorted(this) ## Implicitly converts to a list!
y = sorted(that)
while len(x) and len(y):
if x[0] == y[0]:
other += 1
x.pop(0)
y.pop(0)
elif x[0] < y[0]:
x.pop(0)
else:
y.pop(0)
other -= exact
Using a dictionary I can trim that down to about nine:
other = 0
counters = dict()
for i in this:
counters[i] = counters.get(i,0) + 1
for i in that:
if counters.get(i,0) > 0:
other += 1
counters[i] -= 1
other -= exact
(Using the new "collections.Counter" class (Python3 and slated for Python 2.7?) I could presumably reduce this a little more; three lines here are initializing the counters collection).
It's important to decrement the "counter" when we find a match and it's vital to test for counter greater than zero in our test. If a given letter/symbol appears in "this" once and "that" twice then it must only be counted as a match once.
The first approach is definitely a bit trickier to write (one must be careful to avoid boundaries). Also in a couple of quick benchmarks (testing a million randomly generated pairs of letter patterns) the first approach takes about 70% longer as the one using dictionaries. (Generating the million pairs of strings using random.shuffle() took over twice as long as the slower of the scoring functions, on the other hand).
A formal analysis of the performance of these two functions would be complicated. The first method has two sorts, so that would be 2 * O(nlog(n)) ... and it iterates through at least one of the two strings and possibly has to iterate all the way to the end of the other string (best case O(n), worst case O(2n)) -- force I'm mis-using big-O notation here, but this is just a rough estimate. The second case depends entirely on the perfomance characteristics of the dictionary. If we were using b-trees then the performance would be roughly O(nlog(n) for creation and finding each element from the other string therein would be another O(n*log(n)) operation. However, Python dictionaries are very efficient and these operations should be close to constant time (very few hash collisions). Thus we'd expect a performance of roughly O(2n) ... which of course simplifies to O(n). That roughly matches my benchmark results.
Glancing over the Wikipedia article on "Master Mind" I see that Donald Knuth used an approach which starts similarly to mine (and 10 years earlier) but he added one significant optimization. After gathering every remaining possibility he selects whichever one would eliminate the largest number of possibilities on the next round. I considered such an enhancement to my own program and rejected the idea for practical reasons. In his case he was searching for an optimal (mathematical) solution. In my case I was concerned about playability (on an XT, preferably using less than 64KB of RAM, though I could switch to .EXE format and use up to 640KB). I wanted to keep the response time down in the realm of one or two seconds (which was easy with my approach but which would be much more difficult with the further speculative scoring). (Remember I was working in Pascal, under MS-DOS ... no threads, though I did implement support for crude asynchronous polling of the UI which turned out to be unnecessary)
If I were writing such a thing today I'd add a thread to do the better selection, too. This would allow me to give the best guess I'd found within a certain time constraint, to guarantee that my player didn't have to wait too long for my guess. Naturally my selection/elimination would be running while waiting for my opponent's guesses.
Have you seem Raymond Hettingers attempt? They certainly match up to some of your requirements.
I wonder how his solutions compares to yours.
There is a great site about MasterMind strategy here. The author starts off with very simple MasterMind problems (using numbers rather than letters, and ignoring order and repetition) and gradually builds up to a full MasterMind problem (using colours, which can be repeated, in any order, even with the possibility of errors in the clues).
The seven tutorials that are presented are as follows:
Tutorial 1 - The simplest game setting (no errors, fixed order, no repetition)
Tutorial 2 - Code may contain blank spaces (no errors, fixed order, no repetition)
Tutorial 3 - Hints may contain errors (fixed order, no repetition)
Tutorial 4 - Game started from the middle (no errors, fixed order, no repetition)
Tutorial 5 - Digits / colours may be repeated (no errors, fixed order, each colour repeated at most 4 times)
Tutorial 6 - Digits / colours arranged in random order (no errors, random order, no repetition)
Tutorial 7 - Putting it all together (no errors, random order, each colour repeated at most 4 times)
Just thought I'd contribute my 90 odd lines of code. I've build upon #Jim Dennis' answer, mostly taking away the hint on symetric scoring. I've implemented the minimax algorithm as described on the Mastermind wikipedia article by Knuth, with one exception: I restrict my next move to current list of possible solutions, as I found performance deteriorated badly when taking all possible solutions into account at each step. The current approach leaves me with a worst case of 6 guesses for any combination, each found in well under a second.
It's perhaps important to note that I make no restriction whatsoever on the hidden sequence, allowing for any number of repeats.
from itertools import product, tee
from random import choice
COLORS = 'red ', 'green', 'blue', 'yellow', 'purple', 'pink'#, 'grey', 'white', 'black', 'orange', 'brown', 'mauve', '-gap-'
HOLES = 4
def random_solution():
"""Generate a random solution."""
return tuple(choice(COLORS) for i in range(HOLES))
def all_solutions():
"""Generate all possible solutions."""
for solution in product(*tee(COLORS, HOLES)):
yield solution
def filter_matching_result(solution_space, guess, result):
"""Filter solutions for matches that produce a specific result for a guess."""
for solution in solution_space:
if score(guess, solution) == result:
yield solution
def score(actual, guess):
"""Calculate score of guess against actual."""
result = []
#Black pin for every color at right position
actual_list = list(actual)
guess_list = list(guess)
black_positions = [number for number, pair in enumerate(zip(actual_list, guess_list)) if pair[0] == pair[1]]
for number in reversed(black_positions):
del actual_list[number]
del guess_list[number]
result.append('black')
#White pin for every color at wrong position
for color in guess_list:
if color in actual_list:
#Remove the match so we can't score it again for duplicate colors
actual_list.remove(color)
result.append('white')
#Return a tuple, which is suitable as a dictionary key
return tuple(result)
def minimal_eliminated(solution_space, solution):
"""For solution calculate how many possibilities from S would be eliminated for each possible colored/white score.
The score of the guess is the least of such values."""
result_counter = {}
for option in solution_space:
result = score(solution, option)
if result not in result_counter.keys():
result_counter[result] = 1
else:
result_counter[result] += 1
return len(solution_space) - max(result_counter.values())
def best_move(solution_space):
"""Determine the best move in the solution space, being the one that restricts the number of hits the most."""
elim_for_solution = dict((minimal_eliminated(solution_space, solution), solution) for solution in solution_space)
max_elimintated = max(elim_for_solution.keys())
return elim_for_solution[max_elimintated]
def main(actual = None):
"""Solve a game of mastermind."""
#Generate random 'hidden' sequence if actual is None
if actual == None:
actual = random_solution()
#Start the game of by choosing n unique colors
current_guess = COLORS[:HOLES]
#Initialize solution space to all solutions
solution_space = all_solutions()
guesses = 1
while True:
#Calculate current score
current_score = score(actual, current_guess)
#print '\t'.join(current_guess), '\t->\t', '\t'.join(current_score)
if current_score == tuple(['black'] * HOLES):
print guesses, 'guesses for\t', '\t'.join(actual)
return guesses
#Restrict solution space to exactly those hits that have current_score against current_guess
solution_space = tuple(filter_matching_result(solution_space, current_guess, current_score))
#Pick the candidate that will limit the search space most
current_guess = best_move(solution_space)
guesses += 1
if __name__ == '__main__':
print max(main(sol) for sol in all_solutions())
Should anyone spot any possible improvements to the above code than I would be very much interested in your suggestions.
To work out the "worst" case, instead of using entropic I am looking to the partition that has the maximum number of elements, then select the try that is a minimum for this maximum => This will give me the minimum number of remaining possibility when I am not lucky (which happens in the worst case).
This always solve standard case in 5 attempts, but it is not a full proof that 5 attempts are really needed because it could happen that for next step a bigger set possibilities would have given a better result than a smaller one (because easier to distinguish between).
Though for the "Standard game" with 1680 I have a simple formal proof:
For the first step the try that gives the minimum for the partition with the maximum number is 0,0,1,1: 256. Playing 0,0,1,2 is not as good: 276.
For each subsequent try there are 14 outcomes (1 not placed and 3 placed is impossible) and 4 placed is giving a partition of 1. This means that in the best case (all partition same size) we will get a maximum partition that is a minimum of (number of possibilities - 1)/13 (rounded up because we have integer so necessarily some will be less and other more, so that the maximum is rounded up).
If I apply this:
After first play (0,0,1,1) I am getting 256 left.
After second try: 20 = (256-1)/13
After third try : 2 = (20-1)/13
Then I have no choice but to try one of the two left for the 4th try.
If I am unlucky a fifth try is needed.
This proves we need at least 5 tries (but not that this is enough).
Here is a generic algorithm I wrote that uses numbers to represent the different colours. Easy to change, but I find numbers to be a lot easier to work with than strings.
You can feel free to use any whole or part of this algorithm, as long as credit is given accordingly.
Please note I'm only a Grade 12 Computer Science student, so I am willing to bet that there are definitely more optimized solutions available.
Regardless, here's the code:
import random
def main():
userAns = raw_input("Enter your tuple, and I will crack it in six moves or less: ")
play(ans=eval("("+userAns+")"),guess=(0,0,0,0),previousGuess=[])
def play(ans=(6,1,3,5),guess=(0,0,0,0),previousGuess=[]):
if(guess==(0,0,0,0)):
guess = genGuess(guess,ans)
else:
checker = -1
while(checker==-1):
guess,checker = genLogicalGuess(guess,previousGuess,ans)
print guess, ans
if not(guess==ans):
previousGuess.append(guess)
base = check(ans,guess)
play(ans=ans,guess=base,previousGuess=previousGuess)
else:
print "Found it!"
def genGuess(guess,ans):
guess = []
for i in range(0,len(ans),1):
guess.append(random.randint(1,6))
return tuple(guess)
def genLogicalGuess(guess,previousGuess,ans):
newGuess = list(guess)
count = 0
#Generate guess
for i in range(0,len(newGuess),1):
if(newGuess[i]==-1):
newGuess.insert(i,random.randint(1,6))
newGuess.pop(i+1)
for item in previousGuess:
for i in range(0,len(newGuess),1):
if((newGuess[i]==item[i]) and (newGuess[i]!=ans[i])):
newGuess.insert(i,-1)
newGuess.pop(i+1)
count+=1
if(count>0):
return guess,-1
else:
guess = tuple(newGuess)
return guess,0
def check(ans,guess):
base = []
for i in range(0,len(zip(ans,guess)),1):
if not(zip(ans,guess)[i][0] == zip(ans,guess)[i][1]):
base.append(-1)
else:
base.append(zip(ans,guess)[i][1])
return tuple(base)
main()
Here's a link to pure Python solver for Mastermind(tm): http://code.activestate.com/recipes/496907-mastermind-style-code-breaking/ It has a simple version, a way to experiment with various guessing strategies, performance measurement, and an optional C accelerator.
The core of the recipe is short and sweet:
import random
from itertools import izip, imap
digits = 4
fmt = '%0' + str(digits) + 'd'
searchspace = tuple([tuple(map(int,fmt % i)) for i in range(0,10**digits)])
def compare(a, b, imap=imap, sum=sum, izip=izip, min=min):
count1 = [0] * 10
count2 = [0] * 10
strikes = 0
for dig1, dig2 in izip(a,b):
if dig1 == dig2:
strikes += 1
count1[dig1] += 1
count2[dig2] += 1
balls = sum(imap(min, count1, count2)) - strikes
return (strikes, balls)
def rungame(target, strategy, verbose=True, maxtries=15):
possibles = list(searchspace)
for i in xrange(maxtries):
g = strategy(i, possibles)
if verbose:
print "Out of %7d possibilities. I'll guess %r" % (len(possibles), g),
score = compare(g, target)
if verbose:
print ' ---> ', score
if score[0] == digits:
if verbose:
print "That's it. After %d tries, I won." % (i+1,)
break
possibles = [n for n in possibles if compare(g, n) == score]
return i+1
def strategy_allrand(i, possibles):
return random.choice(possibles)
if __name__ == '__main__':
hidden_code = random.choice(searchspace)
rungame(hidden_code, strategy_allrand)
Here is what the output looks like:
Out of 10000 possibilities. I'll guess (6, 4, 0, 9) ---> (1, 0)
Out of 1372 possibilities. I'll guess (7, 4, 5, 8) ---> (1, 1)
Out of 204 possibilities. I'll guess (1, 4, 2, 7) ---> (2, 1)
Out of 11 possibilities. I'll guess (1, 4, 7, 1) ---> (3, 0)
Out of 2 possibilities. I'll guess (1, 4, 7, 4) ---> (4, 0)
That's it. After 5 tries, I won.
My friend was considering relatively simple case - 8 colors, no repeats, no blanks.
With no repeats, there's no need for the max entropy consideration, all guesses have the same entropy and first or random guessing all work fine.
Here's the full code to solve that variant:
# SET UP
import random
import itertools
colors = ('red', 'orange', 'yellow', 'green', 'blue', 'indigo', 'violet', 'ultra')
# ONE FUNCTION REQUIRED
def EvaluateCode(guess, secret_code):
key = []
for i in range(0, 4):
for j in range(0, 4):
if guess[i] == secret_code[j]:
key += ['black'] if i == j else ['white']
return key
# MAIN CODE
# choose secret code
secret_code = random.sample(colors, 4)
print ('(shh - secret code is: ', secret_code, ')\n', sep='')
# create the full list of permutations
full_code_list = list(itertools.permutations(colors, 4))
N_guess = 0
while True:
N_guess += 1
print ('Attempt #', N_guess, '\n-----------', sep='')
# make a random guess
guess = random.choice(full_code_list)
print ('guess:', guess)
# evaluate the guess and get the key
key = EvaluateCode(guess, secret_code)
print ('key:', key)
if key == ['black', 'black', 'black', 'black']:
break
# remove codes from the code list that don't match the key
full_code_list2 = []
for i in range(0, len(full_code_list)):
if EvaluateCode(guess, full_code_list[i]) == key:
full_code_list2 += [full_code_list[i]]
full_code_list = full_code_list2
print ('N remaining: ', len(full_code_list), '\n', full_code_list, '\n', sep='')
print ('\nMATCH after', N_guess, 'guesses\n')

Categories