We Keep Coding

Suggestions to optimise brute force number solver python

I'm aware of a few other similar threads, but im not sure how to apply them to my example.
I am trying to attack the countdown number game from a brute force algorithm. This is my first attempt at anything like this, so any tips on how to speed up the process up?
I am testing it for situations where the answer is unsolvable given the initial numbers. This eventually will be paired up with a tkinter interface for the full game.
structure should look like this, where we try every order of abcdef and every operation combo for op1-5
from datetime import datetime
import itertools
import math
starttime = datetime.now()
def permutationG(input, s):
if len(s) == len(input): yield s
for i in input:
if i in s: continue
s=s+i
for x in permutationG(input, s): yield x
s=s[:-1]
def op(operator, number1,number2):
string=str(number1)+str(operator)+str(number2)
return eval(string)
a=11
b=10
c=9
d=8
e=7
f=6
targetnumber = 101234
listofnumbers = ['a','b','c','d','e','f']
listprep = ['+','-','*','/']
stringofnumbers = ''.join(str(e) for e in listofnumbers)
numberlocations =[]
for item in permutationG(listofnumbers,''):
numberlocations.append(item)
numberlocations = set(numberlocations)
myarray = itertools.combinations_with_replacement(listprep, 5)
operatorlist = []
for item in myarray:
#for all different numbers find the different permutations
temp = list(itertools.permutations(item))
operatorlist.extend(temp)
#remove duplicates
finaloplist = list(set(operatorlist))
dist=[math.inf]
currentclosestnumber = 0
count=0
looptime=datetime.now()
print('Starting Loop')
for item in numberlocations:
for ops in finaloplist:
initial_value = op(ops[0],item[0],item[1])
for i in range(2,len(item)):
intcheck2 = int(initial_value) - initial_value
if initial_value != targetnumber and initial_value >= 0 and intcheck2 == 0:
newvalue = op(ops[i-1], initial_value, item[i])
else:
break
initial_value = newvalue
attempt = initial_value
intcheck = int(attempt) - attempt
distance = targetnumber - initial_value
if abs(distance) < abs(dist[0]) and intcheck == 0:
currentclosestnumber = attempt
dist[0]=distance
print(attempt)
if targetnumber == attempt:
break
if targetnumber == attempt:
break
endtime = datetime.now()
stringtime= endtime-starttime
#print('Loops: ', count)
if targetnumber == attempt:
print('FOUNDIT!! Target Number = %s Closest Number = %s Time Elapsed = %s' %(targetnumber, currentclosestnumber, stringtime))
elif targetnumber!=attempt:
print('Heres how close: Target Number = %s Closest Number = %s Time Elapsed = %s' %(targetnumber, currentclosestnumber, stringtime))
This outputs a time of roughly a minute and a half.
Another issue is because of the method I'm using (using eval string manipulation) I have no idea to show where the brackets go in the final formula when printed, or how to fit an eval into the zip at the end to show the numbers instead of the letters.
Any guidance is really appreciated.
Note: I have edited the post with the most recent version of the code. This reduced the time to calculate from 1:30 to 0:45. The major change was instead of one long string of calculations I created a for loop for each sequential operation, with an if statement to make sure that if the current value is negative or a decimal it breaks.
This reduces the number of calculations required significantly.

Here is a slower version but without the bugs.
I don't have time to try to optimize, but my thought is that most of the problem is in creating/destroying garbage in splitting subsets.
If I really wanted this to be fast, I think that you could use a dynamic programming approach.
def subset_splits (a_list):
if 0 == len(a_list):
yield ([], [])
else:
for sublist1, sublist2 in subset_splits(a_list[1:]):
yield ([a_list[0]] + sublist1, sublist2)
yield ([a_list[0]] + sublist2, sublist1)
def reachable (numbers):
if 1 == len(numbers):
yield (numbers[0], numbers[0])
else:
for list1, list2 in subset_splits(numbers):
if 0 == len(list2):
continue
for x1, expr1 in reachable(list1):
for x2, expr2 in reachable(list2):
yield x1+x2, (expr1, '+', expr2)
yield x1*x2, (expr1, '*', expr2)
yield x1-x2, (expr1, '-', expr2)
yield x2-x1, (expr2, '-', expr1)
if 0 != x2:
yield x1/x2, (expr1, '/', expr2)
if 0 != x1:
yield x2/x1, (expr2, '/', expr1)
numbers = [1, 2, 3, 4, 5, 6]
target = 10000
best = numbers[0]
if best == target:
print(("Answer: ", numbers[0], numbers[0]))
else:
print(("Best: ", numbers[0], numbers[0]))
done = False;
for s, t in subset_splits(numbers):
if done:
break
for x, expr in reachable(s):
if x == target:
print(("Answer: ", x, expr))
done = True
break
elif abs(target-x) < abs(target-best):
print(("Best: ", x, expr))
best = x
if done:
break
for x, expr in reachable(t):
if x == target:
print(("Answer: ", x, expr))
done = True
break
elif abs(target-x) < abs(best-x):
print(("Best: ", x, expr))
best = x

Algorithm for matching objects

I have 1,000 objects, each object has 4 attribute lists: a list of words, images, audio files and video files.
I want to compare each object against:
a single object, Ox, from the 1,000.
every other object.
A comparison will be something like:
sum(words in common+ images in common+...).
I want an algorithm that will help me find the closest 5, say, objects to Ox and (a different?) algorithm to find the closest 5 pairs of objects
I've looked into cluster analysis and maximal matching and they don't seem to exactly fit this scenario. I don't want to use these method if something more apt exists, so does this look like a particular type of algorithm to anyone, or can anyone point me in the right direction to applying the algorithms I mentioned to this?

I made an example program for how to solve your first question. But you have to implement ho you want to compare images, audio and videos. And I assume every object has the same length for all lists. To answer your question number two it would be something similar, but with a double loop.
import numpy as np
from random import randint
class Thing:
def __init__(self, words, images, audios, videos):
self.words = words
self.images = images
self.audios = audios
self.videos = videos
def compare(self, other):
score = 0
# Assuming the attribute lists have the same length for both objects
# and that they are sorted in the same manner:
for i in range(len(self.words)):
if self.words[i] == other.words[i]:
score += 1
for i in range(len(self.images)):
if self.images[i] == other.images[i]:
score += 1
# And so one for audio and video. You have to make sure you know
# what method to use for determining when an image/audio/video are
# equal.
return score
N = 1000
things = []
words = np.random.randint(5, size=(N,5))
images = np.random.randint(5, size=(N,5))
audios = np.random.randint(5, size=(N,5))
videos = np.random.randint(5, size=(N,5))
# For testing purposes I assign each attribute to a list (array) containing
# five random integers. I don't know how you actually intend to do it.
for i in xrange(N):
things.append(Thing(words[i], images[i], audios[i], videos[i]))
# I will assume that object number 999 (i=999) is the Ox:
ox = 999
scores = np.zeros(N - 1)
for i in xrange(N - 1):
scores[i] = (things[ox].compare(things[i]))
best = np.argmax(scores)
print "The most similar thing is thing number %d." % best
print
print "Ox attributes:"
print things[ox].words
print things[ox].images
print things[ox].audios
print things[ox].videos
print
print "Best match attributes:"
print things[ox].words
print things[ox].images
print things[ox].audios
print things[ox].videos
EDIT:
Now here is the same program modified sligthly to answer your second question. It turned out to be very simple. I basically just needed to add 4 lines:
Changing scores into a (N,N) array instead of just (N).
Adding for j in xrange(N): and thus creating a double loop.
if i == j:
break
where 3. and 4. is just to make sure that I only compare each pair of things once and not twice and don't compary any things with themselves.
Then there is a few more lines of code that is needed to extract the indices of the 5 largest values in scores. I also reformated the printing so it will be easy to confirm by eye that the printed pairs are actually very similar.
Here comes the new code:
import numpy as np
class Thing:
def __init__(self, words, images, audios, videos):
self.words = words
self.images = images
self.audios = audios
self.videos = videos
def compare(self, other):
score = 0
# Assuming the attribute lists have the same length for both objects
# and that they are sorted in the same manner:
for i in range(len(self.words)):
if self.words[i] == other.words[i]:
score += 1
for i in range(len(self.images)):
if self.images[i] == other.images[i]:
score += 1
for i in range(len(self.audios)):
if self.audios[i] == other.audios[i]:
score += 1
for i in range(len(self.videos)):
if self.videos[i] == other.videos[i]:
score += 1
# You have to make sure you know what method to use for determining
# when an image/audio/video are equal.
return score
N = 1000
things = []
words = np.random.randint(5, size=(N,5))
images = np.random.randint(5, size=(N,5))
audios = np.random.randint(5, size=(N,5))
videos = np.random.randint(5, size=(N,5))
# For testing purposes I assign each attribute to a list (array) containing
# five random integers. I don't know how you actually intend to do it.
for i in xrange(N):
things.append(Thing(words[i], images[i], audios[i], videos[i]))
################################################################################
############################# This is the new part: ############################
################################################################################
scores = np.zeros((N, N))
# Scores will become a triangular matrix where scores[i, j]=value means that
# value is the number of attrributes thing[i] and thing[j] have in common.
for i in xrange(N):
for j in xrange(N):
if i == j:
break
# Break the loop here because:
# * When i==j we would compare thing[i] with itself, and we don't
# want that.
# * For every combination where j>i we would repeat all the
# comparisons for j<i and create duplicates. We don't want that.
scores[i, j] = (things[i].compare(things[j]))
# I want the 5 most similar pairs:
n = 5
# This list will contain a tuple for each of the n most similar pairs:
best_list = []
for k in xrange(n):
ij = np.argmax(scores) # Returns a single integer: ij = i*n + j
i = ij / N
j = ij % N
best_list.append((i, j))
# Erease this score so that on next iteration the second largest score
# is found:
scores[i, j] = 0
for k, (i, j) in enumerate(best_list):
# The number 1 most similar pair is the BEST match of all.
# The number N most similar pair is the WORST match of all.
print "The number %d most similar pair is thing number %d and %d." \
% (k+1, i, j)
print "Thing%4d:" % i, \
things[i].words, things[i].images, things[i].audios, things[i].videos
print "Thing%4d:" % j, \
things[j].words, things[j].images, things[j].audios, things[j].videos
print

If your comparison works with "create a sum of all features and find those which the closest sum", there is a simple trick to get close objects:
Put all objects into an array
Calculate all the sums
Sort the array by sum.
If you take any index, the objects close to it will now have a close index as well. So to find the 5 closest objects, you just need to look at index+5 to index-5 in the sorted array.

Quickly counting particles in grid

I've written some python code to calculate a certain quantity from a cosmological simulation. It does this by checking whether a particle in contained within a box of size 8,000^3, starting at the origin and advancing the box when all particles contained within it are found. As I am counting ~2 million particles altogether, and the total size of the simulation volume is 150,000^3, this is taking a long time.
I'll post my code below, does anybody have any suggestions on how to improve it?
Thanks in advance.
from __future__ import division
import numpy as np
def check_range(pos, i, j, k):
a = 0
if i <= pos[2] < i+8000:
if j <= pos[3] < j+8000:
if k <= pos[4] < k+8000:
a = 1
return a
def sigma8(data):
N = []
to_do = data
print 'Counting number of particles per cell...'
for k in range(0,150001,8000):
for j in range(0,150001,8000):
for i in range(0,150001,8000):
temp = []
n = []
for count in range(len(to_do)):
n.append(check_range(to_do[count],i,j,k))
to_do[count][1] = n[count]
if to_do[count][1] == 0:
temp.append(to_do[count])
#Only particles that have not been found are
# searched for again
to_do = temp
N.append(sum(n))
print 'Next row'
print 'Next slice, %i still to find' % len(to_do)
print 'Calculating sigma8...'
if not sum(N) == len(data):
return 'Error!\nN measured = {0}, total N = {1}'.format(sum(N), len(data))
else:
return 'sigma8 = %.4f, variance = %.4f, mean = %.4f' % (np.sqrt(sum((N-np.mean(N))**2)/len(N))/np.mean(N), np.var(N),np.mean(N))

I'll try to post some code, but my general idea is the following: create a Particle class that knows about the box that it lives in, which is calculated in the __init__. Each box should have a unique name, which might be the coordinate of the bottom left corner (or whatever you use to locate your boxes).
Get a new instance of the Particle class for each particle, then use a Counter (from the collections module).
Particle class looks something like:
# static consts - outside so that every instance of Particle doesn't take them along
# for the ride...
MAX_X = 150,000
X_STEP = 8000
# etc.
class Particle(object):
def __init__(self, data):
self.x = data[xvalue]
self.y = data[yvalue]
self.z = data[zvalue]
self.compute_box_label()
def compute_box_label(self):
import math
x_label = math.floor(self.x / X_STEP)
y_label = math.floor(self.y / Y_STEP)
z_label = math.floor(self.z / Z_STEP)
self.box_label = str(x_label) + '-' + str(y_label) + '-' + str(z_label)
Anyway, I imagine your sigma8 function might look like:
def sigma8(data):
import collections as col
particles = [Particle(x) for x in data]
boxes = col.Counter([x.box_label for x in particles])
counts = boxes.most_common()
#some other stuff
counts will be a list of tuples which map a box label to the number of particles in that box. (Here we're treating particles as indistinguishable.)
Using list comprehensions is much faster than using loops---I think the reason is that you're basically relying more on the underlying C, but I'm not the person to ask. Counter is (supposedly) highly-optimized as well.
Note: None of this code has been tested, so you shouldn't try the cut-and-paste-and-hope-it-works method here.

comparing large vectors in python

I have two large vectors (~133000 values) of different length. They are each sortet from small to large values. I want to find values that are similar within a given tolerance. This is my solution but it is very slow. Is there a way to speed this up?
import numpy as np
for lv in range(np.size(vector1)):
for lv_2 in range(np.size(vector2)):
if np.abs(vector1[lv_2]-vector2[lv])<.02:
print(vector1[lv_2],vector2[lv],lv,lv_2)
break

Your algorithm is far from optimal. You compare way too much values. Assume you are at a certain position in vector1 and the current value in vector2 is already more than 0.02 bigger. Why would you compare the rest of vector2?
Start with something like
pos1 = 0
pos2 = 0
Now compare the values at those postions in your vectors. If the difference is too big, move the position of the smaller one fowared and check again. Continue until you reach the end of one vector.

haven't tested it, but the following should work. The idea is to exploit the fact that the vectors are sorted
lv_1, lv_2 = 0,0
while lv_1 < len(vector1) and lv_2 < len(vector2):
if np.abs(vector1[lv_2]-vector2[lv_1])<.02:
print(vector1[lv_2],vector2[lv_1],lv_1,lv_2)
lv_1 += 1
lv_2 += 1
elif vector1[lv_1] < vector2[lv_2]: lv_1 += 1
else: lv_2 += 1

The following code gives a nice increase in performance that depends upon how dense the numbers are. Using a set of 1000 random numbers, sampled uniformly between 0 and 100, it runs about 30 times faster than your implementation.
pos_1_start = 0
for i in range(np.size(vector1)):
for j in range(pos1_start, np.size(vector2)):
if np.abs(vector1[i] - vector2[j]) < .02:
results1 += [(vector1[i], vector2[j], i, j)]
else:
if vector2[j] < vector1[i]:
pos1_start += 1
else:
break
The timing:
time new method: 0.112464904785
time old method: 3.59720897675
Which is produced by the following script:
import random
import numpy as np
import time
# initialize the vectors to be compared
vector1 = [random.uniform(0, 40) for i in range(1000)]
vector2 = [random.uniform(0, 40) for i in range(1000)]
vector1.sort()
vector2.sort()
# the arrays that will contain the results for the first method
results1 = []
# the arrays that will contain the results for the second method
results2 = []
pos1_start = 0
t_start = time.time()
for i in range(np.size(vector1)):
for j in range(pos1_start, np.size(vector2)):
if np.abs(vector1[i] - vector2[j]) < .02:
results1 += [(vector1[i], vector2[j], i, j)]
else:
if vector2[j] < vector1[i]:
pos1_start += 1
else:
break
t1 = time.time() - t_start
print "time new method:", t1
t = time.time()
for lv1 in range(np.size(vector1)):
for lv2 in range(np.size(vector2)):
if np.abs(vector1[lv1]-vector2[lv2])<.02:
results2 += [(vector1[lv1], vector2[lv2], lv1, lv2)]
t2 = time.time() - t_start
print "time old method:", t2
# sort the results
results1.sort()
results2.sort()
print np.allclose(results1, results2)

"Josephus-p‌r‌o‌b‌l‌e‌m" using list in python

I wanted to know if it will be possible to solve the Josepheus problem using list in python.
In simple terms Josephus problem is all about finding a position in a circular arrangement which would be safe if executions were handled out using a skip parameter which is known beforehand.
For eg : given a circular arrangement such as [1,2,3,4,5,6,7] and a skip parameter of 3, the people will be executed in the order as 3,6,2,7,5,1 and position 4 would be the safe.
I have been trying to solve this using list for some time now, but the index positions becomes tricky for me to handle.
a=[x for x in range(1,11)]
skip=2
step=2
while (len(a)!=1):
value=a[step-1]
a.remove(value)
n=len(a)
step=step+skip
large=max(a)
if step>=n:
diff=abs(large-value)
step=diff%skip
print a
Updated the question with code snippet, but i don't think my logic is correct.

Quite simply, you can use list.pop(i) to delete each victim (and get his ID) in a loop. Then, we just have to worry about wrapping the indices, which you can do just by taking the skipped index mod the number of remaining prisoners.
So then, the question solution becomes
def josephus(ls, skip):
skip -= 1 # pop automatically skips the dead guy
idx = skip
while len(ls) > 1:
print(ls.pop(idx)) # kill prisoner at idx
idx = (idx + skip) % len(ls)
print('survivor: ', ls[0])
Test output:
>>> josephus([1,2,3,4,5,6,7], 3)
3
6
2
7
5
1
survivor: 4

In [96]: def josephus(ls, skip):
...: from collections import deque
...: d = deque(ls)
...: while len(d)>1:
...: d.rotate(-skip)
...: print(d.pop())
...: print('survivor:' , d.pop())
...:
In [97]: josephus([1,2,3,4,5,6,7], 3)
3
6
2
7
5
1
survivor: 4
If you do not want to calculate the index, you can use the deque data structure.

My solution uses a math trick I found online here: https://www.youtube.com/watch?v=uCsD3ZGzMgE
It uses the binary way of writing the number of people in the circle and the position where the survivor sits. The result is the same and the code is shorter.
And the code is this:
numar_persoane = int(input("How many people are in the circle?\n")) #here we manually insert the number of people in the circle
x='{0:08b}'.format(int(numar_persoane)) #here we convert to binary
m=list(x) #here we transform it into a list
for i in range(0,len(m)): #here we remove the first '1' and append to the same list
m.remove('1')
m.append('1')
break
w=''.join(m) #here we make it a string again
print("The survivor sits in position",int(w, 2)) #int(w, 2) makes our string a decimal number

if you are looking for the final result only, here is a simple solution.
def JosephusProblem(people):
binary = bin(people) # Converting to binary
winner = binary[3:]+binary[2] # as the output looks like '0b101001'. removing 0b and adding the 1 to the end
print('The winner is',int(winner,2)) #converting the binary back to decimal
If you are looking for the math behind this code, go check out this video:
Josephus Problem(youTube)

it looks worse but easier to understand for beginners
def last(n):
a=[x for x in range(1,n+1)]
man_with_sword = 1
print(a)
while len(a)!=1:
if man_with_sword == a[len(a)-2]: #man_with_sword before last in circle
killed = a[len(a)-1]
a.remove(killed)
man_with_sword=a[0]
elif man_with_sword==a[len(a)-1]: #man_with_sword last in circle
killed = a[0]
a.remove(killed)
man_with_sword=a[0]
else:
i=0
while i < (len(a)//2):
i=a.index(man_with_sword)
killed = a[a.index(man_with_sword)+1]
a.remove(killed)
#pass the sword
man_with_sword=a[i+1] # pass the sword to next ( we killed next)
print (a, man_with_sword) #show who survived and sword owner
i+=1
print (a, man_with_sword,'next circle') #show who survived and sword owner

The total number of persons n and a number k, which indicates that k-1 persons are skipped and a kth person is killed in the circle.
def josephus(n, k):
if n == 1:
return 1
else:
return (josephus(n - 1, k) + k-1) % n + 1
n = 14
k = 2
print("The chosen place is ", josephus(n, k))

This is my solution to your question:
# simple queue implementation<ADT>
class Queue:
def __init__(self):
self.q = []
def enqueue(self,data):
self.q.insert(0,data)
def dequeue(self):
self.q.pop()
def sizeQ(self):
return len(self.q)
def printQ(self):
return self.q
lists = ["Josephus","Mark","Gladiator","Coward"]
to_die = 3
Q = Queue()
# inserting element into Q
for i in lists:
Q.enqueue(i)
# for size > 1
while Q.sizeP() > 1:
for j in range(1,3):
# every third element to be eliminated
Q.enqueue(Q.dequeue())
Q.dequeue()
print(Q.printQ())

def Last_Person(n):
person = [x for x in range(1,n+1)]
x = 0
c = 1
while len(person) > 1:
if x == len(person) - 1:
print("Round ", c, "- Here's who is left: ", person, "Person ", person[x], "killed person", person[0])
person.pop(0)
x = 0
c = c+1
elif x == len(person) - 2:
print("Round ", c, "- Here's who is left: ", person, "Person ", person[x], "killed person", person[x + 1])
person.pop(x+1)
x = 0
c = c + 1
else:
print("Round ", c, "- Here's who is left: ", person, "Person ", person[x], "killed person", person[x + 1])
person.pop(x + 1)
x = x + 1
c = c + 1
print("Person", person[x], "is the winner")
Last_Person(50)