We Keep Coding

Trying to find sets of numbers with all distinct sums; help optimizing algorithm?

I was recently trying to write an algorithm to solve a math problem I came up with (long story how I encountered it): basically, I wanted to come up with sets of P distinct integers such that given a number, there is at most one way of selecting G numbers from the set (repetitions allowed) which sum to that number (or put another way, there are not two distinct sets of G integers from the set with the same sum, called a "collision"). For example, with P, G = 3, 3, the set (10, 1, 0) would work, but (2, 1, 0) wouldn't, since 1+1+1=2+1+0.
I came up with an algorithm in Python that can find and generate these sets, but when I tried it, it runs extremely slowly; I'm pretty sure there is a much more optimized way to do this, but I'm not sure how. The current code is also a bit messy because parts were added organically as I figured out what I needed.
The algorithm starts with these two functions:
import numpy
def rec_gen_list(leng, index, nums, func):
if index == leng-1: #list full
func(nums)
else:
nextMax = nums[index-1];
for nextNum in range(nextMax)[::-1]: # nextMax-1 to 0
nums[index] = nextNum;
rec_gen_list(leng, index+1, nums, func)
def gen_all_lists(leng, first, func):
nums = np.zeros(leng, dtype='int')
nums[0] = first
rec_gen_list(leng, 1, nums, func)
Basically, this code generates all possible lists of distinct integers (with maximum of "first" and minimum 0) and applies some function to them. rec_gen_list is the recursive part; given a partial list and an index, it tries every possible next number in the list less than the last one, and sends that to the next recursion. Once it gets to the last iteration (with the list being full), it applies the given function to the completed list. Note that I stop before the last entry in the list, so it always ends with 0; I enforce that because if you have a list that doesn't contain 0, you can subtract the smallest number from each one in the list to get one that does, so I force them to have 0 to avoid duplicates and make other things I'm planning to do more convenient.
gen_all_lists is the wrapper around the recursive function; it sets up the array and first iteration of the process and gets it started. For example, you could display all lists of 4 distinct numbers between 7 and 0 by calling it as gen_all_lists(4, 7, print). The function included is so that once the lists are generated, I can test them for collisions before displaying them.
However, after coming up with these, I had to modify them to fit with the rest of the algorithm. First off, I needed to keep track of if the algorithm had found any lists that worked; this is handled by the foundOne and foundNew variables in the updated versions. This probably could be done with a global variable, but I don't think it's a significant issue with the slowdown.
In addition, I realized that I could use backtracking to significantly optimize this: if the first 3 numbers out of a long list are something like (100, 99, 98...), that already causes a collision, so I can skip checking all the lists generated from that. This is handled by the G variable (described before) and the test_no_colls function (which tests if a list has any collisions for a certain value of G); each time I make a new sublist, I check it for collisions, and skip the recursive call if I find any.
This is the result of these modifications, used in the current algorithm:
import numpy
def rec_test_list(leng, index, nums, G, func, foundOne):
if index == leng - 1: #list full
foundNew = func(nums)
foundOne = foundOne or foundNew
else:
nextMax = nums[index-1];
for nextNum in range(nextMax)[::-1]: # nextMax-1 to 0
nums[index] = nextNum;
# If already a collision, don't bother going down this tree.
if (test_no_colls(nums[:index+1], G)):
foundNew = rec_test_list(leng, index+1, nums, G, func, foundOne)
foundOne = foundOne or foundNew
return foundOne
def test_all_lists(leng, first, G, func):
nums = np.zeros(leng, dtype='int')
nums[0] = first
return rec_test_list(leng, 1, nums, G, func, False)
For the next two functions, test_no_colls takes a list of numbers and a number G, and determines if there are any "collisions" (two distinct sets of G numbers from the list that add to the same total), returning true if there are none. It starts by making a set that contains the possible scores, then generates every possible distinct set of G indices into the list (repetition allowed) and finds their totals. Each one is checked for in the set; if one is found, there are two combinations with the same total.
The combinations are generated with another algorithm I came up with; this probably could be done the same way as generating the initial lists, but I was a bit confused about the variable scope of the set, so I found a non-recursive way to do it. This may be something to optimize.
The second function is just a wrapper for test_no_colls, printing the input array if it passes; this is used in the test_all_lists later on.
def test_no_colls(nums, G):
possiblePoints=set(()) # Set of possible scores.
ranks = np.zeros(G, dtype='int')
ranks[0] = len(nums) - 1 # Lowest possible rank.
curr_ind = 0
while True: # Repeat until break.
if ranks[curr_ind] >= 0: # Copy over to make the start of the rest.
if curr_ind < G - 1:
copy = ranks[curr_ind]
curr_ind += 1
ranks[curr_ind] = copy
else: # Start decrementing, since we're at the end. We also have a complete list, so test it.
# First, get the score for these rankings and test to see if it collides with a previous score.
total_score = 0
for rank in ranks:
total_score += nums[rank]
if total_score in possiblePoints: # Collision found.
return False
# Otherwise, add the new score to the list.
possiblePoints.add(total_score)
#Then backtrack and continue.
ranks[curr_ind] -= 1
else:
# If the current value is less than 0, we've exhausted the possibilities for the rest of the list,
# and need to backtrack if possible and start with the next lowest number.
curr_ind -= 1;
if (curr_ind < 0): # Backtracked from the start, so we're done.
break
else:
ranks[curr_ind] -= 1 # Start with the next lowest number.
# If we broke out of the loop before returning, no collisions were found.
return True
def show_if_no_colls(nums, games):
if test_no_colls(nums, games):
print(nums)
return True
return False
These are the final functions that wrap everything up. find_good_lists wraps up test_all_lists more conveniently; it finds all lists ranging from 0 to maxPts of length P which have no collisions for a certain G. find_lowest_score then uses this to find the smallest possible maximum value of a list that works for a certain P and G (for example, find_lowest_score(6, 3) finds two possible lists with max 45, [45 43 34 19 3 0] and [45 42 26 11 2 0], with nothing that is all below 45); it also shows some timing data about how long each iteration took.
def find_good_lists(maxPts, P, G):
return test_all_lists(P, maxPts, G, lambda nums: show_if_no_colls(nums, G))
from time import perf_counter
def find_lowest_score(P, G):
maxPts = P - 1; # The minimum possible to even generate a scoring table.
foundSet = False;
while not foundSet:
start = perf_counter()
foundSet = find_good_lists(maxPts, P, G)
end = perf_counter()
print("Looked for {}, took {:.5f} s".format(maxPts, end-start))
maxPts += 1;
So, this algorithm does seem to work, but it runs very slowly; when trying to run lowest_score(7, 3), for example, it starts taking minutes per iteration around maxPts in the 70s or so, even on Google Colab. Does anyone have suggestions for optimizing this algorithm to improve its runtime and time complexity, or better ways to solve the problem? I am interested in further exploration of this (such as filtering the lists generated for other qualities), but am concerned about the time it would take with this algorithm.

How to use recursion for "for" loops

My code is meant to find the longest path in a matrix, where each value is greater than the one previous. However, I've been instructed to not use for loops at all, which is difficult because I have 3, with 2 of them being involved in a nested loop. Is there any way I could only user recursion to solve this?
def path(self, matrix):
res = 1
# for loop to run the function for every element in list
for row in range (len(matrix)):
for col in range (len(matrix[0])):
# pass in the current max and the new spot, and take the max value
res = max(res, self.dfs(matrix, row, col))
# return the max value
return res
# function to compare paths (Depth-First Seach)
def dfs(self, matrix, row, col):
# if spot was visited before, return value from cache
if (row, col) in self.cache:
return self.cache[(row, col)]
# Set a default value of 1
self.cache[(row, col)] = 1
# moving the tile of focus
for rowVal, colVal in self.directions:
newRow = row + rowVal
newCol = col + colVal
# if the pointer can move in a direction (not out of bounds), and is greater: store cache value
if (0 <= newRow < len(matrix)) and (0 <= newCol < len(matrix[0])) and matrix[row][col] < matrix[newRow][newCol]:
self.cache[(row, col)] = max(self.cache[(row, col)], 1 + self.dfs(matrix, newRow, newCol))

Recursion is about a base case and an iterative case. In your situation, think of the smallest matrix you can - an empty matrix. That is your base case. Your function should return a path length of 0 if it is empty.
The iterative case is a bit more difficult, and usually where things become confusing. The key goal of the iterative case is to reduce the size of the problem, usually by the smallest amount possible.
To overly simplify, if you start with a function like this:
def f(ls):
for x in ls:
result = g(x, result)
return result
Then the iterative version looks like this:
def f(ls, result):
if 0 == len(x): # Base Case
return result
else: # Iterative Case
result = g(x, result)
return f(ls[1:], result)
The trick is figuring out what of your internal logic goes into g() and how to represent the result.
Let's take a simpler version of your problem, where we deal with a single array and want to return the longest 'path' in that array. A path is a sequence of integers that are incrementing by one.
So, if we have [0,1,2,3] the expected value is 4. If we have [0,1,1,2,3] the expected value is 3. Similarly, 3 would be expected for the input [0,1,2,2,1]. [3,2,1] should return 0.
The most basic signature we require is this:
def f(ls: list[int]) -> int
Essentially, 'a function that takes a list of ints and returns the length of the longest path'. But we have to remember a bit of extra state to do this properly. Specifically, we need to remember the length of the current path we are on, and the lengths of all paths we have found.
def f(ls: list[int], current_path: int) -> int
Let's examine the base case. A base case is any case where reducing the size of your input (in our case 'input' really refers only to ls) would not yield an easier problem to solve. There are two base cases - if the input list is length 0 or 1. In both these cases, we have no need to shrink the problem any further.
def f(ls: list[int], current_path: int) -> int:
# If there are no elements, current_path is the only valid length
if 0 == len(ls):
return current_path
# If there is one element, increment current_path before returning it, to account for the length that element adds
if 1 == len(ls):
return current_path + 1
These serve to terminate the recursion. Now lets look at the iterative case:
def f(ls: list[int], current_path: int) -> int:
# Base cases
if 0 == len(ls):
return current_path
if 1 == len(ls):
return current_path + 1
# Iterative case - guaranteed that len(ls) >= 2
current_path = current_path + 1 # Increment the current_path to account for the current element.
if ls[1] == ls[0] + 1:
# In this branch we know that the path will continue.
return f(ls[1:], current_path) # ls[1:] reduces our problem space by one element
else:
# In this branch the path ends, because the next element breaks the incremental sequence.
recursive_result = returnf(ls[1:], 0) # Reduce the problem space and start a new path of length 0
return max(recursive_result, current_path) # Choose which path is longer
There is a lot going on here, so I'll break it down in parts. The key thing to remember is that we are going to reduce the problem space - shorten the list - and then recurse with that smaller problem space. The element we are removing is therefore key to determining how we proceed.
Since we are removing one element, we add one to the current path. If the incoming path length was 0, we now have a path length of 1. If it was 3, we now have a path of 4.
Then we check the value of the next element. Is it exactly larger than the current element? If so, we know the path will continue, so we recurse, passing along a list without the current element and the length of our current path.
If it is not exactly one more, we know our current path ends here. In the recursion we pass along a new path length of 0, resetting the counter. But we have to do something with the returned value - decide whether it is larger than the current path as it stands at this element. Hence using max() to choose between the two possibilities.
This gives you a recursive function that iteratively shortens the problem at each step until it finds a base case, at which point it returns - but it returns up through the recursive function, accumulating the results.
(n.b. There are ways to optimize this, clean it up, add default values, etc. I'm skipping that because it doesn't help you think about the recursion.)
Your actual problem is harder. You're going along a two dimensional array. The key insight to have is this: in the iterative step in the example I gave, I looked at all the possible cases for moving forward and chose between them. However, you can go down all possible paths. If you are at a particular element in a two dimensional array, you know that you can go one way or the other - that's two recursive function calls. Because recursion is shortening your problem space, your iterative step can simply trust it will return, and only deal with the results. In your case, that is choosing which of the two recursive calls you made returned the larger result.
(At this point I have to make assumptions about your problem because you included neither a complete specification nor full code.)
def f(matrix: list[list[int], coords: (int, int), current_path: int) -> int:
# Find all possible 'next steps'. For a next step to be valid it must be exactly one greater than the current location.
# Base Case - There are no possible next steps -> return current path + 1
# Increment path
# Iterative cases
# There is only one next step -> recurse passing new coordinates and path length
# There are two or three next steps -> recurse passing new coordinates and path length, then choose which result is the longest.
The difficulty here is that this finds the longest path from any given starting position. To truly find the longest path in the matrix, you would have to add a fourth argument to your function - a list of all the starting positions you have tried. Then you would change your logic for finding the next steps from 'is it strictly one larger' to 'is it strictly one larger or have I not tried starting from that point'?
# Use type aliases so you're clear about the types
type Matrix: list[list[int]]
type Coordinate: (int, int) # x-y coordinates
type Cache: list[Coordinate] # All the places we've started from
def f(matrix: Matrix,
coords: Coordinate,
current_path: int,
starting_points: Cache) -> int:
if 0 == len(matrix):
return current_path
if 1 == len(matrix) and 0 == len(matrix[0]):
return current_path
current_path = current_path + 1 # From here on, we have a valid element at this coordinate
if 1 == len(matrix) and 1 == len(matrix[0]):
return current_Path
moves = get_all_moves(...)
if 0 == len(moves): # This is *also* a base case - the problem cannot be shrunk any further!
return current_path
results = try_each_move(moves) # This is also a recursive function... but a *different* recursive function, in order to avoid using a for loop (or list comprehension)
return max(max(results), current_path)
A few closing notes:
Do try to adhere to python style guides. It makes reading python easier!
Any information you think you need to store outside the function can just be passed as a parameter. Pure recursive functions don't have access to a closure or outer scope. Depending on your case, this may not be the most efficient solution, but it is where you should start until you're much more comfortable with recursion.
Similarly, if copying a value rather than a reference makes it easier for your to reason about what you're doing, do that first. Efficiency is for later. Clarity first.
Recursion often (though not always) easier if you're building up a solution from the return of the recursion call. In the examples here, the max() function is doing that accretion, but you could imagine inverting the approach here and first doing the recursive call, which returns two values - the value of the last element and the length of the path. Then you could decide if you're smaller than that value. I didn't do that here because you'd have to remember two path lengths at a time.
In this specific problem, do take care with the cache. You can't just remember if you've ever visited a coordinate.

Recursion is just a function that keeps calling itself until it reaches a condition that disallows it from continuing.
Here's an example that's themed toward your needs.
""" Emulation Of:
for row in range (len(matrix)):
for col in range (len(matrix[0])):
print(matrix[row][col])
"""
matrix = [[1,2,3],[4,5,6],[7,8,9]]
#wrapping everything in this function makes `i` unreachable
#because it should be managed internally
def process_matrix(r:int, c:int) -> None:
def columns(i:int=0, r:int=0) -> None:
if i==c: return #columns finished
print(matrix[r][i]) #work
columns(i+1, r) #next column
def rows(i:int=0) -> None:
if i==r: return #rows finished
columns(0, i) #recurse all columns for this row
rows(i+1) #next row
rows(0) #start recursion
#use
process_matrix(len(matrix), len(matrix[0]))
If you are trying to retrieve data, you have to return the "recursion call". Otherwise, you'll get None back from the very first call, and the recursion will carry on in a way that is unreachable by your code.
data = [10,20,30,40,50,60]
def where_is_50(i:int=0) -> int:
if data[i] == 50:
return i #stop recursion
return where_is_50(i+1) #next
print(where_is_50())
If it isn't clear, The first time the function is called, it is not 50 so, it returns a call to itself. However, the actual return can't finish until the call does. Essentially, you end up with a string of "active" functions that are all waiting for the call that finds 50. When 50 is found, the return value keeps ascending through all the calls back to the very first one.
Whatever recursive functions you make should have a local reference to the data to traverse. In other words, don't pass your entire matrix on each call. Pass names or indexes recursively.

Python list changing for no reason even with copy method

def return_solved_board(board):
solution = board.copy()
Backtracking recursion loop starts
def solve(board):
for y in range(9):
for x in range(9):
if solution[y][x] == 0:
for number in range(1,10):
if check_rules_at_point(x, y, number, solution):
solution[y][x] = number
solve(solution)
#Where backtracking comes into play, if the solve method fails on the next cell than it resets current one.
solution[y][x] = 0
#Breaks out of recursion to try a differant number in cell prior
return
#Stores a solved copy of the solution into global variable
global returned_information
returned_information = solution.copy()
print(returned_information) #1st print statement
Checks if the board can be solved or not
if not check_for_valid_board(board):
return "This board is not possible to solve."
else:
solve(board)
global returned_information
print(returned_information) #2nd print statement
return returned_information
print(return_solved_board(valid_sudoku_board1))
I am trying to write a program to solve sudoku puzzles. The puzzle is stored in a list of lists. When I run this program the first print statement returns the correct solved list of lists. However, the second return statement returns the original unsolved puzzle. Both of their id's are the same however I can't figure out why returned_information in the second print statement changes back as returned_information is only called once.

You're probably bumping into dicts'/lists' copy() being a shallow copy; use copy.deepcopy(...) instead to do a deep copy.

How to perform a tree search on a connect-4 board and identify the best move?

I need to create a recursive function that iterates through all possible moves of a "4-in-a-row" game at a given depth and finds the best move.
The functions available are:
eval_pos(board, pid) # Returns eval score of a given board and pid(player id, either -1 or 1)
get_legal_moves(board) # Returns all legal moves in a tuple
The goal is to create a recursive function which finds what move gives the best score, given the best play from both parts.
def get_score(board, depth, pid):
moves = get_legal_moves(board)
evals = []
for move in moves:
board[move] = pid
evals.append(eval_pos(board, pid))
board[move] = 0 #0 signifies an empty board space, doing this to prevent filling the board
if depth == 0:
return max(evals)
idx = evals.index(max(evals))
board[moves[idx]] = pid
# Notice we are passing -pid on each new iteration
return get_score(board, depth-1, -pid, start_pid)
What I've come up with so far, does not work at all. It essentially does what I want in reverse and skipping a lot of positions, where it checks the best move for player1 in the first position, then the best move for player2 in the following position and so on. However, it needs to search through ALL the moves and then determine what is best given the best counterplay. Is there an efficient way to accomplish this?

This solution seems to work, however it is extremely slow and can only reach a depth of 5 before it becomes unusable.
def minimax(board, depth, max_turn):
# Base case
if depth == 0:
return eval_pos(board)
eval = eval_pos(board)
if eval is None:
return 0
if abs(eval) == 100:
return eval
moves = get_legal_moves(board)
if max_turn:
return max([minimax(update_board(board, move, 1), depth-1, False) for move in moves])
else:
return min([minimax(update_board(board, move, -1), depth-1, True) for move in moves])

How to limit the depth of recursion in python?

I have a list of elements. I want to know if there are two pairs of elements in the list, in which the elements of the pair have the same value.
My idea is that I first compare all the elements in the list, if a pair is found, remove the pair from the list, then proceed again. Thus I think I can use recursion to do this task, but limit the depth to 2 to solve the problem.
Here is my first try:
recursion_depth=0
def is_twopair(card):
nonlocal recursion_depth
if recursion_depth==2: return True
for i in range(0, len(card)):
for k in range(i+1,len(card)):
if card[i].value==card[k].value:
del card[k], card[i]
recursion_depth+=1
is_twopair(card)
else: continue
else: continue
else: return False
I use the variable recursion_depth to record the the depth of recursion, but then realize that the return command doesn't immediately terminate the function and return true, but returns to its original caller is_twopair(card) instead. So my question is:
Is there a way to immediately terminate the function and return the result true?
Is there a way to limit the depth of recursion?
I know there maybe several ways to work around this. But I want to stay faithful to my idea and use this as an opportunity for learning.

I don't believe you can "break out" of the recursion without some serious black magic, nor do I believe that you should try to. Cascading the return value up the calling chain is typically a fine approach.
I'd personally avoid using a nonlocal variable and keep track of the recursion depth within the scope of each function call.
def remove_pairs(card, count=2, depth=0):
if depth == count:
return card
for i in range(0, len(card)):
for j in range(i+1, len(card)):
if card[i].value == card[j].value:
del card[j], card[i]
return remove_pairs(card, count, depth+1) # add return here
Moving the tracking of the depth into the function itself will allow the function to be called repeatedly from different locations without issue.
Additionally, the code can be cleaned up some using itertools.combinations.
from itertools import combinations
def remove_pairs(cards, count=2, depth=0):
if depth == count:
return cards
for card1, card2 in combinations(cards, 2):
if card1.value == card2.value:
cards.remove(card1)
cards.remove(card2)
return remove_pairs(cards, count, depth+1)

I think the piece you're missing is a return call to pass on the results of a recursive call back up to the previous caller:
if card[i].value==card[k].value:
del card[k], card[i]
recursion_depth+=1
return is_twopair(card) # add return here!
I don't really think recursion is a natural way to solve this problem, but with the above change, it should work. You could avoid needing to use a nonlocal variable by passing the depth as an optional parameter.

yourList = [1,1,2,2,3,4]
yourDict = {}
for i in yourList:
yourDict[i] = yourList.count(i)
This code will return the number of ocurrences for every value in the list so you can determinate the number of pairs..
In this case:
yourDict - - > {1: 2, 2: 2, 3: 1, 4: 1}
The value 1 appear 2 times, the value 2 appear 2 times, the value 3 and 4 appear 1 time.