We Keep Coding

Understanding speed in Python through a competition problem exceeding time limits

I recently have been doing some competition problems. In particular this one:
https://open.kattis.com/problems/knightjump
An abridged statement of the problem:
You are given a two dimensional (square) chess board of size N (1-based indexing). Some of the cells on this board are ‘.’ denoting an empty cell. Some of the cells on this board are ‘#’ denoting a blocked cell, which you are not allowed to visit. Exactly one of the cells on this board is ‘K’ denoting the initial position of the knight. Determine the minimum number of steps required for the Knight to reach cell (1,1) while avoiding cells with ‘#’ in the path.
I know this is a simple BFS problem and have written the following code to solve it:
from collections import deque
from sys import stdin
dirs = ((2,1), (2,-1), (-2,1), (-2,-1), (1,2), (1,-2), (-1,2), (-1,-2))
N = int(stdin.readline())
q = deque()
mat = [list(str(stdin.readline())) for _ in range(N)]
for i in range(N):
for j in range(N):
if mat[i][j] == "K":
inX, inY = i+1, j+1
break
q.append((inX,inY,0))
flag = False
visited = [[0]*N for _ in range(N)]
while q:
ix,iy, moves = q.popleft()
visited[ix-1][iy-1] = 1
if (ix,iy) == (1,1):
print(moves)
flag = True
break
for d in dirs:
dx,dy = d
if 1 <= ix + dx < N+1 and 1 <= iy + dy < N+1 and mat[ix+dx-1][iy+dy-1] != "#" and visited[ix+dx-1][iy+dy-1] == 0:
q.append((ix+dx,iy+dy, moves+1))
if not flag:
print(-1)
All test data I've generated has given a correct answer, but I am recieving a time limit error on the site for some cases (and correct on all other). The input limits are N <= 100. I know that C++ is faster, but I was hoping to use this problem to learn a bit about why my solution times out but many Python solutions on a similar concept perform quick, and hopefully teach me something about Python nuts and bolts I didn't know. For example - I found this solution on someone's Github:
from collections import deque
n = int(input())
grid = [list(input()) for _ in range(n)]
for i in range(n):
for j in range(n):
if grid[i][j] == 'K':
k_i, k_j = i, j
break
visited = [[0 for x in range(n)] for y in range(n)]
# time complexity is number of configs (n^2) multiplied by
# work per configs (iterate through 8 deltas)
def k_jumps(i, j):
q = deque()
q.append((0, i, j))
visited[i][j] = 1
valid = False
while len(q) > 0:
d, i, j = q.popleft()
if (i,j) == (0,0):
print(d)
valid = True
break
deltas = {(-2, 1), (-2, -1), (1, 2), (1, -2), (-1, 2), (2, 1), (2, -1), (-1, -2)}
for delta in deltas:
di, dj = delta
if 0 <= i + di < n and 0 <= j + dj < n and visited[i+di][j+dj] == 0 and grid[i+di][j+dj] != '#':
visited[i+di][j+dj] = 1
q.append((d+1, i+di, j+dj))
if not valid:
print(-1)
k_jumps(k_i, k_j)
(Thanks to user Case Pleog).
Superficially the solutions are the same (BFS) - however the second solution runs in 0.07s, while my solutions times out at over 1s.
While there are some minor differences between our solutions, I can't see what explains such a large time discrepancy. I already made some changes to my code to see if that was the issue, e.g. the line:
if 1 <= ix + dx < N+1 and 1 <= iy + dy < N+1 and mat[ix+dx-1][iy+dy-1] != "#" and visited[ix+dx-1][iy+dy-1] == 0:
and indeed that speeded things up, as before I was using:
if min(point) > 0 and max(point) < N+1 and mat[point[0]+dx-1][point[1]+dy-1] != "#" and visited[point[0]+dx-1][point[1]+dy-1] == 0:
where point was a tuple consisting of ix,iy. Any advice would be appreciated - I would like to know more about what's causing the time difference as its > 10x difference. This is just a hobby, so some of my code may have amateur quirks/inefficiencies which hopefully I can explain. Thanks!

Do visited[ix+dx-1][iy+dy-1] = 1 right away when you do q.append((ix+dx,iy+dy, moves+1)), otherwise you might put that same coordinate into the queue multiple times.

All the possible states for wild tic tac toe (wild Tic-tac-toe combinatorics)

Wild tic-tac-toe is an impartial game similar to tic-tac-toe. However, in this game players can choose to place either X or O on each move
what are all the possible states after the change in that rule? and how can I generate all the valid states using python?
that is my try to solve the question, But it is the wrong way
this is the wrong solution
There are only 3**9, or 19,683 possible combinations of placing x, o, or in the grid, and not all of those are valid.
First, a valid game position in the classic tic tac toe is one where the difference between x and o counts is no more than one since they have to alternate moves, but this is not the case here.
In addition, it's impossible to have a state where both sides have three in a row, so they can be discounted as well. If both have three in a row, then one of them would have won in the previous move.
There's actually another limitation in that it's impossible for one letter X or O to have won in two different ways without a common cell (again, they would have won in a previous move), meaning that:
XXX
OOO
XXX
cannot be achieved, while:
XXX
OOX
OOX
make sense.
so I thought the invalid states are the states where we have two winning lines in the same direction(vertically or horizontally)
here is the code:
import numpy as np
def checkRows(board):
i = -1
for row in board:
i += 1
if len(set(row)) == 1 and set(row) != {''}:
r = row[0]
board[i][0] = board[i][1] = board[i][2] = ''
return r
return 0
def checkinvalide(board):
for newBoard in [board, np.transpose(board)]:
k = 0
result = checkRows(newBoard)
if result:
k += 1
result = checkRows(newBoard)
if result:
k += 1
if k == 2:
return k
return k
def generatelists():
StatesMatrix = np.zeros((3**9,9))
for i in range(3**9):
c = i
for j in range(9):
StatesMatrix[i][j] = c % 3
c //= 3
dic = {}
for i in StatesMatrix:
e += 1
i = ["X" if item == 1 else item for item in i]
i = ["O" if item == 2 else item for item in i]
i = ["" if item == 0 else item for item in i]
dd_board = np.reshape(i, (3, 3))
result = checkinvalide(dd_board)
if result != 2:
dic.update({tuple(i): 0})
k += 1
print(k)
return dic
generatelists()
doing the code generates 19177 states
this is wrong solution, there are states where there are two winning lines not in the same direction
The state XXX, _X_, X_X has three winning lines, and no two of them are parallel

You could apply this logic to determine if a board is valid or not:
If there are multiple three-in-a-rows for a certain symbol, make sure they all overlap at the same cell.
I would represent the board state as an integer: each pair of bits represent a cell. The pair can be 0b00 (empty), 0b01 ("X") or 0b10 ("O"). A board has 9 cells, so 18 bits.
Then I would not generate all states, but perform a depth-first traversal in states by adding a symbol at each recursion level. Once a board is invalid, we can backtrack and so skip a lot of invalid states.
Here is an implementation:
patterns = (
(0x0003F, 0x00015), # Horizontal
(0x00FC0, 0x00540),
(0x3F000, 0x15000),
(0x030C3, 0x01041), # Vertical
(0x0C30C, 0x04104),
(0x30C30, 0x10410),
(0x30303, 0x10101), # Diagonal
(0x03330, 0x01110)
)
def valid(board, side):
found = 0x3FFFF
for mask, three in patterns:
if board & mask == three << side:
found &= three
if not found: # Parallel or overlapping at different cells
return False
return True
def countstates(board=0, side=0, i=0):
if not valid(board, side):
return 0
# Iterate all next cells, and for each, the two possible moves (bits)
# There are 9 cells, so 18 bits:
return 1 + sum(
countstates(board | (1 << j), j % 2, j + 2 - j % 2)
for j in range(i, 18)
)
print(countstates()) # 19035
To make it easier to test a few boards, I used the following code:
def statefromstring(s):
board = 0
for ch in s.replace(" ", ""):
board = board * 4 + ".XO".index(ch)
return board
print(valid(statefromstring("XXX .X. X.X"), 0)) # False

How can i optimize my sudoku generator code to generate sudoku puzzles faster

I modified the sudoku generator code I found at https://medium.com/codex/building-a-sudoku-solver-and-generator-in-python-1-3-f29d3ede6b23 to look like this:
def generateBoard():
print("Generating puzzle...please wait")
time.sleep(1)
gameboard = createEmpty() # creates a list with the three diagonal boxes filled and the rest filled with zeros
gameboard = solve(gameboard) # changes the unsolved board DIRECTLY to a solved board
gameboard = removeNumbers(gameboard,38)
print("\nHere's your puzzle :)")
printBoard(gameboard)
So what the above code does is, it first creates a list in which the three 3x3 subgrids in the leading diagonal are filled with numbers 1 through 9 (sudoku rules) using the createEmpty() function below:
def createEmpty():
board = [[0 for i in range(9)] for j in range(9)]
for i in range(3):
box = shuffle(range(1,10))
for y in range(i*3, i*3+3):
for x in range(i*3, i*3+3):
board[y][x] = box.pop()
return board
This function was only returning a 9x9 list filled with zeros (first line) before, but according to the article i read in that website, it said that using this method of filling the three boxes in the diagonal with numbers before solving would save some time in backtracking so i changed it and it affected the time spent but by only a little amount.
def solve(board):
empty = find_empty(board)
if not empty:
return True
col, row = empty
for num in shuffle(range(1,10)):
if valid(board, num, empty):
board[row][col] = num
if solve(board):
return board
board[row][col] = 0
return False
So after creating the list, it uses the solve() function above👆👆 which uses backtracking to solve the partially filled board.
So after getting a solved sudoku board, i created a function (removeNumbers) to remove numbers from the board to create the puzzle. The function and its dependent functions are below:
def removeNumbers(solvedboard,hints=None):
board = copy.deepcopy(solvedboard)
remaining = 81
if not hints: hints = random.randrange(28, 40)
for y in range(3):
for x in range(3):
boxCoords = [(i,j) for j in range(y*3, (y*3)+3) for i in range(x*3, (x*3)+3)]
for n in range(random.choice(range(1,5))):
a, b = random.choice(boxCoords)
board[b][a] = 0
boxCoords.remove((a,b))
remaining -= 1
coords = [(i,j) for i in range(9) for j in range(9)]
coords = list(filter(lambda p: board[p[0]][p[1]] != 0, coords))
random.shuffle(coords)
for row, col in coords:
if remaining == hints:
break
num = board[row][col]
board[row][col] = 0
n = numberOfSolutions(board, solvedboard)
if n != 1:
board[row][col] = num
continue
remaining -= 1
#printBoard(board)
#print(remaining, hints)
return board
def numberOfSolutions(board, solved=[], ret=False):
solutions = []
# find a solution in which an empty square has a valid value and add it to the list(solutions)
for row in range(9):
for col in range(9):
if board[row][col] == 0:
dupeBoard = copy.deepcopy(board)
solution = getSolution(dupeBoard, row, col)
if not ret and solution != solved:
return 0
if solution:
solutions.append(code(solution))
solutions = list(set(solutions))
return len(solutions) if not ret else solutions
def getSolution(board, row, col):
for n in range(1, 10):
# insert a valid number into the (row,col) and return a solved board or if there are no solutions for that board, return False
if valid(board, n, (col, row)):
board[row][col] = n
if (solvedboard:=solve(board)):
return solvedboard
board[row][col] = 0
return False
def valid(board, num, pos):
x, y = pos
# check rows
if board[y].count(num) > 0:
return False
# check columns
if [board[j][x] for j in range(9)].count(num) > 0:
return False
# 3x3 grid
grid_x, grid_y = x-(x % 3), y-(y % 3)
grid = [board[j][i] for j in range(grid_y, grid_y+3) for i in range(grid_x, grid_x+3)]
if grid.count(num) > 0:
return False
return True
The removeNumbers() function has a local variable hints which represents the number of clues present in the sudoku puzzle and which contains a random number from 28 to 40, with 28 being the hardest and 39 being the simplest.
After removing the numbers from the board, the resulting board is printed out using the printBoard() function below:
def printBoard(board):
print("-"*41)
for y in range(len(board)):
if y % 3 == 0:
print(('||'+'-'*11)*3, end='')
print('||')
print(f"|| {board[y][0] if board[y][0]!=0 else ' '} | {board[y][1] if board[y][1]!=0 else ' '} | {board[y][2] if board[y][2]!=0 else ' '} ||", end='')
print(f" {board[y][3] if board[y][3]!=0 else ' '} | {board[y][4] if board[y][4]!=0 else ' '} | {board[y][5] if board[y][5]!=0 else ' '} ||", end='')
print(f" {board[y][6] if board[y][6]!=0 else ' '} | {board[y][7] if board[y][7]!=0 else ' '} | {board[y][8] if board[y][8]!=0 else ' '} ||")
print(('||'+'-'*11)*3, end='')
print('||')
print('-'*41)
Right now my generator generates simple sudoku puzzles in 30+ seconds and hard sudoku puzzles in 3+ minutes. And the code in that website generates simple in 0.22 secs, medium in 2+ secs and hard in 20+ secs.
But after several debugging, I found out that a large amount of time is spent at the removeNumbers() function. Having said all of that, I'll appreciate it if you could help me to optimize the removeNumbers() function and therefore reduce the amount of time spent generating the sudoku puzzle.

I am looking for a way to make my Minesweeper game add numbers around the bombs

Currently my code adds bombs through random x and y coordinates on my hidden grid. What would I have to add here in order for the code to add numbers if the grid space is touching a bomb?
def addBombNum(bombs,gSize,hiddenGrid,bCount):
fNum = ""
for i in range(bombs):
randPlace = True
while randPlace:
x = random.randint(0,gSize - 1)
y = random.randint(0,gSize - 1)
if hiddenGrid[y][x] == "[-]": hiddenGrid[y][x] = "\033[90m[*]\033[m" ; randPlace = False
if hiddenGrid[y][x] == "\033[90m[*]\033[m": pass
for i in range(gSize):
for j in range(gSize):
if hiddenGrid[i][j] == "\033[90m[*]\033[m": continue
This is my hiddenGrid if needed.
hiddenGrid = [["[-]" for y in range(gSize)] for x in range(gSize)]

I would personally recomment storing the bomb array as a numerical array and having a way to process it into the output afterwards. However, to answer your question directly, what I would recommend doing is each time you place down a bomb, you increment a counter for every cell adjacent to it.
So, at the top of your function, you could first create a number grid: grid = [[0 for y in range(gSize)] for x in range(gSize)].
Next, each time you add a bomb, increment the counter for each grid space that touches it:
for a in range(y - 1, y + 2):
for b in range(x - 1, x + 2):
if 0 <= a < gSize and 0 <= b < gSize:
grid[a][b] += 1
Finally, at the very end, you can just relay this information from the grid to the output. Thus, after the if hiddenGrid[i][j] == "\033[90m[*]\033[m": continue line (which I assume you added to prevent it from overwriting bombs with numbers), you can add this block:
if grid[i][j] > 0:
hiddenGrid[i][j] = "\033[90m[%d]\033[m" % grid[i][j]
A different way you could do it is just add this block to the very end after the bomb skip check:
count = 0
for a in range(i - 1, i + 2):
for b in range(j - 1, j + 2):
if 0 <= a < gSize and 0 <= b < gSize:
if hiddenGrid[a][b] == "\033[90m[*]\033[m":
count += 1
if count > 0:
hiddenGrid[i][j] = "\033[90m[%d]\033[m" % count
This way, you don't need to have the counter grid and don't need to increment it each time. However, having the grid means if you need to use it for other things later on, you have it around and don't need to recompute it every time.

Remove triplets of adjacent numbers from the list [duplicate]

This question already has answers here:
Write a function lines(a) that determines how many balls will be destroyed
(6 answers)
Closed 2 years ago.
Here is the problem. The input is a list of integers. If three adjacent numbers appear next to each other they should be dropped and the operation goes again. Iphone app with the balls of the same colors. The output should be the count of the balls that will be destroyed.
Example:
input = [3,3,4,4,4,3,4]
1st iteration
output: [3,3,3,4]
Final output:
6
4,4,4 on the first iteration, so three balls. and 3,3,3 on the second. Overall six.
My code is below. It will remove the 4,4,4 but will fail after, because list index will quickly go out of range.
def balls(a):
curr_index = 0
removed_count = 0
while len(a) > 2:
if (a[curr_index] == a[curr_index+1]) and (a[curr_index] == a[curr_index+2]):
a.remove(a[curr_index])
a.remove(a[curr_index+1])
a.remove(a[curr_index+2])
curr_index += 1
removed_count += 3
return removed_count
a = [3, 3, 4, 4, 4, 3, 4]
print(balls(a)) # should print 6
Any ideas?

input_values = [3,3,4,4,4,3,4]
values = input_values
while len(values) >= 3:
for x in range(0,len(values)-2):
# if we find 3 values in ar row
if values[x] == values[x+1] and values[x] == values[x+2]:
# update values by cutting out the series
values = values[:x] + values[x+3:]
# break this for loop
break
else:
# for loops can have an else statement
# this means that we came at the end of the for loop
# this if we didn't break the loop (and didn't found a valid triple)
# then we brea
break
#result - amount of removed balls
values, len(input_values) - len(values)

A more generic approach where you can define N
(it will find exact N values)
input_values = [3,3,4,4,4,4,3,3,4]
# amount of neighbours
N = 4
values = input_values
# keep looping as long as we've N values
while len(values) >= N:
# we need this if we break the inner loop
stop = False
# loop over values from left to right
# 3 -> 3 -> 4 -> 4 -> ... until -N + 1
# because that is the last spot that we've N values left
for x in range(0, len(values) - N + 1):
# scout for the next numbers (x+1 x+2 x+3 ....)
# this goes from, x+1 until the end of the serie
# also start counting from 2 - because
# if we've a single match, 2 numbers are equal
for e, y in enumerate(range(x+1, len(values)), start=2):
# if x and y are different, the stop this scouting
# remember y = x+1 x+2 x+3 ....
if values[x] != values[y]:
break
# if we reached this stage, we know that
# x and x+1 x+2 ... are equal
# but also check if we've enough matches
if e == N:
# update values by cutting out the series
values = values[:x] + values[y+1:]
stop = True
break
if stop:
break
else:
# for loops can have an else statement
# this means that we came at the end of the for loop
# this if we didn't break the loop (and didn't found a valid triple)
# then we brea
break
values, len(input_values) - len(values)

Here is one way to do it:
your_list=[3, 3, 4, 4, 4, 3, 4]
def remove_one(l):
if len(l)>2:
for i in range(len(l)-2):
if l[i]==l[i+1]==l[i+2]:
res.extend(l[i:i+3])
l=l[:i]+l[i+3:]
return l
return []
res=[]
while len(your_list)>2:
your_list=remove_one(your_list)
print(len(res))
Output:
6

An approach with a minimal threshold
input_values = [3,3,4,4,4,4,4,3,3,4]
# threshold
N = 3
values = input_values
while len(values) >= N:
# we need this if we break the inner loop
stop = False
# loop over values
for x in range(0, len(values) - N + 1):
# scout for the next numbers (x+1 x+2 x+3 ....)
for e, y in enumerate(range(x+1, len(values)), start=2):
# check if the values of x and x+1 (x+2 x+3 ...) are different
if values[x] != values[y]:
# if so, check if we've reached our threshold of N
if e >= N:
# if this is the case update values by cutting out the series
values = values[:x] + values[y:]
stop = True
break
# if x+n != x then we break this scouting loop
break
# stop is True, this means, that we found N neibouring numbers
# thus we can break this loop, and start over with a new* value
if stop:
break
else:
# for loops can have an else statement
# this means that we came at the end of the for loop
# thus if we didn't break the loop (and didn't found a valid triple)
# then we break
break
values, len(input_values) - len(values)

There are a couple of issues with your code.
The first is you're trying to iterate over the list a single time, but that won't work for your example.
The second is you're assuming the list will always reduce to less than 3, which won't be the case if you have more numbers.
There are also some places you can simplify the syntax, for example:
if (a[curr_index] == a[curr_index+1]) and (a[curr_index] == a[curr_index+2]):
is equivalent to:
if a[curr_index] == a[curr_index + 1] == a[curr_index + 2]:
And
a.remove(a[curr_index])
a.remove(a[curr_index+1])
a.remove(a[curr_index+2])
is equivalent to:
del a[curr_index: curr_index + 3]
So here is one way to accomplish this, thought there are more efficient ways.
def balls(a):
removed = 0
complete = False
# Run until complete is set or the list is too short for a match
while not complete and len(a) >= 3:
for idx, val in enumerate(a):
if val == a[idx + 1] == a[idx + 2]:
del a[idx: idx + 3]
removed += 3
break # Restart every time we find a match
# If it's the last possible match, set complete
if idx == len(a) - 3:
complete = True
break
return removed

I'm little bit late, but my idea is to do the processing in one pass. The idea is that when you drop a tripple, go back two positions to re-check if a new tripple has been created by that change.
Example:
3 3 4 4 4 3
^--- tripple found here, remove it
3 3 3
^--- move the cursor 2 positions left and continue checking
The code:
def drop3(lst):
# drop3 destroys the lst, make a copy if you want to keep it
dropped = 0
cursor = 0
limit = len(lst) - 2
while cursor < limit:
if lst[cursor] == lst[cursor+1] == lst[cursor+2]:
del lst[cursor:cursor+3]
cursor = max(0, cursor-2)
limit -= 3
dropped += 3
else:
cursor += 1
return dropped

A linear-tme solution:
def balls(a):
b = []
for x in a:
b.append(x)
if b[-3:] == [x] * 3:
del b[-3:]
return len(a) - len(b)