Understanding speed in Python through a competition problem exceeding time limits - python

I recently have been doing some competition problems. In particular this one:
https://open.kattis.com/problems/knightjump
An abridged statement of the problem:
You are given a two dimensional (square) chess board of size N (1-based indexing). Some of the cells on this board are ‘.’ denoting an empty cell. Some of the cells on this board are ‘#’ denoting a blocked cell, which you are not allowed to visit. Exactly one of the cells on this board is ‘K’ denoting the initial position of the knight. Determine the minimum number of steps required for the Knight to reach cell (1,1) while avoiding cells with ‘#’ in the path.
I know this is a simple BFS problem and have written the following code to solve it:
from collections import deque
from sys import stdin
dirs = ((2,1), (2,-1), (-2,1), (-2,-1), (1,2), (1,-2), (-1,2), (-1,-2))
N = int(stdin.readline())
q = deque()
mat = [list(str(stdin.readline())) for _ in range(N)]
for i in range(N):
for j in range(N):
if mat[i][j] == "K":
inX, inY = i+1, j+1
break
q.append((inX,inY,0))
flag = False
visited = [[0]*N for _ in range(N)]
while q:
ix,iy, moves = q.popleft()
visited[ix-1][iy-1] = 1
if (ix,iy) == (1,1):
print(moves)
flag = True
break
for d in dirs:
dx,dy = d
if 1 <= ix + dx < N+1 and 1 <= iy + dy < N+1 and mat[ix+dx-1][iy+dy-1] != "#" and visited[ix+dx-1][iy+dy-1] == 0:
q.append((ix+dx,iy+dy, moves+1))
if not flag:
print(-1)
All test data I've generated has given a correct answer, but I am recieving a time limit error on the site for some cases (and correct on all other). The input limits are N <= 100. I know that C++ is faster, but I was hoping to use this problem to learn a bit about why my solution times out but many Python solutions on a similar concept perform quick, and hopefully teach me something about Python nuts and bolts I didn't know. For example - I found this solution on someone's Github:
from collections import deque
n = int(input())
grid = [list(input()) for _ in range(n)]
for i in range(n):
for j in range(n):
if grid[i][j] == 'K':
k_i, k_j = i, j
break
visited = [[0 for x in range(n)] for y in range(n)]
# time complexity is number of configs (n^2) multiplied by
# work per configs (iterate through 8 deltas)
def k_jumps(i, j):
q = deque()
q.append((0, i, j))
visited[i][j] = 1
valid = False
while len(q) > 0:
d, i, j = q.popleft()
if (i,j) == (0,0):
print(d)
valid = True
break
deltas = {(-2, 1), (-2, -1), (1, 2), (1, -2), (-1, 2), (2, 1), (2, -1), (-1, -2)}
for delta in deltas:
di, dj = delta
if 0 <= i + di < n and 0 <= j + dj < n and visited[i+di][j+dj] == 0 and grid[i+di][j+dj] != '#':
visited[i+di][j+dj] = 1
q.append((d+1, i+di, j+dj))
if not valid:
print(-1)
k_jumps(k_i, k_j)
(Thanks to user Case Pleog).
Superficially the solutions are the same (BFS) - however the second solution runs in 0.07s, while my solutions times out at over 1s.
While there are some minor differences between our solutions, I can't see what explains such a large time discrepancy. I already made some changes to my code to see if that was the issue, e.g. the line:
if 1 <= ix + dx < N+1 and 1 <= iy + dy < N+1 and mat[ix+dx-1][iy+dy-1] != "#" and visited[ix+dx-1][iy+dy-1] == 0:
and indeed that speeded things up, as before I was using:
if min(point) > 0 and max(point) < N+1 and mat[point[0]+dx-1][point[1]+dy-1] != "#" and visited[point[0]+dx-1][point[1]+dy-1] == 0:
where point was a tuple consisting of ix,iy. Any advice would be appreciated - I would like to know more about what's causing the time difference as its > 10x difference. This is just a hobby, so some of my code may have amateur quirks/inefficiencies which hopefully I can explain. Thanks!

Do visited[ix+dx-1][iy+dy-1] = 1 right away when you do q.append((ix+dx,iy+dy, moves+1)), otherwise you might put that same coordinate into the queue multiple times.

Related

Finding total amount of paths in a triangle grid (iterative without recursion)

i have a given a triangle grid:
Triangle
For every point (i,j) with i+j being even:
Given recursive function
Now i need to write a iterative function that finds all possible paths from (0,0) to the point (2n,0) given that n ∈ N.
Edit: Solution
import math
A = []
def C_iterativ(n):
C = [[0 for _ in range(n+2)] for _ in range(2*n+1)]
C[0][0] = 1
for i in range(2*n+1):
for j in range(n+1):
C[i][j] += C[i-1][j-1] + C[i-1][j+1]
return C[2*n][0]
for i in range(20):
print(int(1/(i+1)*math.factorial(2*i)/(math.factorial(i)**2)) == C_iterativ(i))
** Edit: Recursive solution**
def C_memo(i, j, memo={}):
if (i, j) in memo:
return memo[(i, j)]
if i == j == 0:
result = 1
elif i == 0 and j > 0:
result = 0
elif i > 0 and j == 0:
result = C_memo(i-1, 1, memo)
elif i > 0 and j > 0:
result = C_memo(i-1, j-1, memo) + C_memo(i-1, j+1, memo)
memo[(i, j)] = result
return result
On a more mathy note, this problem is equivalent to a more famous problem- balancing parenthesis. Ever step up-right is a ( and ever step down-right is a ), with n parenthesis pairs, how many valid sequences of parenthesis are there?
The answer for n is the nth catalan number, which is nck(2n, n) - nck(2n, n+1) (nck being "n choose k"). In python, this comes out to-
from math import comb
def catalan(n):
return comb(2*n, n) - comb(2*n, n+1)
So the answer to "How many distinct shortest paths from (0, 0) to (2n, 0) through my triangular grid are there?" is catalan(n).
For every node other than (0,0), the number of ways to reach that node is the sum of the number of ways to meet its immediate predecessors. There is one way to reach (0,0).
Start with (0,0) -> 1, and a queue containing (1,1). On each iteration, take the node at the front of the queue, update it to be the sum of its predecessors, and add any successors that have all predecessors calculated to the end of the queue.
(0,0) -> 1; queue = [(1,1)]
(1,1) -> 1; queue = [(2,2), (2,0)]
(2,2) -> 1; queue = [(2,0), (3,3)]
(2,0) -> 1; queue = [(3,3), (3,1)]
(3,3) -> 1; queue = [(3,1), (4,4)]
(3,1) -> 2; queue = [(4,4), (4,0), (4,2)]
... and so on

How to optimise a strictly increasing stack based list

So I am trying to solve https://leetcode.com/problems/daily-temperatures
Basically given an array [30,40,50,60]
we need to return the indices of the next increasing value
so output would be [1,1,1,0]
My algorithm is to
while end !=0 read from the array
if last element add 0 to output and stack
Else if temperature[end] > stack top add to stack and add stack top to be the temperature difference.
If temperature[end] <= stack top then we pop every element until this condition breaks and update the stack top and output.
I keep getting a time limit exceeded here, My understanding here is that the complexity is O(N). I am not sure how to optimise it further.
def dailyTemperatures(self, temperatures: List[int]) -> List[int]:
output = []
r = len(temperatures) - 1
stack = []
while r >= 0:
if r == len(temperatures) - 1:
stack.append(r)
output.insert(0, 0)
else:
if temperatures[stack[-1]] > temperatures[r]:
output.insert(0, stack[-1]-r)
stack.append(r)
else:
while stack and temperatures[stack[-1]] <= temperatures[r]:
stack.pop()
if len(stack) == 0:
output.insert(0, 0)
stack.append(r)
else:
output.insert(0, stack[-1]-r)
stack.append((r))
r -= 1
return output
An efficient implementation of a slow algorithm will still scale slowly. Optimizing the stack won't change the big-O. But I'll outline an idea using a different data structure for you to figure out.
In example1 of the problem you start with the array [73,74,75,71,69,72,76,73]. Let's arrange this in terms of maxes of ranges like so:
x = [
[73,74,75,71,69,72,76,73],
[ 74, 75, 72, 76],
[ 75, 76],
[ 76],
]
In other words x[i][j] gives the max of all elements from j * 2**i to (j+1) * 2**i - 1.
Now to find the first rise after k we go "forward and down" until we find a block with a larger (or 0 if there is none) and then "up and forward" until we find where it is. Where:
while going forward and down at (i, j):
if j is even:
if j not at end of x[i]:
move to (i, j+1) then test
else:
answer is 0
else:
move to (i+1, j//2 + 1) then test
while going up and forward at (i, j) with 0 < i:
if answer in range for (i-1, j+j):
move to (i-1, j+j)
else:
move to (i-1, j+j+1)
For example if k was 2 we would do this.
start at (0, 2)
go f&d to (0, 3), check if 75 < x[0][3] (ie 71), stay f&d
go f&d to (1, 2), check if 75 < x[1][2] (ie 72), stay f&d
go f&d to (1, 3), check if 75 < x[1][3] (ie 76), switch to u&f
check if 75 < x[0][6] (ie 76) go u&f to (0, 6)
And now we know that the next one is at position 6. So we get 6 - 2 == 4 there.
What is the performance? The whole data structure is O(n) to build. Each lookup is at worst O(log(n)). So doing the n checks is at worst O(n log(n)). Which should be fast enough.
After considerable testing, your answer IS linear. (And therefore better than what I suggested before.) However in order to meet their time limit you need to microptimize everything. Meaning get rid of every unneeded if, make sure your comparisons are the fastest possible, etc.
I would definitely say that leetcode set the time limit too restrictively for the problem and language performance.
def dailyTemperatures(self, temperatures: List[int]) -> List[int]:
if 0 == len(temperatures):
return []
output = [0]
r = len(temperatures) - 1
stack = [r]
r -= 1
while r >= 0:
while len(stack) and temperatures[stack[-1]] <= temperatures[r]:
stack.pop()
if len(stack) == 0:
output.append(0)
else:
output.append(stack[-1]-r)
stack.append((r))
r -= 1
output.reverse()
return output
My Solution is
def dailyTemperatures(self, temperatures):
res = []
i = 0
while i < len(temperatures) - 1:
j = i + 1
while True:
if j >= len(temperatures):
res.append(0)
break
if temperatures[j] <= temperatures[i]:
j += 1
else:
res.append(j - i)
break
i += 1
res.append(0)
return res

How to backtrack when using a iterative DFS implemented via a stack

I was completing this Leetcode problem: https://leetcode.com/problems/word-search/ and I randomly chose to implement the DFS iteratively with a while loop and a stack but I encountered some inconveniences when backtracking that I wouldn't normally occur if I had done the problem recursively i.e. I could only think of implementing a list (visited_index) to keep track of the indexes I had visited and pop values off to set the boolean matrix visited back to False when backtracking.
from collections import defaultdict
class Solution:
def exist(self, board: List[List[str]], word: str) -> bool:
starting_points = defaultdict(list)
m, n = len(board), len(board[0])
for i in range(m):
for j in range(n):
starting_points[board[i][j]].append((i,j))
start = starting_points[word[0]]
visited = [[False] * n for _ in range(m)]
stack = []
directions = [(1,0), (0,-1), (-1,0), (0,1)]
for s in start:
stack.append((s[0], s[1], 0))
visited_index = [] # EXTRA LIST USED
while stack:
x, y, count = stack.pop()
while len(visited_index) > count:
i, j = visited_index.pop()
visited[i][j] = False # SETTING BACK TO FALSE WHEN BACKTRACKING
if x < 0 or x >= m or y < 0 or y >= n or visited[x][y] or board[x][y] != word[count]:
continue
else:
visited[x][y] = True
visited_index.append((x,y))
if count + 1 == len(word):
return True
for d in directions:
i, j = x + d[0], y + d[1]
stack.append((i,j, count + 1))
else:
stack.clear()
for i in range(m):
for j in range(n):
visited[i][j] = False
return False
I believe that in a recursive approach I could have reset the the visited boolean value back to False at the end of the function without the need to use an extra list. Does anyone have any suggestions to not introduce an extra data structure when doing an iterative DFS with a stack?
I would keep the parent node on the stack for as long there are children being processed. Then when all children have been processed and you pop the parent from the stack you'll have the right moment to also remove the visited mark for that parent.
One way to implement that idea is to put one more information in the tuple that you put on the stack: the last direction that was taken. You can use that info to look for the next direction to take, and if there is a valid direction available, push the current node back on the stack with that new direction, and then push the corresponding child on the stack. The latter gets some default value for that "previous" direction indication. For example -1.
I reworked your code to align with that idea:
class Solution:
def exist(self, board: List[List[str]], word: str) -> bool:
stack = []
m, n = len(board), len(board[0])
for i in range(m):
for j in range(n):
if board[i][j] == word[0]:
# 4th member of tuple is previous direction taken. -1 is none.
stack.append((i, j, 1, -1)) # count=1, side=-1
visited = [[False] * n for _ in range(m)]
directions = [(1,0), (0,-1), (-1,0), (0,1)]
while stack:
x, y, count, side = stack.pop()
# perform the success-check here, so it also works for 1-letter words.
if count == len(word):
return True
visited[x][y] = True # will already be True when side > -1
# find next valid direction
found = False
while side < 3 and not found:
side += 1
dx, dy = directions[side]
i, j = x + dx, y + dy
found = 0 <= i < m and 0 <= j < n and not visited[i][j] and board[i][j] == word[count]
if not found: # all directions processed => backtrack
visited[x][y] = False
continue
stack.append((x, y, count, side)) # put node back on stack
stack.append((i, j, count + 1, -1))
return False

Why this DFS way not working?

Problem description:
Given a 2D grid, each cell is either a wall 'W', an enemy 'E' or empty '0' (the number zero), return the maximum enemies you can kill using one bomb.
The bomb kills all the enemies in the same row and column from the planted point until it hits the wall since the wall is too strong to be destroyed.
Note that you can only put the bomb at an empty cell.
Example:
For the given grid
0 E 0 0
E 0 W E
0 E 0 0
return 3. (Placing a bomb at (1,1) kills 3 enemies)
My DFS solution:
def maxKilledEnemies(grid):
"""
:type grid: List[List[str]]
:rtype: int
"""
l_row, l_col = len(grid), len(grid[0])
visited = [[False] * l_col for _ in range(l_row)] #using this array to avoid duplicate traverse.
def dfs(i, j):
if 0 <= i < l_row and 0 <= j < l_col and not visited[i][j]:
visited[i][j] = True
if grid[i][j] == 'W': #wall return 0
return 0
elif grid[i][j] == '0': #0 means we just ignore this cell and traverse it adjacents
top_val = dfs(i - 1, j)
down_val = dfs(i + 1, j)
left_val = dfs(i, j - 1)
right_val = dfs(i, j + 1)
return left_val + right_val + top_val + down_val
elif grid[i][j] == 'E': # Enemy and we add it by 1
top_val = dfs(i - 1, j)
down_val = dfs(i + 1, j)
left_val = dfs(i, j - 1)
right_val = dfs(i, j + 1)
return left_val + right_val + top_val + down_val + 1
return 0
ret = [0]
for i in range(l_row):
for j in range(l_col):
if not visited[i][j] and grid[i][j] == '0':
val = dfs(i, j)
ret[0] = max(val, ret[0])
return ret[0]
Solution().maxKilledEnemies([["0","E","0","0"],["E","0","W","E"],["0","E","0","0"]]) #return 4 but expect 3.
The idea is quit simple that for every cell which num is 0, we traverse it by 4
directions(Top/Down/Left/Right).
I know there are other ways to solve it more smarter. But I would like to figure out why my way not working?
There are at least three errors in your code:
In each recursion step, you explore all possible directions. This makes your search behave like a flood-fill, but you want to start in all directions from the bomb location you're checking only. After that, recurse (or search iteratively) in the given direction.
You don't check whether the possible bomb location is empty, so your code could place the bomb on an enemy.
You don't reset the visited array between searches, so that effectively only cell (0, 0) is assessed.
One solution is to have two functions,
one for the possible bomb location. Check here whether the cell is empty. If yes, count the victims by recursing north, east, south and west.
One for the "rays" of the bomb. Travel in the given direction and count the enemies until you hit a wall. Because the ray travels in only one direction, you don't need the visited array any longer.
This isn't really depth first search, so instead of calling the second function recursively, you could just use loops.
Probably because the use of DFS is not a solution. When you use DFS you search all reachable spaces in the grid but for your question you should just search the directly horizontal or vertical spaces.
Ex: When using DFS at [0][0] you will find all enemies, not only the ones at [0][1] and [1][0].

Queens on chessboard solved randomly in Python

The idea is to try solve the "queen problem" by totally placing the queens totally random in each row of the chess board, and see how many repetitions it takes to solve it. The chess board can be any size.
My idea is to create s lists, each containing s "empty" characters (underscores). Then for each line randomly pick a position to insert the queen ("I" value) and then mark all the positions below and diagonally down (I'm going row-by-row, so I don't have to bother about rows above) with X. if in any iteration the randomly chosen position for the queen matches with position of any X in that row, I start the new chessboard from scratch.
I have something like this, but it seems to get stuck on line 19 (marked with a comment), but it doesn't give me any error. What can be wrong? Also, is my solution (apart that line) correct?
from random import *
#Success flag
success = 0
#Trials counter
trials = 0
s = input ("enter board size\n")
s = int(s)
block = 1 #blockade
queen = 2 #queen
board = [[0 for x in range(s)] for y in range(s)]
while success == 0:
for y in range (0, s-1):
pos = randint(0,s-1) #line 19
if board[y][pos] != block:
board[y][pos] = queen
a = 1
for z in range (y, s-2):
board[z + 1][pos] = block
if pos - a >= 0:
board[z + 1][pos - a] = block
if pos + a <= s-1:
board[z + 1][pos + a] = block
a = a + 1
success = 1
else:
success = 0
#Printing board
for y in range (0, s-1):
print (board[y])
print ("Number of trials:\n")
print (trials)
Some issues:
The second argument to the range function represents the first number that will not be visited, so most of the time you have it one short.
You need to exit the loop on y at a failed attempt: you don't want to continue with the next row, but restart
You need to reset the board after each failed attempt, or otherwise put: before each attempt
You should build in some safety to exit if no solution is found after many iterations, otherwise you risk it to get stuck. For input 1, 2 or 3, there is no solution.
The trials number is never increased
Instead of blindly choosing a position, you'd better only select from among the available positions (not blocked), or you will be going for an amazing number of trials: for size 8, it would not be unusual to need 100 000 trials without this filtering in place!
See the corrected code, with comments where I made changes:
import random
s = input ("enter board size\n")
s = int(s)
trials = 0
block = 1
queen = 2
# add some maximum to the number of attempts
max_trials = 100000
success = 0
# add safety measure to avoid infinite looping
while success == 0 and trials <= max_trials:
# initialise board before every trial
board = [[0 for x in range(s)] for y in range(s)]
# assume success until failure
success = 1
# count trials
trials += 1
for y in range (0, s): # use correct range
# get the fields that are still available in this row
available = [x for x, i in enumerate(board[y]) if i == 0]
if len(available) == 0:
success = 0
# exit for loop, you want to start a next trial
break
# choose a random position among available spots only
pos = available[random.randint(0, len(available)-1)]
board[y][pos] = queen
a = 1
for z in range (y+1, s): # use correct range
board[z][pos] = block
if pos - a >= 0:
board[z][pos - a] = block
if pos + a < s:
board[z][pos + a] = block
a = a + 1
for y in range (0, s): # use correct range
print (board[y])
print ("Number of trials:", trials)
See it run on repl.it
Here is a short solution based on doing arithmetic on the coordinates of the placed queens:
import random, itertools
def clashes(p,q):
a,b = p
c,d = q
return a == c or b == d or abs(a-c) == abs(b-d)
def solution(queens):
#assumes len(queens) == 8
return not any(clashes(p,q) for p,q in itertools.combinations(queens,2))
def randSolve():
counter = 0
while True:
counter += 1
queens = [(i,random.randint(1,8)) for i in range(1,9)]
if solution(queens): return counter, queens
print(randSolve())
Last time I ran it I got:
(263528, [(1, 4), (2, 7), (3, 3), (4, 8), (5, 2), (6, 5), (7, 1), (8, 6)])
meaning that the first solution was encountered after 263527 failures. On average, you can expect to go through 182360 failures before you get a success.
After you try different row and fail this time, you have to create a new empty board and if success is 0, you should break the for loop as following.
while success == 0:
board = [[0 for x in range(s)] for y in range(s)]
for y in range (0, s):
pos = randint(0,s-1) #line 19
if board[y][pos] != block:
board[y][pos] = queen
for i in range(y+1, s):
board[i][pos] = block
success = 1
else:
success = 0
break
trials += 1
You can follow the same logic to implement diagonal cases.

Categories