Briefly speaking, the recursion stopped halfway though everything else is fine.
The recursion function is shown below (The entire code can be found here):
def DFS(graph, startNode = 0):
global nodesProcessed; global explored; global finishingTime
explored[startNode] = True
currentLeader = startNode
if startNode in graph:
for neighbor in graph[startNode]:
if not explored[neighbor]:
#checkpoint 1
DFS(graph, neighbor)
#checkpoint 2
else: return currentLeader
nodesProcessed += 1
finishingTime[startNode] = nodesProcessed
return currentLeader
The problem is that after a while's recursion, it stopped. The things that confused me are that:
The input is fixed, but where it stops is not fixed. However, it always stopped at around 7000 times of invoke;
All the failed recursions reaches checkpoint 1 but fails to reach checkpoint 2, and the recursion does not execute at all;
It doesn't reach that maximum recursion time at all, I've set the max recursion as sys.setrecursionlimit(10**6)
It runs pretty well on relatively small input (hundreds or thousands of nodes) but stuck on a large graph which has more than 800,000 nodes.
Which are driving me crazy, I can't see a reason why it doesn't work, no error, no stack overflow, just stopped, saying Press any key to continue ... as if it is finished. Anyone has a clue about what could possibly go wrong?
As documentation specifies:
The highest possible limit is platform-dependent. A user may need to
set the limit higher when they have a program that requires deep
recursion and a platform that supports a higher limit. This should be
done with care, because a too-high limit can lead to a crash.
There is a script to check that limit.
You will have to implement non recursive DFS.
Related
I have wrote this code for solving a problem given to me in class, the task was to solve the "toads and frogs problem" using backtracking. My code solves this problem but does not stop once it reaches the solution (it keeps printing "states" showing other paths that are not a solution to the problem), is there a way to do this?. This is the code:
def solution_recursive(frogs):
#Prints the state of the problem (example: "L L L L _ R R R R" in the starting case
#when all the "left" frogs are on the left side and all the "right" frogs are on
#the right side)
show_frogs(frogs)
#If the solution is found, return the list of frogs that contains the right order
if frogs == ["R","R","R","R","E","L","L","L","L"]:
return(frogs)
#If the solution isn't the actual state, then start (or continue) recursion
else:
#S_prime contains possible solutions to the problem a.k.a. "moves"
S_prime = possible_movements(frogs)
#while S_prime contains solutions, do the following
while len(S_prime) > 0:
s = S_prime[0]
S_prime.pop(0)
#Start again with solution s
solution_recursive(s)
Thanks in advancement!
How do I stop my backtracking algorithm once I find an answer?
You could use Python exception facilities for such a purpose.
You could also adopt the convention that your solution_recursive returns a boolean telling to stop the backtrack.
It is also a matter of taste or of opinion.
I'd like to expand a bit on your recursive code.
One of your problems is that your program displays paths that are not the solutions. This is because each call to solution_recursive starts with
show_frogs(frogs)
regardless of whether frogs is the solution or not.
Then, you say that the program is continuing even after the solution has been found. There are two reasons for this, the first being that your while loop doesn't care about whether the solution has been found or not, it will go through all the possible moves:
while len(S_prime) > 0:
And the other reason is that you are reinitializing S_prime every time this function is called. I'm actually quite amazed that it didn't enter an infinite loop just checking the first move over and over again.
Since this is a problem in class, I won't give you an exact solution but these problems need to be resolved and I'm sure that your teaching material can guide you.
I have a problem with a function in a file that I am executing. It is expected to produce an IndexError, which it usually does.
However, when N defined as below becomes large enough the error is not shown and seems to be suppressed. I would like to know why this is happening. Why isn't some error thrown?
from collections import Counter
from sys import setrecursionlimit
def correct(events: list, lens: list=[]):
counts = Counter(events).values()
present = []
while events[0] not in present:
present.append(events.pop(0))
lens.append(len(present))
correct(events, lens)
if __name__ == '__main__':
N = 50000 # enough in my case, increase if the problem does not occur
setrecursionlimit(10000)
result = correct([711321] * N)
I am executing the file on Windows 10 via PowerShell. After the execution of the file, no output is produced. Nonetheless, its return status $? is False, indicating that there is something wrong with the execution.
Note that running this piece of code in a Python shell closes it forcefully.
This is because the sys.setrecursionlimit sets the recursion limit, which the maximum depth on the interpreter stack, to at most the system's recursion limit, which in turn you can get by sys.getrecursionlimit. Actually the CPython interpreter will set the limit to be somewhat bigger than you give it, because when you overflow over the recursive limit, some cleanup function will come in and they will need some stack to recurse too.
When you put it far too high, and because the highest possible limit is platform-dependent, The interpreter will overflow while cleaning up an overflow, and then the interpreter will exit with a fatal error (the interpreter will exit with error, and not your code) "Py_FatalError("Cannot recover from stack overflow.");". And because the interpreter exits, it will not print an error message to you.
location in in CPython
if (tstate->overflowed) {
if (tstate->recursion_depth > recursion_limit + 50) {
/* Overflowing while handling an overflow. Give up. */
Py_FatalError("Cannot recover from stack overflow.");
}
return 0;
This is the code you are looking for:
def correct(events: list, lens: list=[]):
while True:
present = []
while events[0] not in present:
present.append(events.pop(0))
lens.append(len(present))
if __name__ == '__main__':
N = 50000 # enough in my case, increase if the problem does not occur
result = correct([711321] * N)
This code just loops until the IndexError terminates the program.
The reason your original code returned nothing is that you set the recursion limit too high and python itself ran out of stack space and crashed.
The first 'fix' was to not mess with the recursion limit. The second fix was to not make recursive calls, but instead use a while loop.
Imagine I calculate the Fibonacci sequence following the (obviously inefficient) recursive algorithm :
def Fibo(n):
if n <= 1:
return(n)
else:
return(Fibo(n-2) + Fibo(n-1))
then my question is : how does Python known it has to stop the recurrence at n=0 ?
After all, if I call Fibo(-12), Python obviously answers -12, so why does it stop the recursion at n=0 while calling Fibo(12) for instance ?
Edit after a few comments :
This question has nothing to do with the mathematical concept of recurrence. I know recurrence stops at the initialized point. I would like to understand how recurrence are implemented in computer. For me ti is absolutely not clear while a computer should stop while there is no explicit stop command. What prevents here Fibo(0)=Fibo(-1)+Fibo(-2) to continue endlessly ? Because after all I precised that Fibo(-1)=-1, Fibo(-2)=-2, ... and I might want to summ all negative numbers as well ...
I confess in the last case I would prefer do a while loop.
It's functional, so it doesn't run, so it also doesn't stop. You are (still) thinking in terms of iterative programming and assume some kind of loop here which needs to stop at some time. This is not the case.
Instead in this paradigm you just state that the return value is the sum of the prior two numbers. At this point you don't care how the prior numbers are produced, here you just assume that they already exist.
Of course they don't and you will have to compute them as well but still this is not a loop which needs a stop. Instead it is a recursion which has an anchor. With each recursion step the values will become smaller and smaller and once they reach a value below 2 you just return 0 or 1 without any further recursion. This is your anchor.
Feel free to think of it as a "stopping point" but be aware that there is no loop happening which you need to break out of or similar.
This code is the recursive factorial function.
The problem is that if I want to calculate a very large number, it generates this error:
RuntimeError : maximum recursion depth exceeded
import time
def factorial (n) :
if n == 0:
return 1
else:
return n * (factorial (n -1 ) )
print " The factorial of the number is: " , factorial (1500)
time.sleep (3600)
The goal is to do with the recursive function is a factor which can calculate maximum one hour.
This is a really bad idea. Python is not at all well-suited for recursing that many times. I'd strongly recommend you switch this to a loop which checks a timer and stops when it reaches the limit.
But, if you're seriously interested in increasing the recursion limit in cython (the default depth is 1000), there's a sys setting for that, sys.setrecursionlimit. Note as it says in the documentation that "the highest possible limit is platform-dependent" - meaning there's no way to know when your program will fail. Nor is there any way you, I or cython could ever tell whether your program will recurse for something as irrelevant to the actual execution of your code as "an hour." (Just for fun, I tried this with a method that passes an int counting how many times its already recursed, and I got to 9755 before IDLE totally restarted itself.)
Here's an example of a way I think you should do this:
# be sure to import time
start_time = time.time()
counter = 1
# will execute for an hour
while time.time() < start_time + 3600:
factorial(counter) # presumably you'd want to do something with the return value here
counter += 1
You should also keep in mind that regardless of whether you use iteration or recursion, (unless you're using a separate thread) you're still going to be blocking the entire program for the entirety of the hour.
Don't do that. There is an upper limit on how deep your recursion can get. Instead, do something like this:
def factorial(n):
result = 1
for i in range(1, n+1):
result *= i
return result
Any recursive function can be rewritten to an iterative function. If your code is fancier than this, show us the actual code and we'll help you rewrite it.
Few things to note here:
You can increase recursion stack with:
import sys
sys.setrecursionlimit(someNumber) # may be 20000 or bigger.
Which will basically just increase your limit for recursion. Note that in order for it to run one hour, this number should be so unreasonably big, that it is mostly impossible. This is one of the problems with recursion and this is why people think about iterative programs.
So basically what you want is practically impossible and you would rather make it with a loop/while approach.
Moreover your sleep function does not do what you want. Sleep just forces you to wait additional time (frozing your program)
It is a guard against a stack overflow. You can change the recursion limit with sys.setrecursionlimit(newLimit)where newLimit is an integer.
Python isn't a functional language. Rewriting the algorithm iteratively, if possible, is generally a better idea.
So I think I am just fundamentally doing something wrong, but here is a basic example of what I am doing
some variables here
some code here to run once
def runfirst():
do some stuff
runsecond()
def runsecond():
do some different stuff
runthird():
def runthird():
do some more stuff
runfirst():
runfirst()
So it basically pulls some info I need at beginning, and then runs through 3 different variables. What I am doing is pulling info from db, then watching some counts on the db, and if any of those counts goes over a certain number over a time period, it sends email. This is for monitoring purposes, and I need it to run all the time. The problem I get is, all the time it is running, in the background it is throwing errors like "File "asdf.py", line blah, in firstrun"
I think it is complaining because it sees that I am looping through functions, but for what I need this to do, it works perfectly, except for the errors, and eventually killing my script due to maximum recursion depth exceeded. Any help?
You have infinite recursion here. Because you call runfirst from runthird, it keeps going deeper and deeper and none of the functions ever return. You might want to consider putting the functions in a while True loop instead of calling them from each other.
def runfirst():
do some stuff
def runsecond():
do some different stuff
def runthird():
do some more stuff
while True:
runfirst()
runsecond()
runthird()
You're not looping.
You're calling a function that calls another function that calls a third function that calls the first function which calls the second function which calls the third function which again calls the first function...and so on until your stack overflows.