We Keep Coding

Timing out a function in python [duplicate]

This question already has answers here:
Timeout on a function call
(23 answers)
Closed 25 days ago.
I am trying to run an apriori analysis on a series of hashtags scraped from Twitter in python using jupyter lab, and need to find a way to time out a function after a certain period of time. The function is being run using a while loop that incrementally reduces the size of the support value, and stops after ten seconds has passed.
def association_rules(hashtags_list):
# Convert the list of hashtags into a list of transactions
transactions = [hashtags for hashtags in hashtags_list]
# Initialize the support
min_support = 1
# Initialize the confidence
min_confidence = 0.1
# Initialize the lowest support
lowest_support = 1
# Start the timer
start_time = time.time()
while True:
try:
# Find the association rules
association_rules = apriori(transactions, min_confidence=min_confidence, min_support=min_support)
# Convert the association rules into a list
association_rules_list = list(association_rules)
# End the timer
end_time = time.time()
# Calculate the running time
running_time = end_time - start_time
# check if running time is over the maximum time
if running_time >= 10:
break
lowest_support = min_support
if min_support > 0.01:
min_support = min_support - 0.01
else:
min_support = min_support - 0.005
if min_support <= 0:
min_support = 0.01
except Exception as e:
print("An error occurred:", e)
break
return association_rules_list, round(lowest_support, 3)
The problem this causes is because the timeout is called within the loop itself, it is possible for the loop to get hung up if the apriori support value gets too low before hitting the 10 seconds, which often happens with small datasets, so I need an external function to stop the loop.
I've been looking into parallel processing with no success, and still can't really determine if it can even be carried out in Jupyter Lab.
Any ideas on how to stop a function would be appreciated.
Edited to add that I am running on Win 10, which may effect some options.

This is what I have so far, which seems to work, but I could be wrong...
import time
import func_timeout
import tempfile
def test_function():
highest_n = 0
n = 0
temp_file = tempfile.NamedTemporaryFile(delete=False)
start_time = time.time()
while time.time()-start_time < timeout_time:
n += 1
if n > highest_n:
highest_n = n
temp_file.write(str(n).encode() + b' ' + str(highest_n).encode())
temp_file.close()
return highest_n
timeout_time = 5
try:
result = func_timeout.func_timeout(timeout_time, test_function)
print("highest number counted to: ",result)
except func_timeout.FunctionTimedOut:
print("Function Timed Out after ", timeout_time, " seconds")
temp_file = open(temp_file.name, 'r')
n, highest_n = temp_file.read().split()
n = int(n)
highest_n = int(highest_n)
print("highest number counted to: ",highest_n)

Edit: After some discussion in the comments, here's a different suggestion.
You can terminate a function from outside the function by causing an exception to be thrown via the signalling mechanism as in the example below:
import signal
import time
import random
def out_of_time(signum, frame):
raise TimeoutError
def slow_function():
time.sleep(random.randint(1, 10))
return random.random()
signal.signal(signal.SIGALRM, out_of_time)
signal.alarm(5)
v = 0
while True:
try:
v = slow_function()
except TimeoutError:
print("ran out of time")
break
print("v:", v)
What's going on here is that we have a function, slow_function, that will run for an unknown period of time (1-10 seconds). We run that in a while True loop. At the same time, we've set up the signal system to throw a TimeoutError after 5 seconds. So when that happens, we can catch the exception.
A few things to note though:
This does not work on Windows.
There is absolutely NO GUARANTEE that the code will be where you think it will be when the exception is thrown. If the loop is already completed and the interpreter is not currently running slow_function, you don't know what will happen. So you'll need to 'arm' the exception-throwing mechanism somehow, for example by checking the frame parameter that is passed to out_of_time to make sure that the exception is only thrown if the signal comes while we're inside the expected function.
This is kind of the Python equivalent of a goto in the sense that it causes the execution to jump around in unexpected ways.
A better solution would be to insert some code into the function that you want to terminate to periodically check to see if it should keep running.
Change the while loop from while True to:
while time.time()-start_time < 10 and <some other criteria>:
# loop body
Then you can get rid of the break and you can add whatever halting criteria needed to the loop statement.

I think I found a simpler way just using a list rather than a temporary file.
import time
from func_timeout import func_timeout, FunctionTimedOut
start_time = time.time()
temp_list = []
running_time = 3.3
def while_loop():
min_support = 1
lowest_support = 1
while True:
time.sleep(0.1)
if lowest_support > 0.01:
lowest_support -= 0.01
else:
lowest_support -= 0.001
if lowest_support <= 0:
lowest_support = 0.001
temp_list.append(lowest_support)
if time.time() - start_time > running_time:
break
try:
func_timeout(running_time, while_loop)
except FunctionTimedOut:
pass
temp_list[-1]

Check if a function has been called from a loop

Is it possible to check dynamically whether a function has been called from within a loop?
Example:
def some_function():
if called_from_loop:
print("Hey, I'm called from a loop!")
for i in range(0, 1):
some_function()
some_function()
Expected output:
> Hey, I'm called from a loop!

Not sure if this is going to be 100% reliable. It's also rather cumbersome and would have to be written inside the function of interest rather than in its own discrete function for fairly obvious reasons.
We can get the source code for the currently executing .py file
We can also get a stack trace.
Therefore, we can determine the line number in the code from where our function has been called. Thus we can navigate the source code (backwards) taking care to note indentation and see where the indentation decreases and if the line where that occurs starts with either 'for' or 'while' then we were called from within a loop.
So here's the fun:
import traceback
import inspect
import sys
def getws(s):
return len(s) - len(s.lstrip())
def func():
source = inspect.getsource(sys.modules[__name__]).splitlines()
stack = traceback.extract_stack()
lineno = stack[-2].lineno-1
isLooped = False
if (ws := getws(source[lineno])) > 0:
for i in range(lineno-1, -1, -1):
if (_ws := getws(line := source[i])) < ws:
if (tokens := line.split()) and tokens[0] in {'for', 'while'}:
isLooped = True
break
if (ws := _ws) == 0:
break
print('Called from loop' if isLooped else 'Not from loop')
def looper():
for _ in range(1):
# banana
func()
def looper2(): # this code is nonsense
i = 5
while i > 0:
if i < 10:
func()
break
looper()
looper2()
func()
while True:
func()
break
if __name__ == '__main__':
func()
Output:
Called from loop
Called from loop
Not from loop
Called from loop
Not from loop

How to avoid using a global variable in this recursion function and improve my code?

I made the next function to solve the problem of rat maze, where the rat only can move to forward and down, and i needed to find the number of ways possibles. I did it but i want to avoid the global variable "possible_ways". What are the way to improve my code ?
possible_ways = 0
def solve(n,x,y):
if x == n-1 and y == n-1:
global possible_ways
possible_ways = possible_ways+1
return True
if x<=n-1 and y<=n-1:
solve(n,x+1,y)
solve(n,x,y+1)
solve(4,0,0)
print(possible_ways)

In this case you can make the function return a value (for all possible code paths in it — your code returned None whenever the first if condition was untrue):
def solve(n,x,y):
if x == n-1 and y == n-1:
return 1
if x <= n-1 and y <= n-1:
return solve(n,x+1,y) + solve(n,x,y+1)
return 0
possible_ways = solve(4,0,0)
print(possible_ways)

Recursion program flow

I have some textbook code that calls itself recursively. I don't understand the program flow. Here is the code:
def Recur_Factorial_Data(DataArray):
numbers = list(DataArray)
num_counter = 0
list_of_results = []
for num_float in numbers:
n = int(num_float)
1. result = Recur_Factorial(n)
list_of_results.append(result)
def Recur_Factorial(num):
if num == 1:
2. return num
else:
result = num*Recur_Factorial(num-1)
3. return result
if num < 0:
return -1
elif num == 0:
return 0
else:
return 0
In Recur_Factorial_Data, I loop through the data elements and call Recur_Factorial, which returns its value back to the calling function (Recur_Factorial_Data). I would expect that the lines marked 2 ("return num") and 3 ("return result") would always return a value back to the calling function, but that's not the case. For example, where the initial value (from the array DataArray) is 11, the function repeatedly calls itself until num is 1; at that point, the program falls to the line marked 2, but it does not loop back to the line marked 1. Instead, it falls through to the next line. The same happened on the line marked 3 -- I would expect it to return the result back to the calling function, but it does that in some cases and not in others.
I hope this is enough description to understand my question -- I just don't know why each return does not loop back to return the result to the calling function.
EDIT: The question at Understanding how recursive functions work is very helpful, and I recommend it to anyone interested in recursion. My question here is a bit different -- I asked it in the context of the program flow of the Python code where the recursive function is called.

If it call itself recursively 10 times, it will be at the 10th level of recursion and should go back 10 times at the point where there was a recursive call with an intermediate result and only then exit from the recursive function to the place where it was called. Learn more about recursion
Also try to rearrange instructions in Recur_Factorial function in this way:
def Recur_Factorial(num):
if num < 0:
return -1
elif num == 0:
return 0
elif num == 1:
return num
else:
return num * Recur_Factorial(num-1)

recursive function python 3 'maximum recursion depth exceeded'

I am creating a collatz sequence with a recursive function below:
def collatz(n):
if n%2 == 0:
return int(n/2)
else:
return int((3 * n)/2)
From what I understand, a recursive function is a function that basically calls itself. Below I have attempted creating the recursive function with the following:
def collatz(x):
if x == 1:
"Done"
print(x)
x = collatz(x)
return(x)
Where essentially the variable x continues to get passed into the collatz function I defined until it gets to 1. However, every time I run the recursive function it prints 'x' repeatedly and then I get the
collatz(3)
'RecursionError: maximum recursion depth exceeded in comparison'
Which I understand is an infinite loop essentially. I thought by reassigning it to x to the results of the first collatz() it would return the new value and continue till it hit '1' but I can't seem to quite get there.
Any help/tips/advice would be great! Thanks!

Recursive functions have what's known as a "base case" and what's known as a "recursive case." The base case is when you should stop recursing and return an answer.
In this case, the base case is when x==1
def collatz(x):
if x == 1:
return x
and the recursive case is the rest of the time
# continuing from above
else:
if n % 2 == 0:
return collatz(int(n//2))
else:
return collatz(n*3 / 2) # sicut. Note that the collatz sequence
# uses 3n+1 here, not 3n/2 as in the question
N.B. that I change the effective value of x in the next loop through collatz before returning the result of that new call. If you don't, and simply return collatz(x), you'll never reach your base case and recurse forever.

#Roee Gavirel
Here is the final answer based on his answer above:
def collatz(x):
if x == 1:
"Done"
elif x%2 == 0:
x = int(x/2)
print(x)
collatz(x)
else:
x = int((3*x)+1)
print(x)
collatz(x)
collatz(3)
Thanks for all the help!

You show two different implementations of the same function collatz, while you need to combine them.
def collatz(n):
if x == 1:
"Done"
print(x)
if n%2 == 0:
collatz(int(n/2))
else:
collatz(int((3 * n)/2))