I am having a python function that has loop which may fall into infinte loop.
I want to kill the function and print error if it exceeds the time limit let's say 10 sec.
For eg. this is the function that contains infinite loop
def myfunc(data):
while True:
data[0]+=10
return data
data=[57,6,879,79,79,]
On calling from here
print(myfunc(data))
I want it to completely kill the process and print the message. I am working on windows.
Any reference or resource will be helpful.
Thanks
Maybe you can try with func-timeout Python package.
You can install it with the following command:
pip install func-timeout
This code will resolve your problem "I want to kill the function and print error if it exceeds the time limit let's say 10 sec."
from time import process_time as timer
def whatever(function):
start = timer()
infinite loop here:
end = timer()
(your code here)
if end >= 10:
print(whatever)
return
Breakdown: there are a few options for doing this with the time module
At the start of your program import the method
At the beginning of your (infinite loop) start = timer() will call process_time and save the value as start.
end = timer() will continue to send the call to process_time storing new value
put w/e code you want in your loop
if end >= 10: will keep checking the count each loop iteration
print(whatever)
return will automatically terminate the function by escaping if time runs 10 seconds when it is checked next loop.
This doesn't say "how to stop the process_time" from running once its called idk if it continues to run in the background and you have to stop it on your on. But this should answer your question. And you can investigate a bit further with what I've given you.
note: This is not designed to generate "precision" timing for that a more complex method will need to be found
Related
This is for a 3.x python program. One thread is created for a web server - no problems there.
There is another function running in a thread that I would like to run every X seconds unless the prior thread is still running in which case we can skip this iteration. I'm not looking for a break of X seconds between the end of the prior run and the start of a new one.
Desired behavior:
X = 180 seconds (3 minutes)
8:00 am - Code begins
8:01 am - Code completes
8:03 am - Code begins
8:07 am - Code completes (we skipped the 8:06 am run)
8:09 am - Code begins
The following code runs the function (main) every X seconds, but doesn't check to see if there is already a prior thread running.
def sync():
t_sync = threading.Timer(syncInterval, sync)
t_sync.daemon = False
t_sync.start()
main()
What's the best way to accomplish this?
Thanks to PygoNode's idea about is_alive() I was able to get the following working. This probably isn't the most elegant way to do this, but it works.
def sync():
t_sync = threading.Thread(target=main)
t_sync.daemon = False
t_sync.start()
while True:
time.sleep (syncInterval)
if t_sync.is_alive():
logger.warning ('Sync: Prior thread still running. Do not kick off another sync.')
else:
t_sync = threading.Thread(target=main)
t_sync.daemon = False
t_sync.start()
threading.Timer has a is_alive() method. You could refactor your sync() method to return the thread after starting, and then pass that reference to your next sync() call check if the existing is still alive/running.
My timer won't work properly, since it keeps printing forever, not what I want.
I've already tried re-organizing the script in many ways, but the result is always the same.
import time
CommandInput = input()
#execution [uptime]
def uptime(y):
while 1 == 1:
if (CommandInput == "uptime"):
print(y, " seconds")
y = y + 1
time.sleep(1)
uptime(9)
I wanted to make some sort of "background timer" that kept running from when the script was executed up to when it was closed, and if I typed a certain line in the input it would show the current number it is in. The problem is that it keeps printing the timer forever, for every single number it counts. I wanted to do a one-time thing, where you could wait as much as you want and type the input, which would show the number the timer is in.
Your current program has to wait for input() to finish. The compiler does not move past the second line until the user hits enter. Thus the functions starts once the input is finished.
This timer could be done several way, such as threading or timers. There are some examples of that here and here. For threading, you need one process for user inputs and one for timers. But the better and easier way is to to use a timer and store the current time. I believe the following code is somewhat similar to what you want:
import time
start_time = time.time()
def get_time():
current_time = time.time()
total = current_time - start_time
return total
while True:
CommandInput = input()
if CommandInput == "uptime":
y = get_time()
print(str(round(y)) + " seconds")
The variable start_time records the current time at the start. time.time() gets the current time in seconds on the computer. The get_time function is called after an input and sees how much time has pasted by subtracting the current time minus the start time. This way the you do not need multiple loops.
The while True loops just waits for the user input and then prints the time. This then repeats until the program is exited.
To write a script like that you would need to look into a module called asyncio (https://docs.python.org/3/library/asyncio.html), which would allow you to run multiple things at the same time.
Here is a asyncio hello world:
import asyncio
async def main():
print('Hello ...')
await asyncio.sleep(1)
print('... World!')
# Python 3.7+
asyncio.run(main())
I have a multi-threaded SMTP server. Each thread takes care of one client. I need to set a timeout value of 10 seconds on each server thread to terminate dormant or misbehaving clients.
I have used the time.time(), to find the start time and my checkpoint time and the difference gives the running time. But I believe it gives the system time and not the time this thread was running.
Is there a Thread local timer API in Python ?
import threading
stop = 0
def hello():
stop = 1
t=threading.Timer(10,hello)
t.start()
while stop != 1:
print stop
print "stop changed"
This prints 0 (initial stop) in a loop and does not come out of the while loop.
Python has progressed in the 6 years since this question was asked, and in version 3.3 it's introduced a tool for exactly what was being asked for here:
time.clock_gettime(time.CLOCK_THREAD_CPUTIME_ID)
Python 3.7 additionally introduced an analogous time.clock_gettime_ns.
Detailed docs are exactly where you'd expect but the feature is pretty straightforward straight out of the box.
In the python documentation there is no mention of "thread timing". Either the clocks are process-wide or system-wide. In particular time.clock measures process time while time.time returns the system time.
In python3.3 the timings API was revised and improved but still, I can't see any timer that would return the process time taken by a single thread.
Also note that even if possible it's not at all easy to write such a timer.
Timers are OS specific, so you would have to write a different version of the module for every OS. If you want to profile a specific action, just launch it without threads.
When threaded the timing either it runs as expected, or it is a lot slower because of the OS, in which case you can't do nothing about it(at least, if you don't want to write a patch that "fixes" the GIL or removes it safely).
Python 3.7 has added the time.thread_time() method that seems to do what this question needs. According to the docs, it is thread-specific and excludes time spent sleeping.
The hello function's stop value is local, not the global one.
Add the following:
def hello():
global stop
stop = 1
I am posting a sample code which can measure the running time of the thread, you can modify the code, so as to use with your function.
import time
import threading
def hello():
x = 0
while x < 100000000:
pass
x += 1
start = time.clock()
t = threading.Thread(target = hello, args = ())
t.start()
t.join()
end = time.clock()
print "The time was {}".format(end - start)
On my system, it gave a time of 8.34 seconds.
I have written a scraper that does html scraping and then use API to get some data, since its a very lengthy code I haven't put it here. I have implemented random sleep method and using it within my code to monitor throttle. But I want to make sure I don't over run this code, so my idea is to run for an 3-4 hours then taker breather and then run again. I haven't done anything like this in python I was trying to search but not really sure where to start from, it would be great if I get some guidance on this. If python has a specific module link to that would be a great help.
Also is this relevant? I don't I need this level of complication?
Suggestions for a Cron like scheduler in Python?
I have functions for every single scraping task, and I have main method calling all those functions.
You can use a threading.Timer object to schedule an interrupt signal to the main thread after the time is exceeded:
import thread, threading
def longjob():
try:
# do your job
while True:
print '*',
except KeyboardInterrupt:
# do your cleanup
print 'ok, giving up'
def terminate():
print 'sorry, pal'
thread.interrupt_main()
time_limit = 5 # terminate in 5 seconds
threading.Timer(time_limit, terminate).start()
longjob()
Put this in your crontab and run every time_limit + 2 minutes.
You could just note the time you have started and each time you want to run something make sure you haven't exceeded the given maximum. Something like this should get you started:
from datetime import datetime
MAX_SECONDS = 3600
# note the time you have started
start = datetime.now()
while True:
current = datetime.now()
diff = current-start
if diff.seconds >= MAX_SECONDS:
# break the loop after MAX_SECONDS
break
# MAX_SECONDS not exceeded, run more tasks
scrape_some_more()
Here's the link to the datetime module documentation.
I have a set of instructions, say {I} and I would like to perform this set {I}
at predefined time for instance each minute.
I'm not asking how to insert a delay of 1 minutes between to successive executions of
the set {I}, I want to start the instructions {I} each minute independently of the time of execution of {I}.
If I inderstand the following code
import time
while True:
{I}
time.sleep(60)
would simply insert a delay of 60 secs between the end of the execution of {I} and the following one. Is it true? Instead I would like that the set of instructions {I} starts each minute (for instance at 9.00 am, 9.01 am, 9.02 am, etc).
Is it possible to perform such a task inside python, or is it preferable to write a script with {I} that I execute each minutes, for instance, with Crontab?
Thank you in advance
Looks like signal.alarm and signal.signal(signal.SIGALRM, handler) should help you.
If you don't need finer resolution than a minute, cron would be the easiest option. Otherwise you'd end up re-writing something like it.
If you need intervals shorter than a minute, you might consider "timeouts" from the glib library. It has Python bindings. The timeout should then probably start the task in a separate process.
Something like APScheduler might meet your needs.
I'm sure there are other similar packages out there as well.
Chances are, you'd have to instantiate separate threads for every instruction to be run concurrently, and simply dispatch them in your delayed while loop.
You could spawn a thread every second using threading.Timer:
import threading
import time
def do_stuff(count):
print(count)
if c < 10: # Let's build in some way to quit
t = threading.Timer(1.0, do_stuff, args=[count+1])
t.start()
t = threading.Timer(0.0, do_stuff, args=[0])
t.start()
t.join()
Using the sched module is another possibility, but note that the sched.scheduler.run method blocks the main process until the event queue is empty. (So if the do_stuff function takes longer than a second, the next event won't run on time.) If you want nonblocking events, use threading.Timer.