I have a python program that I want to run every 10 seconds, just like cron job. I cannot use sleep in a loop because the time interval would become uncertain. The way I am doing it now is like this:
interval = 10.0
next = time.time()
while True:
now = time.time()
if now < next:
time.sleep(next - now)
t = Thread(target=control_lights,)
t.start()# start a thread
next += interval
It generates a new thread that executes the contro_lights function. The problem is that as time goes, the number of python process grows and takes memory/CPU. Is there any good way to do this? Thanks a lot
may be try use supervisord or god for this script? It is very simple to use and to control a number of you'r processes on UNIX-like operating system
Take a look at a program called The Fat Controller which is a scheduler similar to CRON but has many more options. The interval can be measured from the end of the previous run (like a for loop) or regularly every x seconds, which I think is what you want. Particularly useful in this case is that you can tell The Fat Controller what to do if one of the processes takes longer than x seconds:
run a new instance anyway (increase parallel processes up to a specified maximum)
wait for the previous one to finish
kill the previous one and start a new one
There should be plenty of information in the documentation on how to get it set up.
You can run a cron job every 10 seconds, just set the second param to '0/10'. It will run on 0, 10, 20 etc
#run every 10 seconds from mon-fri, between 8-17
CronTrigger(day_of_week='mon-fri', hour='8-17', second='0/10')
Related
I've got a Python loop that should run every minute, do some data processing and sleep until the next minute is up. However the processing takes variable amount of time, sometimes it's close to zero when there is not much to do, sometimes it takes even 10 or 20 seconds.
To compensate for that I measure the time it takes to run the processing like this:
while True:
time_start = time.time()
do_something() # <-- This takes unknown time
time_spent = time.time() - time_start
time.sleep(60 - time_spent)
It kind of works but over a couple of days it still drifts away by a number of seconds. I guess it happens when the computer (small Raspberry Pi) is busy and delays the start of the loop, then it all starts slipping away.
I don't need do_something() executed exactly every minute, so no need for real-time OS or anything like that, but I don't want one delayed start affect all the subsequent ones either.
Is there some kind of scheduler that can run my function at a predefined rate? Or some more clever way to compensate for the occasional delays?
Playing with the loop a little this seems to work quite well. The trick is to record the start time once before the loop starts, not on every iteration. That way one delayed start won't affect any future ones.
rate_sec = 60
time_start = time.time()
while True:
print("{}".format(datetime.now()))
# Emulate processing time 0s to 20s
time.sleep(random.randint(0, 20))
# Sleep until the next 'rate_sec' multiple
delay = rate_sec - (time.time() - time_start) % rate_sec
time.sleep(delay)
Is sleeping a pre-requisite of your project? I mean, you don't need to have your processing blocked if you want to run the task every ~1 minute.
Since you are on a Raspberry Pi, you can (and probably should) use crontab.
This will give you the most flexibility and allow you to don't have the computer sleeping idle.
To do this, open your cron tab with
crontab -e
And add the following entry
* * * * * /usr/bin/env python3 /path/to/script.py
My script has to run over a day and its core cycle runs 2-3 times per a minute. I used multiprocessing to give a command simultaneously and each of them will be terminated/join within one cycle.
But in reality I found the software end up out of swap memory or computer freezing situation, I guess this is caused by accumulated processes. I can see on another session while running program, python PID abnormally increasing by time. So I just assume this must be something process thing. What I don't understand is how it happens though I made sure each cycle's process has to be finished on that cycle before proceed the next one.
so I am guessing, actual computing needs more time to progress 'terminate()/join()' job, so I should not "reuse" same object name. Is this proper guessing or is there other possibility?
def function(a,b):
try:
#do stuff # audio / serial things
except:
return
flag_for_2nd_cycle=0
for i in range (1500): # main for running long time
#do something
if flag_for_2nd_cycle==1:
while my_process.is_alive():
if (timecondition) < 30: # kill process if it still alive
my_process.terminate()
my_process.join()
flag_for_2nd_cycle=1
my_process=multiprocessing.process(target=function, args=[c,d])
my_process.start()
#do something and other process jobs going on, for example
my_process2 = multiprocessing.process() ##*stuff
my_process2.terminate()
my_process2.join()
Based on your comment, you are controlling three projectors over serial ports.
The simplest way to do that would be to simply open three serial connections (using pySerial). Then run a loop where you check for available data each of the connections and if so, read and process it. Then you send commands to each of the projectors in turn.
Depending on the speed of the serial link you might not need more than this.
I wrote a python script which makes calculation at every hour. I run this script with crontab scheduled for every hour. But there is one more thing to do;
Additionally, I should make calculation once a day by using the results evaluated at every hour. In this context, I defined a thread function which checks the current time is equal to the specified time (15:00 PM, once a day ). If it is, thread function is called and calculation made.
What I wanna ask here is; is this approach applicable ? I mean, running the first script at every hour using crontab, and calling the second function using thread function once a day.
Is there any other way of doing this ?
You may use python library sched.
It efficient way to schelude your operations.
You could do a script which runs all the time and just sleeps for one hour.
import time
hours_ran = 0
while True:
# your stuff here
time.sleep(3600)
hours_ran += 1
if hours_ran == 24:
# calculate stuff
hours_ran = 0
This is not necessarily better tho...
You can setup systemd to manage this process.
I am using python's sched module to run a task periodically, and I think I have come across a bug.
I find that it relies on the time of the system on which the python script is run. For example, let's say that I want to run a task every 5 seconds. If I forward the system time, the scheduled task will run as expected. However, if I rewind the system time to, say 1 day, then the next scheduled task will run in 5 seconds + 1 day.
If you run the script below and then change your system time by a few days back, then you can reproduce the issue. The problem can be reproduced on Linux and Windows.
import sched
import time
import threading
period = 5
scheduler = sched.scheduler(time.time, time.sleep)
def check_scheduler():
print time.time()
scheduler.enter(period, 1, check_scheduler, ())
if __name__ == '__main__':
print time.time()
scheduler.enter(period, 1, check_scheduler, ())
thread = threading.Thread(target=scheduler.run)
thread.start()
thread.join()
exit(0)
Anyone has any python solution around this problem?
From the sched documentation:
class sched.scheduler(timefunc, delayfunc)
The scheduler class defines a generic interface to scheduling events. It needs two functions to actually deal with the “outside
world” — timefunc should be callable without arguments, and return a
number (the “time”, in any units whatsoever). The delayfunc function
should be callable with one argument, compatible with the output of
timefunc, and should delay that many time units. delayfunc will also
be called with the argument 0 after each event is run to allow other
threads an opportunity to run in multi-threaded applications.
The problem you have is that your code uses time.time() as timefunc, whose return value (when called without arguments) is the current system time and is thus affected by re-winding the system clock.
To make your code immune to system time changes you'd need to provide a timefunc which doesn't depend on the system time, start/current timestamps, etc.
You can write your own function, for example one returning the number of seconds since your process is started, which you'd have to actually count in your code (i.e. don't compute it based on timestamp deltas). The time.clock() function might help, if it's based on CPU time counters, but I'm not sure if that's true or not.
There is a significant difference between my question and the given one. If I implement sleep as given on the link (which some people think my question is a duplicate of another one) then my whole app will hang for certain time. On the other hand I am looking for a scheduler so that my app will not hang but just after a certain period of time a .wav file will run. Meanwhile I would be able to do anything using my app. Hope that makes sense.
I am going to build an alarm clock. I think I can do this according to this Algorithm...
take current time using time.clock()
compare current time with the given time.
if current time is not given time then continue taking current time and compare whether current time is the given time or not.
if current time is the given time then play the .wav file.
The problem with this program is it will continuously run until the given time. So I am looking for better idea. Let's say if there is any Python module/class/function which can play a sound file on a given time. Is there any Python module/class/function which can wake up on a given time? Or my algorithm is used usually for all alarm clocks?
As far as I know there is not a good Python module to do this. What you are looking for though is a cron job. It allows you to schedule specific scripts to run at certain times. So your Python script would end up just being the code to play the .wav, and then you would need to create a cron job to tell your computer to execute that script at a certain time each day.
Have a look at the sched module.
Here's an example on how to use it:
import sched, time, datetime
def print_time():
print("The time is now: {}".format(datetime.datetime.now()))
# Run 10 seconds from now
when = time.time() + 10
# Create the scheduler
s = sched.scheduler(time.time)
s.enterabs(when, 1, print_time)
# Run the scheduler
print_time()
print("Executing s.run()")
s.run()
print("s.run() exited")
The time is now: 2015-06-04 11:52:11.510234
Executing s.run()
The time is now: 2015-06-04 11:52:21.512534
s.run() exited