I want to run a function at the start of every minute, without it lagging over time. Using time.sleep(60) eventually lags.
while True:
now = datetime.datetime.now().second
if now == 0:
print(datetime.datetime.now())
The function doesn't take a minute to run so as long as it runs a the beginning it should be fine, I'm not sure if this code is resource-efficient, as its checking every millisecond or so and even if it drifts the if function should correct it.
Repeat scheduling shouldn't really be done in python, especially by using time.sleep. The best way would be to get your OS to schedule running the script, using something like cron if you're on Linux or Task Scheduler if you're on Windows
Assuming that you've examined and discarded operating-based solutions such as cron or Windows Scheduled Tasks, what you suggest will work but you're right in that it's CPU intensive. You would be better off sleeping for one second after each check so that:
It's less resource intensive; and, more importantly
It doesn't execute multiple times per at the start of each minute if the job takes less than a second.
In fact, you could sleep for even longer immediately after the payload by checking how long to the next minute, and use the minute to decide in case the sleep takes you into a second that isn't zero. Something like this may be a good start:
# Ensure we do it quickly first time.
lastMinute = datetime.datetime.now().minute - 1
# Loop forever.
while True:
# Get current time, do payload if new minute.
thisTime = datetime.datetime.now()
if thisTime.minute != lastMinute:
doPayload()
lastMinute = thisTime.minute
# Try to get close to hh:mm:55 (slow mode).
# If payload took more than 55s, just go
# straight to fast mode.
afterTime = datetime.datetime.now()
if afterTime.minute == thisTime.minute:
if afterTime.second < 55:
time.sleep (55 - afterTime.second)
# After hh:mm:55, check every second (fast mode).
time.sleep(1)
Related
I have a program running on a Raspberry Pi, and I want to pull some data from a thermometer every 15 minutes at 0, 15, 30 and 45 minutes past the hour.
I have tried this using a while loop, I previously used time.sleep(900) effectively, but this sometimes drifted away from 0, 15, 30 and 45 minutes past the hour.
At the moment I currently have this;
from datetime import datetime
def run(condition):
while condition == True:
if (datetime.now().minute == (0 or 15 or 30 or 45)):
#perform some task
temperature_store()
For sake of simplicity I have not got into what temperature_store() does, but it reads the temperature from a sensor plugged into the Pi and then prints it.
I want temperature_store() to occur every 15 minutes, but currently, it is happening every second.
I am aware that it is probably because I have the logic/syntax of the while loop wrong, but I cannot figure it out. (Do not have much experience with python scripts and delays in time).
There's two ways to do this: the 'easy' way, and the stable way.
The easy way is simply to do the following:
from datetime import datetime
from time import sleep
def run(condition):
while datetime.now().minute not in {0, 15, 30, 45}: # Wait 1 second until we are synced up with the 'every 15 minutes' clock
sleep(1)
def task():
# Your task goes here
# Functionised because we need to call it twice
temperature_store()
task()
while condition == True:
sleep(60*15) # Wait for 15 minutes
task()
This essentially waits until we are synced up with the correct minute, then executes it, and simply waits 15 minutes before looping. Use it if you wish, it's the simplest way in pure Python. The issues with this are countless, however:
It syncs up with the minute, not the second
It's machine dependent, and may give incorrect readings on some machines
It needs to run continuously!
The second method is to use cron-jobs, as suggested in the comments. This is superior because:
It uses system-level events, not timers, so we can assure the best possible accuracy
Since we're not waiting, there's no room for error
It only runs the function once it gets the aforementioned event setter.
So, to use, simply (assuming you're on Linux):
from crontab import CronTab
cron = CronTab(user='username') # Initialise a new CronTab instance
job = cron.new(command='python yourfile.py') # create a new task
job.minute.on(0, 15, 30, 45) # Define that it should be on every 0th, 15th, 30th and 45th minute
cron.write() # 'Start' the task (i.e trigger the cron-job, but through the Python library instead
(obviously, configure username appropriately)
In yourfile.py, in the same path, simply put your code for temperature_store().
I hope this helps. Obviously, go with the first methods or even the suggestions in the comments if you prefer, but I just felt that the entire loop structure was a bit too fragile, especially on a Raspberry Pi. This should hopefully be something more stable and scalable, should you want to hook up other IoT things to your Pi.
I did the module test in case of the program that prints seconds every 15 seconds.
The below code does not contain sleep().
You can change second to minute in below code to do tasks every 15 minutes.
from datetime import datetime
while True:
a = datetime.now().second
if (a % 15) == 0: # every 15 seconds
print(a) # ---- To do something every 15 seconds ---- #
while True: # discard any milliseconds or duplicated 15 seconds
a = datetime.now().second
if (a % 15) is not 0:
break
But I think that cron or any other schduler modules are good choices.
I have come up with the following answer, using some of the logic from #geza-kerecsenyi 's answer
def run():
first = True
while first == True:
second = True
while second == True:
if datetime.now().minute == 0 or datetime.now().minute ==15 or datetime.now().minute ==30 or datetime.now().minute == 45:
action()
sleep(899)
I am not sure which of these is better in terms of CPU or which is more effective to run, but the logic seems sound on both.
[#geza-kerecsenyi will mark your answer as correct]
I write a lot of code that relies on precise periodic method calls. I've been using Python's futures library to submit calls onto the runtime's thread pool and sleeping between calls in a loop:
executor = ThreadPoolExecutor(max_workers=cpu_count())
def remote_call():
# make a synchronous bunch of HTTP requests
def loop():
while True:
# do work here
executor.submit(remote_call)
time.sleep(60*5)
However, I've noticed that this implementation introduces some drift after a long duration of running (e.g. I've run this code for about 10 hours and noticed about 7 seconds of drift). For my work I need this to run on the exact second, and millisecond would be even better. Some folks have pointed me to asyncio ("Fire and forget" python async/await), but I have not been able to get this working in Python 2.7.
I'm not looking for a hack. What I really want is something akin to Go's time.Tick or Netty's HashedWheelTimer.
Nothing like that comes with Python. You'd need to manually adjust your sleep times to account for time spent working.
You could fold that into an iterator, much like the channel of Go's time.Tick:
import itertools
import time
import timeit
def tick(interval, initial_wait=False):
# time.perf_counter would probably be more appropriate on Python 3
start = timeit.default_timer()
if not initial_wait:
# yield immediately instead of sleeping
yield
for i in itertools.count(1):
time.sleep(start + i*interval - timeit.default_timer())
yield
for _ in tick(300):
# Will execute every 5 minutes, accounting for time spent in the loop body.
do_stuff()
Note that the above ticker starts ticking when you start iterating, rather than when you call tick, which matters if you try to start a ticker and save it for later. Also, it doesn't send the time, and it won't drop ticks if the receiver is slow. You can adjust all that on your own if you want.
How do I have a part of python script(only a method, the whole script runs in 24/7) run everyday at a set-time, exactly at every 20th minutes? Like 12:20, 12:40, 13:00 in every hour.
I can not use cron, I tried periodic execution but that is not as accurate as I would... It depends from the script starting time.
Module schedule may be useful for this. See answer to
How do I get a Cron like scheduler in Python? for details.
You can either put calling this method in a loop, which would sleep for some time
from time import sleep
while True:
sleep(1200)
my_function()
and be triggered once in a while, you could use datetime to compare current timestamp and set next executions.
import datetime
function_executed = False
trigger_time = datetime.datetime.now()
def set_trigger_time():
global function executed = False
return datetime.datetime.now() + datetime.timedelta(minutes=20)
while True:
if function_executed:
triggertime = set_trigger_time()
if datetime.datetime.now() == triggertime:
function_executed = True
my_function()
I think however making a system call the script would be a nicer solution.
Use for example redis for that and rq-scheduler package. You can schedule tasks with specific time. So you can run first script, save to the variable starting time, calculate starting time + 20 mins and if your current script will end, at the end you will push another, the same task with proper time.
I'm scraping (extracting) data from a certain website. The data contains two values that I need, namely (grid) frequency value and time.
The data on the website is being updated every second. I'd like to continuously save these values (append them) into a list or a tuple using python. To do that I tried using schedule library. The following job schedule commands run the data scraping function (socket_freq) every second.
import schedule
schedule.every(1).seconds.do(socket_freq)
while True:
schedule.run_pending()
I'm facing two problems:
I don't know how to restrict the schedule to run during a chosen time interval. For example, i'd like to run it for 5 or 10 minutes. how do I define that? I mean how to I tell the schedule to stop after a certain time.
if I run this code and stop it after few seconds (using break), then I often get multiple entries, for example here is one result, where the first list[ ] in the tuple refers to the time value and the second list[ ] is the values of frequency:
out:
(['19:27:02','19:27:02','19:27:02','19:27:03','19:27:03','19:27:03','19:27:03','19:27:03','19:27:03','19:27:03','19:27:04','19:27:04','19:27:04', ...],
['50.020','50.020','50.020','50.018','50.018','50.018','50.018','50.018','50.018','50.018','50.017','50.017','50.017'...])
As you can see, the time variable is entered (appended) multiple times, although I used a schedule that runs every 1 second. What i'd actually would expect to retrieve is:
out:
(['19:27:02','19:27:03','19:27:04'],['50.020','50.018','50.017'])
Does anybody know how to solve these problems?
Thanks!
(I'm using python 2.7.9)
Ok, so here's how I would tackle these problems:
Try to obtain a timestamp at the start of your program and then simply check if it has been working long enough each time you execute piece of code you are scheduling.
Use time.sleep() to put your program to sleep for a period of time.
Check my example below:
import schedule
import datetime
import time
# Obtain current time
start = datetime.datetime.now()
# Simple callable for example
class DummyClock:
def __call__(self):
print datetime.datetime.now()
schedule.every(1).seconds.do(DummyClock())
while True:
schedule.run_pending()
# 5 minutes == 300 seconds
if (datetime.datetime.now() - start).seconds >= 300:
break
# And here we halt execution for a second
time.sleep(1)
All refactoring is welcome
I have a set of instructions, say {I} and I would like to perform this set {I}
at predefined time for instance each minute.
I'm not asking how to insert a delay of 1 minutes between to successive executions of
the set {I}, I want to start the instructions {I} each minute independently of the time of execution of {I}.
If I inderstand the following code
import time
while True:
{I}
time.sleep(60)
would simply insert a delay of 60 secs between the end of the execution of {I} and the following one. Is it true? Instead I would like that the set of instructions {I} starts each minute (for instance at 9.00 am, 9.01 am, 9.02 am, etc).
Is it possible to perform such a task inside python, or is it preferable to write a script with {I} that I execute each minutes, for instance, with Crontab?
Thank you in advance
Looks like signal.alarm and signal.signal(signal.SIGALRM, handler) should help you.
If you don't need finer resolution than a minute, cron would be the easiest option. Otherwise you'd end up re-writing something like it.
If you need intervals shorter than a minute, you might consider "timeouts" from the glib library. It has Python bindings. The timeout should then probably start the task in a separate process.
Something like APScheduler might meet your needs.
I'm sure there are other similar packages out there as well.
Chances are, you'd have to instantiate separate threads for every instruction to be run concurrently, and simply dispatch them in your delayed while loop.
You could spawn a thread every second using threading.Timer:
import threading
import time
def do_stuff(count):
print(count)
if c < 10: # Let's build in some way to quit
t = threading.Timer(1.0, do_stuff, args=[count+1])
t.start()
t = threading.Timer(0.0, do_stuff, args=[0])
t.start()
t.join()
Using the sched module is another possibility, but note that the sched.scheduler.run method blocks the main process until the event queue is empty. (So if the do_stuff function takes longer than a second, the next event won't run on time.) If you want nonblocking events, use threading.Timer.