I'm trying to send a file after a loop completes while inside an endless loop. Everything works except the trigger for sending the file. How can I check if midnight passed when I was in the loop?
from datetime import date
while(True):
stamp = date.today()
for site in siteList:
# Long series of url requests and data processing
check = date.today()
if stamp != check: # <--- This doesn't work I don't know why
# Send today's file
As far as I can see, your code should work. Perhaps the problem lies elsewhere.
Have you tried printing stamp and check so you can see what values are being used?
Is your timezone set properly?
This is fragile and is going to cause you problems. The only way this will ever fire is if your processing actually spans midnight. This is because you update your check date every loop as well as your original date.
If you are committed to this, set the value of stamp outside the loop, and then update it inside your if block so that your send file function will always fire once per day.
However, starting a script on a machine and leaving it running for a long period of time with no supervision is just going to lead to heartache. You need some kind of job processor- I either have my webapp's work queue fire the periodic task, or if it is really external I have my Jenkins server run it- in either case I can look somewhere and see when things ran last and get notified if something goes wrong.
How can I check if midnight passed when I was in the loop?
Your code already does it if you mean the internal loop: date.today() returns only the date i.e., unless the day ends during the inner loop; stamp and check are always equal.
If you meant the outer loop then move stamp = date.today() outside of the outer loop and update stamp only on if the trigger is executed i.e., if stamp != check. btw, you should use a better names such as last_date and today. You could set last_date = None initially, to trigger the execution on the first pass.
I want it to do something once a day after the inner loop finishes
from datetime import date
last_date = None
while True: # outer loop
for site in siteList: # inner loop
"hours long computations here"
today = date.today()
if last_date is None or last_date < today:
last_date = today # jobs are executed on different days
"send today's file here"
In general, it is more flexible to separate the code that schedules a task from the code that defines the task itself e.g., you could create a cron job, to run something once a day. You could use locks, to wait for the inner loop to finish and to avoid running multiple jobs in parallel (cron doesn't protect against it). Two jobs may happen on the same day if the previous job took more than a day.
Related
In my python Script I need to Stop/Skip the execution of a particular function for a day if the required condition gets false even for a single time. That means if a single error arises while the execution of particular function in my python script ...I want that function to become non-callable for a given duration of time and instead use alternate function for that particular duration. When the duration ends the function becomes again callable
Please share your code next time to receive the proper help...
From what I understood, you need to start storing error times somehow.
if not your_required_condition:
error_time = datetime.datetime.now()
if (datetime.datetime.now() - error_time).days >= 1:
your_other_function()
else:
your_main_function()
You need to make your code check if the condition is verified each time...
I need to write a python program that lets me set reminders for specific times, eg 'remember to take bins out at 2pm', but I can only work out setting a reminder for a certain length of time, not for a given time. I also need to be able to set multiple reminders for multiple times.
Any help would be much appreciated :)
This looks like a homework assignment, so you need to write the code yourself.
You know what time it is now. You know when 2pm is. How much time is there between now and 2pm? Sleep for that long.
Keep a list of all pending alarms. Find the earliest alarm. Remove it from the list. Sleep until that alarm happens. Repeat
You'll probably find Step 2 easier if you use an appropriate data structure like heapq or PriorityQueue. But if the number of alarms is small, a list should do just fine.
The following checks for any new reminders every seconds,
although, after reading Frank's answer, that would be a better solution,
and the best solution is to not use Python at all, and let the operating system manage this by creating a cron job or on Windows a scheduled task.
reminders = [
# Put all of your reminders here
('2021-11-16 02:44:00', 'Take out the garbage'),
('2021-11-17 04:22:00', 'Another reminder')
]
from datetime import datetime
import time
# For performance reasons it's best to perform whatever computations we can before we go into our infinite loop
# In this case let's calculate all of the timestamps
reminders2 = {datetime.fromisoformat(reminder[0]).timestamp(): reminder[1] for reminder in reminders}
while True:
now = time.time()
for timestamp, reminder_msg in reminders2.items():
if timestamp < now:
print(reminder_msg)
del reminders2[timestamp]
# we're not in any hurry, instead of worrying about the consequences of deleting something from the same dictionary we are iterating over
# we can just break and wait for the next go around of the while loop to finish checking the remaining reminders
break
time.sleep(1) # in seconds
I want to use the Advanced Python Scheduler (https://apscheduler.readthedocs.io/en/stable/) in order to scrape popular times from Google every hour (40 grocery stores in list markets, from 6:00 to 18:00) on a specific day.
My code works if I start it manually every hour, but I don't know how to write correct code in order to start the AP Scheduler. It was running for quite some time, but the resulting data frame was empty.
I was looking into the manual and other questions, but I couldn't figure out how to write the code.
from apscheduler.schedulers.blocking import BlockingScheduler
def job():
for market in markets:
data = livepopulartimes.get_populartimes_by_address(market)
current_popularity_ = pd.DataFrame([[date.today().strftime("%Y-%m-%d"),
date.today().strftime("%A"),
datetime.now().strftime("%H:%M:%S"),
pd.DataFrame.from_dict(data, orient='index').loc['current_popularity'].values[0]
]],
columns=['date','day','time','value'])
current_popularity_['market'] = market
current_popularity = current_popularity.append(current_popularity_)
sched = BlockingScheduler()
sched.add_job(
job,
trigger='cron',
hour='06-18',
start_date = '2021-02-04',
end_date = '2021-02-04'
)
sched.start()
So I corrected the dates in my code and tried to run it again.
But again, it didn't work. The resulting data frame was empty.
The code works if I start it manually (data frame is not empty).
So if anyone has an idea what is wrong with the code for the scheduler, please let me know.
I'm seeing this weird issue in my Python code, and not quite sure what is going on. So quick backstory on the code, I've got a function that my UI script calls on startup that runs in a separate thread. This thread is intended to call a function in a separate module that checks to see if the stored timestamp in a database is older than x_value, and does stuff if that's true.
So here's the function (I can post more code later if needed, but this function is where the issue is happening):
def tableUpdateCheck(sensor_manager):
last_check = datetime.now()
print(f"last_check: {last_check.strftime('%m/%d/%Y_%H:%M:%S')}")
while RUN_TABLE_UPDATE_THREAD:
print(f"Now: {datetime.now().strftime('%H:%M:%S')}\tLast check time: {last_check.strftime('%H:%M:%S')}")
if datetime.now() - last_check > timedelta(seconds=60):
print('Running update table update check...')
sensor_manager.updateTables()
last_check = datetime.now()
time.sleep(5)
if __name__ == '__main__':
sensor_mgr = SensorManager()
table_thread = Thread(target=tableUpdateCheck, args=[sensor_mgr])
table_thread.start()
# Start webserver stuff
The resulting output looks like this:
Now: 17:49:13 Last check time: 17:49:13
Now: 17:49:18 Last check time: 17:49:10
Now: 17:49:23 Last check time: 17:49:13
Now: 17:49:28 Last check time: 17:49:10
Now: 17:49:33 Last check time: 17:49:13
Now: 17:49:38 Last check time: 17:49:10
And continues to do this until the if statement runs, and "last_check" goes back to fluctuating between two values.
Any thoughts on why this might be? It doesn't stop the overall program from ultimately doing what it's supposed to do, but it's super annoying.
So for anyone who might stumble across this in the future, it turns out that "app.run_server(debug=True)" in Flask apps causes an extra thread to run at start. I'm not 100% why this is the case, but that seems to be what's causing the problem I'm describing.
However, using a production Flask app, or simply setting the "debug" argument to False on a development server will not start the multiple threads, and this issue goes away.
I have encountered a very weird situation in Python.
I have a while True: loop in a script that constantly grabs real-time data. The data is only meant to be grabbed during business hours, therefore I have a helper function in a module called lib/date.py.
This is the function:
def isTime(utc_now=datetime.datetime.utcnow()):
'''
9:00 AM UTC - 5:00 PM UTC Mon - Fri.
The parameter utc_now can be used to specify a specific
datetime to be checked instead of the utcnow().
'''
week_day = utc_now.weekday()
# no good on Saturdays and Sundays
if 5 == week_day or 6 == week_day:
return False
now_int = (utc_now.hour * 60) + utc_now.minute
if (17 * 60) >= now_int >= (9 * 60):
return True
return False
I realize that the way times are checked is an overkill, but this is a simplified version of my function. My real function also includes a bunch of time zone conversions which make the above necessary.
This is my main script (simplified):
def main():
while True:
do_ten_seconds()
def do_ten_seconds():
log.info("alive")
for i in xrange(10):
if isTime():
log.info("its time")
# do data grabbing
sleep(1)
This works perfectly during the week, stopping every week night and continuing the next morning. However, on the weekend, it does not "recover" from the long break. In other words, isTime never returns True Monday morning.
I checked the log, and the script just prints "alive" every 10 seconds, but the data grabbing part never happens (and "its time" is never printed). Hence, I am assuming this means isTime is returning False.
Could it be that because the function is called so frequently over the weekend, and because it returns False so many times in a row, Python caches the results and never recovers from it?
When I spawn the same instance of code while the main script is still running in the faulty mode where isTime is returning False, the new instance gets True from isTime and begins to work perfectly.
What is going on?
The first time you import date.py, the following line is run once:
def isTime(utc_now=datetime.datetime.utcnow()):
When it's run, the default value for utc_now is set to the time at that instant, and is never recomputed.
Default arguments are calculated at compile time. Make the default value None, and check for that using is None.