I want to use the Advanced Python Scheduler (https://apscheduler.readthedocs.io/en/stable/) in order to scrape popular times from Google every hour (40 grocery stores in list markets, from 6:00 to 18:00) on a specific day.
My code works if I start it manually every hour, but I don't know how to write correct code in order to start the AP Scheduler. It was running for quite some time, but the resulting data frame was empty.
I was looking into the manual and other questions, but I couldn't figure out how to write the code.
from apscheduler.schedulers.blocking import BlockingScheduler
def job():
for market in markets:
data = livepopulartimes.get_populartimes_by_address(market)
current_popularity_ = pd.DataFrame([[date.today().strftime("%Y-%m-%d"),
date.today().strftime("%A"),
datetime.now().strftime("%H:%M:%S"),
pd.DataFrame.from_dict(data, orient='index').loc['current_popularity'].values[0]
]],
columns=['date','day','time','value'])
current_popularity_['market'] = market
current_popularity = current_popularity.append(current_popularity_)
sched = BlockingScheduler()
sched.add_job(
job,
trigger='cron',
hour='06-18',
start_date = '2021-02-04',
end_date = '2021-02-04'
)
sched.start()
So I corrected the dates in my code and tried to run it again.
But again, it didn't work. The resulting data frame was empty.
The code works if I start it manually (data frame is not empty).
So if anyone has an idea what is wrong with the code for the scheduler, please let me know.
Related
I have been given a python script which allows me to mute the volume of my mac at a time I set so that any podcasts I'm listening to as I fall asleep won't wake me up once I am asleep.
However, it will only act once the time given happens on that day, therefore if I try to activate it before midnight, it will mute the laptop immediately, which is not ideal because if I want to use it I have to wait for midnight to pass before I go to sleep.
#!/usr/bin/env python3
import datetime as dt
import osascript
import sys
from threading import Timer
def do_the_biz():
osascript.osascript("set volume output volume 0")
print("Night night")
today = dt.datetime.now()
dateString = today.strftime('%d-%m-%Y') + " " + sys.argv[1]
newDate = today.strptime(dateString,'%d-%m-%Y %H:%M')
delay = (newDate - dt.datetime.now()).total_seconds()
Timer(delay,do_the_biz,()).start()
So, a typical execution of this script looks like this:
$./sleep.py 04:00
and the command line will return the following once it has reached 4am, and then close the program:
Night Night
What I would like is to be able to manipulate the date so that the script will operate at the next available time of, for example, 4am - so it would essentially operate almost in the exact same way an alarm would. For this I could run an entirely different script of sleep.tomorrow.
However, in an ideal world, I would like to be able to:
have the option to have another argument where I specify today or tomorrow and the program acts accordingly;
be able to cancel the process without having to close the terminal.
I am new to python and have been having a hard time understanding the differences between the various datetime functions and the documention hasn't helped, so a brief explanation and comparison between what they input and output and how they interact would be gratefully appreciated.
I have a script that runs every 30 minutes but there is a section I just want to run on the first of the month at 2:00 am. I am using schedule in Python and I can't figure out how to set it for day 1 of the month.
month doesn't seem to be in the defined parameters of schedule to do something like schedule.every().month.at("02:00").do(job2)
Any suggestions? I am using python 2.7
Simplified code:
from safe_schedule import SafeScheduler
import time
def job():
print "I'm working...",
return
def scheduler():
# Schedule every30min routines
print 'Starting Scheduler'
scheduler = SafeScheduler()
scheduler.every(30).minutes.do(job)
#scheduler.every().month.at("02:00").do(job2)
while True:
scheduler.run_pending()
time.sleep(1)
if __name__ == '__main__':
scheduler()
The main contributor of the library discourages this sort of thing, see https://github.com/dbader/schedule/issues/73#issuecomment-167758653.
Yet, if one insists, one can schedule a daily job but run it only if it's the 1st of the month.
from datetime import date
from safe_schedule import SafeScheduler
def job2():
if date.today().day != 1:
return
# actual job body
scheduler = SafeScheduler()
scheduler.every().day.at("02:00").do(job2)
Another alternative is described in one of the issue comments https://github.com/dbader/schedule/issues/73#issuecomment-356769023.
EDIT :
They mention something about this in many issues of the project but a clean solution doesn't seem to exist yet. Also the whole project doesn't seem to be active anymore so I don't think this feature will be implemented anytime soon.
IMO you need to check manually if the current day is the first day of the month in your job in order to do this.
ORIGINAL ANSWER (which is wrong):
I've looked at the documentation and you are right, there isn't any mention of month:
https://schedule.readthedocs.io/en/stable/api.html#schedule.Scheduler
But there is an issue opened on the project that mentions just that:
https://github.com/dbader/schedule/issues/73
You can do this:
scheduler.every().day.if(lambda d,t: d.day == 1 and t.hour == 2).do(x)
I didn't see any mention of Windows Task Scheduler, here, but that could definitely work for you, if you are on Windows, of course.
https://www.digitalcitizen.life/how-create-task-basic-task-wizard
So I have a task that occurs three times a day at a certain time that needs to be executed.
I've set up code that does this using a package called Schedule
https://pypi.python.org/pypi/schedule
What I like about this is I can say, run at 3:00AM every day, or something similar.
However, the issue is, I want my other code to be running at the same time, and not be stuck in the same loop that the Schedule is running in
So right now, it looks something like:
def archerPull():
#insert code for calling archer pull here
with open("LogsForStuffPull.txt", "a") as myfile:
myfile.write("time: " + time.ctime(time.time()))
#this is code for scheduling job to do every day
def schedulingTasks(firstTime, secondTime, thirdTime, fourthTime, fivthTime):
schedule.every().day.at(firstTime).do(archerPull)
schedule.every().day.at(secondTime).do(archerPull)
schedule.every().day.at(thirdTime).do(archerPull)
schedule.every().day.at(fourthTime).do(archerPull)
schedule.every().day.at(fivthTime).do(archerPull)
schedulingTasks("13:46", "13:47", "13:48", "13:49", "13:50")
while True:
schedule.run_pending()
time.sleep(1)
So as you can see, the loop will be True forever, and therefore run the scheduler everyday. But if I want to integrate other stuff with it, will it also be looped forever?
I want the tasks to be indivitual occuring (is asynchronous the word for it)
Please help me out, thanks!
Yeah, I figured this out the same day I asked this question
I used ap-scheduler to do this, my webapp in flask is running well while the backgrounds tasks I needed work great
I'm trying to send a file after a loop completes while inside an endless loop. Everything works except the trigger for sending the file. How can I check if midnight passed when I was in the loop?
from datetime import date
while(True):
stamp = date.today()
for site in siteList:
# Long series of url requests and data processing
check = date.today()
if stamp != check: # <--- This doesn't work I don't know why
# Send today's file
As far as I can see, your code should work. Perhaps the problem lies elsewhere.
Have you tried printing stamp and check so you can see what values are being used?
Is your timezone set properly?
This is fragile and is going to cause you problems. The only way this will ever fire is if your processing actually spans midnight. This is because you update your check date every loop as well as your original date.
If you are committed to this, set the value of stamp outside the loop, and then update it inside your if block so that your send file function will always fire once per day.
However, starting a script on a machine and leaving it running for a long period of time with no supervision is just going to lead to heartache. You need some kind of job processor- I either have my webapp's work queue fire the periodic task, or if it is really external I have my Jenkins server run it- in either case I can look somewhere and see when things ran last and get notified if something goes wrong.
How can I check if midnight passed when I was in the loop?
Your code already does it if you mean the internal loop: date.today() returns only the date i.e., unless the day ends during the inner loop; stamp and check are always equal.
If you meant the outer loop then move stamp = date.today() outside of the outer loop and update stamp only on if the trigger is executed i.e., if stamp != check. btw, you should use a better names such as last_date and today. You could set last_date = None initially, to trigger the execution on the first pass.
I want it to do something once a day after the inner loop finishes
from datetime import date
last_date = None
while True: # outer loop
for site in siteList: # inner loop
"hours long computations here"
today = date.today()
if last_date is None or last_date < today:
last_date = today # jobs are executed on different days
"send today's file here"
In general, it is more flexible to separate the code that schedules a task from the code that defines the task itself e.g., you could create a cron job, to run something once a day. You could use locks, to wait for the inner loop to finish and to avoid running multiple jobs in parallel (cron doesn't protect against it). Two jobs may happen on the same day if the previous job took more than a day.
I'm working on a Python program which will be running on users computer. The program should be idle as much as possible between actions (3 per day). Actions should be at morning, afternoon and in the night.
So I want to tell the program that it has to run (to instantiate a class and do some methods from it) at three certain times every day.
I know about one way but it is an overkill I think.
while True:
time = time.now()
if time in [08:00 am, 12:30pm, 08:00 pm]:
#Do what you have to do
Can you advise me the best approach to program like this? This program will run not on my computer so I would like to manage everything as a one component.
EDIT: Martin advise me to use cron job. Is it possible to set everything in a Python code so I don't have to change anything on a clients PC?
Since you are on Windows, consider using the SCHTASKS command:
Schedules commands and programs to run periodically or at a
specific time. Adds and removes tasks from the schedule, starts and
stops tasks on demand, and displays and changes scheduled tasks.
Your command should look like:
schtasks /create /tn <TaskName> /tr <TaskRun> /sc hourly [/mo {1 - 23}] [/st <HH:MM>] [/sd <StartDate>] [/ed <EndDate>]
/mo is the repeat interval
For the complete documentation refer to:
https://technet.microsoft.com/en-ca/library/cc725744.aspx
Only if you do not have access to clients PC , you can do something like - calculate the time now using datetime library , then take the different between the next time you want to run the program and now , then sleep for that much amount of seconds using time.sleep . When you wake up , again run whatever you want , and then again do same thing calculate time between now and next time you want to run the program.
Example code -
from datetime import datetime
import time
while True:
nw = datetime.now()
nw
>> datetime.datetime(2015, 6, 22, 17, 57, 5, 668002)
t = datetime(nw.year,nw.month,nw.day,22,0,0,0) # this is where you calculate next time you want to run the program.
t
>> datetime.datetime(2015, 6, 22, 22, 0)
td = t - nw
td.seconds
>> 14574
time.sleep(14574)
#<do your stuff>
Please note in above code , >> indicates the printed value of the previous statements, and they are not part of the code.
Though I am not sure, if there are any side-effects (bad-effects) of making a process sleep for very long.
If you have access to client's PC , or can make him run a task scheduler or in unix , cronjob , consider that as the better option.