I have python code that I am developing for a website that, among other things, creates an excel sheet and then converts it into a json file. I need for this code to run continuously unless it is killed by the website administrator.
To this end, I am using APscheduler.
The code runs perfectly without APscheduler but when I attempt to add the rest of the code one of two things happens; 1) It runs forever and will not stop despite using "ctrl+C" and I need to stop it using task manager or 2) It only runs once, and then it stops
Code That doesn't Stop:
from apscheduler.scheduler import Scheduler
import logging
import time
logging.basicConfig()
sched = Scheduler()
sched.start()
(...)
code to make excel sheet and json file
(...)
#sched.interval_schedule(seconds = 15)
def job():
excelapi_final()
while True:
time.sleep(10)
sched.shutdown(wait=False)
Code that stops running after one time:
from apscheduler.scheduler import Scheduler
import logging
import time
logging.basicConfig()
sched = Scheduler()
(...)
#create excel sheet and json file
(...)
#sched.interval_schedule(seconds = 15)
def job():
excelapi_final()
sched.start()
while True:
time.sleep(10)
sched.shutdown(wait=False)
I understand from other questions, a few tutorials and the documentation that sched.shutdown should allow for the code to be killed by ctrl+C - however that is not working. Any ideas? Thanks in advance!
You could use the standalone mode:
sched = Scheduler(standalone=True)
and then start the scheduler like this:
try:
sched.start()
except (KeyboardInterrupt):
logger.debug('Got SIGTERM! Terminating...')
Your corrected code should look like this:
from apscheduler.scheduler import Scheduler
import logging
import time
logging.basicConfig()
sched = Scheduler(standalone=True)
(...)
code to make excel sheet and json file
(...)
#sched.interval_schedule(seconds = 15)
def job():
excelapi_final()
try:
sched.start()
except (KeyboardInterrupt):
logger.debug('Got SIGTERM! Terminating...')
This way the program will stop when Ctrl-C is pressed
You can gracefully shut it down:
import signal
from apscheduler.scheduler import Scheduler
import logging
import time
logging.basicConfig()
sched = Scheduler()
(...)
#create excel sheet and json file
(...)
#sched.interval_schedule(seconds = 15)
def job():
excelapi_final()
sched.start()
def gracefully_exit(signum, frame):
print('Stopping...')
sched.shutdown()
signal.signal(signal.SIGINT, gracefully_exit)
signal.signal(signal.SIGTERM, gracefully_exit)
Related
Following this article I have managed to do the following example code:
import subprocess
import time
from apscheduler.schedulers.background import BackgroundScheduler
class Test:
#classmethod
def test_method(cls):
subprocess.call(['notepad.exe'])
#classmethod
def test_stop(cls, scheduler):
print(scheduler.state)
scheduler.shutdown()
scheduler.remove_all_jobs()
print(scheduler.state)
scheduler = BackgroundScheduler()
scheduler.add_job(Test.test_method, 'interval', seconds=1, id='My Job')
scheduler.start()
time.sleep(10)
print("After Sleep")
Test.test_stop(scheduler)
The problem is that after the tenth try to open "Notepad" the program just stucks and does not end.
Can someone explain what is happening and why the script does not end?
P.S. If it helps, I am runnig it on Windows.
I found the problem:
The problem is that subprocess.call() "calls" the notepad and waits for its completion.
What it menas in simple words is that the program will end when the Notepad opened from this script is closed.
I need to simulate one program 100 times, I need to have one simulation per hour (The duration of one simulation is 1 hour). From every simulation, I want to generate different json. My program must be stopped only be tapping 'exit' I try too to make that works:
from apscheduler.schedulers.blocking import BlockingScheduler
import os
def simulation():
os.system("./program")
print ("tshark -i tun0 -T ek > path/packets_one_hour.json")
scheduler = BlockingScheduler()
scheduler.add_job(some_job, 'interval', hours=1)
scheduler.start()
My problem is that my script is running without stopping. I can't generate 100 different json files. I must put quit after every hour in order to generate a new and different json. But I don't know how to do that.
Refering to this example from apscheduler docs, you could do the following.
from datetime import datetime
import os
import threading
from apscheduler.schedulers.blocking import BlockingScheduler
def json_dump():
# Put here your dump script
pass
def simulation():
os.system("./program")
t = threading.Timer(3600, json_dump)
t.start()
if __name__ == '__main__':
scheduler = BlockingScheduler()
scheduler.add_job(simulation, 'interval', hours=1, id='id_sim')
print('Press Ctrl+{0} to exit'.format('Break' if os.name == 'nt' else 'C'))
try:
scheduler.start()
except (KeyboardInterrupt, SystemExit):
pass
Honestly, haven't tried it. Let me know if it works.
I am importing Schedule into Flask. My project contains WSGI however I know little about the relationship between Flask and WSGI. Now I have three main files:
wsgi.py: Automatically generated by other tool.
app.py : I put client request here.
test.py: Used to test Schedule.
I want to start a task which is a long task when server launch. Here is the part of wsgi.py:
# -*- coding: utf-8 -*-
from threading import Thread
import test
t = Thread(target=test.job)
t.start()
if __name__ == '__main__':
...
As you see I start a thread and let the job work in it.Here is my test.py.
import schedule
def job():
schedule.every(1).seconds.do(pr)
def pr():
print("I'm working...")
My problem is that the job never starts.
I find out my problem.I never let schedule execute jobs. Now wsgi.py looks like this.
# -*- coding: utf-8 -*-
from threading import Thread
import test
schedule.every(1).seconds.do(test.job)
t = Thread(target=test.run_schedule)
t.start()
if __name__ == '__main__':
...
And test.py:
import schedule
import time
start_time = time.time()
def job():
print("I'm working..." + str(time.time() - start_time))
def run_schedule():
while True:
schedule.run_pending()
time.sleep(1)
In order to work in separate thread, I create a thread and in this thread I loop every 1ms. In loop, schedule invoke run_pending to call the job if time out (in my case it's 1s).
I'm trying to use the APScheduler library to execute a job at two certain times: 1am PST and 2am PST.
Sort of lost, I am trying to figure out what am doing wrong. Here is my code so far:
#!/usr/bin/python
import sys
import re
import urllib
import csv
import urllib2
import logging
import datetime
from apscheduler.schedulers.blocking import BlockingScheduler
sched = BlockingScheduler()
logging.basicConfig()
def main():
#Make a rest call, store the output in a csv, close the file.
...
f.close()
sched.add_job(my_job, 'date', run_date = '2015-12-15 01:00:00', args=['text'])
try:
sched.start()
print(datetime.datetime.now())
except (KeyboardInterrupt):
logger.debug('Terminating...')
print (datetime.datetime.now())
The current time is 12:16 PST, I just ran the above script and it executed and created the file prior to the job time of 12:30:30pm; this is for testing purposes.
So, I'm trying to figure out what I am doing wrong with regards to using APScheduler.
I want to improve this code, what are some things I can change?
If you want to run your code at certain times every time, you should be using the cron trigger:
sched.add_job(my_job, 'cron', hour='1,2', args=['text'])
For better debugging:
logging.basicConfig(level=logging.DEBUG)
I have a job.py which has the following code.
import datetime
import logging
import sys
import os
from apscheduler.scheduler import Scheduler
from src.extractors.pExtractor import somejob
def run_job():
start = datetime.datetime.now()
logging.debug('Proposal extraction job starting')
somejob.main()
end = datetime.datetime.now()
duration = end - start
logging.debug('job completed , took ' + str(duration.seconds) + ' seconds')
def main():
logging.basicConfig(filename='/tmp/pExtractor.log', level=logging.DEBUG,format='%(levelname)s[%(asctime)s]: %(message)s')
sched = Scheduler()
sched.start()
sched.add_interval_job(run_job, minutes=2)
if __name__ == '__main__':
main()
When I run this on the command prompt, it exits immediately:
INFO[2012-04-03 13:31:02,825]: Started thread pool with 0 core threads
and 20 maximum threads INFO[2012-04-03 13:31:02,827]: Scheduler
started INFO[2012-04-03 13:31:02,827]: Added job "run_job (trigger:
cron[minute='2'], next run at: 2012-04-03 14:02:00)" to job store
"default" INFO[2012-04-03 13:31:02,828]: Shutting down thread pool
How can I makde this run as a daemon?
Write your main() as below.
def main():
[... your_code_as_in_your_question ...]
while (True):
pass
Additionally it shouldn't hurt to consider PEP 3143.