I'm seeing this weird issue in my Python code, and not quite sure what is going on. So quick backstory on the code, I've got a function that my UI script calls on startup that runs in a separate thread. This thread is intended to call a function in a separate module that checks to see if the stored timestamp in a database is older than x_value, and does stuff if that's true.
So here's the function (I can post more code later if needed, but this function is where the issue is happening):
def tableUpdateCheck(sensor_manager):
last_check = datetime.now()
print(f"last_check: {last_check.strftime('%m/%d/%Y_%H:%M:%S')}")
while RUN_TABLE_UPDATE_THREAD:
print(f"Now: {datetime.now().strftime('%H:%M:%S')}\tLast check time: {last_check.strftime('%H:%M:%S')}")
if datetime.now() - last_check > timedelta(seconds=60):
print('Running update table update check...')
sensor_manager.updateTables()
last_check = datetime.now()
time.sleep(5)
if __name__ == '__main__':
sensor_mgr = SensorManager()
table_thread = Thread(target=tableUpdateCheck, args=[sensor_mgr])
table_thread.start()
# Start webserver stuff
The resulting output looks like this:
Now: 17:49:13 Last check time: 17:49:13
Now: 17:49:18 Last check time: 17:49:10
Now: 17:49:23 Last check time: 17:49:13
Now: 17:49:28 Last check time: 17:49:10
Now: 17:49:33 Last check time: 17:49:13
Now: 17:49:38 Last check time: 17:49:10
And continues to do this until the if statement runs, and "last_check" goes back to fluctuating between two values.
Any thoughts on why this might be? It doesn't stop the overall program from ultimately doing what it's supposed to do, but it's super annoying.
So for anyone who might stumble across this in the future, it turns out that "app.run_server(debug=True)" in Flask apps causes an extra thread to run at start. I'm not 100% why this is the case, but that seems to be what's causing the problem I'm describing.
However, using a production Flask app, or simply setting the "debug" argument to False on a development server will not start the multiple threads, and this issue goes away.
Related
In my python Script I need to Stop/Skip the execution of a particular function for a day if the required condition gets false even for a single time. That means if a single error arises while the execution of particular function in my python script ...I want that function to become non-callable for a given duration of time and instead use alternate function for that particular duration. When the duration ends the function becomes again callable
Please share your code next time to receive the proper help...
From what I understood, you need to start storing error times somehow.
if not your_required_condition:
error_time = datetime.datetime.now()
if (datetime.datetime.now() - error_time).days >= 1:
your_other_function()
else:
your_main_function()
You need to make your code check if the condition is verified each time...
I'm really new to programming in general and very inexperienced, and I'm learning python as I think it's more simple than other languages. Anyway, I'm trying to use Flask-Ask with ngrok to program an Alexa skill to check data online (which changes a couple of times per hour). The script takes four different numbers (from a different URL) and organizes it into a dictionary, and uses Selenium and phantomjs to access the data.
Obviously, this exceeds the 8-10 second maximum runtime for an intent before Alexa decides that it's taken too long and returns an error message (I know its timing out as ngrok and the python log would show if an actual error occurred, and it invariably occurs after 8-10 seconds even though after 8-10 seconds it should be in the middle of the script). I've read that I could just reprompt it, but I don't know how and that would only give me 8-10 more seconds, and the script usually takes about 25 seconds just to get the data from the internet (and then maybe a second to turn it into a dictionary).
I tried putting the getData function right after the intent that runs when the Alexa skill is first invoked, but it only runs when I initialize my local server and just holds the data for every new Alexa session. Because the data changes frequently, I want it to perform the function every time I start a new session for the skill with Alexa.
So, I decided just to outsource the function that actually gets the data to another script, and make that other script run constantly in a loop. Here's the code I used.
import time
def getData():
username = '' #username hidden for anonymity
password = '' #password hidden for anonymity
browser = webdriver.PhantomJS(executable_path='/usr/local/bin/phantomjs')
browser.get("https://gradebook.com") #actual website name changed
browser.find_element_by_name("username").clear()
browser.find_element_by_name("username").send_keys(username)
browser.find_element_by_name("password").clear()
browser.find_element_by_name("password").send_keys(password)
browser.find_element_by_name("password").send_keys(Keys.RETURN)
global currentgrades
currentgrades = []
gradeids = ['2018202', '2018185', '2018223', '2018626', '2018473', '2018871', '2018886']
for x in range(0, len(gradeids)):
try:
gradeurl = "https://www.gradebook.com/grades/"
browser.get(gradeurl)
grade = browser.find_element_by_id("currentStudentGrade[]").get_attribute('innerHTML').encode('utf8')[0:3]
if grade[2] != "%":
grade = browser.find_element_by_id("currentStudentGrade[]").get_attribute('innerHTML').encode('utf8')[0:4]
if grade[1] == "%":
grade = browser.find_element_by_id("currentStudentGrade[]").get_attribute('innerHTML').encode('utf8')[0:1]
currentgrades.append(grade)
except Exception:
currentgrades.append('No assignments found')
continue
dictionary = {"class1": currentgrades[0], "class2": currentgrades[1], "class3": currentgrades[2], "class4": currentgrades[3], "class5": currentgrades[4], "class6": currentgrades[5], "class7": currentgrades[6]}
return dictionary
def run():
dictionary = getData()
time.sleep(60)
That script runs constantly and does what I want, but then in my other script, I don't know how to just call the dictionary variable. When I use
from getdata.py import dictionary
in the Flask-ask script it just runs the loop and constantly gets the data. I just want the Flask-ask script to take the variable defined in the "run" function and then use it without running any of the actual scripts defined in the getdata script, which have already run and gotten the correct data. If it matters, both scripts are running in Terminal on a MacBook.
Is there any way to do what I'm asking about, or are there any easier workarounds? Any and all help is appreciated!
It sounds like you want to import the function, so you can run it; rather than importing the dictionary.
try deleting the run function and then in your other script
from getdata import getData
Then each time you write getData() it will run your code and get a new up-to-date dictionary.
Is this what you were asking about?
This issue has been resolved.
As for the original question, I didn't figure out how to make it just import the dictionary instead of first running the function to generate the dictionary. Furthermore, I realized there had to be a more practical solution than constantly running a script like that, and even then not getting brand new data.
My solution was to make the script that gets the data start running at the same time as the launch function. Here was the final script for the first intent (the rest of it remained the same):
#ask.intent("start_skill")
def start_skill():
welcome_message = 'What is the password?'
thread = threading.Thread(target=getData, args=())
thread.daemon = True
thread.start()
return question(welcome_message)
def getData():
#script to get data here
#other intents and rest of script here
By design, the skill requested a numeric passcode to make sure I was the one using it before it was willing to read the data (which was probably pointless, but this skill is at least as much for my own educational reasons as for practical reasons, so, for the extra practice, I wanted this to have as many features as I could possibly justify). So, by the time you would actually be able to ask for the data, the script to get the data will have finished running (I have tested this and it seems to work without fail).
So I have a task that occurs three times a day at a certain time that needs to be executed.
I've set up code that does this using a package called Schedule
https://pypi.python.org/pypi/schedule
What I like about this is I can say, run at 3:00AM every day, or something similar.
However, the issue is, I want my other code to be running at the same time, and not be stuck in the same loop that the Schedule is running in
So right now, it looks something like:
def archerPull():
#insert code for calling archer pull here
with open("LogsForStuffPull.txt", "a") as myfile:
myfile.write("time: " + time.ctime(time.time()))
#this is code for scheduling job to do every day
def schedulingTasks(firstTime, secondTime, thirdTime, fourthTime, fivthTime):
schedule.every().day.at(firstTime).do(archerPull)
schedule.every().day.at(secondTime).do(archerPull)
schedule.every().day.at(thirdTime).do(archerPull)
schedule.every().day.at(fourthTime).do(archerPull)
schedule.every().day.at(fivthTime).do(archerPull)
schedulingTasks("13:46", "13:47", "13:48", "13:49", "13:50")
while True:
schedule.run_pending()
time.sleep(1)
So as you can see, the loop will be True forever, and therefore run the scheduler everyday. But if I want to integrate other stuff with it, will it also be looped forever?
I want the tasks to be indivitual occuring (is asynchronous the word for it)
Please help me out, thanks!
Yeah, I figured this out the same day I asked this question
I used ap-scheduler to do this, my webapp in flask is running well while the backgrounds tasks I needed work great
I'm trying to send a file after a loop completes while inside an endless loop. Everything works except the trigger for sending the file. How can I check if midnight passed when I was in the loop?
from datetime import date
while(True):
stamp = date.today()
for site in siteList:
# Long series of url requests and data processing
check = date.today()
if stamp != check: # <--- This doesn't work I don't know why
# Send today's file
As far as I can see, your code should work. Perhaps the problem lies elsewhere.
Have you tried printing stamp and check so you can see what values are being used?
Is your timezone set properly?
This is fragile and is going to cause you problems. The only way this will ever fire is if your processing actually spans midnight. This is because you update your check date every loop as well as your original date.
If you are committed to this, set the value of stamp outside the loop, and then update it inside your if block so that your send file function will always fire once per day.
However, starting a script on a machine and leaving it running for a long period of time with no supervision is just going to lead to heartache. You need some kind of job processor- I either have my webapp's work queue fire the periodic task, or if it is really external I have my Jenkins server run it- in either case I can look somewhere and see when things ran last and get notified if something goes wrong.
How can I check if midnight passed when I was in the loop?
Your code already does it if you mean the internal loop: date.today() returns only the date i.e., unless the day ends during the inner loop; stamp and check are always equal.
If you meant the outer loop then move stamp = date.today() outside of the outer loop and update stamp only on if the trigger is executed i.e., if stamp != check. btw, you should use a better names such as last_date and today. You could set last_date = None initially, to trigger the execution on the first pass.
I want it to do something once a day after the inner loop finishes
from datetime import date
last_date = None
while True: # outer loop
for site in siteList: # inner loop
"hours long computations here"
today = date.today()
if last_date is None or last_date < today:
last_date = today # jobs are executed on different days
"send today's file here"
In general, it is more flexible to separate the code that schedules a task from the code that defines the task itself e.g., you could create a cron job, to run something once a day. You could use locks, to wait for the inner loop to finish and to avoid running multiple jobs in parallel (cron doesn't protect against it). Two jobs may happen on the same day if the previous job took more than a day.
I am trying to have my code detect when a flashdrive is plugged in and then continue the code. I am currently using "os.path.exists". When I start the code with the flashdrive plugged in, the code functions fine, however, if I start when the flashdrive is unplugged, and attempt to plug it in while the code is running, the code never checks to see if the flashdrive is plugged in and keeps forcing the code to sleep. How can I get the code to work?
import os
import sys
import datetime
from datetime import datetime
import shutil
import time
#Wait for FlashDrive to be connected
if os.path.exists("F:\"):
connected = 1
else:
connected = 0
while connected == 0:
print "..."
time.sleep(10)
#Get current date
currentdate=datetime.now().strftime("%m-%d-%Y")
print "Photos saved: " + currentdate
#Copy and rename DCIM
src = "F:/Pictures"
dst = "C:/Users/Josh/Desktop/photos/" + currentdate
shutil.copytree(src, dst)
The code is supposed to be a loop and execute every time an iPhone connects and never stop running, but I cannot get the code to work if it does not really check for the flashdrive.
Cycle with some arbitrary sleeps isn't a good idea (at all). It makes your program less responsive to the event, because it will take at least N ms to catch an event fired at the start of the iteration*. Also it wastes CPU due to a large amount of API calls.
Create a window.
Listen to WM_DEVICECHANGE message in your message loop. It will fire every time your device configuration changed, but won't tell you, how.
On such event, ask for current configuration.
You can find a tutorial here. Also, take a look at the similar answer on SO.
(*) Actually sleep will test on each next system tick if time_passed >= sleep_timeout. If so, it will return to the program. Problem is that system tick could be 1/18 of second on an old PC (56 ms), so you'll never have 10 ms delay.
Your problem is htat you set the connected variable outside the loop so it's never updated.
Try:
while not os.path.exists('F:\'):
print("...")
time.sleep(10)
--edit---
Then, wait for it to be removed at the end:
while os.path.exists('F:\'):
print("...")
time.sleep(10)
And, finally, wrap the entire thing in a big while True: so that the whole program repeats.
(Again, I do agree this is a 'hackish' and inefficent way to do this task)