We Keep Coding

Dash running in a PyQt QThread() - most appropriate way to re-initialize with new data

At the moment I have a PyQt5 app running Dash in a QThread like so:
class MapView(QObject):
message = pyqtSignal(str)
shutting_down = pyqtSignal(bool)
def __init__(self, data):
super().__init__()
self.app = dash.Dash(__name__)
self.data = data
#manual callbacks
self.app.callback(
Output('hover-data', 'children'),
Input('basic-interactions', 'hoverData'))(self.display_hover_data)
self.app.callback(
Output('page-content', 'children'),
Input('url', 'pathname'))(self.shutdown)
#...more callbacks
def shutdown(self, pathname):
if pathname != '/shutdown':
return
print("Trying to shutdown map dash")
func = request.environ.get('werkzeug.server.shutdown')
if func is None:
raise RuntimeError('Not running with the Werkzeug Server')
func()
self.shutting_down.emit(True)
#.....lots of stuff like graphs, the layout, a function to start the server etc
and then the threading code in my main pyqt5 app is:
def show_map(self):
self.mapthread = QThread()
self.map = map_dash.MapView(self.data)
self.map.moveToThread(self.mapthread)
self.map.message.connect(self.update_output)
self.map.shutting_down.connect(self.close_map_thread)
self.mapthread.finished.connect(self.open_project)
self.mapthread.started.connect(self.map.run)
self.mapthread.start()
self.browser.setUrl(QUrl('http://127.0.0.1:8050'))
self.update_output("Map plotted.")
My initial problem here was that if I tried to run show_map() with new data when Dash was already running (the user goes to File -> Open when a project is already running) , I would get the fatal Qthread: destroyed while thread is still running. So I embarked along the path of shutting down the flask server, then closing the thread, then going back to the function that opens a new project. It's like this:
User goes to File -> Open
def open_project() checks mapthread.isRunning()
It's not running so it opens a new project, creates a new QThread and new MapView instance
User goes to File -> Open again
The check in (2) returns True so the flask server is asked to shut down
After the server has shut down, the shutting_down signal causes the thread to be asked to quit() (I'm not sure how robust this is because it isn't a signal from flask, just a line after I've asked it to shut down. But it seems to work for now).
Once the thread has finished, the thread emits finished() which calls open_project() again
open_project this time sees that the thread is not running and allows the user to open a new file.
4 to 8 doesn't take too long to run but it all seems a bit complicated and the Dash layout glitches a bit for whatever reason. As it stands, when the QThread finishes under any other circumstances it would call open_project (although I could probably work around that). Is there a better way to do this? Can I feed the existing map instance new data somehow? I have read the documentation on dcc.Interval but that doesn't seem a great way to do it either...
Update (as per comments below):
Now what I'm doing is passing new data to the thread self.map.data = new_data, then using a callback on url to re-draw the map and refresh the layout. Exactly like this:
elif pathname == '/refresh':
self.draw_map()
self.set_layout()
But here is the problem. self.set_layout() refreshes dash with the OLD figure. I have verified that self.draw_map() is drawing a new figure. But calling /refresh a second time does cause dash to use the new figure. So is dash storing the old figure in a cache that hasn't been updated in the fraction of a second it takes to set the layout again? How can I make dash wait for the cache to be updated?

One way to do this is to pass the new data into the dash instance
self.map.data = new_data
but this does not refresh the layout and you can't simply call
self.map.set_layout() from the main thread. Instead you have to use the url callback like this:
def shutdown(self, pathname):
if pathname == '/shutdown':
print("Trying to shutdown map dash")
func = request.environ.get('werkzeug.server.shutdown')
if func is None:
raise RuntimeError('Not running with the Werkzeug Server')
func()
self.shutting_down.emit(True)
elif pathname == '/refresh':
self.draw_map()
self.set_layout()
of course it would be more appropriate to call that method something like handle_url() now.
But there is a further problem. I'm guessing due to some caching issue, dash displays the old version of the figure unless the user manually calls refresh again. The solution is to add a dcc.Interval callback that updates the figure (it calls draw_map()). It doesn't need to be doing this every second, you can set the interval to 999999, for example. In fact you can set 'max_intervals=1' and it still works. As long as the callback is there, it will be called the first time the page refreshes and the figure will be updated.
In fact, with that in place, the draw_map() and set_layout() functions don't even need to be called in the url callback. So the code now looks like this:
def shutdown(self, pathname):
if pathname == '/shutdown':
print("Trying to shutdown map dash")
func = request.environ.get('werkzeug.server.shutdown')
if func is None:
raise RuntimeError('Not running with the Werkzeug Server')
func()
self.shutting_down.emit(True)
elif pathname == '/refresh':
print("I'm doing nothing")
And it works. But the url callback itself IS necessary.

Dearpygui status box not updating properly

I am using dearpygui as an interface for a webscraping tool. I have a "status box" which I want to use to communicate messages to the user. It is setup as a read-only, multi-line input text box and is designed to update with the old value and append the new message. If I run the program, it will work but the updates to the status box do not show up until the webscraping(download_reports function) is finished. The gui is not updating each time the update_status function is called. What is the best way to get this to update and display when called and not when the entire process has finished?
def window_month_end_reports(sender, data):
with window('Month End'):
for building in buildings:
add_checkbox(building)
add_spacing()
add_button("Done", callback=run_month_end_reports)
add_spacing()
add_separator()
add_text("Select reports")
for report in reports_list:
add_checkbox(report)
def run_month_end_reports(sender, data):
item_list = []
building_list = []
delete_item('Month End')
for item in reports_list: # LIST OF REPORTS SELECTED
if get_value(item):
item_list.append(item)
for building in buildings:
if get_value(building):
building_list.append(building)
update_status('Running month end reports')
download_reports(building_list, item_list)
def update_status(message):
set_value('##status box', get_value('##status box') + str(message) + '\n')
with window("Main Window"):
add_button("Month End Reports", callback=window_month_end_reports)
add_text("Status Box:")
add_input_text("##status box", readonly=True, multiline=True, height=500)
start_dearpygui(primary_window="Main Window")

Probably, the best way to solve this is to use an async task to complete the download reports function. You could use run_async_function to launch the download. You can find an example of the long task management in the documentation: async functions

Job Scheduling in Django

I need to implement a scheduled task in our Django app. DBader's schedule seems to be a good candidate for the job, however when run it as part of a Django project, it doesn't seem to produce the desired effect.
Specifically, this works fine as an independent program:
import schedule
import time
import logging
log = logging.getLogger(__name__)
def handleAnnotationsWithoutRequests(settings):
'''
From settings passed in, grab job-ids list
For each job-id in that list, perform annotation group/set logic [for details, refer to handleAnnotationsWithRequests(requests, username)
sans requests, those are obtained from db based on job-id ]
'''
print('Received settings: {}'.format(str(settings)))
def job():
print("I'm working...")
#schedule.every(3).seconds.do(job)
#schedule.every(2).seconds.do(handleAnnotationsWithoutRequests, settings={'a': 'b'})
invoc_time = "10:33"
schedule.every().day.at(invoc_time).do(handleAnnotationsWithoutRequests, settings={'a': 'b'})
while True:
schedule.run_pending()
time.sleep(1)
But this (equivalent) code run in Django context doesn't result in an invocation.
def handleAnnotationsWithoutRequests(settings):
'''
From settings passed in, grab job-ids list
For each job-id in that list, perform annotation group/set logic [for details, refer to handleAnnotationsWithRequests(requests, username)
sans requests, those are obtained from db based on job-id ]
'''
log.info('Received settings: {}'.format(str(settings)))
def doSchedule(settings):
'''
with scheduler library
Based on time specified in settings, invoke .handleAnnotationsWithoutRequests(settings)
'''
#settings will need to be reconstituted from the DB first
#settings = {}
invocationTime = settings['running_at']
import re
invocationTime = re.sub(r'([AaPp][Mm])', "", invocationTime)
log.info("Invocation time to be used: {}".format(invocationTime))
schedule.every().day.at(invocationTime).do(handleAnnotationsWithoutRequests, settings=settings)
while True:
schedule.run_pending()
time.sleep(1)
so the log from handleAnnotationsWithoutRequests() doesn't appear on the console.
Is this scheduling library compatible with Django? Are there any usage samples that one could refer me to?
I'm suspecting some thread issues are at work here. Perhaps there are better alternatives to be used? Suggestions are welcome.
Thank you in advance.

For web servers, you probably don't want something that runs in-process:
An in-process scheduler for periodic jobs [...]
https://github.com/Tivix/django-cron has proven a working solution.
There's also the heavyweight champion Celery and Celerybeat.

I do this a lot with Django Commands
The pattern I use is to setup a new Django command in my app and then make it a long-running process inside a never-ending while() loop.
I the loop iterates continuously with a custom defined sleep(1) timer.
The short version is here, with a bit of pseudo-code thrown in. You can see a working version of this pattern in my Django Reference Implementation.
class Command(BaseCommand):
help = 'My Long running job'
def handle(self, *args, **options):
self.stdout.write(self.style.SUCCESS(f'Starting long-running job.'))
while True:
if conditions met for job:
self.job()
sleep(5)
def job(self):
self.stdout.write(self.style.SUCCESS(f'Running the job...'))

action callback for libnotify not working

I am trying to develop a desktop notification system with python and Libnotify.
I found this tutorial and tried the example with an action callback, below is my code modification
#initialization of libnotify
Notify.init("Py-Notification")
#creating object
summary = "Wake Up!"
body = "Meeting at 9pm"
notification = Notify.Notification.new(
summary, body
)
# Define a callback function
def my_callback_func():
webbrowser.open('http://google.com')
#adding actions
notification.add_action(
"action_click",
"Reply to Alarm",
my_callback_func,
None # Arguments
)
notification.show()
The callback function is not called whenever I click on the "Reply to Alarm" button
Any help.

You need to integrate with D BUS to receive events from your notifications.
Here is good explanation use Glib for it.

Kivy: How to create a 'blocking' popup/modalview?

I did find a question on this on stackoverflow, here, but I find it just doesn't answer the question, as for me, neither the Popup nor the ModalView actually 'blocks'. What I mean is, execution is moving through a function, like:
def create_file(self):
modal = ModalView(title="Just a moment", size_hint=(0.5, 0.5))
btn_ok = Button(text="Save & continue", on_press=self.save_file)
btn_no = Button(text="Discard changes", on_press=modal.dismiss)
box = BoxLayout()
box.add_widget(btn_ok)
box.add_widget(btn_no)
modal.add_widget(box)
modal.open()
print "back now!"
self.editor_main.text = ""
new = CreateView()
new.open()
And the print statement prints "back now!" and the rest of the function is immediately executed, despite the fact the ModalView just opened. I also tried this using a Popup instead of a ModalView, with the same result. I would like execution in the function to pause while I interact with the Popup/ModalView. Is there a way for this to be done built into kivy? Must I use threads? Or will I need to just find some other workaround?

You can't block like that, as that would stop the event loop and you wouldn't be able to interact with your app anymore. The easiest fix is to split this into two functions, and use on_dismiss to continue:
def create_file(self):
modal = ModalView(title="Just a moment", size_hint=(0.5, 0.5))
btn_ok = Button(text="Save & continue", on_press=self.save_file)
btn_no = Button(text="Discard changes", on_press=modal.dismiss)
box = BoxLayout()
box.add_widget(btn_ok)
box.add_widget(btn_no)
modal.add_widget(box)
modal.open()
modal.bind(on_dismiss=self._continue_create_file)
def _continue_create_file(self, *args):
print "back now!"
self.editor_main.text = ""
new = CreateView()
new.open()
It's also possible to use Twisted to make the function asynchronous, though that's a little more complicated.