I am rookie to Python threading and am looking for guidance on how to implement mulithreading on an application with two classes, GUI and weblogic that need need to run in parallel.
The GUI class is all about user interaction and will have code like "if you click here run this weblogic method." The GUI must be in its own thread because the weblogic method can be slow and so waiting for it to complete will cause the GUI to seem unresponsive. The good news is that GUI code has no ability to modify any data inside the weblogic thread and so locking is not an issue. (The GUI just calls a weblogic method with the appropriate variables and then processes the resulting output.)
The web logic class includes multiple methods and constructors and some of the methods can be time consuming to run. The class also includes constructors that are vital to all the methods in the class. (Think userid, password, API access url and the like)
So what I am looking to implement is a way to instantiate the two threads in parallel. Hence, the GUI would run in its own thread and issue commands to the weblogic thread which would respond accordingly.
I started looking at the Threading module and while I have gotten it to work for simplistic functions, I have not been successful in running an entire class inside a thread. Since I am new to all of this, I am looking for guidance on the right strategy. Should I even be using Threading? Is there better approach?
Here is some pseudo code to illustrate what I am trying to accomplish:
GUI Code:
class GUI():
def createGUI(self):
# create GUI layout here
def button1(self):
# if clicked
outputtodisplay = weblogic.filelist()
# display outputtodisplay
def button2(self)
# if clicked assume that we have a file selector box which provides a filename
weblogic.uploadfile(filename)
# show an upload success indicator
Weblogic:
class weblogic():
def __init__(self, count, instance):
# run some stuff to setup
self.uid = xxx
self.pwd = yyy
self.url = zzz
def listfiles():
# get file list using uid, pwd, url
return output
def uploadfile(self,filename):
# upload file using uid, pwd, url
return success/failure
Related
I am trying to do an app using tkinter that gets information from some internet sources after entering stuff in an input box and then clicking a "load" button.
The button function reads the stuff string from the input box. After its loaded it retrieves some info from the internet (which is blocking), and then update some labels with this information.
Obviously, when clicking the load button, the app freezes a micro second due to the requests blocking the flow of the program.
Once the info is retrieved, and the labels updated, some other labels would need to keep retrieving data from the internet constantly. To do this I have them done a bit like this:
Note: the print statements are done for testing, so I can see them on the console
def update_price_label(self):
# TODO fix .after() duplicate
print("Updating stuff")
price = self.get_formatted_price(self.stuff) # this is another function being called, passing an argument of the stuff that has been loaded by the load button, this function returns price
self.PriceValue.configure(text=price) # updates the price label with the price obtained from the function above
self.PriceValue.after(1000, self.update_price_label) # sets a scheduler to run this function to update the price label each second
Above there is a function that is called upon clicking "load" for a label that needs to be updated all the time, this function calls another function that receives an argument, and then returns the price. Then the label is updated, and then its scheduled in an endless loop using the after() method of the priceValue label control. This way, the price gets updated all the time
def get_formatted__price(self, stuff):
price = RETRIEVE PRICE # this is not a real function, but this is the request being done to the server to get the price
return f"{price:.12f} # sets decimal numbers
This function is called by update_price_label(), receives an argument and returns the price
As you can see I have divided the label update functions vs the actual functions that actually retrieve the info from a server. So the first function is responsible for calling another function to retrieve the information, update the label, and then reschedule itself again using the after() method of each label widget.
There are like 5 functions like this that need to update several labels on the app, connecting to different sources and keeping the information up to date. All of them are scheduled using after() and run on the same intervals (1 second).
Obviously, due to not using any threading, the app freezes a lot when information is being requested due to them being blocking in nature.
So I need to implement threading, or any form of concurrency. And I am unable to find any good tutorials on this, or at least that fits my needs for an app that fetches information from sources on a regular basis.
I am still grasping the concepts of threading and concurrency in general, and maybe there are other ways such as asynchronism or other methods of concurrency that I don't know yet and might be better suited. But Threading seems to be the one that seems to be mostly used with Tkinter.
I assume each of these request functions would need a thread. Like this:
get_formatted_price_thread = Thread(target=get_formatted_price, args=(stuff), daemon=True) # calling a thread with an argument on the function and setting daemon to true, so it closes when I close the app
So I have tried to create threads on one of them as an example and I have found a few limitations such as:
There's no way to get the returned value of the get_formatted_price() function directly. So the other way could be just letting the function in a thread to change the label value. Or wrapping the whole label update function in a thread. But as I read everywhere, Tkinter is not thread safe. Meaning that updating tkinter widgets may work fine on some operative systems but not in others.
Besides that, I seem to struggle on to how to turn the structure of this app to something that works well with threading or any kind of concurrency. As I need to have all the labels updated as soon as new info is retrieved.
My other concern is that threads are controlled by the operative system. As it is the one that decides when threads are launched, and how that could impact my apps performance when fetching data.
I have also checked queues and the Queue library, but I am not sure if this what would help me, as each price update would be put into the queue by the thread, and retrieved by the label widget. But the information could be outdated as the queue gets the first element of the queue.
So my question here is what would I need to change here to achieve what I need. And if threading is what I need to go on, or if maybe I would need to try another approach.
Any example source code of apps that do what I need to would be very appreciated. After all, retrieving information, and keeping widgets up to date with that information should be like a pretty common use case.
Another approach I have thought of is creating a data structure, such as a Python dictionary, or an object. Each server fetching function would run in a thread in an endless loop as a daemon, and would write to the dictionary. Then the label widget update functions, since they are scheduled, would read the data on the dictionary and update the labels accordingly. But I think this approach might be messy, and probably there would be a delay on updating the labels vs the information on the dictionaries, unless a smaller after() scheduler timer is set. Or maybe all solutions are messy by default
Thank you.
I would solve this by creating a data structure, creating a function that can update the display based on the current values in the data structure, and then binding to an event that calls this function. Then, create a thread that updates this data structure and emits the event when the data changes.
Here's a contrived example that calls a web service once a second and updates a simple data structure with the time and timezone information. Whenever the data changes it emits a <<Tick>> event that triggers an update of the display.
I'm not an expert on writing threaded tkinter code, and it's my understanding that except in a very few circumstances it is unsafe to run any tkinter code in a thread other than the one where the widgets were created. One exception is that it's safe for additional threads to generate events, since the events themselves get handled in the main GUI thread. I'm guessing it's also safe to call the winfo_exists function since it doesn't modify any internal data structures.
This example kills itself after 10 seconds so as to not hammer the server for too long.
import requests
import tkinter as tk
from tkinter.font import Font
from threading import Thread
import time
class ThreadedClock(tk.Frame):
data = {"time": "", "tz": ""}
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.time_label = tk.Label(self, width=12, font=Font(size=32))
self.tz_label = tk.Label(self, text="GMT")
self.time_label.pack(side="top", fill="x")
self.tz_label.pack(side="top", fill="x")
# call the refresh function on every <<Tick>> event
self.bind("<<Tick>>", self.refresh)
# start a thread to update the data and generate <<Tick>> events
self.thread = Thread(target=self.get_data, daemon=True)
self.running = True
self.thread.start()
def get_data(self):
while self.winfo_exists():
now = time.time()
response = requests.get(
"https://timeapi.io/api/Time/current/zone?timeZone=GMT"
)
t = response.json()
timestr = f"{t['hour']:02}:{t['minute']:02}:{t['seconds']:02}"
self.data = {"time": timestr, "tz": t["timeZone"]}
self.event_generate("<<Tick>>")
delta = time.time() - now
time.sleep(delta)
def refresh(self, event=None):
self.time_label.configure(text=self.data["time"])
self.tz_label.configure(text=f"timezone: {self.data['tz']}")
if __name__ == "__main__":
root = tk.Tk()
root.after(10000, root.destroy)
clock = ThreadedClock(root)
clock.pack(fill="both", expand=True)
root.mainloop()
I need to send updates to tkinter from another thread, which should be handled immediately. Sadly Tkinter freezes whenever I try multithreading.
My Attempt:
I've read through several Tkinter-threaded pages but didn't find anything that works well, since most of them try creating a new thread on button.click(), which doesn't help here.
I tried not calling .mainloop() and instead calling the update functions myself, whenever an update comes in:
#GUI
def update(self, string):
self._interactor.config(text=string)
#interactor is a button
self._tk.update_idletasks()
self._tk.update()
This works fine until I use a loop with sleep() in the master to constantly update the text. GUI and master are frozen during sleep().
So I tried to use a threaded timer as discussed here.
#MASTER
gui=gui_remote(self)
def changetext():
text=self.gettextsomewhere()
self._gui.update(text)
loop=RepeatedTimer(5, changetext)
But this just leads to the following error, thrown by Tkinter:
RuntimeError: main thread is not in main loop
Having a hard time on how to solve this. Is it possible to call a GUI class on main thread and still have proper access to its functions?
How I got to this point:
For my project, I need a button, which represents several Buttons.
Every y (eg. 1.5) seconds, the displayed text should be updated to a new one from the outside.
Also, I want to keep GUI, Controller and Data separated, (using blueprint methods) so that later adjustments on each of them will be easier.
I already got it to work, using TK's .after() function, but I had to use GUI and controlling functions closely together.
My Plan
Have a GUI class, which is updateable from another object via simple public functions. The other object (the master) should be able to create a GUI object and call the GUI's update functions with new data every y seconds.
When the GUI button is clicked, simply call a certain method at master every time:
#GUI example
from tkinter import Tk, Button, Frame, Label
class gui_sample:
def __init__(self, master):
"""This is the very simple GUI"""
self._master=master
self._tk=Tk()
self._interactor= Button(self._tk, text="Apfelsaft", command=self._click)
self._interactor.pack()
self._tk.mainloop()
def update(self, string):
"""Handle interactor update"""
self._interactor.config(text=string)
def _click(self):
self._master.click()
#MASTER
from gui_module import *
class Controller:
def __init__(self):
self._gui=gui_sample(self)
self._run()
def _run(self):
#call this every 5 seconds
new_text=self.gettextfromsomewhere()
self._gui.update(new_text)
def click():
#do something
pass
#this code is just a blueprint it probably does nothing
My Problem:
I don't want the master to use TK functions since I might switch to another UI module later and keep the master's functionality. The master will constantly loop through what's being displayed next and needs to be accessible at the same time. Using loops with sleep() isn't a good idea since they will block both, the master and the GUI. Calling .mainloop() is also problematic since it will block all other programs. The gui should always respond to updates and not ask for them.
I have a program that contains two main classes:
A GUI class that inherits from QWidget, and has several user input and output fields (graphs, spin boxes etc.)
A Serial Monitor class that inherits from QObject and contains various loops for continuously polling serial attached devices in accordance with a set of parameters (sample period, polling commands, etc.)
An instance of the Serial Monitor class is created from within the GUI class and moved to a secondary thread like so:
# Create Serial monitor instance in this thread
self.serial_monitor = Serial_Monitor(
formatter=self.poll,
prompt="2 poll\n")
# Create a secondary QThread to run the serial monitor
self.serial_thread = QThread()
# Move the serial monitor to the secondary thread and start
self.serial_monitor.moveToThread(self.serial_thread)
self.serial_thread.start()
My question arises when trying to directly pass the output of user interface items (spin boxes, buttons etc.) to variables in the Serial Monitor. My main aims are:
I want to avoid creating several pyqtSlot functions in the Serial Monitor just to set each variable, as it makes adding further variables a faff and bloats the code.
I would also like to keep all setting of Serial Monitor variables in the correct thread, rather than setting them in the GUI command thread.
My initial solution to directly setting variables from the GUI was this:
self.set_box_period.valueChanged.connect(
lambda val: setattr(self.serial_monitor, "sample_period", val))
It avoids creating any new methods in the Serial Monitor, and appears to set the variables just fine, however I am not sure if it is "thread safe", and further more am not sure how to find out.
After some digging I found out that you can re-implement the __setattr__ method within a class like so:
#pyqtSlot()
def __setattr__(self, name, value):
"""
------------------------------------------------------------------------
Re-implementation of "getattr" method to correctly wire up to slots
------------------------------------------------------------------------
"""
self.__dict__[name] = value
self.logger.info(F"Set {name} to {value}")
The logging output indicates that every Serial Monitor variable is set within the Serial Monitor thread, so this solution does work, however I am not sure if it is necessary.
To sum up, do I need to worry about re-implementing __setattr__ in my Serial Monitor class, or is my initial lambda implementation perfectly safe to use?
Thanks in advance!
I currently run a daemon thread that grabs all cell values, calculates if there's a change, and then writes out dependent cells in a loop, ie:
def f():
while not event.is_set():
update()
event.wait(15)
Thread(target=f).start()
This works, but the looped get-all calls are significant I/O.
Rather than doing this, it would be much cleaner if the thread was notified of changes by Google Sheets. Is there a way to do this?
I rephrased my comment on gspread GitHub's Issues:
Getting a change notification from Google Sheets is possible with help of installable triggers in Apps Script. You set up a custom function in the Scripts editor and assign a trigger event for this function. In this function you can fetch an external url with UrlFetchApp.fetch.
On the listening end (your web server) you'll have a handler for this url. This handler will do the job. Depending on the server configuration (many threads or processes) make sure to avoid possible race condition.
Also, I haven't tested non browser-triggered updates. If Sheets trigger the same event for this type of updates there could be a case for infinite loops.
I was able to get this working by triggering an HTTP request whenever Google Sheets detected a change.
On Google Sheets:
function onEdit (e) {
UrlFetchApp.fetch("http://myaddress.com");
}
Python-side (w/ Tornado)
import tornado.ioloop
import tornado.web
class MainHandler(tornado.web.RequestHandler):
def get(self):
on_edit()
self.write('Updating.')
def on_edit():
# Code here
pass
app = tornado.web.Application([(r'/', MainHandler)])
app.listen(#port here)
tornado.ioloop.IOLoop.current().start()
I don't think this sort of functionality should be within the scope of gspread, but I hope the documentation helps others.
I am fairly new in writing bigger programs in Python (I was only writing short scripts before). The program I'm about to write recives data from an external device, saves it to database and displays it in GUI (continuously). Since handling the data is time consuming I want to use threads (PySide QtCore QThreads to be specific). The best way to do this that I can think of is to set up two threads, one for the database processes and one for the handling of serial port, with GUI running in the MainThread. Now I've read a whole bunch of stuff about proper/improper QThreading, starting with Maya's post and digging deeper, up to this post where I found that:
When to subclass and when not to?
If you do not really need an event loop in the thread, you should subclass.
If you need an event loop and handle signals and slots within the thread, you may not need to subclass.
Question 1: Now the first thing I don't know is which way to go (subclassing or not). My guess is subclassing, since (Question2), but I'm not entirely sure and for now I'm sticking to moveToThread().
Question 2: The other thing I have no clue about is how to make the actions in my threads event-driven (the SerialThread receives the data from the device, sends it via signal to the DatabaseThread, the DatabaseThread collects the data and puts it in the database).
Question 3: When using moveToThread() I get AttributeError: 'PySide.QtCore.QThread' object has no attribute 'my_awesome_method' for every method I write for my class. It's clear to me that I don't understand the principle of operation here. How do I implement my methods that I want my_class to have while using my_class.moveToThread(my_thread)?
I've found all tutorials possible on QThreads and C++, so this discussion on QThreads in Python is interesting, but does not explain everything I want to know. I need a simple example with signals, slots and an exact explanation on how to use run(). Do I decorate it? Why? How? How does it work?
The short version of my code looks like this:
from PySide import QtCore
from app.thrads.gui.gui import Gui #my own class
from app.threads.db.db import Database #my own class
from app.threads.serial.serial import Serial #my own class
class AppManager():
def __init__(self):
self.gui = Gui()
self.db = Database()
self.db_thread = QtCore.QThread()
self.db.moveToThread(self.db_thread)
self.db_thread.start()
self.serial = Serial()
self.serial_thread = QtCore.QThread()
self.serial.moveToThread(self.serial_thread)
self.serial_thread.start()
and my Database and Serial class look somewhat like this:
from PySide import QtCore
from .db_signal import DbSignal
class Database(QtCore.QObject):
def __init__(self):
super(Database, self).__init__()
self.signal = DbSignal()
def my_awesome_method(self):
''' does something awesome '''
pass
Question 1
I think the best way is the easiest: don't subclass, just create two different threads. In the first, move the Database object, in the second the Serial one. As it, you won't make mistakes in the implementation of your sub-classed threads,and bug fixing will be quicker.
Question 2
I don't know the architecture of your application, but you could do as follows, after creating the threads and moving the worker objects in them:
self.serial_thread.started.connect(self.serial.start)
self.serial_thread.finished.connect(self.serial.stop)
self.db_thread.started.connect(self.db.start)
self.serial_thread.finished.connect(self.db.stop)
self.serial.data_ready.connect(self.db.handle_new_data)
# Starting the database first, as it we won't lose a single packet of data
self.db_thread.start()
self.serial_thread.start()
In fact, the advantage of QThreads is that they doesn't really modify the code.
Question 3
I think the problem is you're trying to call my_awesome_method with the QThread where your data base or your serial listener has been moved to. On the contrary, you should call the method with the object itself, as if it wasn't in a different thread !
# Bad!
obj.moveToThread(thread)
thread.method_of_obj(param)
# Good!
obj.moveToThread(thread)
obj.method_of_obj(param)