I'm new to threading and python. I would like to understand how to pass multiple arguments from one class to another class in python using threading.
I'm using a main thread to call a class- Process then inside the run I'm doing some business logic and calling another class- build using thread and passing multiple arguments.
The run of build class is getting executed but Inside the build class, I'm unable to access those arguments and hence not able to proceed further.
Not sure if my approach is right? Any suggestions will be appreciated.
Below is my main class :
from threading import Thread
import logging as log
from process import Process
if __name__ == '__main__':
try:
proc = Process()
proc.start()
except Exception as e:
#log some error
Inside Process:
#all the dependencies are imported
class Process(Thread):
'''
classdocs
'''
def __init__(self):
'''
Constructor
'''
Thread.__init__(self)
#other intializations
def run(self):
#some other logic
self.notification(pass_some_data)
#inside notification I'm calling another thread
def notification(self,passed_data):
#passed data is converted dict1
#tup1 is being formed from another function.
#build is a class, and if i don't pass None, i get groupname error.
th = build(None,(tup1,),(dict1,))
th.start()
#inside build
class build(Thread):
def _init_(self,tup1,dict1):
super(build,self).__init__(self)
self.tup1 = tup1
self.dict1 = dict1
def run(self):
#some business logic
#I'm unable to get the arguments being passed here.
Related
In a custom class I have the following code:
class CustomClass():
triggerQueue: multiprocessing.Queue
def __init__(self):
self.triggerQueue = multiprocessing.Queue()
def poolFunc(queueString):
print(queueString)
def listenerFunc(self):
pool = multiprocessing.Pool(5)
while True:
try:
queueString = self.triggerQueue.get_nowait()
pool.apply_async(func=self.poolFunc, args=(queueString,))
except queue.Empty:
break
What I intend to do is:
add a trigger to the queue (not implemented in this snippet) -> works as intended
run an endless loop within the listenerFunc that reads all triggers from the queue (if any are found) -> works as intended
pass trigger to poolFunc which is to be executed asynchronosly -> not working
It works as soon as I source my poolFun() outside of the class like
def poolFunc(queueString):
print(queueString)
class CustomClass():
[...]
But why is that so? Do I have to pass the self argument somehow? Is it impossible to perform it this way in general?
Thank you for any hint!
There are several problems going on here.
Your instance method, poolFunc, is missing a self parameter.
You are never properly terminating the Pool. You should take advantage of the fact that a multiprocessing.Pool object is a context manager.
You're calling apply_async, but you're never waiting for the results. Read the documentation: you need to call the get method on the AsyncResult object to receive the result; if you don't do this before your program exits your poolFunc function may never run.
By making the Queue object part of your class, you won't be able to pass instance methods to workers.
We can fix all of the above like this:
import multiprocessing
import queue
triggerQueue = multiprocessing.Queue()
class CustomClass:
def poolFunc(self, queueString):
print(queueString)
def listenerFunc(self):
results = []
with multiprocessing.Pool(5) as pool:
while True:
try:
queueString = triggerQueue.get_nowait()
results.append(pool.apply_async(self.poolFunc, (queueString,)))
except queue.Empty:
break
for res in results:
print(res.get())
c = CustomClass()
for i in range(10):
triggerQueue.put(f"testval{i}")
c.listenerFunc()
You can, as you mention, also replace your instance method with a static method, in which case we can keep triggerQueue as part of the class:
import multiprocessing
import queue
class CustomClass:
def __init__(self):
self.triggerQueue = multiprocessing.Queue()
#staticmethod
def poolFunc(queueString):
print(queueString)
def listenerFunc(self):
results = []
with multiprocessing.Pool(5) as pool:
while True:
try:
queueString = self.triggerQueue.get_nowait()
results.append(pool.apply_async(self.poolFunc, (queueString,)))
except queue.Empty:
break
for r in results:
print(r.get())
c = CustomClass()
for i in range(10):
c.triggerQueue.put(f"testval{i}")
c.listenerFunc()
But we still need to reap the pool_async results.
Okay, I found an answer and a workaround:
the answer is based the anser of noxdafox to this question.
Instance methods cannot be serialized that easily. What the Pickle protocol does when serialising a function is simply turning it into a string.
For a child process would be quite hard to find the right object your instance method is referring to due to separate process address spaces.
A functioning workaround is to declare the poolFunc() as static function like
#staticmethod
def poolFunc(queueString):
print(queueString)
I'm aware of this structure
class MyThread(QThread):
def __init__(self):
super().__init__()
def run():
# do stuff
t = MyThread()
t.start()
With regular threading.Thread you can do something like this:
def stuff():
# do stuff
t = threading.Thread(target=stuff)
t.start()
Any way to do this in pyqt5 with QThreads? Something like this:
t = Qthread(target=stuff)
t.start()
I tried that but I got this error:
TypeError: 'target' is an unknown keyword argument
You can add the function to a custom argument in the __init__, create an instance attribute for its reference and then run it in the run.
class MyThread(QThread):
def __init__(self, target=None):
super().__init__()
self.target = target
def run():
if self.target:
self.target()
def stuff():
# do something
t = MyThread(target=stuff)
t.start()
Be aware that access to UI elements is not allowed in external threads, so don't use the threaded function to do anything related to UI: reading values and properties is unreliable, and writing can cause your program to crash.
I know this is an old question and have already found answers in other questions like this thread here. However, I have some problems applying it in my case.
The way I have things right now are the following: I have my MainWindow class where I can input some data. Then I have a Worker class which is a PySide2.QtCore.QThread object. To this class I pass some input data from the MainWindow. Inside this Worker class I have a method which sets up some ODEs, which in another method of the Worker class are being solved by scipy.integrate.solve_ivp. When the integration is done, I send the results via a signal back to the MainWindow. So the code roughly looks like this:
import PySide2
from scipy.integrate import solve_ivp
class Worker(QtCore.QThread):
def __init__(self,*args,**kwargs):
super(Worker,self).__init__()
"Here I collect input parameters"
def run(self):
"Here I call solve_ivp for the integration and send a signal with the
solution when it is done"
def ode_fun(self,t,c):
"Function where the ode equations are set up"
class Ui_MainWindow(QtWidgets.QMainWindow):
def __init__(self):
"set up the GUI"
self.btnStartSimulation.clicked.connect(self.start_simulation) #button to start the integration
def start_simulation(self):
self.watchthread(Worker)
self.thread.start()
def watchthread(self,worker):
self.thread = worker("input values")
"connect to signals from the thread"
Now I understand, that using the multiprocessing module I should be able to run the thread with the integration on another processor core to make it faster and make the GUI less laggy. However, from the link above I am not sure how I should apply this module or even how to restructure my code. Do I have to put the code that I now have in my Worker class into another class or am I somehow able to apply the multiprocessing module on my existing thread?
Any help is greatly appreciated!
Edit:
The new code looks like this:
class Worker(QtCore.QThread):
def __init__(self,*args,**kwargs):
super(Worker,self).__init__()
self.operation_parameters = args[0]
self.growth_parameters = args[1]
self.osmolality_parameters = args[2]
self.controller_parameters = args[3]
self.c_zero = args[4]
def run(self):
data = multiprocessing.Queue()
input_dict = {"function": self.ode_fun_vrabel_rushton_scaba_cont_co2_oxygen_biomass_metabol,
"time": [0, self.t_final],
"initial values": self.c_zero}
data.put(input_dict)
self.ode_process = multiprocessing.Process(target=self.multi_process_function, args=(data,))
self.ode_process.start()
self.solution = data.get()
def multi_process_function(self,data):
self.message_signal = True
input_dict = data.get()
solution = solve_ivp(input_dict["function"], input_dict["time"],
input_dict["initial values"], method="BDF")
data.put(solution)
def ode_fun(self,t,c):
"Function where the ode equations are set up"
(...) = self.operation_parameters
(...) = self.growth_parameters
(...) = self.osmolality_parameters
(...) = self.controller_parameters
Is it okay if I access the parameters in the ode_fun function via self."parameter_name"? Or do I also have to pass them with the data-parameter?
With the current code I receive the following error: TypeError: can't pickle Worker objects
You could call it from your worker like this:
import PySide2
from scipy.integrate import solve_ivp
import multiprocessing
class Worker(QtCore.QThread):
def __init__(self,*args,**kwargs):
super(Worker, self).__init__()
self.ode_process = None
"Here I collect input parameters"
def run(self):
"Here I call solve_ivp for the integration and send a signal with the solution when it is done"
data = multiprocessing.Queue()
data.put("all objects needed in the process, i would suggest a single dict from which you extract all data")
self.ode_process = multiprocessing.Process(target="your heavy duty function", args=(data,))
self.ode_process.start() # this is non blocking
# if you want it to block:
self.ode_process.join()
# make sure you remove all input data from the queue and fill it with the result, then to get it back:
results = data.get()
print(results) # or do with it what you want to do...
def ode_fun(self, t, c):
"Function where the ode equations are set up"
class Ui_MainWindow(QtWidgets.QMainWindow):
def __init__(self):
"set up the GUI"
self.btnStartSimulation.clicked.connect(self.start_simulation) #button to start the integration
def start_simulation(self):
self.watchthread(Worker)
self.thread.start()
def watchthread(self,worker):
self.thread = worker("input values")
"connect to signals from the thread"
Also beware that you would overwrite the running process now every time you press to start the simulation. You may want to use some sort of lock for that.
I'm curious how can I get the status of thread that have been started in a separate class in python.
So currently I have:
class VideoCapture:
def record:
Thread(name='uploading', target=self.upload, args=(upload_queue)).start()
In a seperate file main.py I have an instance of VideoCapture.
I want to be able to check the status of the thread "uploading" by typing something like VideoCapture.uploading.isAlive(). However I get the error that VideoCapture has no object uploading. So how can I access it?
Store the thread as something that is part of the class, then after that you have a means of accessing it later on.
class VideoCapture:
def __init__():
self.uploading = None
def record:
self.uploading = Thread(name='uploading', target=self.upload, args=(upload_queue)).start()
Now somewhere else you have:
video_capture = VideoCapture()
video_capture.record()
if video_capture.uploading.isAlive():
# do something
There's method is_alive() in Thread object, so basically you need just use it:
class VideoCapture:
def record(self):
# don't forget to add it on __init__
self.uploading = Thread(name='uploading', target=self.upload, args=(upload_queue)).start()
tmp = VideoCapture()
tmp.record()
tmp.uploading.is_alive() # here it is
First of all, here are my two python files:
sred.py:
import _thread,time
class Thread:
def __init__(self,time:int,say:str):
self.time=time
self.say=say
def create():
id = _thread.get_ident()
for i in range(5):
print("HALLO", id)
return
from sred import Thread
import time,_thread
_thread.start_new_thread(Thread.create,())
The second one:
main.py
from sred import Thread
import time,_thread
_thread.start_new_thread(Thread.create,())
when executing this it doesn't print anything out, why?
UPDATE:
import _thread
class Thread:
#classmethod
def create():
id = _thread.get_ident()
for i in range(5):
print("HALLO", id)
return
main.py:
from sred import Thread
import time,_thread
_thread.start_new_thread(Thread().create,())
Is this now right, or is there still something wrong?
The create method is missing self as a parameter -- it looks like it should also be a #classmethod if you want to call it as it's written now. Note that your __init__ method is never getting called, because you never instantiate any Thread objects. You may want it to read:
_thread.start_new_thread(Thread().create, ())
i.e., instantiate a thread, then pass its create method to be executed in the new thread. I'm not sure what's happening, but I suspect that something is erroring and the stacktrace is being suppressed by something.
Also, you need to delete the space after the for statement -- it's significant, and it should be throwing you a syntax error about an unexpected indent.
EDIT:
This version runs on my machine:
import _thread
class Thread:
def create(self):
id = _thread.get_ident()
for i in range(5):
print("HALLO", id)
return
_thread.start_new_thread(Thread().create, ())