I am trying to run a function in the background till some work is done in the main function and then finish the thread. I have implemented the threading logic in a separate class and the main in another file. But every time I run it the target function only seems to run once and then waits
Here is the main function
from ThreadRipper import *
thread_obj=ThreadRipper()
thread_obj.start_thread()
squish.snooze(10)
print("Main Continuing")
thread_obj.stop_thread()
And the implemented class is as follows
class ThreadRipper():
def __init__(self):
lib_path="iCOMClient.dll"
self.vx = IcomVx(lib_path)
config = ConfigParser.SafeConfigParser(allow_no_value=True)
config.readfp(open("vx_logger_config.cfg"))
self.vx.connect(config.get("icom","ip"), timeout_millis = 30000)
self.t = threading.Thread(target=self.task_to_do, args=(self.vx,))
def stop_thread(self):
self.t.do_run=False
self.t.join()
def start_thread(self):
self.t.start()
def task_to_do(self,arg):
current_thread=threading.currentThread()
while getattr(current_thread,"do_run",True):
with open("vx.txt",'a') as f:
f.write(str(arg.get_next_message()._rawmsg)+"\n")
time.sleep(1)
print("Stopping")
arg.disconnect()
When I run this I get the vx file created but with only one entry, I expect it to be written to continuously till the while loop exits. I am quite new at threading and may have understood it incorrectly. Please advise
Thank you
The reason is probably because
print("Stopping")
arg.disconnect()
are both inside the while loop. After disconnecting, arg doesn't seem to produce any more messages.
(Unless, of course, your code in the question is not what you really have, but in this case, you surely would have edited your question so it matches.)
Related
Simply put, I want to properly implement threading in a Python GTK application. This is in order to prevent UI freezing due to functions/code taking a long time to finish running. Hence, my approach was to move all code which took a long time to run into separate functions, and run them in their separate threads as needed. This however posed a problem when trying to run the functions in sequence.
For example, take a look at the following code:
class Main(Gtk.Window):
def __init__(self):
super().__init__()
self.button = Gtk.Button(label='button')
self.add(self.button)
self.button.connect('clicked', self.main_function)
def threaded_function(self):
time.sleep(20)
print('this is a threaded function')
def first_normal_function(self):
print('this is a normal function')
def second_normal_function(self):
print('this is a normal function')
def main_function(self, widget):
self.first_normal_function()
self.threaded_function()
self.second_normal_function()
Pressing the button starts main_function which then starts 3 functions in sequence. threaded_function represents a function which would take a long time to complete. Running this as is will freeze the UI. Hence it should be threaded as such:
...
...
def main_function(self, widget):
self.first_normal_function()
thread = threading.Thread(target=self.threaded_function)
thread.daemon = True
thread.start()
self.second_normal_function()
What should happen is that the following first_normal_function should run, then threaded_function in a background thread - the UI should remain responsive as the background thread is working. Finally, second_normal_function should run, but only when threaded_function is finished.
The issue with this is that the functions will not run in sequence. The behaviour I am looking for could be achieved by using thread.join() however this freezes the UI.
So I ask, what's the proper way of doing this? This is a general case, however it concerns the general issue of having code which takes a long time to complete in a graphical application, while needing code to run sequentially. Qt deals with this by using signals, and having a QThread emit a finished signal. Does GTK have an equivalent?
I'm aware that this could be partially solved using Queue , with a put() and get() in relevant functions, however I don't understand how to get this to work if the main thread is calling anything other than functions.
EDIT: Given that it's possible to have threaded_function call second_normal_function using GLib.idle_add, let's take an example where in main_function, the second_normal_function call is replaced with a print statement, such that:
def main_function(self, widget):
self.first_normal_function()
thread = threading.Thread(target=self.threaded_function)
thread.daemon = True
thread.start()
print('this comes after the thread is finished')
...
...
...
#some more code here
With GLib.idle_add, the print statement and all the code afterwards would need to be moved into a separate function. Is it possible to avoid moving the print statement into its own function while maintaining sequentiality, such that the print statement remains where it is and still gets called after threaded_function is finished?
Your suggestion on how to do this was very close to the actual solution, but it's indeed not going to work.
In essence, what you'll indeed want to do, is to run the long-running function in a different thread. That'll mean you get 2 threads: one which is running the main event loop that (amongs other things) updates your UI, and another thread which does the long-running logic.
Of course, that bears the question: how do I notify the main thread that some work is done and I want it to react to that? For example, you might want to update the UI while (or after) some complex calculation is going on. For this, you can use GLib.idle_add() from within the other thread. That function takes a single callback as an argument, which it will run as soon as it can ("on idle").
So a possibility to use here, would be something like this:
class Main(Gtk.Window):
def __init__(self):
super().__init__()
self.button = Gtk.Button(label='button')
self.add(self.button)
self.button.connect('clicked', self.main_function)
thread = threading.Thread(target=self.threaded_function)
thread.daemon = True
thread.start()
def threaded_function(self):
# Really intensive stuff going on here
sleep(20)
# We're done, schedule "on_idle" to be called in the main thread
GLib.idle_add(self.on_idle)
# Note, this function will be run in the main loop thread, *not* in this one
def on_idle(self):
second_normal_function()
return GLib.SOURCE_REMOVE # we only want to run once
# ...
For more context, you might want to read the pygobject documentation on threading and concurrency
I have a script, let's say "sensors.py" in which I have a Class "Meas", that reads measurements from several sensors. These comes from serial ports, program makes some calculations on them and changes the class "self.variable_a value" and another self variables also. The readings are in continuous mode, i.e. the program automatically waits for a message to come from the sensor to the serial port and read whole line (it's done via pyserial library). Some transmit readings at frequency of 10Hz, others 20Hz or 100Hz. This is a really big and messy class, therefore I put it in a separate file.
In my "main.py" script I import this "sensors" file and instantiate the "Meas" Class. Now I have a problem. How can I run immediately some "on_changed_var_a" function in "main" script, only when the "variable_a" in "Meas" object has changed - without consuming CPU power with while loop (constatly checking whether by any chance the variable has not changed) or waiting with time.sleep()? I need to get the sensors readings changes and then run another functions in "main" script in the most efficient way, as fast as possible. Thanks in advance!
EDIT: added example files
"sensors.py" file:
import random
import time
import threading
running = True
class Meas1:
def __init__(self, xyz):
self.xyz = xyz
self.var_a = None
thr1 = threading.Thread(target=self.readings, daemon=True)
thr1.start()
def readings(self):
while running:
# simulating 5Hz sensor readings:
self.var_a = self.xyz * random.randint(1, 1000)
print(self.var_a)
time.sleep(0.2)
"main.py" file:
import time
import sensors
import threading
class MainClass:
def __init__(self):
print("started")
self.sensor1 = sensors.Meas1(xyz=7)
thr_ksr = threading.Thread(target=self.thr_keep_script_running, daemon=True)
thr_ksr.start()
# in this part I would like to run the on_changed_var_a function, immediately when var_a changes
thr_ksr.join()
def on_changed_var_a(self):
print("var_a changed: ", self.sensor1.var_a)
def thr_keep_script_running(self, t=10):
time.sleep(t)
sensors.running = False
print("stopped, sleeping 1 sec")
time.sleep(1)
mc = MainClass()
Not sure why this is tagged mutithreading. You need this function to be run on different thread?
To the problem. The easiest way would be to make Meas call function you will pass to it.
You could make variable_a a property and then in it's setter call the function you want. Function could be passed and assigned to self.call_on_a_change attr for example.
Edit:
I don't think there is a way to make function execute on different thread (well, you could start a new one for that purpose, which sounds like a great solution to me).
Another problem with threads is that you give control to the system. It decides when and for how long which thread runs. So "as fast as possible" is constrained by that.
Nonetheless, you could create a threading.Lock and try to acquire it from main thread. Then in the reading thread upon change you could release the Lock and allow main thread to execute all call_on_a_change. Something like this:
import time
import threading
lock = threading.Lock()
# change to locked
lock.acquire()
a_change_callbacks = []
def on_changed_var_a(new_a):
print(new_a)
def readings():
a_change_callbacks.append(lambda: on_changed_var_a('first `a` change'))
lock.release()
time.sleep(5)
a_change_callbacks.append(lambda: on_changed_var_a('second `a` change'))
lock.release()
time.sleep(5)
a_change_callbacks.append(lambda: on_changed_var_a('third `a` change'))
lock.release()
thr = threading.Thread(target=readings, daemon=True)
thr.start()
while True:
lock.acquire()
for callback in list(a_change_callbacks):
callback()
a_change_callbacks.remove(callback)
if not thr.is_alive():
break
It's not your class model, but I hope it's enough to show the idea :D
The goal is to create a docx document in parallel with the process of executing the rest of the program.
The "first" function should just call the asynchronous "second" which will create the docx.
Now i use modules asyncio, multiprocessing, concurrent.futures, but isn't create the docx:
def first(self, event):
pool = ThreadPoolExecutor(max_workers=multiprocessing.cpu_count())
loop = asyncio.get_event_loop()
loop.run_in_executor(pool, self.second)
async def second(self):
document = Document()
document.save('test.docx')
I'm sure the problem is with the "first" function, the way it calls "second", but one man said to me, that it's not the fault of asynchrony. Until I found the closest method to the solution, I was constantly faced with the problem that the document is created only after the completion of the entire program execution process - that's not the goal.
I'm working on an old project that doesn't have time to fix; there are a lot of errors in basic things inside, so the browser didn't help - it need something specific for the situation. Even so, please tell me how to solve the problem.
Thanks.
There's no need to create second async. I will presume that you can change it to regular function.
You probably just want to start file creation in background OS thread:
def first():
with ThreadPoolExecutor(max_workers=1) as executor:
fut = executor.submit(second) # start `second` in background
# rest of the program
fut.result() # make sure `second` is finished
def second():
document = Document()
document.save('test.docx')
In case bottleneck is disk I/O this should do the trick. In case bottleneck is CPU, you should consider using ProcessPoolExecutor instead of ThreadPoolExecutor.
Here's reproducible code to play with:
import time
from concurrent.futures import ThreadPoolExecutor
def first():
with ThreadPoolExecutor(max_workers=1) as executor:
fut = executor.submit(second) # start `second` in background
print('Rest of the program started')
time.sleep(2) # rest of the program
print('Rest of the program finished')
fut.result() # make sure `second` is finished
def second():
time.sleep(1) # create doc
print('Doc created')
first()
I'm having a problem that I can't figure out how to solve. Probably the solution is also simple, but at the moment it does not appear in my head. Ah, I didn't know what to write in the title, so if you want to change how you think it's lawful, do it =)
I put a simplified version of the code, where the problem appears:
from pyglet.gl import *
import threading, time, os
FPS = 120.0
class ucgm(pyglet.window.Window):
window_on_show = False
grid_set = 32, 18
old_set = 1024, 576
new_set = old_set
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.__video_set()
self.__resource = Resource_Load(self)
s_l = threading.Thread(target=self.__resource.start_load())
s_l.start()
def on_show(self):
print("1")
self.window_on_show = True
print("2")
def on_draw(self):
self.clear()
def update(self, dt):
pass
def on_close(self):
os._exit(0)
def __video_set(self):
self.__platform = pyglet.window.get_platform()
self.__default_display = self.__platform.get_default_display()
self.__default_screen = self.__default_display.get_default_screen()
self.set_size(self.new_set[0], self.new_set[1])
self.location = self.__default_screen.width // 2 - self.new_set[0] // 2, self.__default_screen.height // 2 - self.new_set[1] // 2
self.set_location(self.location[0], self.location[1])
class Resource_Load():
def __init__(self, main):
self.engine = main
self.loop_load = True
def start_load(self):
while self.loop_load:
if self.engine.window_on_show:
self.loop_load = False
print("3")
else:
print("4")
time.sleep(1)
print("5")
if __name__ == "__main__":
uc = ucgm()
pyglet.clock.schedule_interval(uc.update, 1 / FPS)
pyglet.app.run()
Before talking about the problem, I would like to explain what my intention was. What I thought was to create a class parallel to the main one that took care of loading all the resources used. In the meantime, it had to "dialogue" with the main class and modify a variable of it that showed the progress of the upload. Some time ago I asked a similar question on Stackoverflow about something similar (Link Question). The difference is that in that question the parallel function was internal to the main class. But given the amount of resources to load, I decided to dedicate a separate class to him. Once everything was written, I thought of another problem. Although it was written later, the Thread started before the window appeared completely. I thought of adding a simple time.sleep set with a few seconds. But then I reasoned that there was the possibility of a delay, caused by excessive use of RAM by the user, and therefore the time set was not sufficient. So I modified, adding a while loop with the condition that if the window was visible, it would have ended the loop and continued loading, otherwise it would have waited a second. nd here we are finally in the problem. The script as it is written does not work. Or rather, it remains stuck in the cycle, despite being open in a Thread. I knew, from my previous question, that the Thread should not directly touch Pyglet, and in fact it does not happen, since it is only reading the status of a variable. So I don't understand why I'm getting this result. The cycle seems to be not happening in parallel and therefore blocks the on_show function which fails to update the variable and thus to break the cycle. Do you have any suggestions for me to solve the problem I'm having? Thanks for your help
Instead of passing a reference to the start_load function on this line:
s_l = threading.Thread(target=self.__resource.start_load())
...that line is actually calling that function (notice the ()), which is running that infinite loop on the main thread. Get rid of that (); threading.Thread will do the call for you:
s_l = threading.Thread(target=self.__resource.start_load)
Additionally, there are better ways to wait for something to happen in one thread from another. You should consider using one of the thread synchronization object provided in Python, for example, a threading.Event:
class ucgm(pyglet.window.Window):
window_opened = threading.Event()
// ...
def on_show(self):
self.window_opened.set()
// ...
class Resource_Load():
// ...
def start_load(self):
self.engine.window_opened.wait()
print("done")
// ...
I'm writing an emulator for a vintage computer system in Python, and I'm having some trouble with an exception thrown when trying to "restart" the emulator core thread after coming out of a halt condition. The "run processor" method, part of a larger class looks something like this:
def run_processor(self):
processor = self
class processor_thread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.daemon = True
def run(self):
while processor.running:
#main loop for instruction fetch and decode is here
self.processor_thread = processor_thread()
self.running = True
self.processor_thread.start()
I have a main function where I load a memory image into the processor, set the program counter, and run a test program on the emulator. This outputs some stuff to the console, and eventually the processor's "HLT" instruction sets "processor.running" to False, terminating processor_thread.
This works OK, but where I'm running in to trouble is a test of restarting the processor by calling run_processor a second time:
processor = Processor(memory, scheduler, interrupts, teleprinter)
processor.program_counter = 128
processor.run_processor()
while processor.processor_thread.isAlive():
pass
processor.program_counter = 128
processor.run_processor()
The first instance runs fine, but when the run_processor method is called a second time I get the following error:
Exception in thread Thread-3 (most likely raised during interpreter
shutdown)
How can I rectify this? Thanks.
EDIT: I broke the code down to its bare essentials and found it actually works OK. I didn't notice that my HALT method was actually written in such a way that it shut down all the processor peripheral threads, including the thread that runs the teleprinter emulator. Without the teleprinter to output to it looks like the emulator core crashed on the second test.
Here's the test case:
import threading
import time
class Processor(object):
def __init__(self):
self.running = False
def halt_processor(self):
self.running = False
def run_processor(self):
processor = self
class processor_thread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.daemon = True
def run(self):
while processor.running:
#main loop for instruction fetch and decode is here
print "Hello, I am the main loop doing my thing 1"
time.sleep(1)
print "Hello, I am the main loop doing my thing 2"
time.sleep(1)
print "I will halt now."
processor.halt_processor()
self.processor_thread = processor_thread()
self.running = True
self.processor_thread.start()
def test1():
processor = Processor()
processor.run_processor()
def test2():
processor = Processor()
processor.run_processor()
while processor.processor_thread.isAlive():
pass
processor.run_processor()
def main():
test2()
if __name__ == '__main__':
main()
Works OK.
Firstly, you can ignore this error. If your program works, any errors during shutdown could be ignored. I'm fully with you that this isn't the way things should be and that a clean solution is called for.
What I think is missing is the call to thread.join(). You are currently busy-waiting for the thread to return false from isAlive() (which should be is_alive(), btw), using join() is cleaner and shorter.
BTW: There is no need to derive from the Thread class, which makes the code a bit shorter. I personally find it clearer, too, because the Python object is just a handle to the thread and not the thread itself.
The way I see it, the main thread ends just after starting again the Processor thread.
Have you tried to put a while after starting it?
P.S.: Also, some nice working code would make debugging much easier for those that want to help.