I'm writing an emulator for a vintage computer system in Python, and I'm having some trouble with an exception thrown when trying to "restart" the emulator core thread after coming out of a halt condition. The "run processor" method, part of a larger class looks something like this:
def run_processor(self):
processor = self
class processor_thread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.daemon = True
def run(self):
while processor.running:
#main loop for instruction fetch and decode is here
self.processor_thread = processor_thread()
self.running = True
self.processor_thread.start()
I have a main function where I load a memory image into the processor, set the program counter, and run a test program on the emulator. This outputs some stuff to the console, and eventually the processor's "HLT" instruction sets "processor.running" to False, terminating processor_thread.
This works OK, but where I'm running in to trouble is a test of restarting the processor by calling run_processor a second time:
processor = Processor(memory, scheduler, interrupts, teleprinter)
processor.program_counter = 128
processor.run_processor()
while processor.processor_thread.isAlive():
pass
processor.program_counter = 128
processor.run_processor()
The first instance runs fine, but when the run_processor method is called a second time I get the following error:
Exception in thread Thread-3 (most likely raised during interpreter
shutdown)
How can I rectify this? Thanks.
EDIT: I broke the code down to its bare essentials and found it actually works OK. I didn't notice that my HALT method was actually written in such a way that it shut down all the processor peripheral threads, including the thread that runs the teleprinter emulator. Without the teleprinter to output to it looks like the emulator core crashed on the second test.
Here's the test case:
import threading
import time
class Processor(object):
def __init__(self):
self.running = False
def halt_processor(self):
self.running = False
def run_processor(self):
processor = self
class processor_thread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.daemon = True
def run(self):
while processor.running:
#main loop for instruction fetch and decode is here
print "Hello, I am the main loop doing my thing 1"
time.sleep(1)
print "Hello, I am the main loop doing my thing 2"
time.sleep(1)
print "I will halt now."
processor.halt_processor()
self.processor_thread = processor_thread()
self.running = True
self.processor_thread.start()
def test1():
processor = Processor()
processor.run_processor()
def test2():
processor = Processor()
processor.run_processor()
while processor.processor_thread.isAlive():
pass
processor.run_processor()
def main():
test2()
if __name__ == '__main__':
main()
Works OK.
Firstly, you can ignore this error. If your program works, any errors during shutdown could be ignored. I'm fully with you that this isn't the way things should be and that a clean solution is called for.
What I think is missing is the call to thread.join(). You are currently busy-waiting for the thread to return false from isAlive() (which should be is_alive(), btw), using join() is cleaner and shorter.
BTW: There is no need to derive from the Thread class, which makes the code a bit shorter. I personally find it clearer, too, because the Python object is just a handle to the thread and not the thread itself.
The way I see it, the main thread ends just after starting again the Processor thread.
Have you tried to put a while after starting it?
P.S.: Also, some nice working code would make debugging much easier for those that want to help.
Related
Simply put, I want to properly implement threading in a Python GTK application. This is in order to prevent UI freezing due to functions/code taking a long time to finish running. Hence, my approach was to move all code which took a long time to run into separate functions, and run them in their separate threads as needed. This however posed a problem when trying to run the functions in sequence.
For example, take a look at the following code:
class Main(Gtk.Window):
def __init__(self):
super().__init__()
self.button = Gtk.Button(label='button')
self.add(self.button)
self.button.connect('clicked', self.main_function)
def threaded_function(self):
time.sleep(20)
print('this is a threaded function')
def first_normal_function(self):
print('this is a normal function')
def second_normal_function(self):
print('this is a normal function')
def main_function(self, widget):
self.first_normal_function()
self.threaded_function()
self.second_normal_function()
Pressing the button starts main_function which then starts 3 functions in sequence. threaded_function represents a function which would take a long time to complete. Running this as is will freeze the UI. Hence it should be threaded as such:
...
...
def main_function(self, widget):
self.first_normal_function()
thread = threading.Thread(target=self.threaded_function)
thread.daemon = True
thread.start()
self.second_normal_function()
What should happen is that the following first_normal_function should run, then threaded_function in a background thread - the UI should remain responsive as the background thread is working. Finally, second_normal_function should run, but only when threaded_function is finished.
The issue with this is that the functions will not run in sequence. The behaviour I am looking for could be achieved by using thread.join() however this freezes the UI.
So I ask, what's the proper way of doing this? This is a general case, however it concerns the general issue of having code which takes a long time to complete in a graphical application, while needing code to run sequentially. Qt deals with this by using signals, and having a QThread emit a finished signal. Does GTK have an equivalent?
I'm aware that this could be partially solved using Queue , with a put() and get() in relevant functions, however I don't understand how to get this to work if the main thread is calling anything other than functions.
EDIT: Given that it's possible to have threaded_function call second_normal_function using GLib.idle_add, let's take an example where in main_function, the second_normal_function call is replaced with a print statement, such that:
def main_function(self, widget):
self.first_normal_function()
thread = threading.Thread(target=self.threaded_function)
thread.daemon = True
thread.start()
print('this comes after the thread is finished')
...
...
...
#some more code here
With GLib.idle_add, the print statement and all the code afterwards would need to be moved into a separate function. Is it possible to avoid moving the print statement into its own function while maintaining sequentiality, such that the print statement remains where it is and still gets called after threaded_function is finished?
Your suggestion on how to do this was very close to the actual solution, but it's indeed not going to work.
In essence, what you'll indeed want to do, is to run the long-running function in a different thread. That'll mean you get 2 threads: one which is running the main event loop that (amongs other things) updates your UI, and another thread which does the long-running logic.
Of course, that bears the question: how do I notify the main thread that some work is done and I want it to react to that? For example, you might want to update the UI while (or after) some complex calculation is going on. For this, you can use GLib.idle_add() from within the other thread. That function takes a single callback as an argument, which it will run as soon as it can ("on idle").
So a possibility to use here, would be something like this:
class Main(Gtk.Window):
def __init__(self):
super().__init__()
self.button = Gtk.Button(label='button')
self.add(self.button)
self.button.connect('clicked', self.main_function)
thread = threading.Thread(target=self.threaded_function)
thread.daemon = True
thread.start()
def threaded_function(self):
# Really intensive stuff going on here
sleep(20)
# We're done, schedule "on_idle" to be called in the main thread
GLib.idle_add(self.on_idle)
# Note, this function will be run in the main loop thread, *not* in this one
def on_idle(self):
second_normal_function()
return GLib.SOURCE_REMOVE # we only want to run once
# ...
For more context, you might want to read the pygobject documentation on threading and concurrency
I have a script, let's say "sensors.py" in which I have a Class "Meas", that reads measurements from several sensors. These comes from serial ports, program makes some calculations on them and changes the class "self.variable_a value" and another self variables also. The readings are in continuous mode, i.e. the program automatically waits for a message to come from the sensor to the serial port and read whole line (it's done via pyserial library). Some transmit readings at frequency of 10Hz, others 20Hz or 100Hz. This is a really big and messy class, therefore I put it in a separate file.
In my "main.py" script I import this "sensors" file and instantiate the "Meas" Class. Now I have a problem. How can I run immediately some "on_changed_var_a" function in "main" script, only when the "variable_a" in "Meas" object has changed - without consuming CPU power with while loop (constatly checking whether by any chance the variable has not changed) or waiting with time.sleep()? I need to get the sensors readings changes and then run another functions in "main" script in the most efficient way, as fast as possible. Thanks in advance!
EDIT: added example files
"sensors.py" file:
import random
import time
import threading
running = True
class Meas1:
def __init__(self, xyz):
self.xyz = xyz
self.var_a = None
thr1 = threading.Thread(target=self.readings, daemon=True)
thr1.start()
def readings(self):
while running:
# simulating 5Hz sensor readings:
self.var_a = self.xyz * random.randint(1, 1000)
print(self.var_a)
time.sleep(0.2)
"main.py" file:
import time
import sensors
import threading
class MainClass:
def __init__(self):
print("started")
self.sensor1 = sensors.Meas1(xyz=7)
thr_ksr = threading.Thread(target=self.thr_keep_script_running, daemon=True)
thr_ksr.start()
# in this part I would like to run the on_changed_var_a function, immediately when var_a changes
thr_ksr.join()
def on_changed_var_a(self):
print("var_a changed: ", self.sensor1.var_a)
def thr_keep_script_running(self, t=10):
time.sleep(t)
sensors.running = False
print("stopped, sleeping 1 sec")
time.sleep(1)
mc = MainClass()
Not sure why this is tagged mutithreading. You need this function to be run on different thread?
To the problem. The easiest way would be to make Meas call function you will pass to it.
You could make variable_a a property and then in it's setter call the function you want. Function could be passed and assigned to self.call_on_a_change attr for example.
Edit:
I don't think there is a way to make function execute on different thread (well, you could start a new one for that purpose, which sounds like a great solution to me).
Another problem with threads is that you give control to the system. It decides when and for how long which thread runs. So "as fast as possible" is constrained by that.
Nonetheless, you could create a threading.Lock and try to acquire it from main thread. Then in the reading thread upon change you could release the Lock and allow main thread to execute all call_on_a_change. Something like this:
import time
import threading
lock = threading.Lock()
# change to locked
lock.acquire()
a_change_callbacks = []
def on_changed_var_a(new_a):
print(new_a)
def readings():
a_change_callbacks.append(lambda: on_changed_var_a('first `a` change'))
lock.release()
time.sleep(5)
a_change_callbacks.append(lambda: on_changed_var_a('second `a` change'))
lock.release()
time.sleep(5)
a_change_callbacks.append(lambda: on_changed_var_a('third `a` change'))
lock.release()
thr = threading.Thread(target=readings, daemon=True)
thr.start()
while True:
lock.acquire()
for callback in list(a_change_callbacks):
callback()
a_change_callbacks.remove(callback)
if not thr.is_alive():
break
It's not your class model, but I hope it's enough to show the idea :D
I am trying to run a function in the background till some work is done in the main function and then finish the thread. I have implemented the threading logic in a separate class and the main in another file. But every time I run it the target function only seems to run once and then waits
Here is the main function
from ThreadRipper import *
thread_obj=ThreadRipper()
thread_obj.start_thread()
squish.snooze(10)
print("Main Continuing")
thread_obj.stop_thread()
And the implemented class is as follows
class ThreadRipper():
def __init__(self):
lib_path="iCOMClient.dll"
self.vx = IcomVx(lib_path)
config = ConfigParser.SafeConfigParser(allow_no_value=True)
config.readfp(open("vx_logger_config.cfg"))
self.vx.connect(config.get("icom","ip"), timeout_millis = 30000)
self.t = threading.Thread(target=self.task_to_do, args=(self.vx,))
def stop_thread(self):
self.t.do_run=False
self.t.join()
def start_thread(self):
self.t.start()
def task_to_do(self,arg):
current_thread=threading.currentThread()
while getattr(current_thread,"do_run",True):
with open("vx.txt",'a') as f:
f.write(str(arg.get_next_message()._rawmsg)+"\n")
time.sleep(1)
print("Stopping")
arg.disconnect()
When I run this I get the vx file created but with only one entry, I expect it to be written to continuously till the while loop exits. I am quite new at threading and may have understood it incorrectly. Please advise
Thank you
The reason is probably because
print("Stopping")
arg.disconnect()
are both inside the while loop. After disconnecting, arg doesn't seem to produce any more messages.
(Unless, of course, your code in the question is not what you really have, but in this case, you surely would have edited your question so it matches.)
I've been playing along with threads in Python, and I came across something interesting with the following code:
import time
import threading
class Update(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.stop_event = threading.Event()
def join(self, timeout=None):
self.stop_event.set()
threading.Thread.join(self, timeout)
def run(self):
while not self.stop_event.isSet():
print("test")
thread = Update()
thread.start()
This code randomly stops even if I don't call the join() method. As a result, I get different outputs like these:
test#debian:~/$ python3 test.py
test
test
test
test
test#debian:~/$ python3 test.py
test
test
test
test
test
test
test
test#debian:~/$ python3 test.py
test
test
Why is this code randomly stopping? I thought that only by setting stop_event this thread would stop.
You already got the essential answer, but there's a detail you need to be aware of: when the main program ends, as part of shutdown processing Python calls .join() on all non-daemon threads created by the threading module. You overrode .join(), so Python calls your .join(). That in turn sets the event, and so your .run() method exits silently.
When the main thread ends the program ends.
The number of times that the thread loops before the main one stops is fairly arbitrary. (up to the OS to schedule)
I'm trying to wrap the blocking calls in pyaudio with a thread to give me non-blocking access through queues. However, the problem I have is not with pyaudio, or queues, but with the issue of trying to test a thread. In keeping with "strip the example down to the minimum possible", all the pyaudio stuff has vanished, to leave only the thread class, and its instantiation in a main.
What I was hoping for was an object that I could create, and leave to get on with its stuff in the background, while I do control things with the console or tk. I figure the following max-stripped down example should have the thread doing stuff, while main runs and asks me if it is working. The raw_input prompt never appears. I would not be surprised at this if I was running it from IDLE, which is not thread safe, but I get the same behaviour if I run the script directly from the OS. I was prepared to see the raw input prompt disappear up the screen pushed by 'running' prints, but not even that happens. The prompt never appears. What's going on? It does respond to ctrl-C and to closing the window, but I'd still like to be able to see main running.
import threading
import time
class TestThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.running=True
self.run()
def run(self):
while self.running:
time.sleep(0.5)
print 'running'
def stop(self):
self.running=False
if __name__=='__main__':
tt=TestThread()
a=raw_input('simple stuff working ? -- ')
tt.stop()
You should start the thread with self.start() instead of self.run(). In this case you are just running the thread function like any other normal function.
Normally you do not inherit from Thread. Instead, you use Thread(target=func2run).start()