Relative newcomer to wxPython and threading, hence confusion.
I have 5 "steps" that must be performed on my target data. The steps must be performed sequentially in a given order.
In order to keep the GUI responsive, I've coded each step as an individual thread.
When I step through the process manually (using a button to start each step) I can watch each step update the display when finished.
I'm lost as to how to automate the calling of each of the threads sequentially. If I were to create a button labeled "Do all 5 steps in a row", what would the code called by the button look like?
I've tried simply calling each of the threaded methods sequentially, but they all attempt to operate on the data at once. Each thread uses a 'with' and a threading.Lock() to prevent other threads from accessing the data while a given thread is running, but the threads do not appear to be executing in the correct order.
What is the proper way to write
call_thread1()
call_thread2()
call_thread3()
..etc.
that will always execute the threads in the given order and only after each thread is done without blocking wx?
Have a waiting step member of your gui as a list and a thread finished handler that if there is a next task pops it off the list and starts the thread.
In __init__(self) :
self.TaskList = []
In OnDoAllClicked(self, evt):
self.TaskList.extend([call_thread1, call_thread2, call_thread3, etc])
CheckNextTask()
In CheckNextTask(self) :
if len(self.TaskList) > 0:
current = self.TaskList[0]
del self.TaskList[0]
current(callback=self.CheckNextTask)
In each thread:
while not Done:
# Do whatever
wx.CallAfter(callbacK)
I think I would go with a different approach. I would call a single thread that has each of the five functions in it. Then that thread can call each function in order and when each function returns, it can send an update to the GUI using one of the GUI's thread-safe methods (wx.CallAfter, wx.PostEvent).
Here are a couple of resources for more information regarding wxPython and threads:
http://wiki.wxpython.org/LongRunningTasks
http://www.blog.pythonlibrary.org/2010/05/22/wxpython-and-threads/
Related
I am trying to call a thread I define in a function from another function. Here is the first function, its purpose is to create and start a thread:
def startThread(func):
listen = threading.Thread(target = func)
listen.start()
I am trying to implement a function that will close the thread created in that first function, how should I go about it? I don't know how to successfully pass the thread.
def endThread(thread):
thread.exit()
Thank you!
This problem is almost FAQ material.
To summarise, there is no way to kill a thread from the outside. You can of course pass the thread object to any function you want, but threading library is missing kill and exit calls.
There are more or less two distinct ways around this, depending on what your thread does.
The first method is to make it so that your thread co-operates. This approach is discussed here: Is there any way to kill a Thread in Python? This method adds a check to your thread loop and a way to raise a "stop signal", which will then cause the thread to exit from the inside when detected.
This method works fine if your thread is a relatively busy loop. If it is something that is blocking in IO wait, not so much, as your thread could be blocking in a read call for days or weeks before receiving something and executing the signal check part. Many IO calls accept a timeout value, and if it is acceptable to wait a couple of seconds before your thread exits, you can use this to force the exit check every N seconds without making your thread a busy loop.
The other approach is to replace threads with processes. You can force kill a subprocess. If you can communicate with your main program with queues instead of shared variables, this is not too complicated, either. If your program relies heavily on sharing global variables, this would require a major redesign.
If your program is waiting in IO loops, you need instantaneous termination and you are using shared global variables, then you are somewhat out of luck, as you either need to accept your threads not behaving nicely or you need to redesign some parts of your code to untangle either the IO wait or shared variables.
I have a GUI running in the main thread with a second thread constantly taking input from a shared Queue. Multiple background threads are doing different processes and the second thread takes all of their output and updates the GUI. So far, no problems have come about, but there are countless warnings Tkinter is not thread safe.
With that being the case, I can copy all of this code into another constant loop, but I can't find out where I could put it so it's also in the main thread and does not lock up the GUI. What is the right way to do this?
Thanks.
Tkinter has an event loop that runs. You can leverage that by writing a function that continuously adds itself to the event queue periodically.
The solution looks roughly like this:
def some_func():
<do your work here>
# call this function again in 1 second
root.after(1000, some_func)
Once you call it once, it will continue to be called once a second. In production code you'll need to add a bit of bulletproofing, such as checking a flag that gets set when the app is shutting down.
This isn't exactly recursion, so you don't have to worry about running out of stack space since all this does is adds a new job in the event queue to run no sooner than one second after it was placed in the queue.
My code spawns a number of threads to manage communications with a number of I/O boards. Generally the threads receive events from the boards and update external data sources as necessary. The threads (1 or more) are invoked as:
phThreadDict[devId] = ifkit(self, phDevId, phIpAddr, phIpPort, phSerial)
phThreadDict[devId].start()
This works fine. However, in some cases I also need the thread to send a message to the boards. The thread contains a method that does the work and I call that method, from the main thread, as: (this example turns on a digital output)
phThreadDict[devId].writeDigitalOutput(digitalOut, True)
this is the method contained in the thread:
def writeDigitalOutput(self,index, state):
interfaceKit.setOutputState(index, state)
threading.enumerate() produces:
{134997634: <ifkit(Thread-1, started daemon)>, 554878244: <ifkit(Thread-3, started daemon)>, 407897606: <tempsensor(Thread-4, started daemon)>}
and the instance is:
<ifkit(Thread-3, started daemon)>
This works fine if I have only one thread. But, if I have multiple threads, only one is used - the choice appears to be made at random when the program starts.
I suspect that storing the thread identifier in the dict is the problem, but still, it works with one thread.
Instead of storing your threads in a "simple" associative array maybe you should instantiate a threadpool beforehand (you can find an example of implementation here h**p://code.activestate.com/recipes/577187-python-thread-pool/ or directly use the following lib http://pypi.python.org/pypi/threadpool).
Also instantiate a "watchdog", each of your thread will hold a reference to this watchdog, so when your threads need to do their callback they'll send back the info to this watchdog. (beware of the deadlock, look at http://dabeaz.blogspot.fr/2009/11/python-thread-deadlock-avoidance_20.html).
Note : sorry for the lame "h**p" but SO won't let me post more than 2 links....
I am fairly new to Python programming and Threads isn't my area of expertise. I have a problem for which i would hope that people here can help me out with.
Task: as a part of my master thesis, i need to make a mixed reality game which involves multiplayer capability. in my game design, each player can set a bunch of traps, each of which is active for a specific time period e.g. 30 secs. In order to maintain a consistent game state across all the players, all the time check needs to be done on the server side, which is implemented in Python.
I decided to start a python thread, everytime a new trap is laid by a player and run a timer on the thread. All this part is fine, but the real problem arises when i need to notify the main thread that the time is up for this particular trap, so that i can communicate the same to the client (android device).
i tried creating a queue and inserting information into the queue when the task is done, but i cant do a queue.join() since it will put the main thread on hold till the task is done and this is not what i need nor is it ideal in my case, since the main thread is constantly communicating with the client and if it is halted, then all the communication with the players will come to a standstill.
I need the secondary thread, which is running a timer, to tell the main thread, as soon as the time runs out that the time has run out and send the ID of the trap, so that i can pass this information to the android client to remove it. How can i achieve this ??
Any other suggestions on how this task can be achieved without starting a gazillion threads, are also welcome.. :) :)
Thanks in advance for the help..
Cheers
i have finally found a nice little task scheduler written in python, which actually is quite light and quite handy to schedule events for a later time or date with a callback mechanism, which allows the child thread to pass-back a value to the main thread notifying the main thread of its status and whether the job was successfully done or not.
people out there, who need a similar functionality as the one in the question and dont want to haggle around with threads can use this scheduler to schedule their events and get a callback when the event is done
here is the link to APScheduler
It may be easier to have the timers all done in the main thread - have a list of timers that you keep appending new ones to. Each timer doesn't actually do anything, it just has a time when it goes off - which is easier if you work in arbitrary 'rounds' than in real time, but still doable. Each interval, the mainloop should check all of them, and see if it is time (or past time) for them to expire - if it is, remove them from the list (of course, be careful about removing items from a list you're iterating over - it mightn't do what you expect).
If you have a lot of timers, and by profiling you find out that running through all of them every interval is costing you too much time, a simple optimisation would be to keep them in a heapq - this will keep them sorted for you, so you know after the first one that hasn't expired yet that none of the rest have either. Something like:
while True:
if not q:
break
timer = heapq.heappop(q)
if timer.expiry <= currenttime:
# trigger events
else:
heapq.heappush(q)
break
This does still cost you one unnecessary pop/push pair, but its hard to see how you would do better - again, doing something like:
for timer in q:
if timer.expiry <= currenttime:
heapq.heappop(timer)
# trigger events
else:
break
Can have subtle bugs because list iterators (functions in heapq work on sequences and use side effects, rather than there being a full-fledged heapq class for some reason) work by keeping track of what index they're up to - so if you remove the current element, you push everything after it one index to the left and end up skipping the next one.
The only important thing is that currenttime is consistently updated each interval in the main loop (or, if your heart is set on having it in real time, based on the system clock), and timer.expiry is measured in the same units - if you have a concept of 'rounds', and a trap lasts six rounds, when it is placed you would do heapq.heappush(q, Timer(expiry=currenttime+6).
If you do want to do it the multithreaded way, your way of having a producer/consumer queue for cleanup will work - you just need to not use Queue.join(). Instead, as the timer in a thread runs out, it calls q.put(), and then dies. The mainloop would use q.get(False), which will avoid blocking, or else q.get(True, 0.1) which will block for at most 0.1 seconds - the timeout can be any positive number; tune it carefully for the best tradeoff between blocking long enough that clients notice and having events go off late because they only just missed being in the queue on time.
The main thread creates a queue and a bunch of worker threads that are
pulling tasks from the queue. As long as the queue is empty all worker
threads block and do nothing. When a task is put into the queue a random
worker thread acquires the task, does it job and sleeps as soon as its
ready. That way you can reuse a thread over and over again without
creating a new worker threads.
When you need to stop the threads you put a kill object into the queue
that tells the thread to shut down instead of blocking on the queue.
I do a time-consuming processing in the _init callback function. Therefore, at the GUI, when I activate the check-button, the program freezes, while the processing runs to completion, and then the check mark appears and execution resumes normally.
self.init = gtk.CheckButton("Init", False)
self.init.connect("toggled", self._init)
def _init(self, w):
....
How do I make the GUI more responsive while the processing takes place ?
And is there a way I can make the GUI change the state of the check-box right in the beginning and then enter the heavy loop ?
You have several solutions:
Make the computation in a worker thread, which will send the result to the main thread. But don't change GUI items in the worker thread. Like most toolkits, GTK is not thread safe. I don't really like that one, as I don't really like threads, as they tend to make debugging harder...
Split the processing in an idle handle. When I say split, I really mean to split it in several steps that take much less time to compute. You should use an automaton with several states taking each much less time. Putting a big fat calculus in an idle handler is like calling sleep(5) in that handler: the GUI message pump would be blocked for 5 seconds, with no way to handle redrawing events... Here's an example of lazy loading data using this technique.
Force a call to the message pump on a regular basis using gtk_main_iteration.
The examples I gave you are in C, but the spirit is the same in python.
In the handler, register the rest of the init as idle-handler and return. gtk.idle_add IIRC. That way it will run after the gui events are handled. Check documentation to see which value you need to return so the function is not started again.