New to SO, please forgive any etiquette errors (point out if there are!).
I'm working on a script that is running on the programs main UI thread. That being said, I need to avoid all blocking calls to ensure user can still interact. I do not have access to the UI event loop so any busy loop solutions aren't possible in my situation.
I have a simple background thread that is communicating to another app and gathering data, and storing in a simple array for consumption. Each time this data is updated I need to use the data to modify the UI (this must run in main thread). Ideally the background thread would emit a signal each time the data is updated then in the main thread a listener would handle this and modify the UI. A busy loop is not an option everything must be asyncronous/event based.
I have the data gathering continuossly running in the background using threading.timer(..). However since this runs in a seperate thread, the UI operations need to be called externally to this.
def _pollLoop(self):
if (self._isCubeControl):
self._getNewData()
#in a perfect world, updateUI() would be here
self._pollThread = threading.Timer(0.1,self._pollLoop)
self._pollThread.start()
I need a way for this pollLoop to callback to main thread so I can update the UI. I've tried direct callbacks within the pollLoop but the callback are ran within the seperate thread causing errors.
Looking for a way to attach listener to the data object so that on change updateUI() can be ran IN MAIN THREAD.
Thanks for any help you can offer! If this is at all vague please let me know
Update
Based off of #CAB's answer I'm now trying to implement an Observer Pattern. The difficulty is that the Observable is to be ran in a spawned thread while the Observer update function must run in the main thread. I've implemented the example chad lung (http://www.giantflyingsaucer.com/blog/?p=5117).
import threading
class Observable(object):
def __init__(self):
self.observers = []
def register(self, observer):
if not observer in self.observers:
self.observers.append(observer)
def unregister(self, observer):
if observer in self.observers:
self.observers.remove(observer)
def unregister_all(self):
if self.observers:
del self.observers[:]
def update_observers(self, *args, **kwargs):
for observer in self.observers:
observer.update(*args, **kwargs)
thread = threading.Timer(4,self.update_observers).start()
from abc import ABCMeta, abstractmethod
class Observer(object):
__metaclass__ = ABCMeta
#abstractmethod
def update(self, *args, **kwargs):
pass
class myObserver(Observer):
def update(self, *args, **kwargs):
'''update is called in the source thread context'''
print(str(threading.current_thread()))
observable = Observable()
observer = myObserver()
observable.register(observer)
observable.update_observers('Market Rally', something='Hello World')
What I get in response is:
<_MainThread(MainThread, started 140735179829248)>
<_Timer(Thread-1, started 123145306509312)>
<_Timer(Thread-2, started 123145310715904)>
So clearly the Observer is running in the spawned thread and not main. Anyone have another method for me? :) Once again I cannot have a busy loop to periodically check for value change (i wish.. :( ) This script is running overtop a UI and I do not have access to the GUI event loop, so everything needs to be asynchronous and non-blocking.
Let's build on that example from http://www.giantflyingsaucer.com/blog/?p=5117.
from abc import ABCMeta, abstractmethod
class Observer(object):
__metaclass__ = ABCMeta
#abstractmethod
def update(self, *args, **kwargs):
pass
The onus is then on the Observer implementation to disconnect the threads. Let's say we do this using a simplistic thread. (syntax might be off, I'm cramming this in and need to catch a bus).
from observer import Observer
from threading import Thread
class myObserver(Observer):
def update(self, *args, **kwargs):
'''update is called in the source thread context'''
Thread(target=self.handler, args=(self,*args), kwargs=**kwargs).start()
def handler(self, *args, **kwargs):
'''handler runs in an independent thread context'''
pass # do something useful with the args
Related
I am trying to run multiple tasks in queue. The tasks come on user input. What i tried was creating a singleton class with ThreadPoolExecutor property and adding tasks into it. The tasks are added fine, but it looks like only the first addition of set of tasks works. The following are added but not executed.
class WebsiteTagScrapper:
class __WebsiteTagScrapper:
def __init__(self):
self.executor = ThreadPoolExecutor(max_workers=5)
instance = None
def __new__(cls): # __new__ always a classmethod
if not WebsiteTagScrapper.instance:
WebsiteTagScrapper.instance = WebsiteTagScrapper.__WebsiteTagScrapper()
return WebsiteTagScrapper.instance
I used multiprocess in one of my project without using celery, cause i think it was overkill for my use.
Maybe you could do something like this:
from multiprocessing import Process
class MyQueuProcess(Process):
def __init__(self):
super(MyQueuProcess, self).__init__()
self.tasks = []
def add_task(self, task):
self.tasks.append(task)
def run(self):
for task in self.tasks:
#Do your task
You just have to create an instance in your view, set up your task and then run(). Also if you need to access your database, you will need to import django in your child and then make a django.setup().
I'm working on a GUI application, developed in Python and its UI library : PySide2 (Qt wrapper for Python)
I have a heavy computation function I want to put on another thread in order to not freeze my UI. The Ui should show "Loading" and when the function is over, receive from it it's results and update the UI with it.
I've tried a lot of different codes, a lot of examples are working for others but not me, is it PySide2 fault ? (For example this is almost what I want to do : Updating GUI elements in MultiThreaded PyQT)
My code is :
class OtherThread(QThread):
def __init__(self):
QThread.__init__(self)
def run(self):
print 'Running......'
self.emit(SIGNAL("over(object)"), [(1,2,3), (2,3,4)])
#Slot(object)
def printHey( obj):
print 'Hey, I\'ve got an object ',
print obj
thr = OtherThread()
self.connect(thr,SIGNAL("over(object)"),printHey)
thr.start()
My code is working if I use primitives such as bool or int but not with object. I see 'Running....' but never the rest.
Hope someone can enlighten me
You can't define signals dynamically on a class instance. They have to be defined as class attributes. You should be using the new-style signals and slot syntax.
class OtherThread(QThread):
over = QtCore.Signal(object)
def run(self):
...
self.over.emit([(1,2,3), (2,3,4)])
class MyApp(QtCore.QObject)
def __init__(self):
super(MyApp, self).__init__()
self.thread = OtherThread(self)
self.thread.over.connect(self.on_over)
self.thread.start()
#QtCore.Slot(object)
def on_over(self, value):
print 'Thread Value', value
Is there a way to run an arbitrary method whenever a new thread is started in Python (2.7)? My goal is to use setproctitle to set an appropriate title for each spawned thread.
Just inherit from threading.Thread and use this class instead of Thread - as long as you have control over the Threads.
import threading
class MyThread(threading.Thread):
def __init__(self, callable, *args, **kwargs):
super(MyThread, self).__init__(*args, **kwargs)
self._call_on_start = callable
def start(self):
self._call_on_start()
super(MyThread, self).start()
Just as a coarse sketch.
Edit
From the comments the need arose to kind of "inject" the new behavior into an existing application. Let's assume you have a script that itself imports other libraries. These libraries use the threading module:
Before importing any other modules, first execute this;
import threading
import time
class MyThread(threading.Thread):
_call_on_start = None
def __init__(self, callable_ = None, *args, **kwargs):
super(MyThread, self).__init__(*args, **kwargs)
if callable_ is not None:
self._call_on_start = callable_
def start(self):
if self._call_on_start is not None:
self._call_on_start
super(MyThread, self).start()
def set_thread_title():
print "Set thread title"
MyThread._call_on_start = set_thread_title()
threading.Thread = MyThread
def calculate_something():
time.sleep(5)
print sum(range(1000))
t = threading.Thread(target = calculate_something)
t.start()
time.sleep(2)
t.join()
As subsequent imports only do a lookup in sys.modules, all other libraries using this should be using our new class now. I regard this as a hack, and it might have strange side effects. But at least it's worth a try.
Please note: threading.Thread is not the only way to implement concurrency in python, there are other options like multiprocessing etc.. These will be unaffected here.
Edit 2
I just took a look at the library you cited and it's all about processes, not Threads! So, just do a :%s/threading/multiprocessing/g and :%s/Thread/Process/g and things should be fine.
Use threading.setprofile. You give it your callback and Python will invoke it every time a new thread starts.
Documentation here.
I have a UUT class which instantiates Worker objects, and calls their do_stuff() method.
The Worker objects uses a Provider object for two things:
Calls methods on the provider object to do some stuff
Gets notifications from the provider by subscribing a method with the provider's events
When a worker gets a notification, it processes it, an notifies the UUT object, which in reponse can create more Worker objects.
I've already tested each class on its own, and I want to test UUT+Worker together. For that, I intend to mock-out Provider.
import mock
import unittest
import provider
class Worker():
def __init__(self, *args):
resource.default_resource.subscribe('on_spam', self._on_spam) # I'm going to patch 'resource.default_resource'
def do_stuff(self):
self.resource.do_stuff()
def _on_spam(self, message):
self._tell_uut_to_create_more_workers(message['num_of_new_workers_to_create'])
class UUT():
def __init__(self, *args):
self._workers = []
def gen_worker_and_do_stuff(self, *args)
worker = Worker(*args)
self._workers.append(resource)
worker.do_stuff()
class TestCase1(unittest.TestCase):
#mock.patch('resource.default_resource', spec_set=resource.Resource)
def test_1(self, mock_resource):
uut = UUT()
uut.gen_worker_and_do_stuff('Egg') # <-- say I automagically grabbed the resulting Worker into self.workers
self.workers[0]._on_spam({'num_of_new_workers_to_create':5}) # <-- I also want to get hold of the newly-created workers
Is there a way to grab the worker objects generated by uut, without directly accessing the _workers list in uut (which is an implementation detail)?
I guess I can do it in Worker.__init__, where the worker subscribes to provider events, so I guess the question reduces to:
How to I extract the self in the callee, when calling resource.default_resource.subscribe('on_spam', self._on_spam)?
As an application of the Dependency Inversion principle, I'd pass the Worker class as a dependency to UUT:
class UUT():
def __init__(self, make_worker=Worker):
self._workers = []
self._make_worker = make_worker
def gen_worker_and_connect(self, *args)
worker = self._make_worker(*args)
self._workers.append(resource)
worker.connect()
Then provide anything you want from the test instead of Worker. This own function could share the created object with the test scope. Besides solving this particular problem, that would also make the dependency explicit and independent of the UUT implementation. And you would not need to mock the resource thing as well, which makes the test dependent on things unrelated to the class under test.
I'm building a PyGTK application with several widgets that when changed, need to notify other widgets about the change. I would like to avoid code like this:
def on_entry_color_updated(self, widget):
self.paint_tools_panel.current_color_pane.update_color()
self.main_window.status_bar.update_color()
self.current_tool.get_brush().update_color()
And do something like this instead:
def on_entry_color_updated(self, widget):
self.update_notify('color-changed')
The status bar, current color pane and current tool would subscribe to that notification event and act accordingly. From what I can tell, the GObject signaling mechanism only allows me to register a callback on a particular widget, so each object that wants to receive a notification has to be aware of that widget.
Does GTK provide such a system or should I build it myself? Developers of Shotwell, a photo organization application for GNOME, had to build their own signaling mechanism, if I understand their design doc correctly. Searching here on SO didn't turn out any definitive answers.
Edit:
Clarification why I think GObject signaling is not what I need (or just a part of what I need). With GObject, I need to explicitly connect an object to another object, like so:
emitter.connect('custom-event', receiver.event_handler)
So in my application, I would have to do this:
class ColorPane(gtk.Something):
def __init__(self, application):
# init stuff goes here...
application.color_pallette.connect('color-changed', self.update_color)
def update_color(self, widget):
"""Show the new color."""
pass
class StatusBar(gtk.Something):
def __init__(self, application):
# init stuff goes here...
application.color_pallette.connect('color-changed', self.update_color)
def update_color(self, widget):
"""Show the new color name."""
pass
class Brush(gtk.Something):
def __init__(self, application):
# init stuff goes here...
application.color_pallette.connect('color-changed', self.update_color)
def update_color(self, widget):
"""Draw with new color."""
pass
In other words, I have to pass the application object or some other object that knows about the color_pallete to other objects in my application so that they connect to color_pallette signals. This is the kind of coupling that I want to avoid.
For one, you could create a custom subclass of GObject, which provides some custom signals. The following example is a slightly adapted version of the one given in the linked article:
import pygtk
pygtk.require('2.0')
import gobject
class Car(gobject.GObject):
__gsignals__ = {
'engine-started': (gobject.SIGNAL_RUN_LAST, gobject.TYPE_NONE, ()),
'engine-stopped': (gobject.SIGNAL_RUN_LAST, gobject.TYPE_NONE, ()),
}
def __init__(self):
gobject.GObject.__init__(self)
self._state = 0
def start(self):
if not self._state:
self._state = 1
self.emit('engine-started')
def stop(self):
if self._state:
self._state = 0
self.emit('engine-stopped')
gobject.type_register(Car)
def kill_switch(c):
def callback(*unused, **ignored):
c.stop()
return callback
def on_start(*unused, **ignored):
print "Started..."
def on_stop(*unused, **ignored):
print "Stopped..."
some_car = Car()
some_car.connect('engine-started', on_start)
some_car.connect('engine-started', kill_switch(some_car))
some_car.connect('engine-stopped', on_stop)
some_car.start()
Another approach would be to take advantage of one of the many event/signalling packages already on PyPI, for example:
Zope Event
Louie
PyDispatcher
Darts Events
Trellis
GObjects don't have to be widgets. For example, your application class can be a GObject which defines signals that other widgets connect to.
Also, I don't think you understood the Shotwell design document correctly. It looks to me like their signalling system is 100% GObject signalling system, just with particular guarantees about the order in which signals are handled. As they say in their design document, such things are possible in plain GObject, but Vala makes it easier to code it their way.