We Keep Coding

Run only one Instance of a Thread

I am pretty new to Python and have a question about threading.
I have one function that is called pretty often. This function starts another function in a new Thread.
def calledOften(id):
t = threading.Thread(target=doit, args=(id))
t.start()
def doit(arg):
while true:
#Long running function that is using arg
When calledOften is called everytime a new Thread is created. My goal is to always terminate the last running thread --> At all times there should be only one running doit() Function.
What I tried:
How to stop a looping thread in Python?
def calledOften(id):
t = threading.Thread(target=doit, args=(id,))
t.start()
time.sleep(5)
t.do_run = False
This code (with a modified doit Function) worked for me to stop the thread after 5 seconds.
but i can not call t.do_run = False before I start the new thread... Thats pretty obvious because it is not defined...
Does somebody know how to stop the last running thread and start a new one?
Thank you ;)

I think you can decide when to terminate the execution of a thread from inside the thread by yourself. That should not be creating any problems for you. You can think of a Threading manager approach - something like below
import threading
class DoIt(threading.Thread):
def __init__(self, id, stop_flag):
super().__init__()
self.id = id
self.stop_flag = stop_flag
def run(self):
while not self.stop_flag():
pass # do something
class CalledOftenManager:
__stop_run = False
__instance = None
def _stop_flag(self):
return CalledOftenManager.__stop_run
def calledOften(self, id):
if CalledOftenManager.__instance is not None:
CalledOftenManager.__stop_run = True
while CalledOftenManager.__instance.isAlive():
pass # wait for the thread to terminate
CalledOftenManager.__stop_run = False
CalledOftenManager.__instance = DoIt(id, CalledOftenManager._stop_flag)
CalledOftenManager.__instance.start()
# Call Manager always
CalledOftenManager.calledOften(1)
CalledOftenManager.calledOften(2)
CalledOftenManager.calledOften(3)
Now, what I tried here is to make a controller for calling the thread DoIt. Its one approach to achieve what you need.

python design pattern queue with workers

I'm currently working on a project that involves three components,
an observer that check for changes in a directory, a worker and an command line interface.
What I want to achieve is:
The observer, when a change happens send a string to the worker (add a job to the worker's queue).
The worker has a queue of jobs and forever works on his queue.
Now I want the possibility to run a python script to check the status of the worker (number of active jobs, errors and so on)
I don't know how to achieve this with python in terms of which component to use and how to link the three components.
I though as a singleton worker where the observer add a job to a queue but 1) I was not able to write a working code and 2) How can I fit the checker in?
Another solution that I thought of may be multiple child processes from a father that has the queue but I'm a bit lost...
Thanks for any advices

I'd use some kind of observer pattern or publish-subscribe pattern. For the former you can use for example the Python version of ReactiveX. But for a more basic example let's stay with the Python core. Parts of your program can subscribe to the worker and receive updates from the process via queues for example.
import itertools as it
from queue import Queue
from threading import Thread
import time
class Observable(Thread):
def __init__(self):
super().__init__()
self._observers = []
def notify(self, msg):
for obs in self._observers:
obs.put(msg)
def subscribe(self, obs):
self._observers.append(obs)
class Observer(Thread):
def __init__(self):
super().__init__()
self.updates = Queue()
class Watcher(Observable):
def run(self):
for i in it.count():
self.notify(i)
time.sleep(1)
class Worker(Observable, Observer):
def run(self):
while True:
task = self.updates.get()
self.notify((str(task), 'start'))
time.sleep(1)
self.notify((str(task), 'stop'))
class Supervisor(Observer):
def __init__(self):
super().__init__()
self._statuses = {}
def run(self):
while True:
status = self.updates.get()
print(status)
self._statuses[status[0]] = status[1]
# Do something based on status updates.
if status[1] == 'stop':
del self._statuses[status[0]]
watcher = Watcher()
worker = Worker()
supervisor = Supervisor()
watcher.subscribe(worker.updates)
worker.subscribe(supervisor.updates)
supervisor.start()
worker.start()
watcher.start()
However many variations are possible and you can check the various patterns which suits you most.

python create single thread to do one task then quit

I am runing a web project.
I want to start a thread to do one job if some event happen.
Before,I always write multithreading program.Such as below
class FetchThread(threading.Thread):
def __init__(self, profile_queue):
....
def run(self):
while true:
....
But here is only one task,I don't want a thread running all the time.
I want the thread just do one thing, then quit.But I always see others write while True in Thread.run, I don't know if method 2 is a good way .Anyone can help?
1.
class FetchThread(threading.Thread):
def __init__(self, profile_queue):
....
def stop(self):
self.__running = False
def run(self):
while self.__running:
....
self.stop()
2.
class FetchThread(threading.Thread):
def __init__(self, profile_queue):
....
def run(self):
....

If you can encapsulate your entire task into a single function, you can just use the default Thread class and pass the right arguments.
def a_task(callback):
# do your task here
callback()
def some_callback():
# some way to notify that the task is completed
pass
task = Thread(target=a_task,
kwargs={'callback': some_callback})
task.start()
If your web project involves Tornado, a previous answer I provided might be of interest to you.

Put the job you want to do in a regular function and call it like so:
>>> def job_func(arg):
... do_work(arg)
>>> Thread(target=job_func, args=(arg,)).start()

Repeat function at an interval?

I'm making a wxPython app that I need to update a value from the internet every 15 seconds. Is there any way I can have a function to set the value, and make it run in the background at this interval, without interrupting the program?
EDIT: Here's what I'm trying:
import thread
class UpdateThread(Thread):
def __init__(self):
self.stopped = False
UpdateThread.__init__(self)
def run(self):
while not self.stopped:
downloadValue()
time.sleep(15)
def downloadValue():
print x
UpdateThread.__init__()

What you want is to add a thread that runs your task at a specified pace.
You may have a look at this great answer here : https://stackoverflow.com/a/12435256/667433 to help you achieve this.
EDIT : Here is the code that should work for you :
import time
from threading import Thread # This is the right package name
class UpdateThread(Thread):
def __init__(self):
self.stopped = False
Thread.__init__(self) # Call the super construcor (Thread's one)
def run(self):
while not self.stopped:
self.downloadValue()
time.sleep(15)
def downloadValue(self):
print "Hello"
myThread = UpdateThread()
myThread.start()
for i in range(10):
print "MainThread"
time.sleep(2)
Hope it helps

I have made something similar to this:
-you need a thread to run in the background .
-And a define a 'custom' event , so that the tread can notify the UI when needed
Create the custom WX event
(MyEVENT_CHECKSERVER, EVT_MYEVENT_CHECKSERVER) =
wx.lib.newevent.NewEvent()
on UI "init" you can bind the event , and start the thread
# bind the custom event
self.Bind(EVT_MYEVENT_CHECKSERVER, self.foo)
# and start the worker thread
checkServerThread = threading.Thread(target=worker_checkServerStatus
,args=(self,) )
checkServerThread.daemon = True
checkServerThread.start()
the worker thread can be something like this ,ps. caller is the UI instance
def worker_checkServerStatus(caller):
while True:
# check the internet code here
evt = MyEVENT_CHECKSERVER(status='Some internet Status' ) #make a new event
wx.PostEvent(caller, evt) # send the event to the UI
time.sleep(15) #ZZZzz for a bit
Edit: miss read the question...

Another way to do that is with a timer:
import threading
stopNow = 0
def downloadValue():
print("Running downloadValue")
if not stopNow: threading.Timer(15,downloadValue).start()
downloadValue()
This is a classic pattern for repeating a function: the function itself adds a timed call to itself. To start the cycle, call the function (it returns immediately). To break the cycle set stopNow to 1.

Python multiprocessing with twisted's reactor

I am working on a xmlrpc server which has to perform certain tasks cyclically. I am using twisted as the core of the xmlrpc service but I am running into a little problem:
class cemeteryRPC(xmlrpc.XMLRPC):
def __init__(self, dic):
xmlrpc.XMLRPC.__init__(self)
def xmlrpc_foo(self):
return 1
def cycle(self):
print "Hello"
time.sleep(3)
class cemeteryM( base ):
def __init__(self, dic): # dic is for cemetery
multiprocessing.Process.__init__(self)
self.cemRPC = cemeteryRPC()
def run(self):
# Start reactor on a second process
reactor.listenTCP( c.PORT_XMLRPC, server.Site( self.cemRPC ) )
p = multiprocessing.Process( target=reactor.run )
p.start()
while not self.exit.is_set():
self.cemRPC.cycle()
#p.join()
if __name__ == "__main__":
import errno
test = cemeteryM()
test.start()
# trying new method
notintr = False
while not notintr:
try:
test.join()
notintr = True
except OSError, ose:
if ose.errno != errno.EINTR:
raise ose
except KeyboardInterrupt:
notintr = True
How should i go about joining these two process so that their respective joins doesn't block?
(I am pretty confused by "join". Why would it block and I have googled but can't find much helpful explanation to the usage of join. Can someone explain this to me?)
Regards

Do you really need to run Twisted in a separate process? That looks pretty unusual to me.
Try to think of Twisted's Reactor as your main loop - and hang everything you need off that - rather than trying to run Twisted as a background task.
The more normal way of performing this sort of operation would be to use Twisted's .callLater or to add a LoopingCall object to the Reactor.
e.g.
from twisted.web import xmlrpc, server
from twisted.internet import task
from twisted.internet import reactor
class Example(xmlrpc.XMLRPC):
def xmlrpc_add(self, a, b):
return a + b
def timer_event(self):
print "one second"
r = Example()
m = task.LoopingCall(r.timer_event)
m.start(1.0)
reactor.listenTCP(7080, server.Site(r))
reactor.run()

Hey asdvawev - .join() in multiprocessing works just like .join() in threading - it's a blocking call the main thread runs to wait for the worker to shut down. If the worker never shuts down, then .join() will never return. For example:
class myproc(Process):
def run(self):
while True:
time.sleep(1)
Calling run on this means that join() will never, ever return. Typically to prevent this I'll use an Event() object passed into the child process to allow me to signal the child when to exit:
class myproc(Process):
def __init__(self, event):
self.event = event
Process.__init__(self)
def run(self):
while not self.event.is_set():
time.sleep(1)
Alternatively, if your work is encapsulated in a queue - you can simply have the child process work off of the queue until it encounters a sentinel (typically a None entry in the queue) and then shut down.
Both of these suggestions means that prior to calling .join() you can send set the event, or insert the sentinel and when join() is called, the process will finish it's current task and then exit properly.