I'm currently in the process of writing some server code in python and I have a method which begins an infinite loop to serve requests ie:
class s:
def serve(self):
while True:
# do stuff
When I call this code I do something like:
a = s()
a.serve()
My question is -- is there way to send a message to 'serve' to disrupt the loop from outside the method. I don't want to simply kill the program. Help much appreciated.
One option I've thought of is rather than:
while True:
// do something
Could do:
while self.serving: # and then update self.serving as appropriate
But there's probably a better solution.
Yes this is possible:
class Server:
def __init__():
self.alive = True
def serve(self):
while self.alive:
# do stuff
def die():
self.alive = False
Now just call die() outside the thread running serve(..).
srv = Server()
thread = Thread(target=srv.serve)
thread.start()
time.sleep(some_time) # or do something else.
srv.die() # or do this from some outside process.
I don't understand exactly what you're trying to do but could you have a simple flag rather than:
while True:
it could be
example = True
while example:
Then set example to false when you want to end the loop.
Related
I am pretty new to Python and have a question about threading.
I have one function that is called pretty often. This function starts another function in a new Thread.
def calledOften(id):
t = threading.Thread(target=doit, args=(id))
t.start()
def doit(arg):
while true:
#Long running function that is using arg
When calledOften is called everytime a new Thread is created. My goal is to always terminate the last running thread --> At all times there should be only one running doit() Function.
What I tried:
How to stop a looping thread in Python?
def calledOften(id):
t = threading.Thread(target=doit, args=(id,))
t.start()
time.sleep(5)
t.do_run = False
This code (with a modified doit Function) worked for me to stop the thread after 5 seconds.
but i can not call t.do_run = False before I start the new thread... Thats pretty obvious because it is not defined...
Does somebody know how to stop the last running thread and start a new one?
Thank you ;)
I think you can decide when to terminate the execution of a thread from inside the thread by yourself. That should not be creating any problems for you. You can think of a Threading manager approach - something like below
import threading
class DoIt(threading.Thread):
def __init__(self, id, stop_flag):
super().__init__()
self.id = id
self.stop_flag = stop_flag
def run(self):
while not self.stop_flag():
pass # do something
class CalledOftenManager:
__stop_run = False
__instance = None
def _stop_flag(self):
return CalledOftenManager.__stop_run
def calledOften(self, id):
if CalledOftenManager.__instance is not None:
CalledOftenManager.__stop_run = True
while CalledOftenManager.__instance.isAlive():
pass # wait for the thread to terminate
CalledOftenManager.__stop_run = False
CalledOftenManager.__instance = DoIt(id, CalledOftenManager._stop_flag)
CalledOftenManager.__instance.start()
# Call Manager always
CalledOftenManager.calledOften(1)
CalledOftenManager.calledOften(2)
CalledOftenManager.calledOften(3)
Now, what I tried here is to make a controller for calling the thread DoIt. Its one approach to achieve what you need.
I've come across an unusual problem in regards to updating variables. I've built a simple class object to help me with some network sniffing. I wanted to make a parallel process which allows me to run some network tests and capture the traffic generated using python so I can extend the program to do amazing things. I'm using scapy's sniffing function to help with the interface sniffing.
Scapy's sniffer allows you to pass a function into itself function that allows you to create a 'stop sniffing' condition. In my case I've created function stop_filter and I wish to stop the Scapy sniff function by simply updating the self.stop_sniffing instance variable. I've presented the program output below, which shows self.stop_sniffing getting set to True in Function stop, but is then set back to False (or is not updated at all) when printed in stop_filter. I have no clue why this is happening and no solution comes to mind as it's such a weird problem.
If anyone with fresh eyes can see what insane thing I've done here it would be greatly appreciated!
from scapy.all import *
from multiprocessing import Process
class DatasetSniffer:
def __init__(self, iface, local_dir='.'):
self.iface = iface
self.master = None
self.local_dir = local_dir
self.stop_sniffing = False # Never updates! why!?
self.writer = PcapWriter(local_dir+"/master.pcap", append=True, sync=True)
def stop_filter(self, p):
# Note: 'p' gets passed in by Scapy function 'sniff'
print self.stop_sniffing
# Return 'True' to stop sniffer
return self.stop_sniffing
def sniff(self):
sniff(store=0, prn=self.writer.write, iface=self.iface, stop_filter=self.stop_filter)
def start(self):
self.master = Process(target=self.sniff)
self.master.start()
def stop(self):
self.stop_sniffing = True
# Shows that self.stop_sniffing is 'True'
print self.stop_sniffing
self.master.join()
if __name__ == "__main__":
interface = 'en3'
sniffer = DatasetSniffer(interface)
sniffer.start()
# some process
time.sleep(5)
sniffer.stop()
Shell output:
sudo python sniffing.py
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
False
True
False
False
False
False
The Problem
You are not using multiple threads in this example code you are using multiple processes.
Here you have two separate processes, that do not share memory:
the original process
a new process, started by multiprocessing.Process.start
this process will have been started by forking the original process, creating a copy of its memory at the time of the fork. They do not "share" memory.
Now, when you call DatasetSniffer.stop within your original process, this will not alter the value of stop_sniffing in the new ("master") process.
How to Communicate Then?
When using multiprocessing, you can communicate using a Pipe. Something like this:
readable_pipe, writable_pipe = multiprocessing.Pipe(duplex=False)
process = Process(target=do_something)
Now, our original process can send a message by writing to the pipe:
writable_pipe.send("stop")
while the new process can check for messages using:
if readable_pipe.poll():
msg = readable_pipe.recv()
Try working this into your code.
Thanks for all your suggestions. After a glass of inspiration I managed to knock up this script. Probably a nicer way to approach my problem without making too many changes. So this code allows the threads to use the stop function outside the class, thus allowing all the asynchronous tasks to use the stop_filter.
Found this information in the link below. Hopfully this post will be useful to someone else!
http://www.tutorialspoint.com/python/python_multithreading.htm
Cheers!
import threading
from scapy.all import *
from datetime import datetime
directory = str(datetime.now().strftime("%Y%m%d%H%M%S"))
os.makedirs(directory)
DatasetSnifferExit = 0
class DatasetSniffer(threading.Thread):
def __init__(self, iface, local_dir='.', filename=str(datetime.now())):
self.iface = iface
self.filename = filename
self.local_dir = local_dir
self.stop_sniffing = False
self.writer = PcapWriter(local_dir+"/"+filename+".pcap", append=True, sync=True)
threading.Thread.__init__(self)
def run(self):
sniff_interface(self.writer.write, self.iface)
def stop_filter(p):
if DatasetSnifferExit:
return True
else:
return False
def sniff_interface(write, iface):
sniff(store=0, prn=write, iface=iface, stop_filter=stop_filter)
if __name__ == "__main__":
DatasetSnifferExit = False
# Create new threads
pcap1 = DatasetSniffer('en3', directory, "master")
pcap2 = DatasetSniffer('en0', directory, "slave")
# Start new Threads
pcap1.start()
pcap2.start()
# Do stuff
time.sleep(10)
# Finished doing stuff
DatasetSnifferExit = True
I'm making a wxPython app that I need to update a value from the internet every 15 seconds. Is there any way I can have a function to set the value, and make it run in the background at this interval, without interrupting the program?
EDIT: Here's what I'm trying:
import thread
class UpdateThread(Thread):
def __init__(self):
self.stopped = False
UpdateThread.__init__(self)
def run(self):
while not self.stopped:
downloadValue()
time.sleep(15)
def downloadValue():
print x
UpdateThread.__init__()
What you want is to add a thread that runs your task at a specified pace.
You may have a look at this great answer here : https://stackoverflow.com/a/12435256/667433 to help you achieve this.
EDIT : Here is the code that should work for you :
import time
from threading import Thread # This is the right package name
class UpdateThread(Thread):
def __init__(self):
self.stopped = False
Thread.__init__(self) # Call the super construcor (Thread's one)
def run(self):
while not self.stopped:
self.downloadValue()
time.sleep(15)
def downloadValue(self):
print "Hello"
myThread = UpdateThread()
myThread.start()
for i in range(10):
print "MainThread"
time.sleep(2)
Hope it helps
I have made something similar to this:
-you need a thread to run in the background .
-And a define a 'custom' event , so that the tread can notify the UI when needed
Create the custom WX event
(MyEVENT_CHECKSERVER, EVT_MYEVENT_CHECKSERVER) =
wx.lib.newevent.NewEvent()
on UI "init" you can bind the event , and start the thread
# bind the custom event
self.Bind(EVT_MYEVENT_CHECKSERVER, self.foo)
# and start the worker thread
checkServerThread = threading.Thread(target=worker_checkServerStatus
,args=(self,) )
checkServerThread.daemon = True
checkServerThread.start()
the worker thread can be something like this ,ps. caller is the UI instance
def worker_checkServerStatus(caller):
while True:
# check the internet code here
evt = MyEVENT_CHECKSERVER(status='Some internet Status' ) #make a new event
wx.PostEvent(caller, evt) # send the event to the UI
time.sleep(15) #ZZZzz for a bit
Edit: miss read the question...
Another way to do that is with a timer:
import threading
stopNow = 0
def downloadValue():
print("Running downloadValue")
if not stopNow: threading.Timer(15,downloadValue).start()
downloadValue()
This is a classic pattern for repeating a function: the function itself adds a timed call to itself. To start the cycle, call the function (it returns immediately). To break the cycle set stopNow to 1.
Im trying to write a module for Python that prints out text for my program and displays a progress bar while i do something in the background. Im using the 'threading' module currently but open to suggestions if something else will make it easier.
what i want to know is two fold, how should i call this class elegantly and how should i stop these threads im creating?
this is what im currently doing:
tmp1 = textprint("hello this is text")
tmp1.start()
# do something
tmp1.stop()
these are the options ive considered and looked into so far:
using thread.name to find the name of the thread or having the thread
return a name to kill afterwards. OR passing a number for similar
thread identification afterwards. (a bit cumbersome and not my
favourite solution.)
sending a thread.event ? - from reading the docs i see an event can
be sent, perhaps that can be used to stop it?
or a with statement but im unclear how to use it in this context, plus i find most of the python docs extremely confusing and not written for me at all.
what i would like to do is something like this:
echo('hello') (prints progress bar etc)
- and then when i want to stop it echo.stop()
the obv. problem there though is that the stop function doesnt know which thread it is trying to stop.
Here is a skeleton of what im trying to do:
import time
import string
import threading
class print_text(threading.Thread):
def __init__(self,arg=None):
super(print_text,self).__init__()
self._stop = False
self.arg=arg
def run (self):
# start thread for text
print self.txt
while not self._stop:
print "rude words"
def echo (self,txt):
self.txt=txt
self.start()
def stop(self):
self._stop = True
def stopped(self):
return self._stop == True
def __enter__(self):
print "woo"
return thing
def __exit__(self, type, value, traceback):
return isinstance(value, TypeError)
if __name__ == '__main__':
print_text.start.echo('this is text') # dunt werk
with print_text.echo('this is text'):
time.sleep(3)
print "done"
and then call it like so:
echo('this is text')
i also guess to do this i would have to
import echo from print_text
the WITH way of doing things suggests putting an __enter__ and __exit__ bit in. i tried them and they didnt work and also, i didnt know what i was doing, really appreciate any help, thanks.
You were very close to having working code. There just needed to be a few minor fixups:
print_text is a class. It should be instantiated with print_text()
The start method returns an instance of print_text, you need to save that
in order to call stop and echo on it: t = print_text()
The enter method needs to return self instead of thing.
The exit method should either set _stop or call stop().
The echo method should return self so that it can be used with the with-statement.
Here is some working code that includes those minor edits:
import time
import string
import threading
class print_text(threading.Thread):
def __init__(self, arg=None):
super(print_text,self).__init__()
self._stop = False
self.arg=arg
def run (self):
# start thread for text
print self.txt
while not self._stop:
print "rude words"
def echo (self, txt):
self.txt=txt
self.start()
return self
def stop(self):
self._stop = True
def stopped(self):
return self._stop == True
def __enter__(self):
print "woo"
return self
def __exit__(self, type, value, traceback):
self._stop = True
return isinstance(value, TypeError)
if __name__ == '__main__':
t = print_text()
t.echo('this is text')
time.sleep(3)
t.stop()
with print_text().echo('this is text'):
time.sleep(3)
print "done"
The best way to stop a thread in Python is to politely ask it to stop. The best way to pass new data to a thread is with the Queue module.
Both are used in the code in this post, which demonstrates socket communication from a Python thread but is otherwise relevant to your question. If you read the code carefully you'll notice:
Using threading.Event() which is set by a method call from outside, and which the thread periodically checks to know if it was asked to die.
Using Queue.Queue() for both passing commands to the thread and receiving responses from it.
A thread name is useful if you could potentially have multiple subthreads running the same target at once and want to ensure that all of them are stopped. It seems like a useful generalization and doesn't seem too cumbersome to me, but beauty is in the eye of the beholder :-). The following:
starts a subthread to print a message and start a progressbar
stops the subthread using a name given when it was started.
It is much simpler code. Does it do what you want?
import time, threading
class print_text:
def __init__(self):
pass
def progress(self):
while not self._stop: # Update progress bar
print(".", sep="", end="")
time.sleep(.5)
def echo(self, arg="Default"): # Print message and start progress bar
print(arg)
self._stop = False
threading.Thread(target=self.progress, name="_prog_").start()
def stop(self):
self._stop = True
for t in threading.enumerate():
if t.name == "_prog_":
t.join()
tmp1 = print_text()
tmp1.echo("hello this is text")
time.sleep(10)
tmp1.stop()
print("Done")
I have an FTP function that traces the progress of running upload but my understanding of threading is limited and i have been unable to implement a working solution... I'd like to add a GUI progress bar to my current Application by using threading. Can someone show me a basic function using asynchronous threads that can be updated from another running thread?
def ftpUploader():
BLOCKSIZE = 57344 # size 56 kB
ftp = ftplib.FTP()
ftp.connect(host)
ftp.login(login, passwd)
ftp.voidcmd("TYPE I")
f = open(zipname, 'rb')
datasock, esize = ftp.ntransfercmd(
'STOR %s' % os.path.basename(zipname))
size = os.stat(zipname)[6]
bytes_so_far = 0
print 'started'
while 1:
buf = f.read(BLOCKSIZE)
if not buf:
break
datasock.sendall(buf)
bytes_so_far += len(buf)
print "\rSent %d of %d bytes %.1f%%\r" % (
bytes_so_far, size, 100 * bytes_so_far / size)
sys.stdout.flush()
datasock.close()
f.close()
ftp.voidresp()
ftp.quit()
print 'Complete...'
Here's a quick overview of threading, just in case :) I won't go into too much detail into the GUI stuff, other than to say that you should check out wxWidgets. Whenever you do something that takes a long time, like:
from time import sleep
for i in range(5):
sleep(10)
You'll notice that to the user, the entire block of code seems to take 50 seconds. In those 5 seconds, your application can't do anything like update the interface, and so it looks like it's frozen. To solve this problem, we use threading.
Usually there are two parts to this problem; the overall set of things you want to process, and the operation that takes a while, that we'd like to chop up. In this case, the overall set is the for loop and the operation we want chopped up is the sleep(10) function.
Here's a quick template for the threading code, based on our previous example. You should be able to work your code into this example.
from threading import Thread
from time import sleep
# Threading.
# The amount of seconds to wait before checking for an unpause condition.
# Sleeping is necessary because if we don't, we'll block the os and make the
# program look like it's frozen.
PAUSE_SLEEP = 5
# The number of iterations we want.
TOTAL_ITERATIONS = 5
class myThread(Thread):
'''
A thread used to do some stuff.
'''
def __init__(self, gui, otherStuff):
'''
Constructor. We pass in a reference to the GUI object we want
to update here, as well as any other variables we want this
thread to be aware of.
'''
# Construct the parent instance.
Thread.__init__(self)
# Store the gui, so that we can update it later.
self.gui = gui
# Store any other variables we want this thread to have access to.
self.myStuff = otherStuff
# Tracks the paused and stopped states of the thread.
self.isPaused = False
self.isStopped = False
def pause(self):
'''
Called to pause the thread.
'''
self.isPaused = True
def unpause(self):
'''
Called to unpause the thread.
'''
self.isPaused = False
def stop(self):
'''
Called to stop the thread.
'''
self.isStopped = True
def run(self):
'''
The main thread code.
'''
# The current iteration.
currentIteration = 0
# Keep going if the job is active.
while self.isStopped == False:
try:
# Check for a pause.
if self.isPaused:
# Sleep to let the os schedule other tasks.
sleep(PAUSE_SLEEP)
# Continue with the loop.
continue
# Check to see if we're still processing the set of
# things we want to do.
if currentIteration < TOTAL_ITERATIONS:
# Do the individual thing we want to do.
sleep(10)
# Update the count.
currentIteration += 1
# Update the gui.
self.gui.update(currentIteration,TOTAL_ITERATIONS)
else:
# Stop the loop.
self.isStopped = True
except Exception as exception:
# If anything bad happens, report the error. It won't
# get written to stderr.
print exception
# Stop the loop.
self.isStopped = True
# Tell the gui we're done.
self.gui.stop()
To call this thread, all you have to do is:
aThread = myThread(myGui,myOtherStuff)
aThread.start()