multithreading in Python: extremely high CPU consumption - python

Below is the code of a simple timer that prints out a new number in the command line every 0.1 sec, and -- simultaneously -- awaits for a keypress (Enter). As soon as Enter is pressed, the timer is switched into pause, stops printing and waits for another Enter keypress that would resume counting / printing.
The problem I face is that while in the 'pause' state, CPU consumption grows to almost 100% (and seems to be below 1% while running / printing out).
Why is it? What am I doing wrong?
Here goes the code:
import time, thread, sys
time_passed = 0
l = []
trig = True
def input(l):
""" Waits for a keypress """
raw_input()
l.append(None)
def timer():
""" Prints out a number every millisecond """
global time_passed
time.sleep(.1)
time_passed += 1
sys.stdout.write(format(time_passed)+"\r")
sys.stdout.flush()
def running():
""" Toggles play/pause state if Enter is pressed """
global trig, l
while trig:
timer()
if l:
l = []
trig = False
return trig, l
def stopped():
""" Toggles pause/play state if Enter is pressed """
global trig, l
while not trig:
if l:
l = []
trig = True
return trig, l
def main():
""" Waits for a keypress while either running or stopping the timer """
global l, trig
while True:
thread.start_new_thread(input, (l,))
if trig: # The timer is running
running()
else: # The timer is stopped
stopped()
main()
Thank you.

Well, if its in a "stopped" state you run this:
def stopped():
""" Toggles pause/play state if Enter is pressed """
global trig, l
while not trig:
if l:
l = []
trig = True
return trig, l
this loop has no pause/wait. So it runs at full speed without getting back to system, while within the running you do call time.sleep. Putting time.sleep(0.1) before the if will make it work as you wish.
Also a friendly personal note, you could refactor it a little bit. Two functions with the same description working on globals make it a little bit hard to understand =).
I'd make something like:
import time, thread, sys, threading
time_passed = 0
l = []
def input(l):
""" Waits for a keypress """
raw_input()
l.append(None)
def timer():
""" Prints out a number every millisecond """
global time_passed
time_passed += 1
sys.stdout.write(format(time_passed)+"\r")
sys.stdout.flush()
def running(state):
""" Toggles play/pause state if Enter is pressed """
global l
while True:
time.sleep(.1)
if state:
timer()
if l:
l = []
state = not state
return state
def main():
""" Waits for a keypress while either running or stopping the timer """
global l
state = True
while True:
thread.start_new_thread(input, (l,))
state = running(state)
main()
and I would think how to signal the toggle better. I somehow don't like the idea of returning from the function, you don't need really to return from running, just flip the state. But it is just personal preference I guess.

Related

Periodical interruptions Python

I sumarize my problem through this piece of code. When I end my program by closing the tkinter main window, I need the whole program ends, but the loops goes on executing until the functions is over. I suppose there is a way to force these functions ends too. I think there is a way to detect the program was ended, so I could end the functions.
import threading
import time
from tkinter import *
def loop1_10():
for i in range(1, 11):
time.sleep(1)
print(i)
def loop1_10_b():
for i in range(1, 11):
time.sleep(2)
print(i)
threading.Thread(target=loop1_10).start()
threading.Thread(target=loop1_10_b).start()
MainWindow = Tk()
MainWindow.mainloop()
The other way to handle this is to make the threads "daemons". A daemon thread will be forcibly closed when the app exits; it doesn't block the app.
threading.Thread(target=loop1_10, daemon=True).start()
threading.Thread(target=loop1_10_b, daemon=True).start()
Note that I'm not saying one is better or worse than the other. Each option has its uses.
Add a protocol, WM_DELETE_WINDOW, to your MainWindow, where you use define a function you defined, on_close() that gets called once the tkinter window is closed.
The on_close() function will redefine the global variable end from False into True, and in each for loop, if the end variable's value is True, return out of them:
import threading
import time
from tkinter import *
def loop1_10():
for i in range(1, 11):
if end:
return
time.sleep(1)
print(i)
def loop1_10_b():
for i in range(1, 11):
if end:
return
time.sleep(2)
print(i)
end = False
def on_closing():
global end
end = True
MainWindow.destroy()
threading.Thread(target=loop1_10).start()
threading.Thread(target=loop1_10_b).start()
MainWindow = Tk()
MainWindow.protocol("WM_DELETE_WINDOW", on_closing)
MainWindow.mainloop()
But there is still a problem with the above code; if the end = True happened right before the time.sleep() call(s), the last time.sleep()(s) will still make the program wait for a second or two before terminating.
To fix this, use time.time() and a while loop to manually check how much time has passed before continuing each for loop:
import threading
import time
from tkinter import *
def loop1_10():
for i in range(1, 11):
old_time = time.time()
while True:
if end:
return
if time.time() - old_time < 1:
continue
break
print(i)
def loop1_10_b():
for i in range(1, 11):
old_time = time.time()
while True:
if end:
return
if time.time() - old_time < 2:
continue
break
print(i)
end = False
def on_closing():
global end
end = True
MainWindow.destroy()
threading.Thread(target=loop1_10).start()
threading.Thread(target=loop1_10_b).start()
MainWindow = Tk()
MainWindow.protocol("WM_DELETE_WINDOW", on_closing)
MainWindow.mainloop()
But do note from this comment by #kindall:
use time.time() and a while loop -- don't do this, it'll use up an entire CPU core waiting for the loop to exit. this will not only eat battery unnecessarily, but since Python only uses one CPU core due to the Global Interpreter Lock, it will make the rest of the program sluggish, too

Stop button for breaking while loop within Ipywidgets ecosystem

Let's assume the following problem: we have an Ipywidget button and a progress bar. On clicking the button, a function work() is executed, which merely fills the progress bar until completing it, then reverses the process and empties it out. As it stands, such a function runs continuously. The following code snippet provides the corresponding MWE:
# importing packages.
from IPython.display import display
import ipywidgets as widgets
import time
import functools
# setting 'progress', 'start_button' and 'Hbox' variables.
progress = widgets.FloatProgress(value=0.0, min=0.0, max=1.0)
start_button = widgets.Button(description="start fill")
Hbox = widgets.HBox(children=[start_button, progress])
# defining 'on_button_clicked_start()' function; executes 'work()' function.
def on_button_clicked_start(b, start_button, progress):
work(progress)
# call to 'on_button_clicked_start()' function when clicking the button.
start_button.on_click(functools.partial(on_button_clicked_start, start_button=start_button, progress=progress))
# defining 'work()' function.
def work(progress):
total = 100
i = 0
# while roop for continuous run.
while True:
# while loop for filling the progress bar.
while progress.value < 1.0:
time.sleep(0.01)
i += 1
progress.value = float(i)/total
# while loop for emptying the progress bar.
while progress.value > 0.0:
time.sleep(0.01)
i -= 1
progress.value = float(i)/total
# display statement.
display(Hbox)
The aim is to include "Stop" and "Resume" buttons, so that the while loops are broken whenever the first is clicked, and the execution is resumed when pressing the second one. Can this be done without employing threading, multiprocessing or asynchronicity?
Here's an answer I derived by means of the threading package and based on the background-working-widget example given in https://ipywidgets.readthedocs.io/en/latest/examples/Widget%20Asynchronous.html. It's certainly not optimized, and probably not good-practice-compliant. Anybody coming up with a better answer is welcomed to provide it.
# importing packages.
import threading
from IPython.display import display
import ipywidgets as widgets
import time
# defining progress bar 'progress', start, stop and resume buttons
# 'start_button', 'stop_button' and 'resume_button', and horizontal
# box 'Hbox'.
progress = widgets.FloatProgress(value=0.0, min=0.0, max=1.0)
start_button = widgets.Button(description="start fill")
stop_button = widgets.Button(description="stop fill/empty")
resume_button = widgets.Button(description="resume fill/empty")
Hbox = widgets.HBox(children=[start_button, stop_button, resume_button, progress])
# defining boolean flags 'pause' and 'resume'.
pause = False
restart = False
# defining 'on_button_clicked_start()' function.
def on_button_clicked_start(b):
# setting global variables.
global pause
global thread
global restart
# conditinoal for checking whether the thread is alive;
# if it isn't, then start it.
if not thread.is_alive():
thread.start()
# else, pause and set 'restart' to True for setting
# progress bar values to 0.
else:
pause = True
restart = True
time.sleep(0.1)
restart = False
# conditional for changing boolean flag 'pause'.
if pause:
pause = not pause
# defining 'on_button_clicked_stop()' function.
def on_button_clicked_stop(b):
# defining global variables.
global pause
# conditional for changing boolean flag 'pause'.
if not pause:
pause = not pause
# defining 'on_button_clicked_resume()' function.
def on_button_clicked_resume(b):
# defining global variables.
global pause
global restart
# conditional for changing boolean flags 'pause' and 'restart'
# if necessary.
if pause:
if restart:
restart = False
pause = not pause
# call to 'on_button_clicked_start()' function when clicking the button.
start_button.on_click(on_button_clicked_start)
# call to 'on_button_clicked_stop()' function when clicking the button.
stop_button.on_click(on_button_clicked_stop)
# call to 'on_button_clicked_resume()' function when clicking the button.
resume_button.on_click(on_button_clicked_resume)
# defining the 'work()' function.
def work(progress):
# setting global variables.
global pause
i = 0
i_m1 = 0
# setting 'total' variable.
total = 100
# infinite loop.
while True:
# stop/resume conditional.
if not pause:
# filling the progress bar.
if (i == 0) or i > i_m1 and not pause:
time.sleep(0.1)
if i == i_m1:
pass
else:
i_m1 = i
i += 1
progress.value = float(i)/total
# emptying the progress bar.
if (i == 101) or i < i_m1 and not pause:
time.sleep(0.1)
if i == i_m1:
pass
else:
i_m1 = i
i -= 1
progress.value = float(i)/total
else:
if restart:
i = 0
i_m1 = 0
# setting the thread.
thread = threading.Thread(target=work, args=(progress,))
# displaying statement.
display(Hbox)
After trying using asynchronous, I run into too many problems. Much better approach is to use generator approach, also mentioned in documentation https://ipywidgets.readthedocs.io/en/latest/examples/Widget%20Asynchronous.html
Here is the code I came up with
from functools import wraps
from IPython.core.display import display
import ipywidgets as widgets
def yield_for_change(widget, attribute):
def f(iterator):
#wraps(iterator)
def inner():
i = iterator()
def next_i(change):
try:
print([w.description for w in widget])
i.send(change)
except StopIteration as e:
for w in widget:
w.unobserve(next_i, attribute)
for w in widget:
w.on_click(next_i)
w.observe(next_i, attribute)
# start the generator
next(i)
return inner
return f
btn_true = widgets.Button(description="True",style={'button_color':'green'} )
btn_false = widgets.Button(description="False",style={'button_color':'red'})
btn_list = [btn_true, btn_false]
buttons = widgets.HBox(btn_list)
value_loop = widgets.Label()
out = widgets.Output()
#yield_for_change(btn_list, 'description')
def f():
for i in range(10):
print('did work %s'%i)
x = yield
print('generator function continued with value %s'%x)
f()
display(widgets.VBox([buttons, out]))

Python: threading

I want to start thread multiple times, but only when it is not running.
There is a simple model what I am trying:
import threading
import time
def up (x, r):
time.sleep(3)
r['h'] = x + 1
hum = {'h' : 0}
while True:
print(hum['h'])
H = threading.Thread(target = up, args=(hum['h'],hum))
H.daemon=True
if not H.isAlive():
H.start()
print(threading.active_count())
Also what i don't understand is that:
When I run program it prints: 0. Then after 3 seconds it prints:1 and so on after every 3 second it is increased by 1.
But I thought it would print: 0. Then after 3 second it would print: 1. And then immediately increasing fast.
Because after starting first thread it would immediately start the next one and so on. why does this happen?
How not to start a thread 'up', if it's already running?
Not sure if I got your question completely, but here are some thoughts.
When I run your code I get an increasing number of active threads, as you are creating a new thread every time, checking its status (which will always be not alive) and then starting it.
What you want to do instead is to check the status of the last run thread and if that's not alive start a new one. In order to do that you should create a new thread if the old one is done:
def up (x, r):
time.sleep(3)
r['h'] = x + 1
def main():
hum = {'h' : 0}
H = threading.Thread(target = up, args=(hum['h'],hum))
H.daemon=True
while True:
# print(hum['h'])
if not H.isAlive():
H = threading.Thread(target = up, args=(hum['h'],hum))
H.daemon=True
H.start()
print(threading.active_count())
What happens in your code:
Print the value of hum['h']
Create a thread (note you create it, you are not starting it yet)
Set the value of a property
If the thread is not started then start it
Print the count of active threads (active, NOT started)
Since you replace the H variable every time, you'll have a new thread every time that gets immediately started.
If you add a print that says "starting" in the if for the is alive, you'll see that it gets called every time.
You can use join() to wait for the thread to finish:
import threading
import time
def up (x, r):
time.sleep(3)
r['h'] = x + 1
hum = {'h' : 0}
while True:
print(hum['h'])
H = threading.Thread(target = up, args=(hum['h'],hum))
H.daemon=True
H.start()
H.join()
print(threading.active_count())
If you don't want to wait, you can just save the current running thread in a variable and check it in the loop:
import threading
import time
def up (x, r):
time.sleep(3)
r['h'] = x + 1
hum = {'h' : 0}
current_thread = None
while True:
print(hum['h'])
if current_thread is None:
current_thread = threading.Thread(target = up, args=(hum['h'],hum))
current_thread.daemon=True
current_thread.start()
elif not current_thread.isAlive():
current_thread = threading.Thread(target = up, args=(hum['h'],hum))
current_thread.daemon=True
current_thread.start()

How do I schedule a one-time script to run x minutes from now? (alternative for 'at')

(Background: I'd like to control a light source with a motion sensor. Light should turn off x minutes after last detected motion. The framework is in place, scheduling is what remains to be done.)
Currently, when motion is detected the light gets turned on and a job to turn it off in 'now + x minutes' is scheduled. Whenever motion is detected during the x minutes the job gets removed from the queue and a new one is set up, extending effectively the time the light stays on.
I tried the "at" command but job handling is quite clunky. Whenever a job is removed from the queue an email gets sent. I looked at the Python crontab module but it would need much additional programming (handling relative time, removing old cronjobs, etc.) and seems to be slower.
What are my alternatives (bash, python, perl)?
-- Edit: My python skills are at beginner level, here's what I put together:
#!/usr/bin/env python2.7
# based on http://raspi.tv/2013/how-to-use-interrupts-with-python-on-the-raspberry-pi-and-rpi-gpio-part-2
# more than 160 seconds without activity are required to re-trigger action
import time
from subprocess import call
import os
import RPi.GPIO as GPIO
PIR = 9 # data pin of PIR sensor (in)
LED = 7 # positive pin of LED (out)
timestamp = '/home/pi/events/motiontime' # file to store last motion detection time (in epoch)
SOUND = '/home/pi/events/sounds/Hello.wav' # reaction sound
# GPIO setup
GPIO.setmode(GPIO.BCM)
GPIO.setup(PIR,GPIO.IN, pull_up_down=GPIO.PUD_DOWN)
GPIO.setup(LED,GPIO.OUT)
# function which gets called when motion is reported (sensor includes own delay-until-hot again
# and sensibility settings
def my_callback(channel):
now = time.time() # store current epoch time in variable 'now'
f = open(timestamp, "r")
then = float(f.readline()) # read last detection time from file
difference = now - then # calculate time that has passed
call(['/home/pi/bin/kitchenlights.sh', '-1']) # turn light on
call(['/home/pi/bin/lighttimer.sh']) # schedule at job to turn lights off
if difference > 160: # if more than 160 seconds without activity have passed then...
GPIO.output(LED, True) # turn on LED
if not os.path.isfile("/home/pi/events/muted"): # check if system is muted, else
call(['/usr/bin/mplayer', '-really-quiet', '-noconsolecontrols', SOUND]) # play sound
GPIO.output(LED, False) # turn of LED
f = open(timestamp, "w")
f.write(repr(now)) # update timestamp
f.close()
else: # when less than 160 seconds have passed do nothing and
f = open(timestamp, "w")
f.write(repr(now)) # update timestamp (thus increasing the interval of silence)
f.close()
GPIO.add_event_detect(PIR, GPIO.RISING,callback=my_callback,bouncetime=100) # add rising edge detection on a channel
while True:
time.sleep(0.2)
pass
Now that questions come in I think I could put a countdown in the while loop, right? How would that work?
I would approach this with the threading module. To do this, you'd set up the following thread class:
class CounterThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.count = 0
self.start()
def run(self):
while self.count < COUNTLIMIT:
time.sleep(0.1)
self.count += 0.1
#Call function to turn off light here
return
def newSig(self):
self.count = 0
This is a thread which everytime it recieves a new signal (the thread's newSig function is called), the counter restarts. If the COUNTLIMIT is reached (how long you want to wait in seconds), then you call the function to turn off the light.
Here's how you'd incorporate this into your code:
import threading
from subprocess import call
import os
import time
import RPi.GPIO as GPIO
PIR = 9 # data pin of PIR sensor (in)
LED = 7 # positive pin of LED (out)
SOUND = '/home/pi/events/sounds/Hello.wav' # reaction sound
COUNTLIMIT = 160
countThread = None
WATCHTIME = 600 #Run for 10 minutes
# GPIO setup
GPIO.setmode(GPIO.BCM)
GPIO.setup(PIR,GPIO.IN, pull_up_down=GPIO.PUD_DOWN)
GPIO.setup(LED,GPIO.OUT)
#------------------------------------------------------------
class CounterThread(threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
self.count = 0
self.start()
def run(self):
call(['/home/pi/bin/kitchenlights.sh', '-1']) # turn light on
while self.count < COUNTLIMIT:
time.sleep(0.1)
self.count += 0.1
call(['/home/pi/bin/kitchenlights.sh', '-0'])
threadKiller()
return
def newSig(self):
self.count = 0
#------------------------------------------------------------
def my_callback(channel):
'''function which gets called when motion is reported (sensor includes own delay-until-hot again and sensibility settings'''
global countThread
try:
countThread.newSig()
except:
countThread = CounterThread()
#------------------------------------------------------------
def threadKiller():
global countThread
countThread = None
#------------------------------------------------------------
def main():
GPIO.add_event_detect(PIR, GPIO.RISING,callback=my_callback,bouncetime=100) # add rising edge detection on a channel
t = 0
while t < WATCHTIME:
t += 0.1
time.sleep(0.1)
#------------------------------------------------------------
if __name__ == "__main__": main()
I don't have any way to test this, so please let me know if there is anything that breaks. Since you said you're new to Python I made a few formatting changes to make your code a bit prettier. These things are generally considered to be good form, but are optional. However, you need to be careful about indents, because as you have them in your question, your code should not run (it will throw an IndentError)
Hope this helps

Python wait x secs for a key and continue execution if not pressed

I'm a n00b to python, and I'm looking a code snippet/sample which performs the following:
Display a message like "Press any key to configure or wait X seconds to continue"
Wait, for example, 5 seconds and continue execution, or enter a configure() subroutine if a key is pressed.
Thank you for your help!
Yvan Janssens
If you're on Unix/Linux then the select module will help you.
import sys
from select import select
print "Press any key to configure or wait 5 seconds..."
timeout = 5
rlist, wlist, xlist = select([sys.stdin], [], [], timeout)
if rlist:
print "Config selected..."
else:
print "Timed out..."
If you're on Windows, then look into the msvcrt module. (Note this doesn't work in IDLE, but will in cmd prompt)
import sys, time, msvcrt
timeout = 5
startTime = time.time()
inp = None
print "Press any key to configure or wait 5 seconds... "
while True:
if msvcrt.kbhit():
inp = msvcrt.getch()
break
elif time.time() - startTime > timeout:
break
if inp:
print "Config selected..."
else:
print "Timed out..."
Edit Changed the code samples so you could tell whether there was a timeout or a keypress...
Python doesn't have any standard way to catch this, it gets keyboard input only through input() and raw_input().
If you really want this you could use Tkinter or pygame to catch the keystrokes as "events". There are also some platform-specific solutions like pyHook. But if it's not absolutely vital to your program, I suggest you make it work another way.
If you combine time.sleep, threading.Thread, and sys.stdin.read you can easily wait for a specified amount of time for input and then continue.
t = threading.Thread(target=sys.stdin.read(1) args=(1,))
t.start()
time.sleep(5)
t.join()
Here's how I did it:
import threading
import time
import sys
class MyThread(threading.Thread):
def __init__(self, threadID, name, counter, f):
super().__init__()
self.threadID = threadID
self.name = name
self.counter = counter
self.func = f
def run(self):
self.func()
class KeyboardMonitor:
def __init__(self):
# Setting a boolean flag is atomic in Python.
# It's hard to imagine a boolean being
# anything else, with or without the GIL.
# If inter-thread communication is anything more complicated than
# a couple of flags, you should replace low level variables with
# a thread safe buffer.
self.keepGoing = True
def wait4KeyEntry(self):
while self.keepGoing:
s = input("Type q to quit: ")
if s == "q":
self.keepGoing = False
def mainThread(self, f, *args, **kwargs):
"""Pass in some main function you want to run, and this will run it
until keepGoing = False. The first argument of function f must be
this class, so that that function can check the keepGoing flag and
quit when keepGoing is false."""
keyboardThread = MyThread(1, "keyboard_thread", 0, self.wait4KeyEntry)
keyboardThread.start()
while self.keepGoing:
f(self, *args, **kwargs)
def main(keyMonitorInst, *args, **kwargs):
while keyMonitorInst.keepGoing:
print("Running again...")
time.sleep(1)
if __name__ == "__main__":
uut = KeyboardMonitor()
uut.mainThread(main)
Rather than make a blocking call time out, my approach is to start a thread that waits for the user to enter input, while another thread does something else. The two processes communicate through a small number of atomic operations: in this case, setting a boolean flag. For anything more complicated than atomic operations, obviously you should replace the atomic variable with a threadsafe buffer of some kind.

Categories