How do you get tkinter to work with asyncio? My studies suggest this general question does resolve into the specific problem of getting tkinter to await a coroutine function.
Context
If tkinter's event loop is blocked the loop will freeze until the blocking function returns. If the event loop also happens to be running a GUI that will freeze as well. The traditional solution to this problem is to move any blocking code into a thread.
The new asyncio module is able to schedule threaded calls using the coroutine function asyncio.to_thread(coro). I gather this avoids the difficulties of writing correct threaded code.
Baseline: blocked.py
As a starting point I wrote a baseline program (See code below). It creates a tkinter event loop which attempts to
destroy itself and end the program after 2000ms. That attempt is thwarted by a blocking function which runs for 4s.
The program output is:
08:51:57: Program started.
08:51:58: blocking_func started.
08:52:02: blocking_func completed.
08:52:02: Tk event loop terminated.
08:52:02: Program ended.
Process finished with exit code 0
1st try: async_blocked.py
The blocking code has been refactored as a coroutine function so there are two event loops - tkinter's and asyncio's. The function blocking_io_handler is scheduled onto tkinter's event loop which runs it successfully. The coroutine function blocking_func is scheduled onto asyncio's loop where it starts successfully.
The problem is it doesn't start until after tkinter's event loop has terminated. Asyncio's loop was available throughout the execution of the coroutine function main so it was available when tk_root.mainloop() was executed. In spite of this asyncio was helpless because control was not yielded by an await statement during the execution of tk_root.mainloop. It had to wait for the await asyncio.sleep(3) statement which ran later and, by then, tkinter had stopped running.
At that time the await expression returns control to the async loop for three seconds — enough to start the four second blocking_func but not enough for it to finish.
08:38:22: Program started.
08:38:22: blocking_io_handler started.
08:38:22: blocking_io_handler completed.
08:38:24: Tk event loop terminated.
08:38:24: blocking_func started.
08:38:27: Program ended.
Process finished with exit code 0
2nd try: asyncth_blocked.py
This code replaces the function asyncio.create_task with the coroutine function asyncio.to_thread. This fails
with a runtime warning:
07:26:46: Program started.
07:26:47: blocking_io_handler started.
07:26:47: blocking_io_handler completed.
RuntimeWarning: coroutine 'to_thread' was never awaited
asyncio.to_thread(blocking_func)
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
07:26:49: Tk event loop terminated.
07:26:49: Program ended.
> Process finished with exit code 0
3rd try: asyncth_blocked_2.py
asyncio.to_thread must be awaited because it is a coroutine function and not a regular function:
await asyncio.to_thread(blocking_func).
Since the await keyword is a syntax error inside a regular function, def blocking_io_handler has to be changed into a coroutine function: async def blocking_io_handler.
These changes are shown in asyncth_blocked_2.py which produces this output:
07:52:29: Program started.
RuntimeWarning:
coroutine 'blocking_io_handler' was never awaited
func(*args)
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
07:52:31: Tk event loop terminated.
07:52:31: Program ended.
Process finished with exit code 0
Conclusion
For tkinter to work with asyncio the scheduled function call tk_root.after(0, blocking_io_handler) has to be somehow turned into a scheduled coroutine function call. This is the only way the asycio loop will have a chance to run scheduled async
tasks.
Is it possible?
Code
"""blocked.py"""
import time
import tkinter as tk
def timestamped_msg(msg: str):
print(f"{time.strftime('%X')}: {msg}")
def blocking_func():
timestamped_msg('blocking_func started.')
time.sleep(4)
timestamped_msg('blocking_func completed.')
def main():
timestamped_msg('Program started.')
tk_root = tk.Tk()
tk_root.after(0, blocking_func)
tk_root.after(2000, tk_root.destroy)
tk_root.mainloop()
timestamped_msg('Tk event loop terminated.')
timestamped_msg('Program ended.')
if __name__ == '__main__':
main()
"""async_blocked.py"""
import asyncio
import time
import tkinter as tk
def timestamped_msg(msg: str):
print(f"{time.strftime('%X')}: {msg}")
async def blocking_func():
timestamped_msg('blocking_func started.')
await asyncio.sleep(4)
timestamped_msg('blocking_func completed.')
def blocking_io_handler():
timestamped_msg('blocking_io_handler started.')
asyncio.create_task(blocking_func())
timestamped_msg('blocking_io_handler completed.')
async def main():
timestamped_msg('Program started.')
tk_root = tk.Tk()
tk_root.after(0, blocking_io_handler)
tk_root.after(2000, tk_root.destroy)
tk_root.mainloop()
timestamped_msg('Tk event loop terminated.')
await asyncio.sleep(3)
timestamped_msg('Program ended.')
if __name__ == '__main__':
asyncio.run(main())
"""asyncth_blocked.py"""
import asyncio
import time
import tkinter as tk
def timestamped_msg(msg: str):
print(f"{time.strftime('%X')}: {msg}")
async def blocking_func():
timestamped_msg('blocking_func started.')
await asyncio.sleep(4)
timestamped_msg('blocking_func completed.')
def blocking_io_handler():
timestamped_msg('blocking_io_handler started.')
asyncio.to_thread(blocking_func)
timestamped_msg('blocking_io_handler completed.')
async def main():
timestamped_msg('Program started.')
tk_root = tk.Tk()
tk_root.after(0, blocking_io_handler)
tk_root.after(2000, tk_root.destroy)
tk_root.mainloop()
timestamped_msg('Tk event loop terminated.')
timestamped_msg('Program ended.')
if __name__ == '__main__':
asyncio.run(main())
"""asyncth_blocked_2.py"""
import asyncio
import time
import tkinter as tk
def timestamped_msg(msg: str):
print(f"{time.strftime('%X')}: {msg}")
async def blocking_func():
timestamped_msg('blocking_func started.')
await asyncio.sleep(4)
timestamped_msg('blocking_func completed.')
async def blocking_io_handler():
timestamped_msg('blocking_io_handler started.')
await asyncio.to_thread(blocking_func)
timestamped_msg('blocking_io_handler completed.')
async def main():
timestamped_msg('Program started.')
tk_root = tk.Tk()
tk_root.after(0, blocking_io_handler)
tk_root.after(2000, tk_root.destroy)
tk_root.mainloop()
timestamped_msg('Tk event loop terminated.')
timestamped_msg('Program ended.')
if __name__ == '__main__':
asyncio.run(main())
Tkinter's Problem with Blocking IO Calls
The statement asyncio.sleep(60) will block tkinter for a minute if both are running in the same thread.
Blocking coroutine functions cannot run in the same thread as tkinter.
Similarly, the statement time.sleep(60) will block both tkinter and asyncio for a minute if all three are running in the same
thread.
Blocking non-coroutine functions cannot run in the same thread as either tkinter or asyncio.
Sleep commands have been used here to simplify this example of the blocking problem. The principles shown are applicable to internet or database accesses.
Solution
A solution is to create three distinct environments and take care when moving data between them.
Environment 1 - Main Thread
This is Python's MainThread. It's where Python starts and Tkinter lives. No blocking code can be allowed in this environment.
Environment 2 - Asyncio's Thread
This is where asyncio and all its coroutine functions live. Blocking functions are only allowed if they are coroutine
functions.
Environment 3 - Multiple single use threads
This is where non-coroutine blocking functions run. Since these are capable of blocking each other each needs
its own thread.
Data
Data returned from blocking IO to tkinter should be returned in threadsafe queues using a producer/consumer pattern.
Arguments and return values should not be passed between environments using regular functions. Use the threadsafe calling protocols provided by Python as illustrated below.
Wrong code
func(*args, **kwargs)
return_value = func(*args, **kwargs)
print(*args, **kwargs)
Correct code
threading.Thread(func, *args, **kwargs).start()
The return_value is not directly available. Use a queue.
future = asyncio.run_coroutine_threadsafe(func(*args, **kwargs), loop)
return_value = future.result().
print: Use a threadsafe queue to move printable objects to a single print thread. (See the SafePrinter context
maanger in the code below).
The Polling Problem
With tkinter, asyncio, and threading all running together there are three event loops controlling different stuff. Bad things can
happen when they mix. For example threading's Queue.get() will block environment 1 where tkinter's loop is trying to
control events. In this particular case, Queue.get_nowait() has to be used with polling via tkinter's after command. See the code below for other examples of unusual polling of queues.
GUI
Console output
0.001s In Print Thread of 2 without a loop: The SafePrinter is open for output.
0.001s In MainThread of 2 without a loop --- main starting
0.001s In Asyncio Thread of 3 without a loop --- aio_main starting
0.001s In MainThread of 3 without a loop --- tk_main starting
0.305s In Asyncio Thread of 3 with a loop --- manage_aio_loop starting
0.350s In MainThread of 3 without a loop --- tk_callbacks starting
0.350s In MainThread of 3 without a loop --- tk_callback_consumer starting
0.350s In Asyncio Thread of 3 with a loop --- aio_blocker starting. block=3.1s.
0.350s In MainThread of 3 without a loop --- aio_exception_handler starting. block=3.1s
0.351s In MainThread of 3 without a loop --- aio_exception_handler starting. block=1.1s
0.351s In Asyncio Thread of 4 with a loop --- aio_blocker starting. block=1.1s.
0.351s In IO Block Thread (3.2s) of 4 without a loop --- io_exception_handler starting. block=3.2s.
0.351s In IO Block Thread (3.2s) of 4 without a loop --- io_blocker starting. block=3.2s.
0.351s In IO Block Thread (1.2s) of 5 without a loop --- io_exception_handler starting. block=1.2s.
0.351s In IO Block Thread (1.2s) of 5 without a loop --- io_blocker starting. block=1.2s.
0.351s In MainThread of 5 without a loop --- tk_callbacks ending - All blocking callbacks have been scheduled.
1.451s In Asyncio Thread of 5 with a loop --- aio_blocker ending. block=1.1s.
1.459s In MainThread of 5 without a loop --- aio_exception_handler ending. block=1.1s
1.555s In IO Block Thread (1.2s) of 5 without a loop --- io_blocker ending. block=1.2s.
1.555s In IO Block Thread (1.2s) of 5 without a loop --- io_exception_handler ending. block=1.2s.
3.450s In Asyncio Thread of 4 with a loop --- aio_blocker ending. block=3.1s.
3.474s In MainThread of 4 without a loop --- aio_exception_handler ending. block=3.1s
3.553s In IO Block Thread (3.2s) of 4 without a loop --- io_blocker ending. block=3.2s.
3.553s In IO Block Thread (3.2s) of 4 without a loop --- io_exception_handler ending. block=3.2s.
4.140s In MainThread of 3 without a loop --- tk_callback_consumer ending
4.140s In MainThread of 3 without a loop --- tk_main ending
4.141s In Asyncio Thread of 3 with a loop --- manage_aio_loop ending
4.141s In Asyncio Thread of 3 without a loop --- aio_main ending
4.141s In MainThread of 2 without a loop --- main ending
4.141s In Print Thread of 2 without a loop: The SafePrinter has closed.
Process finished with exit code 0
Code
""" tkinter_demo.py
Created with Python 3.10
"""
import asyncio
import concurrent.futures
import functools
import itertools
import queue
import sys
import threading
import time
import tkinter as tk
import tkinter.ttk as ttk
from collections.abc import Iterator
from contextlib import AbstractContextManager
from dataclasses import dataclass
from types import TracebackType
from typing import Optional, Type
# Global reference to loop allows access from different environments.
aio_loop: Optional[asyncio.AbstractEventLoop] = None
def io_blocker(task_id: int, tk_q: queue.Queue, block: float = 0) -> None:
""" Block the thread and put a 'Hello World' work package into Tkinter's work queue.
This is a producer for Tkinter's work queue. It will run in a special thread created solely for running this
function. The statement `time.sleep(block)` can be replaced with any non-awaitable blocking code.
Args:
task_id: Sequentially issued tkinter task number.
tk_q: tkinter's work queue.
block: block time
Returns:
Nothing. The work package is returned via the threadsafe tk_q.
"""
safeprint(f'io_blocker starting. {block=}s.')
time.sleep(block)
# Exceptions for testing handlers. Uncomment these to see what happens when exceptions are raised.
# raise IOError('Just testing an expected error.')
# raise ValueError('Just testing an unexpected error.')
work_package = f"Task #{task_id} {block}s: 'Hello Threading World'."
tk_q.put(work_package)
safeprint(f'io_blocker ending. {block=}s.')
def io_exception_handler(task_id: int, tk_q: queue.Queue, block: float = 0) -> None:
""" Exception handler for non-awaitable blocking callback.
It will run in a special thread created solely for running io_blocker.
Args:
task_id: Sequentially issued tkinter task number.
tk_q: tkinter's work queue.
block: block time
"""
safeprint(f'io_exception_handler starting. {block=}s.')
try:
io_blocker(task_id, tk_q, block)
except IOError as exc:
safeprint(f'io_exception_handler: {exc!r} was handled correctly. ')
finally:
safeprint(f'io_exception_handler ending. {block=}s.')
async def aio_blocker(task_id: int, tk_q: queue.Queue, block: float = 0) -> None:
""" Asynchronously block the thread and put a 'Hello World' work package into Tkinter's work queue.
This is a producer for Tkinter's work queue. It will run in the same thread as the asyncio loop. The statement
`await asyncio.sleep(block)` can be replaced with any awaitable blocking code.
Args:
task_id: Sequentially issued tkinter task number.
tk_q: tkinter's work queue.
block: block time
Returns:
Nothing. The work package is returned via the threadsafe tk_q.
"""
safeprint(f'aio_blocker starting. {block=}s.')
await asyncio.sleep(block)
# Exceptions for testing handlers. Uncomment these to see what happens when exceptions are raised.
# raise IOError('Just testing an expected error.')
# raise ValueError('Just testing an unexpected error.')
work_package = f"Task #{task_id} {block}s: 'Hello Asynchronous World'."
# Put the work package into the tkinter's work queue.
while True:
try:
# Asyncio can't wait for the thread blocking `put` method…
tk_q.put_nowait(work_package)
except queue.Full:
# Give control back to asyncio's loop.
await asyncio.sleep(0)
else:
# The work package has been placed in the queue so we're done.
break
safeprint(f'aio_blocker ending. {block=}s.')
def aio_exception_handler(mainframe: ttk.Frame, future: concurrent.futures.Future, block: float,
first_call: bool = True) -> None:
""" Exception handler for future coroutine callbacks.
This non-coroutine function uses tkinter's event loop to wait for the future to finish.
It runs in the Main Thread.
Args:
mainframe: The after method of this object is used to poll this function.
future: The future running the future coroutine callback.
block: The block time parameter used to identify which future coroutine callback is being reported.
first_call: If True will cause an opening line to be printed on stdout.
"""
if first_call:
safeprint(f'aio_exception_handler starting. {block=}s')
poll_interval = 100 # milliseconds
try:
# Python will not raise exceptions during future execution until `future.result` is called. A zero timeout is
# required to avoid blocking the thread.
future.result(0)
# If the future hasn't completed, reschedule this function on tkinter's event loop.
except concurrent.futures.TimeoutError:
mainframe.after(poll_interval, functools.partial(aio_exception_handler, mainframe, future, block,
first_call=False))
# Handle an expected error.
except IOError as exc:
safeprint(f'aio_exception_handler: {exc!r} was handled correctly. ')
else:
safeprint(f'aio_exception_handler ending. {block=}s')
def tk_callback_consumer(tk_q: queue.Queue, mainframe: ttk.Frame, row_itr: Iterator):
""" Display queued 'Hello world' messages in the Tkinter window.
This is the consumer for Tkinter's work queue. It runs in the Main Thread. After starting, it runs
continuously until the GUI is closed by the user.
"""
# Poll continuously while queue has work needing processing.
poll_interval = 0
try:
# Tkinter can't wait for the thread blocking `get` method…
work_package = tk_q.get_nowait()
except queue.Empty:
# …so be prepared for an empty queue and slow the polling rate.
poll_interval = 40
else:
# Process a work package.
label = ttk.Label(mainframe, text=work_package)
label.grid(column=0, row=(next(row_itr)), sticky='w', padx=10)
finally:
# Have tkinter call this function again after the poll interval.
mainframe.after(poll_interval, functools.partial(tk_callback_consumer, tk_q, mainframe, row_itr))
def tk_callbacks(mainframe: ttk.Frame, row_itr: Iterator):
""" Set up 'Hello world' callbacks.
This runs in the Main Thread.
Args:
mainframe: The mainframe of the GUI used for displaying results from the work queue.
row_itr: A generator of line numbers for displaying items from the work queue.
"""
safeprint('tk_callbacks starting')
task_id_itr = itertools.count(1)
# Create the job queue and start its consumer.
tk_q = queue.Queue()
safeprint('tk_callback_consumer starting')
tk_callback_consumer(tk_q, mainframe, row_itr)
# Schedule the asyncio blocker.
for block in [3.1, 1.1]:
# This is a concurrent.futures.Future not an asyncio.Future because it isn't threadsafe. Also,
# it doesn't have a wait with timeout which we shall need.
task_id = next(task_id_itr)
future = asyncio.run_coroutine_threadsafe(aio_blocker(task_id, tk_q, block), aio_loop)
# Can't use Future.add_done_callback here. It doesn't return until the future is done and that would block
# tkinter's event loop.
aio_exception_handler(mainframe, future, block)
# Run the thread blocker.
for block in [3.2, 1.2]:
task_id = next(task_id_itr)
threading.Thread(target=io_exception_handler, args=(task_id, tk_q, block),
name=f'IO Block Thread ({block}s)').start()
safeprint('tk_callbacks ending - All blocking callbacks have been scheduled.\n')
def tk_main():
""" Run tkinter.
This runs in the Main Thread.
"""
safeprint('tk_main starting\n')
row_itr = itertools.count()
# Create the Tk root and mainframe.
root = tk.Tk()
mainframe = ttk.Frame(root, padding="15 15 15 15")
mainframe.grid(column=0, row=0)
# Add a close button
button = ttk.Button(mainframe, text='Shutdown', command=root.destroy)
button.grid(column=0, row=next(row_itr), sticky='w')
# Add an information widget.
label = ttk.Label(mainframe, text=f'\nWelcome to hello_world*4.py.\n')
label.grid(column=0, row=next(row_itr), sticky='w')
# Schedule the 'Hello World' callbacks
mainframe.after(0, functools.partial(tk_callbacks, mainframe, row_itr))
# The asyncio loop must start before the tkinter event loop.
while not aio_loop:
time.sleep(0)
root.mainloop()
safeprint(' ', timestamp=False)
safeprint('tk_callback_consumer ending')
safeprint('tk_main ending')
async def manage_aio_loop(aio_initiate_shutdown: threading.Event):
""" Run the asyncio loop.
This provides an always available asyncio service for tkinter to make any number of simultaneous blocking IO
calls. 'Any number' includes zero.
This runs in Asyncio's thread and in asyncio's loop.
"""
safeprint('manage_aio_loop starting')
# Communicate the asyncio loop status to tkinter via a global variable.
global aio_loop
aio_loop = asyncio.get_running_loop()
# If there are no awaitables left in the queue asyncio will close.
# The usual wait command — Event.wait() — would block the current thread and the asyncio loop.
while not aio_initiate_shutdown.is_set():
await asyncio.sleep(0)
safeprint('manage_aio_loop ending')
def aio_main(aio_initiate_shutdown: threading.Event):
""" Start the asyncio loop.
This non-coroutine function runs in Asyncio's thread.
"""
safeprint('aio_main starting')
asyncio.run(manage_aio_loop(aio_initiate_shutdown))
safeprint('aio_main ending')
def main():
"""Set up working environments for asyncio and tkinter.
This runs in the Main Thread.
"""
safeprint('main starting')
# Start the permanent asyncio loop in a new thread.
# aio_shutdown is signalled between threads. `asyncio.Event()` is not threadsafe.
aio_initiate_shutdown = threading.Event()
aio_thread = threading.Thread(target=aio_main, args=(aio_initiate_shutdown,), name="Asyncio's Thread")
aio_thread.start()
tk_main()
# Close the asyncio permanent loop and join the thread in which it runs.
aio_initiate_shutdown.set()
aio_thread.join()
safeprint('main ending')
#dataclass
class SafePrinter(AbstractContextManager):
_time_0 = time.perf_counter()
_print_q = queue.Queue()
_print_thread: threading.Thread | None = None
def __enter__(self):
""" Run the safeprint consumer method in a print thread.
Returns:
Thw safeprint producer method. (a.k.a. the runtime context)
"""
self._print_thread = threading.Thread(target=self._safeprint_consumer, name='Print Thread')
self._print_thread.start()
return self._safeprint
def __exit__(self, __exc_type: Type[BaseException] | None, __exc_value: BaseException | None,
__traceback: TracebackType | None) -> bool | None:
""" Close the print and join the print thread.
Args:
None or the exception raised during the execution of the safeprint producer method.
__exc_type:
__exc_value:
__traceback:
Returns:
False to indicate that any exception raised in self._safeprint has not been handled.
"""
self._print_q.put(None)
self._print_thread.join()
return False
def _safeprint(self, msg: str, *, timestamp: bool = True, reset: bool = False):
"""Put a string into the print queue.
'None' is a special msg. It is not printed but will close the queue and this context manager.
The exclusive thread and a threadsafe print queue ensure race free printing.
This is the producer in the print queue's producer/consumer pattern.
It runs in the same thread as the calling function
Args:
msg: The message to be printed.
timestamp: Print a timestamp (Default = True).
reset: Reset the time to zero (Default = False).
"""
if reset:
self._time_0 = time.perf_counter()
if timestamp:
self._print_q.put(f'{self._timestamp()} --- {msg}')
else:
self._print_q.put(msg)
def _safeprint_consumer(self):
"""Get strings from the print queue and print them on stdout.
The print statement is not threadsafe, so it must run in its own thread.
This is the consumer in the print queue's producer/consumer pattern.
"""
print(f'{self._timestamp()}: The SafePrinter is open for output.')
while True:
msg = self._print_q.get()
# Exit function when any producer function places 'None'.
if msg is not None:
print(msg)
else:
break
print(f'{self._timestamp()}: The SafePrinter has closed.')
def _timestamp(self) -> str:
"""Create a timestamp with useful status information.
This is a support function for the print queue producers. It runs in the same thread as the calling function
so the returned data does not cross between threads.
Returns:
timestamp
"""
secs = time.perf_counter() - self._time_0
try:
asyncio.get_running_loop()
except RuntimeError as exc:
if exc.args[0] == 'no running event loop':
loop_text = 'without a loop'
else:
raise
else:
loop_text = 'with a loop'
return f'{secs:.3f}s In {threading.current_thread().name} of {threading.active_count()} {loop_text}'
if __name__ == '__main__':
with SafePrinter() as safeprint:
sys.exit(main())
Related
I want to execute a task after certain time, so I have tried a countdown timer with a condition of being finished (when countdown variable = 0, the task is performed). The thing is that I don't want to stop the execution of the main program while performing the countdown. I have tried this:
import time
def countdown(num_of_secs):
while(num_of_secs):
time.sleep(1)
num_of_secs -= 1
return num_of_secs
So, I run my code setting a number of seconds to the countdown, and when this countdown reaches the 0 value, a task must be executed. Using this code (it uses a while), when I call my function "countdown" it stops the execution of the main program, so it is the same as a big time.sleep. I want to carry out this countdown in the background, without stopping other actions until the countdown finishes and the task starts.
Thank you
Another alternative is by using threading.
I've got a simple example here with 2 Threads where the working thread is waiting for the countdown thread to finish and starting. The Main is still working fine.
import threading
import time
def do_something():
countdown_thread.join()
print("Starting Task")
time.sleep(3)
print("Finished Task")
def countdown(num_of_secs):
while(num_of_secs):
time.sleep(1)
num_of_secs -= 1
print(num_of_secs)
if __name__ == '__main__':
countdown_thread = threading.Thread(target=countdown, args=(3,))
work_thread = threading.Thread(target=do_something)
countdown_thread.start()
work_thread.start()
while True:
print("Main doing something")
time.sleep(1)
Example picture for multithreading: Sequential vs Threading
Usually python only has a single program flow, so every instruction needs to complete before the next one can get executed.
For your case you need asynchronicity, with e.g. asyncio.sleep(5) as a separate task in the same event loop.
import asyncio
async def sleeper():
print('Holding...')
await asyncio.sleep(5)
print('Doing Work!')
async def work():
print('Doing work')
print('while')
print('the other guy is sleeping')
async def main():
await asyncio.gather(sleeper(), work())
asyncio.run(main())
The most common and easiest way to implement this would be with a Timer object from the threading library. It would go as follows:
import threading
import time
i = 0
done = False
def show_results():
print("results from GPIO readings")
print("=)")
global done
done = True # signal end of while loop
def read_GPIO():
print("reading GPIO...")
t = threading.Timer(60, show_results) # task will trigger after 60 seconds
t.start()
# your while loop would go here
read_GPIO() # do work
while not done:
print("waiting", i) # doing work while waiting for timer
time.sleep(1)
i += 1
pass
Notice that the time library is used only for illustrative purposes. You could also start the timer recursively to check periodically GPIOs and print results or trigger an event. For more information on the threading library or the Timer object check the docs
I am trying to implement asyncio with pysimplegui in python.
In this GUI example, two buttons (button2 and button3) emulate a long task to accomplish.
Goal:
Be able to go back to the GUI interface even if a called function (through button) takes time to return the result.
Expected result:
If button2 or button3 or both are pushed, they both continue to execute their tasks and users can go back to the GUI to carry on with other tasks.
Current result:
As soon as either button2 or button3 is pushed, the tasks is blocked and contionue until the end and the GUI hangs until the end.
import PySimpleGUI as sg
import asyncio
import time
sg.theme('Light Blue 3')
# This design pattern simulates button callbacks
# This implementation uses a simple "Dispatch Dictionary" to store events and functions
# The callback functions
async def button1():
print('Button 1 callback')
return 'nothing'
async def button2():
print('Button 2 callback')
for i in range(1,20):
await asyncio.sleep(3)
print(f"Button 2: {i}")
return f"button2 end"
async def button3():
print('Button 3 callback')
for i in range(1,10):
await asyncio.sleep(3)
print(f"Button 3: {i}")
return f"button3: end"
# Lookup dictionary that maps button to function to call
dispatch_dictionary = {'1':button1, '2':button2, '3':button3}
# Layout the design of the GUI
layout = [[sg.Text('Please click a button', auto_size_text=True)],
[sg.Button('1'), sg.Button('2'), sg.Button('3'), sg.Quit()]]
# Show the Window to the user__TIMEOUT__
window = sg.Window('Button callback example', layout)
# Event loop. Read buttons, make callbacks
while True:
# Read the Window
event, values = window.read()
if event in ('Quit', sg.WIN_CLOSED):
break
if event == '__TIMEOUT__':
continue
# Lookup event in function dictionary
if event in dispatch_dictionary:
func_to_call = dispatch_dictionary[event] # get function from dispatch dictionary
print(asyncio.run(func_to_call()))
else:
print('Event {} not in dispatch dictionary'.format(event))
window.close()
# All done!
sg.popup_ok('Done')
I thought that I applied async/wait according to the rules. Did I miss something?
asyncio.run() executes one coroutine and blocks until it is done. It does NOT start a parallel thread to run the coroutine in.
You have two options:
Don't use asyncio, use Threading to start a new thread for each long operation.
Start a thread with an asyncio eventloop in it at the start of the program, then use asyncio.run_coroutine_threadsafe() to schedule a coroutine onto the event loop from the main GUI thread.
I will explain option 2 here.. Example at the start of your program:
from threading import Thread
def asyncloop(loop):
# Set loop as the active event loop for this thread
asyncio.set_event_loop(loop)
# We will get our tasks from the main thread so just run an empty loop
loop.run_forever()
# create a new loop
loop = asyncio.new_event_loop()
# Create the new thread, giving loop as argument
t = Thread(target=asyncloop, args=(loop,))
# Start the thread
t.start()
Later in the button event code (in main thread):
asyncio.run_coroutine_threadsafe(func_to_call(), loop)
This will schedule the coroutine to run as a parallel task inside the thread we created.
I need help please, I use schedule.every().day.at("17:40").do(my_function) and I would like my program to run normally and when the schedule.every().day.at("17:40").do(my_function) arrives, it executes the associated function but then it comes back in my loop and wait for another day etc.... I dont know how to do it because i think schedule.every().day.at("17:40").do(my_function) need
while1:
schedule.run_pending()
time.sleep(1)
So i dont know how to changes this 3 lignes to make my programme work.
Thanks!
You would have to run it in separated threading or multiprocessing.
But first you should check documentation because I found in Common Questions:
How to continuously run the scheduler without blocking the main thread?
They created class Scheduler which put it in thread and you need to run run_continuously()
But I use it to created shorter example
import schedule
import time
import threading
# --- functions ---
stop_running = threading.Event() # to control loop in thread
def run_continuously(scheduler, interval=1):
#print('starting loop in thread')
while not stop_running.is_set():
schedule.run_pending()
time.sleep(interval)
#print('stoping loop in thread')
def job():
print("I'm working...")
# --- main ---
schedule.every(1).minutes.do(job)
# run schedule in thread
schedule_in_thread = threading.Thread(target=run_continuously, args=(schedule,))
schedule_in_thread.start()
# run other code
#print('starting main loop')
try:
while True:
print("other code")
time.sleep(3)
except KeyboardInterrupt as ex:
print('stoping', ex)
#print('stoping main loop')
# stop schedule in thread
stop_running.set() # to stop loop in `run_continuously`
schedule_in_thread.join() # wait until thread finish
I use try/except with KeyboardInterrupt only to gracefully stop program when I press Ctrl+C - and code may stop thread.
What I want to do is, in the Python console:
Create an asyncio future
Attach a callback to this future
Set the future value (and so get at the same time the callback result)
To do this, I tried that, in the Python console:
from threading import Thread
import asyncio
def start_loop(loop):
asyncio.set_event_loop(loop)
loop.run_forever()
new_loop = asyncio.new_event_loop()
t = Thread(target=start_loop, args=(new_loop,))
t.start()
fut = new_loop.create_future()
fut.add_done_callback(lambda _: print("Hey"))
fut.set_result(True)
I expect "Hey" to be printed, but actually I get nothing.
Additional information:
When I do:
import asyncio
loop = asyncio.get_event_loop()
fut = loop.create_future()
fut.add_done_callback(lambda x: print("Hey"))
fut.set_result(True)
loop.run_forever()
I get the expected result, but I had to call run_forever while the future is already set.
My questions are:
Why my first block of code does not do the job as expected ?
Is there a hypothetical syntax like the following existing ?
import asyncio
loop = asyncio.get_event_loop()
fut = loop.create_future()
# Hypothetical part
loop.run_forever_in_background()
# End of hypothetical part
fut.add_done_callback(lambda x: print("Hey"))
fut.set_result(True)
My high level purpose is to play with futures and asyncio directly in the console, to get a better understanding of it.
Pay attention when playing with asyncio loops, futures etc. in multithreading context.
Almost all asyncio objects are not thread safe, which is
typically not a problem unless there is code that works with them from
outside of a Task or a callback.
...
To schedule a coroutine object from a different OS thread, the
run_coroutine_threadsafe() function should be used. It returns a
concurrent.futures.Future to access the result:
from threading import Thread
import asyncio
def start_loop(loop):
asyncio.set_event_loop(loop)
loop.run_forever()
new_loop = asyncio.new_event_loop()
t = Thread(target=start_loop, args=(new_loop,))
t.start()
async def add_future():
fut = new_loop.create_future()
fut.add_done_callback(lambda _: print("Hey"))
fut.set_result(True)
asyncio.run_coroutine_threadsafe(add_future(), loop=new_loop)
Test:
$ python3 test.py
Hey
Another option is using loop.call_soon_threadsafe(callback, *args, context=None).
https://docs.python.org/3/library/asyncio-dev.html#asyncio-multithreading
I'm using python to create a script which runs and interacts with some processes simultaneously. For that I'm using asyncio to implement this parallelism. The main problem is how to run another cleanup routine when a KeyboardInterrupt or a SIGINT occurs.
Here's an example code I wrote to show the problem:
import asyncio
import logging
import signal
from time import sleep
class Process:
async def start(self, arguments):
self._process = await asyncio.create_subprocess_exec("/bin/bash", *arguments)
return await self._process.wait()
async def stop(self):
self._process.terminate()
class BackgroundTask:
async def start(self):
# Very important process which needs to run while process 2 is running
self._process1 = Process()
self._process1_task = asyncio.create_task(self._process1.start(["-c", "sleep 100"]))
self._process2 = Process()
self._process2_task = asyncio.create_task(self._process2.start(["-c", "sleep 50"]))
await asyncio.wait([self._process1_task, self._process2_task], return_when=asyncio.ALL_COMPLETED)
async def stop(self):
# Stop process
await self._process1.stop()
# Call a cleanup process which cleans up process 1
cleanup_process = Process()
await cleanup_process.start(["-c", "sleep 10"])
# After that we can stop our second process
await self._process2.stop()
backgroundTask = BackgroundTask()
async def main():
await asyncio.create_task(backgroundTask.start())
logging.basicConfig(level=logging.DEBUG)
asyncio.run(main(), debug=True)
This code creates a background task which starts two processes (in this example two bash sleep commands) and waits for them to finish. This works fine and both command are running in parallel.
The main problem is the stop routine. I'd like to run the stop method when the program receives a SIGINT or KeyboardInterrupt, which first stops the process1, then starts a cleanup method and stops process2 afterwards. This is necessary because the cleanup command depends on process2.
What I've tried (instead of the asyncio.run() and the async main):
def main():
try:
asyncio.get_event_loop().run_until_complete(backgroundTask.start())
except KeyboardInterrupt:
asyncio.get_event_loop().run_until_complete(backgroundTask.stop())
main()
This of course doens't work as expected, because as soon as an KeyboardInterrupt exception occours the backgroundTask.start Task is canceled and the backgroundTask.stop is started in the main loop, so my processes are canceled and can't stopped properly.
So is there a way to detect the KeyboardInterrupt without canceling the current main loop and run my backgroundTask.stop method instead?
You want to add a signal handler as shown in this example in the docs:
import asyncio
import functools
import os
import signal
def ask_exit(signame, loop):
print("got signal %s: exit" % signame)
loop.stop()
async def main():
loop = asyncio.get_running_loop()
for signame in {'SIGINT', 'SIGTERM'}:
loop.add_signal_handler(
getattr(signal, signame),
functools.partial(ask_exit, signame, loop))
await asyncio.sleep(3600)
print("Event loop running for 1 hour, press Ctrl+C to interrupt.")
print(f"pid {os.getpid()}: send SIGINT or SIGTERM to exit.")
asyncio.run(main())
That's a bit of an overcomplicated/outdated example though, consider it more like this (your coroutine code goes where the asyncio.sleep call is):
import asyncio
from signal import SIGINT, SIGTERM
async def main():
loop = asyncio.get_running_loop()
for signal_enum in [SIGINT, SIGTERM]:
loop.add_signal_handler(signal_enum, loop.stop)
await asyncio.sleep(3600) # Your code here
asyncio.run(main())
At this point a Ctrl + C will break the loop and raise a RuntimeError, which you can catch by putting the asyncio.run call in a try/except block like so:
try:
asyncio.run(main())
except RuntimeError as exc:
expected_msg = "Event loop stopped before Future completed."
if exc.args and exc.args[0] == expected_msg:
print("Bye")
else:
raise
That's not very satisfying though (what if something else caused the same error?), so I'd prefer to raise a distinct error. Also, if you're exiting on the command line, the proper thing to do is to return the proper exit code (in fact, the code in the example just uses the name, but it's actually an IntEnum with that numeric exit code in it!)
import asyncio
from functools import partial
from signal import SIGINT, SIGTERM
from sys import stderr
class SignalHaltError(SystemExit):
def __init__(self, signal_enum):
self.signal_enum = signal_enum
print(repr(self), file=stderr)
super().__init__(self.exit_code)
#property
def exit_code(self):
return self.signal_enum.value
def __repr__(self):
return f"\nExitted due to {self.signal_enum.name}"
def immediate_exit(signal_enum, loop):
loop.stop()
raise SignalHaltError(signal_enum=signal_enum)
async def main():
loop = asyncio.get_running_loop()
for signal_enum in [SIGINT, SIGTERM]:
exit_func = partial(immediate_exit, signal_enum=signal_enum, loop=loop)
loop.add_signal_handler(signal_enum, exit_func)
await asyncio.sleep(3600)
print("Event loop running for 1 hour, press Ctrl+C to interrupt.")
asyncio.run(main())
Which when Ctrl + C'd out of gives:
python cancelling_original.py
⇣
Event loop running for 1 hour, press Ctrl+C to interrupt.
^C
Exitted due to SIGINT
echo $?
⇣
2
Now there's some code I'd be happy to serve! :^)
P.S. here it is with type annotations:
from __future__ import annotations
import asyncio
from asyncio.events import AbstractEventLoop
from functools import partial
from signal import Signals, SIGINT, SIGTERM
from sys import stderr
from typing import Coroutine
class SignalHaltError(SystemExit):
def __init__(self, signal_enum: Signals):
self.signal_enum = signal_enum
print(repr(self), file=stderr)
super().__init__(self.exit_code)
#property
def exit_code(self) -> int:
return self.signal_enum.value
def __repr__(self) -> str:
return f"\nExitted due to {self.signal_enum.name}"
def immediate_exit(signal_enum: Signals, loop: AbstractEventLoop) -> None:
loop.stop()
raise SignalHaltError(signal_enum=signal_enum)
async def main() -> Coroutine:
loop = asyncio.get_running_loop()
for signal_enum in [SIGINT, SIGTERM]:
exit_func = partial(immediate_exit, signal_enum=signal_enum, loop=loop)
loop.add_signal_handler(signal_enum, exit_func)
return await asyncio.sleep(3600)
print("Event loop running for 1 hour, press Ctrl+C to interrupt.")
asyncio.run(main())
The advantage of a custom exception here is that you can then catch it specifically, and avoid the traceback being dumped to the screen
try:
asyncio.run(main())
except SignalHaltError as exc:
# log.debug(exc)
pass
else:
raise