We Keep Coding

Tkinter freezes while trying to read stdout from subprocess

I started a Tkinter application but I'm with problems with buffering. I searched the solution but didn't find it.
Correlated links:
Calling python script with subprocess popen and flushing the data
Python C program subprocess hangs at "for line in iter"
Python subprocess standard output to a variable
As an exemple, this app has two buttons: Start and Stop.
When I press the button Start, the spammer.py is called as a subprocess and when I press the button Stop the program must be killed.
# spammer.py
import time
counter = 0
while counter < 40: # This process will last 10 second maximum
print("counter = %d" % counter)
counter += 1
time.sleep(0.25) # 4 messages/second
While the PrintNumbers.py is running, I want the spammer's output be storage in a variable inside the Tkinter to be used in realtime. But once I try to read the buffer with myprocess.stdout.readline, it stucks and it doesn't continue until the subprocess finish and as consequence for exemple I cannot click on the Stop button.
I read that the function is waiting for EOF to continue, and I tried to use tokens as shown here, and the function that should continue when it finds a or a \n, but it did not work.
The Tkinter exemple is bellow. After I click at the Start button, I instantly see the message Started reading stdout, and after 10 seconds it shows a lot of messages, while I wanted to show every message over time.
# PrintNumbers.py
import Tkinter as tk
import subprocess
class App:
def __init__(self, root):
self.root = root
self.myprocess = None
self.logmessages = []
self.createButtons()
self.timedUpdate()
def createButtons(self):
self.ButtonsFrame = tk.Frame(self.root, width=600, height=400)
self.ButtonsFrame.pack()
self.startbutton = tk.Button(self.ButtonsFrame, text="Start",
command=self.__ClickOnStarButton)
self.stopbutton = tk.Button(self.ButtonsFrame, text="Stop",
command=self.__ClickOnStopButton)
self.startbutton.pack()
self.stopbutton.pack()
self.startbutton["state"] = "normal"
self.stopbutton["state"] = "disable"
def __ClickOnStarButton(self):
print("Click on Start Button")
self.startbutton["state"] = "disable"
self.stopbutton["state"] = "normal"
self.startProcess()
def __ClickOnStopButton(self):
print("Click on Stop Button")
self.startbutton["state"] = "normal"
self.stopbutton["state"] = "disable"
self.killProcess()
def startProcess(self):
command = "python spammer.py"
self.myprocess = subprocess.Popen(command, stdout=subprocess.PIPE, bufsize=1)
def killProcess(self):
self.myprocess.terminate()
self.myprocess.wait()
self.myprocess = None
def timedUpdate(self):
if self.myprocess is not None: # There's a process running
self.getLogText() # Will get the info from spammer.py
self.treatOutput() # Do whatever we want with the data
root.after(200, self.timedUpdate) # Every 0.2 seconds we will update
def getLogText(self):
if self.myprocess is None: # There's no process running
return
# The problem is here
print("Started reading stdout")
for line in iter(self.myprocess.stdout.readline, ''):
print(" Inside the loop. line = '%s'" % line)
self.logmessages.append(line)
print("Finished reading stdout")
def treatOutput(self):
# Any function that uses the spammer's output
# it's here just to test
while len(self.logmessages):
line = self.logmessage.pop(0)
line = line.replace("counter = ", "")
mynumber = int(line)
if mynumber % 3:
print(mynumber)
if __name__ == "__main__":
root = tk.Tk()
app = App(root)
root.mainloop()
How can I read the output without getting stuck? I'm still using python 2.7, and I don't know if it's the problem either.

Python Tkinter account for lost time

I've read similar questions that have been answered:
Getting realtime output from ffmpeg to be used in progress bar (PyQt4, stdout)
Progressbar to show amount of music played
How to measure elapsed time in Python?
Here's my window:
Problem is song ends when counter still has 20% to go. I know the reason is primarily due to system call to check if process is still running pgrep ffplay 10 times every second. Secondary reason is simply Python and Tkinter overhead.
To "band-aid fix" the problem I used 1.24 deciseconds instead of 1 every decisecond as my code illustrates now:
def play_to_end(self):
'''
Play single song, checking status every decisecond
Called from:
self.play_forever() to start a new song
self.pp_toggle() to restart song after pausing
'''
while True:
if not self.top2_is_active: return # Play window closed?
root.update() # Process other events
if self.pp_state is "Paused":
time.sleep(.1) # Wait until playing
continue
PID = os.popen("pgrep ffplay").read() # Get PID for ffplay
if len(PID) < 2: # Has song ended?
return # Song has ended
#self.current_song_time += .1 # Add decisecond
self.current_song_time += .124 # Add 1.24 deciseconds
# compensatation .24
self.current_progress.set(str('%.1f' % self.current_song_time) + \
" seconds of: " + str(self.DurationSecs))
root.update() # Process other events
root.after(100) # Sleep 1 decisecond
The problem with this band-aid fix is it is highly machine dependent. My machine is a Skylake for example. Also it is highly dependent on what other processes are running at the same time. When testing my machine load was relatively light:
How can I programmatically account for lost time in order to increment elapsed time accurately?
Perhaps there is a better way of simply querying ffplay to find out song progress?
As an aside (I know it's frowned upon to ask two questions at once) why can't I simply check if PID is null? I have tried .rstrip() and .strip() after .read() to no avail with checking PID equal to "" or None. If ffplay every has a process ID under 10 program will misbehave.

You can use subprocess.Popen() to execute ffplay and redirect stderr to PIPE, then you can read the progress from stderr and update the progress label.
Below is an example:
import tkinter as tk
import subprocess as subp
import threading
class MediaPlayer(tk.Tk):
def __init__(self):
super().__init__()
self.init_ui()
self.proc = None
self.protocol('WM_DELETE_WINDOW', self.quit)
def init_ui(self):
self.current_progress = tk.StringVar()
self.progress = tk.Label(self, textvariable=self.current_progress)
self.progress.grid(row=0, column=0, columnspan=2)
btn_close = tk.Button(self, text='Stop', width=20, command=self.stop_playing)
btn_close.grid(row=1, column=0, sticky='ew')
btn_play = tk.Button(self, text='Play', width=20, command=self.play_song)
btn_play.grid(row=1, column=1, sticky='ew')
def play_to_end(self):
self.proc = subp.Popen(
['ffplay', '-nodisp', '-hide_banner', '-autoexit', self.current_song_path],
stderr=subp.PIPE, bufsize=1, text=1
)
duration = ''
while self.proc.poll() is None:
msg = self.proc.stderr.readline().strip()
if msg:
if msg.startswith('Duration'):
duration = msg.split(',')[0].split(': ')[1]
else:
msg = msg.split()[0]
if '.' in msg:
elapsed = float(msg)
mins, secs = divmod(elapsed, 60)
hrs, mins = divmod(mins, 60)
self.current_progress.set('Play Progress: {:02d}:{:02d}:{:04.1f} / {}'.format(int(hrs), int(mins), secs, duration))
print('done')
self.proc = None
def play_song(self):
self.current_song_path = '/path/to/song.mp3'
if self.proc is None:
threading.Thread(target=self.play_to_end, daemon=True).start()
def stop_playing(self):
if self.proc:
self.proc.terminate()
def quit(self):
self.stop_playing()
self.destroy()
app = MediaPlayer()
app.mainloop()

Update Tkinter widget from main thread after worker thread completes

I need to update the GUI after a thread completes and call this update_ui function from main thread (like a software interrupt maybe?). How can a worker thread call a function in the main thread?
Sample code:
def thread():
...some long task
update_ui() #But call this in main thread somehow
def main():
start_new_thread(thread)
...other functionality
def update_ui():
Tkinter_widget.update()
I tried to use Queue or any flag accessible to both threads but I have to wait/poll continuously to check if the value has been updated and then call the function - this wait makes the UI unresponsive. e.g.
flag = True
def thread():
...some long task
flag = False
def main():
start_new_thread(thread)
while(flag): sleep(1)
update_ui()
...other functionality

Your code appears to be somewhat hypothetical. Here is some that accomplishes that does what you describe. It creates three labels and initializes their text. It then starts three threads. Each thread updates the tkinter variable associated with the label created in the main thread after a period of time. Now if the main thread really needs to do the updating, queuing does work, but the program must be modified to accomplish that.
import threading
import time
from tkinter import *
import queue
import sys
def createGUI(master, widget_var):
for i in range(3):
Label(master, textvariable=widget_var[i]).grid(row=i, column=0)
widget_var[i].set("Thread " + str(i) + " started")
def sometask(thread_id, delay, queue):
print("Delaying", delay)
time.sleep(delay)
tdict = {'id': thread_id, 'message': 'success'}
# You can put simple strings/ints, whatever in the queue instead
queue.put(tdict)
return
def updateGUI(master, q, widget_var, td):
if not q.empty():
tdict = q.get()
widget_var[tdict['id']].set("Thread " + str(tdict['id']) + " completed with status: " + tdict['message'])
td.append(1)
if len(td) == 3:
print("All threads completed")
master.after(1000, timedExit)
else:
master.after(100, lambda w=master,que=q,v=widget_var, tcount=td: updateGUI(w,que,v,td))
def timedExit():
sys.exit()
root = Tk()
message_q = queue.Queue()
widget_var = []
threads_done = []
for i in range(3):
v = StringVar()
widget_var.append(v)
t = threading.Thread(target=sometask, args=(i, 3 + i * 3, message_q))
t.start()
createGUI(root, widget_var)
updateGUI(root,message_q, widget_var, threads_done)
root.mainloop()

Python GUI stays frozen waiting for thread code to finish running

I have a python GUI program that needs to do a same task but with several threads. The problem is that I call the threads but they don't execute parallel but sequentially. First one executes, it ends and then second one, etc. I want them to start independently.
The main components are:
1. Menu (view)
2. ProcesStarter (controller)
3. Process (controller)
The Menu is where you click on the "Start" button which calls a function at ProcesStarter.
The ProcesStarter creates objects of Process and threads, and starts all threads in a for-loop.
Menu:
class VotingFrame(BaseFrame):
def create_widgets(self):
self.start_process = tk.Button(root, text="Start Process", command=lambda: self.start_process())
self.start_process.grid(row=3,column=0, sticky=tk.W)
def start_process(self):
procesor = XProcesStarter()
procesor_thread = Thread(target=procesor.start_process())
procesor_thread.start()
ProcesStarter:
class XProcesStarter:
def start_process(self):
print "starting new process..."
# thread count
thread_count = self.get_thread_count()
# initialize Process objects with data, and start threads
for i in range(thread_count):
vote_process = XProcess(self.get_proxy_list(), self.get_url())
t = Thread(target=vote_process.start_process())
t.start()
Process:
class XProcess():
def __init__(self, proxy_list, url, browser_show=False):
# init code
def start_process(self):
# code for process
When I press the GUI button for "Start Process" the gui is locked until both threads finish execution.
The idea is that threads should work in the background and work in parallel.

you call procesor.start_process() immediately when specifying it as the target of the Thread:
#use this
procesor_thread = Thread(target=procesor.start_process)
#not this
procesor_thread = Thread(target=procesor.start_process())
# this is called right away ^
If you call it right away it returns None which is a valid target for Thread (it just does nothing) which is why it happens sequentially, the threads are not doing anything.

One way to use a class as the target of a thread is to use the class as the target, and the arguments to the constructor as args.
from threading import Thread
from time import sleep
from random import randint
class XProcesStarter:
def __init__(self, thread_count):
print ("starting new process...")
self._i = 0
for i in range(thread_count):
t = Thread(
target=XProcess,
args=(self.get_proxy_list(), self.get_url())
)
t.start()
def get_proxy_list(self):
self._i += 1
return "Proxy list #%s" % self._i
def get_url(self):
self._i += 1
return "URL #%d" % self._i
class XProcess():
def __init__(self, proxy_list, url, browser_show=False):
r = 0.001 * randint( 1, 5000)
sleep(r)
print (proxy_list)
print (url)
def main():
t = Thread( target=XProcesStarter, args=(4, ) )
t.start()
if __name__ == '__main__':
main()
This code runs in python2 and python3.
The reason is that the target of a Thread object must be a callable (search for "callable" and "__call__" in python documentation for a complete explanation).
Edit The other way has been explained in other people's answers (see Tadhg McDonald-Jensen).

I think your issue is that in both places you're starting threads, you're actually calling the method you want to pass as the target to the thread. That runs its code in the main thread (and tries to start the new thread on the return value, if any, once its done).
Try:
procesor_thread = Thread(target=procesor.start_process) # no () after start_process
And:
t = Thread(target=vote_process.start_process) # no () here either

I want to handle closing xfce4-notifyd notification

import pynotify
import gobject
def on_clicked(notification, signal_text):
print "1: " + str(notification)
print "2: " + str(signal_text)
notification.close()
def on_closed(notification):
print "on_closed"
notification.close()
def show_notification(title, body):
n = pynotify.Notification(title, body)
n.add_action("button", "Test button", on_clicked)
n.connect("closed", on_closed)
n.show()
if __name__ == '__main__':
pynotify.init('TestApp')
global loop
loop = gobject.MainLoop()
# first case
notify = pynotify.Notification("1_notify", "test")
notify.add_action("button", "Test button", on_clicked)
notify.connect("closed", on_closed)
notify.show()
# second case
show_notification("2_notify", "test")
loop.run()
Sorry for my bad English. I want to handle closing xfce4-notifyd notification. In the first case, the function "on_closed()" works. Why in the second case it does not work?
This only works well in one namespace?

It does not work because the Notification object goes out of scope when show_notification() returns and is freed. You can make it work by e.g. returning the Notification object from the function and storing it in a variable in main body.