We Keep Coding

TKINTER - How to keep the main loop from waiting a validate command?

I have an username entry field that validates it's contents by checking if it exists within the domain, the only problem is that this call takes about 1 ~ 2 seconds to return, is there a way to use threading to keep the mainloop from waiting the validate command?
Example code:
import tkinter as tk
import subprocess
from typing import NoReturn
class GUI(tk.Tk):
def __init__(self) -> None:
super().__init__()
self.geometry('300x300')
self.title('test app')
self.__CreateWidgets()
def Start(self) -> None:
self.mainloop()
def __CreateWidgets(self) -> None:
self.__errorlabel = tk.Label(
self,
text = '',
foreground = 'red'
)
self.__errorlabel.grid(column = 0, row = 0)
self.__userIDentry = tk.Entry(
self,
validate = 'focusout',
validatecommand = (self.register(self.__UserIDValidator), '%P'), #This is what I wan't to change.
invalidcommand = lambda: self.__errorlabel.config(
text = 'Invalid user ID'
)
)
self.__userIDentry.grid(column = 0, row = 1)
self.__userIDentry.bind('<FocusIn>', lambda _: self.__errorlabel.config(
text = ''
)
)
tk.Entry( #This is just a blank entry to click on to test the previous entry validation.
self,
).grid(column = 0, row = 2)
def __UserIDValidator(self, string:str) -> bool:
#This function takes about 1 ~ 2 seconds to return:
flag = False
if string:
p = subprocess.Popen(
f'net user /domain {string}',
stdout = subprocess.PIPE,
stderr = subprocess.PIPE
)
error = p.stderr.read()
if not error:
flag = True
return flag
def main() -> NoReturn:
app = GUI()
app.Start()
if __name__ == '__main__':
main()
Running App
You should be able to copy and run the code above if you're on a domain, it basically sends a net user /domain command and checks if there was any error on the request, the program freezes until the validation is complete, so is there a way to run the validation and keep the mainloop from getting stuck?

I had a similar problem, I fixed it with threading:
import threading
def my_function():
something_somewhat_intensive()
def my_function_thr():
thr = threading.Thread(target=my_functioon)
thr.start()
Then you call my_function_thr() to begin that process on a thread. I don't know how this interacts with the import 'subprocess' at all, but this fixed a lot of problems that I had.
It should also be mentioned that if your process is resource intensive, and a bit more than like saving a file or something, use Multiprocessing. For a lot of processes, especially with just wanting the Tkinter window to not freeze, I do recommend threading.

Python Tkinter account for lost time

I've read similar questions that have been answered:
Getting realtime output from ffmpeg to be used in progress bar (PyQt4, stdout)
Progressbar to show amount of music played
How to measure elapsed time in Python?
Here's my window:
Problem is song ends when counter still has 20% to go. I know the reason is primarily due to system call to check if process is still running pgrep ffplay 10 times every second. Secondary reason is simply Python and Tkinter overhead.
To "band-aid fix" the problem I used 1.24 deciseconds instead of 1 every decisecond as my code illustrates now:
def play_to_end(self):
'''
Play single song, checking status every decisecond
Called from:
self.play_forever() to start a new song
self.pp_toggle() to restart song after pausing
'''
while True:
if not self.top2_is_active: return # Play window closed?
root.update() # Process other events
if self.pp_state is "Paused":
time.sleep(.1) # Wait until playing
continue
PID = os.popen("pgrep ffplay").read() # Get PID for ffplay
if len(PID) < 2: # Has song ended?
return # Song has ended
#self.current_song_time += .1 # Add decisecond
self.current_song_time += .124 # Add 1.24 deciseconds
# compensatation .24
self.current_progress.set(str('%.1f' % self.current_song_time) + \
" seconds of: " + str(self.DurationSecs))
root.update() # Process other events
root.after(100) # Sleep 1 decisecond
The problem with this band-aid fix is it is highly machine dependent. My machine is a Skylake for example. Also it is highly dependent on what other processes are running at the same time. When testing my machine load was relatively light:
How can I programmatically account for lost time in order to increment elapsed time accurately?
Perhaps there is a better way of simply querying ffplay to find out song progress?
As an aside (I know it's frowned upon to ask two questions at once) why can't I simply check if PID is null? I have tried .rstrip() and .strip() after .read() to no avail with checking PID equal to "" or None. If ffplay every has a process ID under 10 program will misbehave.

You can use subprocess.Popen() to execute ffplay and redirect stderr to PIPE, then you can read the progress from stderr and update the progress label.
Below is an example:
import tkinter as tk
import subprocess as subp
import threading
class MediaPlayer(tk.Tk):
def __init__(self):
super().__init__()
self.init_ui()
self.proc = None
self.protocol('WM_DELETE_WINDOW', self.quit)
def init_ui(self):
self.current_progress = tk.StringVar()
self.progress = tk.Label(self, textvariable=self.current_progress)
self.progress.grid(row=0, column=0, columnspan=2)
btn_close = tk.Button(self, text='Stop', width=20, command=self.stop_playing)
btn_close.grid(row=1, column=0, sticky='ew')
btn_play = tk.Button(self, text='Play', width=20, command=self.play_song)
btn_play.grid(row=1, column=1, sticky='ew')
def play_to_end(self):
self.proc = subp.Popen(
['ffplay', '-nodisp', '-hide_banner', '-autoexit', self.current_song_path],
stderr=subp.PIPE, bufsize=1, text=1
)
duration = ''
while self.proc.poll() is None:
msg = self.proc.stderr.readline().strip()
if msg:
if msg.startswith('Duration'):
duration = msg.split(',')[0].split(': ')[1]
else:
msg = msg.split()[0]
if '.' in msg:
elapsed = float(msg)
mins, secs = divmod(elapsed, 60)
hrs, mins = divmod(mins, 60)
self.current_progress.set('Play Progress: {:02d}:{:02d}:{:04.1f} / {}'.format(int(hrs), int(mins), secs, duration))
print('done')
self.proc = None
def play_song(self):
self.current_song_path = '/path/to/song.mp3'
if self.proc is None:
threading.Thread(target=self.play_to_end, daemon=True).start()
def stop_playing(self):
if self.proc:
self.proc.terminate()
def quit(self):
self.stop_playing()
self.destroy()
app = MediaPlayer()
app.mainloop()

TKinter GUI freezes until subprocess ends and realtime output to text Widget

I am trying to add some features to a GUI i created some time ago, in particular the function I need is a text widget where the terminal commands I send show their output.
The redirector class looks like this at the moment:
class StdRed(object):
def __init__(self, textwid):
self.text_space = textwid
def write(self, text):
self.text_space.config(state=NORMAL)
self.text_space.insert(END,text)
self.text_space.see(END)
self.text_space.update_idletasks()
self.text_space.config(state=DISABLED)
def flush(self):
pass
and indeed it works. I replaced the os.system(...) command to open terminal commands with
a = subprocess.Popen(command, stdout=PIPE, stderr=STDOUT, shell=True)
and I read stdout through:b = a.stdout.read() without a single problem (unfortunately i need that shell=True, otherwise some programs i need to call fail miserably).
After that I tried to have a realtime output on the tkinter text widget, so I changed b -->
while True:
b = a.stdout.readline().rstrip()
if not b:
break
print b
but it seems that the output appears only when the called process ends, i.e. a simple C software like
for(int i = 0; i<100000; i++){
cout << i << '\n';}
will print very slowly (I remark slowly given that a simple "ls" command will be printed line by line very slowly too) all the numbers at the end of the for cycle.
Other than that I noticed that the GUI is frozen while the programs called through subprocess are run. Any ideas on how to solve these problems?
EDIT:
I created a simple terminal which runs commands using the multiprocessing class and Popen:
from Tkinter import *
from multiprocessing import Process, Pipe, Queue
import sys
from subprocess import PIPE, Popen, STDOUT
root = Tk()
root.title("Test Terminal")
root.resizable(False, False)
class StdRed(object):
def __init__(self, textwid):
self.text_space = textwid
def write(self, text):
self.text_space.config(state=NORMAL)
self.text_space.insert(END,text)
self.text_space.see(END)
self.text_space.update_idletasks()
self.text_space.config(state=DISABLED)
def flush(self):
pass
terminal = Frame(root, bd=2, relief=GROOVE)
terminal.grid(row=0, sticky='NSEW')
TERM = Label(terminal, text='TERMINAL', font='Helvetica 16 bold')
TERM.grid(row=0, pady=10, sticky='NSEW')
termwid = Text(terminal, height=10)
termwid.grid(row=1, sticky='NSEW')
termwid.configure(state=DISABLED, font="Helvetica 12")
sys.stdout = StdRed(termwid)
enter = StringVar()
enter.set("")
termen = Entry(terminal, textvariable=enter)
queue = Queue(maxsize=1)
a = None
def termexe(execute):
a = Popen(execute, shell=True, stdout=PIPE, stderr=STDOUT)
while True:
line = a.stdout.readline().rstrip()
if not line:
break
else:
queue.put(line)
queue.put('')
def labterm(thi):
if queue.empty():
if thi != None:
if thi.is_alive():
root.after(0,lambda:labterm(thi))
else:
pass
else:
pass
else:
q = queue.get()
print q
root.after(0,lambda:labterm(thi))
def comter(event=None, exe=None, seq=None):
global enter
if seq == 1:
if exe != None:
th = Process(target=termexe, args=(exe,))
th.daemon = True
th.start()
labterm(th)
th.join()
else:
pass
else:
if exe != None:
th = Process(target=termexe, args=(exe,))
th.daemon = True
th.start()
labterm(th)
else:
th = Process(target=termexe, args=(enter.get(),))
th.daemon = True
th.start()
enter.set('')
labterm(th)
def resetterm():
global termwid
termwid.config(state=NORMAL)
termwid.delete(1.0, END)
termwid.config(state=DISABLED)
termen.bind('<Return>', comter)
resterm = Button(terminal, text="Clear", command=resetterm)
terbut = Button(terminal, text="Command", command=comter)
termen.grid(row=2, sticky='NSEW')
terbut.grid(row=3, sticky='NSEW')
resterm.grid(row=4, sticky='NSEW')
root.mainloop()
The problem is the acquisition is still not in real-time.
Running from the entry in the software the program:
#include <iostream>
using namespace std;
int main()
{
int i = 0;
while(1)
{
cout << i << '\n';
i++;
int a = 0;
while(a < 10E6)
{
a++;
}
}
}
leads to nothing inside the text widget for a while and, after some time, the output appears suddenly. Any ideas on how to solve this problem?

The solution here is to use threading, otherwise the script wait till the job is done to make the GUI responsive again. With threading, your program will be running both the job and the GUI at the same time, an example of code:
import threading
def function():
pass
t = threading.Thread(target=function)
t.daemon = True # close pipe if GUI process exits
t.start()
I used this std redirector:
class StdRedirector():
"""Class that redirects the stdout and stderr to the GUI console"""
def __init__(self, text_widget):
self.text_space = text_widget
def write(self, string):
"""Updates the console widget with the stdout and stderr output"""
self.text_space.config(state=NORMAL)
self.text_space.insert("end", string)
self.text_space.see("end")
self.text_space.config(state=DISABLED)

I tried using threads as suggested by #Pau B (switched to multiprocessing in the end), and I solved indeed the problem of the stuck GUI. The issue now is that running the program
for(int i = 0; i<100000; i++){ cout << i << '\n';}
doesn't return a realtime output, but it seems it is buffered and then appears after some time into the text widget. The code I'm using looks like this:
class StdRed(object):
def __init__(self, textwid):
self.text_space = textwid
def write(self, text):
self.text_space.config(state=NORMAL)
self.text_space.insert(END,text)
self.text_space.see(END)
self.text_space.update_idletasks()
self.text_space.config(state=DISABLED)
def termexe(execute):
a = Popen(execute, shell=True, stdout=PIPE, stderr=STDOUT)
while True:
line = a.stdout.readline().rstrip()
if not line:
break
else:
queue.put(line)
queue.put('')
def labterm():
global th
if queue.empty():
if th != None:
if th.is_alive():
root.after(0,labterm)
else:
pass
else:
pass
else:
q = queue.get()
print q
root.after(1,labterm)
def comter(event=None):
global enter
global th
if th != None:
if not th.is_alive():
th = Process(target=termexe, args=(enter.get(),))
th.start()
else:
pass
else:
th = Process(target=termexe, args=(enter.get(),))
th.start()
enter.set('')
labterm()
where comter() is called by a button or binded to 'Return' inside a text entry.

Maybe this could help others, I solved the issue replacing '\n' with endl. It seems cout in the while loop is buffered and the stdout flush is called only after a while, while with endl the function is called after every cycle

Python - run two commands at the same time

I am new to Python and am having trouble with this piece of code:
while true:
rand = random.choice(number)
print(rand)
enter_word = input("Write something: ")
time.sleep(5)
I want to be able to input words in the console while, at the same time, have random numbers appear in the console. But a new number only appears once I input a word. What is the best way to make both these commands run at the same time?
Do I need to make a thread or is there something simpler I can do?
And if I need to make a thread can you please give a little help on how I would create it?
Thanks in advance

This can be achieved by using the multiprocessing module in python, please find the code below
#!/usr/bin/python
from multiprocessing import Process,Queue
import random
import time
def printrand():
#Checks whether Queue is empty and runs
while q.empty():
rand = random.choice(range(1,100))
time.sleep(1)
print rand
if __name__ == "__main__":
#Queue is a data structure used to communicate between process
q = Queue()
#creating the process
p = Process(target=printrand)
#starting the process
p.start()
while True:
ip = raw_input("Write something: ")
#if user enters stop the while loop breaks
if ip=="stop":
#Populating the queue so that printramd can read and quit the loop
q.put(ip)
break
#Block the calling thread until the process whose join()
#method is called terminates or until the optional timeout occurs.
p.join()

To wait for input and to display some random output at the same time, you could use a GUI (something with an event loop):
#!/usr/bin/env python3
import random
from tkinter import E, END, N, S, scrolledtext, Tk, ttk, W
class App:
password = "123456" # the most common password
def __init__(self, master):
self.master = master
self.master.title('To stop, type: ' + self.password)
# content frame (padding, etc)
frame = ttk.Frame(master, padding="3 3 3 3")
frame.grid(column=0, row=0, sticky=(N, W, E, S))
# an area where random messages to appear
self.textarea = scrolledtext.ScrolledText(frame)
# an area where the password to be typed
textfield = ttk.Entry(frame)
# put one on top of the other
self.textarea.grid(row=0)
textfield.grid(row=1, sticky=(E, W))
textfield.bind('<KeyRelease>', self.check_password)
textfield.focus() # put cursor into the entry
self.update_textarea()
def update_textarea(self):
# insert random Unicode codepoint in U+0000-U+FFFF range
character = chr(random.choice(range(0xffff)))
self.textarea.configure(state='normal') # enable insert
self.textarea.insert(END, character)
self.textarea.configure(state='disabled') # disable editing
self.master.after(10, self.update_textarea) # in 10 milliseconds
def check_password(self, event):
if self.password in event.widget.get():
self.master.destroy() # exit GUI
App(Tk()).master.mainloop()

I want to be able to input words in the console while, at the same time, have random numbers appear in the console.
#!/usr/bin/env python
import random
def print_random(n=10):
print(random.randrange(n)) # print random number in the range(0, n)
stop = call_repeatedly(1, print_random) # print random number every second
while True:
word = raw_input("Write something: ") # ask for input until "quit"
if word == "quit":
stop() # stop printing random numbers
break # quit
where call_repeatedly() is define here.
call_repeatedly() uses a separate thread to call print_random() function repeatedly.

you have to run two concurrent threads at the same time in order to get rid of such blocking. looks like there are two interpreters that run your code and each of them executes particular section of your project.

Background Process Locking up GUI Python

I have a background Process (using Process from multiprocessing) that is pushing objects to my GUI, however this background process keeps locking up the GUI and the changes being pushed are never being displayed. The objects are being put in to my queue, however the update method in my GUI isn't being called regularly. What can I do make the GUI update more regularly? My GUI is written in Tkinter.
My background process has a infinite loop within it because I always need to keep reading the USB port for more data, so basically my code looks like this:
TracerAccess.py
import usb
from types import *
import sys
from multiprocessing import Process, Queue
import time
__idVendor__ = 0xFFFF
__idProduct__ = 0xFFFF
END_POINT = 0x82
def __printHEXList__(list):
print ' '.join('%02x' % b for b in list)
def checkDeviceConnected():
dev = usb.core.find(idVendor=__idVendor__, idProduct=__idProduct__)
if dev is None:
return False
else:
return True
class LowLevelAccess():
def __init__(self):
self.rawIn = []
self.tracer = usb.core.find(idVendor=__idVendor__, idProduct=__idProduct__)
if self.tracer is None:
raise ValueError("Device not connected")
self.tracer.set_configuration()
def readUSB(self):
"""
This method reads the USB data from the simtracer.
"""
try:
tmp = self.tracer.read(END_POINT, 10000,None, 100000).tolist()
while(self.checkForEmptyData(tmp)):
tmp = self.tracer.read(END_POINT, 10000,None, 100000).tolist()
self.rawIn = tmp
except:
time.sleep(1)
self.readUSB()
def checkForEmptyData(self, raw):
if(len(raw) == 10 or raw[10] is 0x60 or len(raw) == 11):
return True
else:
return False
class DataAbstraction:
def __init__(self, queue):
self.queue = queue
self.lowLevel = LowLevelAccess()
def readInput(self):
while True:
self.lowLevel.readUSB()
raw = self.lowLevel.rawIn
self.queue.put(raw)
ui.py
from Tkinter import *
import time
import TracerAccess as io
from multiprocessing import Process, Queue
from Queue import Empty
from math import ceil
def findNumberOfLines(message):
lines = message.split("\n")
return len(lines)
class Application(Frame):
def addTextToRaw(self, text, changeColour=False, numberOfLines=0):
self.rawText.config(state=NORMAL)
if changeColour is True:
self.rawText.insert(END,text, 'oddLine')
else:
self.rawText.insert(END,text)
self.rawText.config(state=DISABLED)
def updateRaw(self, text):
if(self.numberOfData() % 2 is not 0):
self.addTextToRaw(text, True)
else:
self.addTextToRaw(text)
def startTrace(self):
self.dataAbstraction = io.DataAbstraction(self.queue)
self.splitProc = Process(target=self.dataAbstraction.readInput())
self.stopButton.config(state="normal")
self.startButton.config(state="disabled")
self.splitProc.start()
def pollQueue(self):
try:
data = self.queue.get(0)
self.dataReturned.append(data)
self.updateRaw(str(data).upper())
self.rawText.tag_config("oddLine", background="#F3F6FA")
except Empty:
pass
finally:
try:
if(self.splitProc.is_alive() is False):
self.stopButton.config(state="disabled")
self.startButton.config(state="normal")
except AttributeError:
pass
self.master.after(10, self.pollQueue)
def stopTrace(self):
self.splitProc.join()
self.stopButton.config(state="disabled")
self.startButton.config(state="normal")
def createWidgets(self):
self.startButton = Button(self)
self.startButton["text"] = "Start"
self.startButton["command"] = self.startTrace
self.startButton.grid(row = 0, column=0)
self.stopButton = Button(self)
self.stopButton["text"] = "Stop"
self.stopButton["command"] = self.stopTrace
self.stopButton.config(state="disabled")
self.stopButton.grid(row = 0, column=1)
self.rawText = Text(self, state=DISABLED, width=82)
self.rawText.grid(row=1, columnspan=4)
def __init__(self, master):
Frame.__init__(self, master)
self.queue = Queue()
self.master.after(10, self.pollQueue)
self.pack()
self.dataReturned = []
self.createWidgets()
def numberOfData(self):
return len(self.dataReturned)
Main.py
import ui as ui
if __name__ == "__main__":
root = Tk()
root.columnconfigure(0,weight=1)
app = ui.Application(root)
app.mainloop()
So the background thread never finishes, however when I end the process the UI starts to be displayed before closing. The problem could have appeared because of my design for the TracerAccess.py module as I developed this after moving straight form java and little to no design experience for python.

What multiprocess.Process does, internally, is really a fork(), which effectively duplicated your process. You can perhaps visualize it as:
/ ["background" process] -------------\
[main process] --+ +-- [main process]
\ [main process continued] -----------/
p.join() attempts to "join" the two processes back to one. This effectively means: waiting until the background process is finished. Here's the actual (full) code from the .join() function:
def join(self, timeout=None):
'''
Wait until child process terminates
'''
assert self._parent_pid == os.getpid(), 'can only join a child process'
assert self._popen is not None, 'can only join a started process'
res = self._popen.wait(timeout)
if res is not None:
_current_process._children.discard(self)
Note how self._popen.wait is called.
This is obviously not what you want.
What you probably want, in the context of TKinter, is use the tk event loop, for example like this (Python 3, but the concept also works on Python 2)
from multiprocessing import Process, Queue
import time, tkinter, queue, random, sys
class Test:
def __init__(self, root):
self.root = root
txt = tkinter.Text(root)
txt.pack()
self.q = Queue()
p = Process(target=self.bg)
p.start()
self.checkqueue()
print('__init__ done', end=' ')
def bg(self):
print('Starting bg loop', end=' ')
n = 42
while True:
# Burn some CPU cycles
(int(random.random() * 199999)) ** (int(random.random() * 1999999))
n += 1
self.q.put(n)
print('bg task finished', end=' ')
def checkqueue(self):
try:
print(self.q.get_nowait(), end=' ')
except queue.Empty:
print('Queue empty', end=' ')
sys.stdout.flush()
# Run myself again after 1 second
self.root.after(1000, self.checkqueue)
root = tkinter.Tk()
Test(root)
root.mainloop()
You don't call the .join(), and instead use the .after() method, which schedules a function to run after n microseconds (if you've ever used Javascript, then think setTimeout()) to read the queue.
Depending on the actual content of your bg() function, you may not event need multiprocesing at all, just scheduling a function with .after() may be enough.
Also see:
http://tkinter.unpythonic.net/wiki/UsingTheEventLoop