I'm using python 2.7 and have built a UI using Tkinter. I'm using threads and queues to keep the UI responsive while the main script is working. The basic summary is the script reads a text file, parses out some information on it and puts that info in a dictionary and a list in the dictionary, then uses that info to send TCP modbus request (using pyModbus). It then writes the responses/results to a text file. The results also get printed a Text widget included in the UI. The updates to the Text widget is handled by the mainloop.
I'm still fairly new to threads and queues and I'm having trouble figuring out this issue.
The problem I'm running into is I need to include a ~10ms sleep after it loops through each item in the list for the UI to remain responsive. If I include the sleep time it works as expected, if not it freezes up until the threaded process is finished then updates the UI all at once (as it would if threads weren't used). The 10ms sleep can be slightly shorter. Any amount longer also works.
Here's the code that handles updating the log:
textQueue = Queue.Queue()
def TextDisplay(message, disTime="false", myColor="black", bgColor="white"):
textQueue.put([message, disTime, myColor, bgColor])
class LogUI:
def __init__(self, master):
self.master = master
'''other ui elements, not relevent'''
self.mainLogFrame = Frame(self.master)
self.mainLogFrame.pack(side="top", fill="both", expand="yes", padx=5, pady=2)
self.logText = Text(self.mainLogFrame, height=2)
self.logText.pack(side="left", fill="both", expand="yes", padx=5, pady=2)
self.ThreadSafeTextDisplay()
def ThreadSafeTextDisplay(self):
while not textQueue.empty():
tempText = textQueue.get(0)
message = tempText[0]
disTime = tempText[1]
myColor = tempText[2]
bgColor = tempText[3]
message = str(message) + "\n"
'''bunch of formating stuff'''
logUI.logText.insert(END, message)
print message
#NOTE: tried to include a sleep time here, no effect
self.logText.after(10, self.ThreadSafeTextDisplay)
Here's the non-threaded function that's called when the user clicks a button.
def ParseInputFile():
'''non-threaded function, called when user clicks button'''
inputList = []
inputFile = mainUI.fullInFileEntry.get()
with open(inputFile, 'r') as myInput:
'''open file and put contents in list'''
for line in myInput:
inputList.append(line.strip())
outFile = mainUI.outFileEntry.get().strip() + '.txt'
i = 1
tableBol = False
inputDict = {}
inputKeys = []
tableID = None
for item in inputList:
'''parses out inputKeys, inputDict using regular expressions'''
runInputGetQueue.put([inputKeys, inputDict, outFile, inputFile])
Here's the threaded function that receives the parsed information and handles the modbus request (note: i tried commenting out the actual modbus request, no effect):
def RunInputThread():
time.sleep(.1)
while 1:
while not runInputGetQueue.empty():
tempGet = runInputGetQueue.get(0)
inputKeys = tempGet[0]
inputDict = tempGet[1]
outFile = tempGet[2]
inputFile = tempGet[3]
outFile = open(outFile, 'w')
TextDisplay('< Start of %s input file > ' % inputFile, True, 'blue')
for key in inputKeys:
'''loops through the keys in the dictionary'''
TextDisplay(key) #just used as an example.
for lineIndex in range(len(inputDict[key]['tableLines'])):
'''lots of code that loops thorugh the lines of input file, frequently calls the TextDisplay() function'''
TextDisplay(inputDict[key][lineIndex]) #just used as an example.
time.sleep(0.01) #UI will become unresponseive if not included.
outFile.close()
time.sleep(0.001)
Found a way to get the UI mostly responsive. As stated in the comments above, the queue was receiving stuff to fast the function would be constantly working causing the UI to lock up. I made it so it will print at most 5 messages before taking a 1ms break and recalling the function which allows the UI to 'catch up.' The messages are printed almost as fast as they come in.
The UI will be slightly non-responsive if you move it or resize it. I have no issues interacting with other UI elements while this is running though.
You could also change the while loop to a if statement if you don't mind it being slow. The one process i ran went from 14 seconds with an if statement down to around 5 or 6 seconds using the code below. It would be same as if you changed the pullCount break point to 1 instead of 5.
def ThreadSafeTextDisplay(self):
pullCount = 0
while not textQueue.empty():
pullCount += 1
tempText = textQueue.get(0)
message = tempText[0]
disTime = tempText[1]
myColor = tempText[2]
bgColor = tempText[3]
message = str(message) + "\n"
'''bunch of formatting stuff'''
logUI.logText.insert(END, message)
print message
if pullCount >= 5: break #you can change the 5 value to whatever you want, the higher the number the faster stuff will print but the UI will start to become unresponsive.
self.logText.after(1, self.ThreadSafeTextDisplay)
Related
I have a python program which has a function takeScreenshot that takes a screenshot of 10 webpages that are inputted. I want to use threading to make the web scraping part of taking the screenshot be executed in the background while the program goes on inputting more webpages. After taking 10 screenshots, they should be displayed in the program.
The question is how to make the program display them after the last takeScreenshot thread (the tenth thread) is done so as not to cause an error? In other words, how to make sure that all the threads are finished? I tried to make a list of all the threads that started and make them .join() after inputting the last webpage (in the last loop). However, this makes the program freeze after inputting the last webpage.
threads=[]
n=0
while n<10:
webpage = input("Enter the webpage")
thread = threading.Thread(target = takeScreenshot, args = webpage)
thread.start()
threads.append(thread)
if n==9:
for thread in threads:
thread.join()
n++
I tried to investigate more, so I discovered that the program freezes in the last loop when it sets an attribute of a class equal to the screenshot: self.graph = PhotoImage(file='screenshot.png'). Note that the screenshot of the last webpage is normally downloaded, so the error isn't due to the absence of the screenshot. The previous line of code is included in the takeScreenshot function.
Here's the takeScreenshot method (it's a part of a class called scraping:
ublockPath = r'C:\Users\Bassel Attia\Documents\Trading Core\1.37.2_0'
chromeOptions = Options()
chromeOptions.add_argument("--log-level=3")
chromeOptions.add_argument('load-extension=' + ublockPath)
self.driver = webdriver.Chrome(ChromeDriverManager().install(),
chrome_options=chromeOptions)
self.driver.get(webpage)
self.driver.get_screenshot_as_file('screenshot.png')
self.image = Image.open('screenshot.png')
#crop screenshot
area = (20, 290, 1250, 800)
croppedImage=self.image.crop(area)
os.remove('currentStock.png')
croppedImage.save('screenshot.png')
self.image = Image.open('screenshot.png')
#resizeImage
newHeight = 300
newWidth = int(newHeight / self.image.height * self.image.width)
resizedImage = self.image.resize((newWidth, newHeight))
os.remove('currentStock.png')
resizedImage.save('screenshot.png')
self.image = Image.open('screenshot.png')
self.image.close()
self.driver.quit()
By inspecting the code you provided unfortunately i wasn't able to reproduce the issue, but there are some things that could be problematic:
the file currentStock.png is deleted twice (which i'm surprised it doesn't raise an exception the second time you try to delete it)
you keep overwriting the same 'screenshot.png' file
args is not a list
If this could help, here it is a minimum working example:
import os, io, threading, uuid
from PIL import Image
from selenium import webdriver
def screen(wid, webpage):
opts = webdriver.FirefoxOptions()
opts.add_argument('--headless')
print(wid, 'starting webdriver')
driver = webdriver.Firefox(options=opts)
driver.get(webpage)
print(wid, 'taking screenshot')
image_data = driver.get_screenshot_as_png()
image = Image.open(io.BytesIO(image_data))
area = (20, 290, 1250, 800)
cropped = image.crop(area)
h = 300
w = int(h / cropped.height * cropped.width)
resized = cropped.resize((w, h))
if not os.path.isdir('screen'):
os.mkdir('screen')
fname = os.path.join('screen', f'{uuid.uuid4()}.png')
resized.save(fname)
print(wid, f'screenshot saved to {fname}')
driver.quit()
def main():
threads = []
for wid in range(4):
webpage = input('Webpage: ')
thread = threading.Thread(target = screen, args = [wid, webpage])
thread.start()
threads.append(thread)
for i, thread in enumerate(threads):
thread.join()
print('joined thread : ', i)
if __name__ == '__main__':
main()
so I am making a program on tkinter that gets a response from a server and depending on the answer, it will change the background color, to either green for success or red for error, the problem is that I realized that when running the code, the windows.after() method doesn't wait till is done to continue and when I do the request for the server, it have to do it three times to check if the response is correct, and it is suppossed to change the window background color each time, but it is only doing it one time. And not only the background color changing fails, also I want to change a label's text when it is doing the request,but it does it really quick and I'm not able to diferentiate the changes, so the question is: how can I
How can I make the program wait until one line finishes running to go to the next one and not everything happens at the same time and so fast?
Here is a piece of my code, I removed the request part because I'm trying to solve this problem first:
# import gpiozero
# import picamera
import json
import requests
import tkinter as tk
with open("config.json") as file:
config = json.load(file)
ENDPOINT = config["ENDPOINT"]
USUARIO = config["USUARIO"]
ESTACION = config["ESTACION"]
TIEMPO_ESPERA = config["TIEMPO_ESPERA"]
PIN_RELE = config["PIN_RELE"]
PATH_SALIDA = ENDPOINT + "Salida.getTicket/" + ESTACION + "/" + USUARIO + "/"
barcode = ""
# RELAY = gpiozero.OutputDevice(PIN_RELE, active_high=True, initial_value=False)
# CAMERA = picamera.PiCamera()
def check_scan_barcode(event=None):
info_label.config(text = "Wait...")
barcode = barcode_entry.get()
barcode_entry.delete(0, "end")
for i in range(3):
response = get_request(ENDPOINT + barcode)
if response["data"] == "True":
success()
open_barrier()
else:
error()
info_label.config(text = "Scan barcode")
def get_request(url):
response = requests.get(url)
response.raise_for_status()
response = response.json()
return response
def normal():
window.configure(bg="white")
info_label.configure(bg="white")
def success():
window.configure(bg="green")
info_label.configure(bg="green")
window.after(1000, normal)
def error():
window.configure(bg="red")
info_label.configure(bg="red")
window.after(1000, normal)
def open_barrier(barcode):
# CAMERA.capture(f"/home/pi/Pictures{barcode}.jpg")
# RELAY.on()
# window.after(TIEMPO_ESPERA, RELAY.off)
pass
window = tk.Tk()
# window.attributes('-fullscreen', True)
info_label = tk.Label(window, text= "Scan barcode.", font=("Arial", 40))
info_label.pack()
barcode_entry = tk.Entry(window, width=50)
barcode_entry.bind('<Return>', check_scan_barcode)
barcode_entry.pack(expand=True)
barcode_entry.focus()
window.mainloop()
my project is to build a scanner that send the sensor value on Serial port. I want to control the scanner using a GUI Tkinter, so I want to check if something is coming on the serial continuously. However, I used the .after() function that works when something is sent but when the scanner is "waiting" and nothing is sent, the GUI freezes and I can't to do anything until something is sent.
Thanks,
Here's the code triggered by the main button:
def Acqard():
global flag
flag = 1
arduinoData.write(b'1')
log.insert(END, "Début du scan\n")
root.after(10, write_data)
And here are the functions that save the DATA in a txt file:
def write_data():
global file_stream
file_stream = open("data.txt", "a") # mode write ou append ?
write_lines()
def write_lines():
global after_id
data = arduinoData.read()
try:
data2 = data.decode("utf-8")
file_stream.write(data2)
print(data2)
if data2 == "F":
root.after_cancel(after_id)
print("Ca a marché")
stopacq()
after_id = root.after(10, write_data)
except:
return
And here's the function that stops the scanner:
def stopacq():
global flag, file_stream
flag = 0
root.after_cancel(after_id)# Annulation des rappels automatiques de write_lines
file_stream.close()
file_stream = None
arduinoData.write(b'3')
log.insert(END, "Scan interrompu\n")
I want to print a message on the UI when the user wishes to exit.
This is a part of code relevant to it
if word in quit_words:
user_to_quit = True
Print = "Pleasure serving you!"
print_bot_reply(Print)
time.sleep(5)
root.destroy()
The print bot_reply function is as follows:
def print_bot_reply(response):
response = "\nKalpana: " + str(response)
label = Label(frame,text = response,bg="#b6efb1", borderwidth=5, relief="raised")
label.pack(anchor = "w")
The window is closing after 5 seconds, as desired but not displaying the message.
Please point me the mistake or suggest some other method to do this
Thanks!
I am adapting the Python script in this project (expanded below) to a point where it updates a JSON file's elements, instead of the InitialState streamer. However, with the multiple threads that are opened by the script, it is impossible to succinctly write the data from each thread back to the file as it would be read, changed, and written back to the file in all threads at the same time. As there can only be one file, no version will ever be accurate as the last thread would override all others.
Question: How can I update the states in the JSON based in each thread (simultaneously) without it affecting the other thread's writing operation or locking up the file?
JSON file contains the occupant's status that I would like to manipulate with the python script:
{
"janeHome": "false",
"johnHome": "false",
"jennyHome": "false",
"jamesHome": "false"
}
This is the python script:
import subprocess
import json
from time import sleep
from threading import Thread
# Edit these for how many people/devices you want to track
occupant = ["Jane","John","Jenny","James"]
# MAC addresses for our phones
address = ["11:22:33:44:55:66","77:88:99:00:11:22","33:44:55:66:77:88","99:00:11:22:33:44"]
# Sleep once right when this script is called to give the Pi enough time
# to connect to the network
sleep(60)
# Some arrays to help minimize streaming and account for devices
# disappearing from the network when asleep
firstRun = [1] * len(occupant)
presentSent = [0] * len(occupant)
notPresentSent = [0] * len(occupant)
counter = [0] * len(occupant)
# Function that checks for device presence
def whosHere(i):
# 30 second pause to allow main thread to finish arp-scan and populate output
sleep(30)
# Loop through checking for devices and counting if they're not present
while True:
# Exits thread if Keyboard Interrupt occurs
if stop == True:
print ("Exiting Thread")
exit()
else:
pass
# If a listed device address is present print
if address[i] in output:
print(occupant[i] + "'s device is connected")
if presentSent[i] == 0:
# TODO: UPDATE THIS OCCUPANT'S STATUS TO TRUE
# Reset counters so another stream isn't sent if the device
# is still present
firstRun[i] = 0
presentSent[i] = 1
notPresentSent[i] = 0
counter[i] = 0
sleep(900)
else:
# If a stream's already been sent, just wait for 15 minutes
counter[i] = 0
sleep(900)
# If a listed device address is not present, print and stream
else:
print(occupant[i] + "'s device is not connected")
# Only consider a device offline if it's counter has reached 30
# This is the same as 15 minutes passing
if counter[i] == 30 or firstRun[i] == 1:
firstRun[i] = 0
if notPresentSent[i] == 0:
# TODO: UPDATE THIS OCCUPANT'S STATUS TO FALSE
# Reset counters so another stream isn't sent if the device
# is still present
notPresentSent[i] = 1
presentSent[i] = 0
counter[i] = 0
else:
# If a stream's already been sent, wait 30 seconds
counter[i] = 0
sleep(30)
# Count how many 30 second intervals have happened since the device
# disappeared from the network
else:
counter[i] = counter[i] + 1
print(occupant[i] + "'s counter at " + str(counter[i]))
sleep(30)
# Main thread
try:
# Initialize a variable to trigger threads to exit when True
global stop
stop = False
# Start the thread(s)
# It will start as many threads as there are values in the occupant array
for i in range(len(occupant)):
t = Thread(target=whosHere, args=(i,))
t.start()
while True:
# Make output global so the threads can see it
global output
# Reads existing JSON file into buffer
with open("data.json", "r") as jsonFile:
data = json.load(jsonFile)
jsonFile.close()
# Assign list of devices on the network to "output"
output = subprocess.check_output("arp-scan -interface en1 --localnet -l", shell=True)
temp = data["janeHome"]
data["janeHome"] = # RETURNED STATE
data["johnHome"] = # RETURNED STATE
data["jennyHome"] = # RETURNED STATE
data["jamesHome"] = # RETURNED STATE
with open("data.json", "w") as jsonFile:
json.dump(data, jsonFile)
jsonFile.close()
# Wait 30 seconds between scans
sleep(30)
except KeyboardInterrupt:
# On a keyboard interrupt signal threads to exit
stop = True
exit()
I think we can all agree that the best idea would be to return the data from each thread to the main and write it to the file in one location but here is where it gets confusing, with each thread checking for a different person, how can the state be passed back to main for writing?