We Keep Coding

Threading reading a serial port in Python (with a GUI)

I want to trigger an event whenever there is data to be read from a serial port while running a GUI. The pySerial module apparently has experimental functionality for that, but it isn't particularly well documented (I couldn't find any useful examples in the API).
This question appears to deal with the same or at least very similar task, but doesn't provide instructions to replicate it or working code examples.
I came up with this code:
import tkinter as tk
import serial
import threading
# Create GUI window
window = tk.Tk()
# Initialize the port
myPort = serial.Serial('/dev/ttyUSB0')
# Function to call whenever there is data to be read
def readFunc(port):
port.readline()
print('Line read')
# Configure threading
t1 = threading.Thread(target = readFunc, args=[myPort])
t1.start()
# Main loop of the window
window.mainloop()
Running it does indeed trigger the event, but only once. Why is that? Is there a "recommended" way to do this as by using the functionality of pySerial itself?
Alternatively, I would also run the function to read and process data on an event like you can with GUI elements. If that is the better solution, how would that be done?
Related question (unanswered), probably makes this question a duplicate
Edit: Here is a minimal example derived from the answer below that changes the text of a label whenever data is read to the incoming data:
import tkinter as tk
from serial import Serial
from serial.threaded import ReaderThread, Protocol
app = tk.Tk()
label = tk.Label(text="A Label")
label.pack()
class SerialReaderProtocolRaw(Protocol):
port = None
def connection_made(self, transport):
"""Called when reader thread is started"""
print("Connected, ready to receive data...")
def data_received(self, data):
"""Called with snippets received from the serial port"""
updateLabelData(data)
def updateLabelData(data):
data = data.decode("utf-8")
label['text']=data
app.update_idletasks()
# Initiate serial port
serial_port = Serial("/dev/ttyACM0")
# Initiate ReaderThread
reader = ReaderThread(serial_port, SerialReaderProtocolRaw)
# Start reader
reader.start()
app.mainloop()

Your main concern is to be thread safe, when You are updating GUI from another running Thread.
To achieve this, we can use .after() method, which executes callback for any given tk widget.
Another part of Your request is to use Threaded serial reader.
This can be achieved by using ReaderThread accompanied with Protocol.
You can pick two protocols:
raw data reader protocol, which reads data as they come
line reader protocol, which enables us to read lines of data
Here is working code example, with two protocols mentioned above, so You can pick which one suits You. Just remember, that all data coming from serial port are just raw bytes.
import tkinter as tk
from serial import Serial
from serial.threaded import ReaderThread, Protocol, LineReader
class SerialReaderProtocolRaw(Protocol):
tk_listener = None
def connection_made(self, transport):
"""Called when reader thread is started"""
if self.tk_listener is None:
raise Exception("tk_listener must be set before connecting to the socket!")
print("Connected, ready to receive data...")
def data_received(self, data):
"""Called with snippets received from the serial port"""
self.tk_listener.after(0, self.tk_listener.on_data, data.decode())
class SerialReaderProtocolLine(LineReader):
tk_listener = None
TERMINATOR = b'\n\r'
def connection_made(self, transport):
"""Called when reader thread is started"""
if self.tk_listener is None:
raise Exception("tk_listener must be set before connecting to the socket!")
super().connection_made(transport)
print("Connected, ready to receive data...")
def handle_line(self, line):
"""New line waiting to be processed"""
# Execute our callback in tk
self.tk_listener.after(0, self.tk_listener.on_data, line)
class MainFrame(tk.Frame):
def __init__(self, *args, **kwargs):
super().__init__(*args, **kwargs)
self.listbox = tk.Listbox(self)
self.listbox.pack()
self.pack()
def on_data(self, data):
print("Called from tk Thread:", data)
self.listbox.insert(tk.END, data)
if __name__ == '__main__':
app = tk.Tk()
main_frame = MainFrame()
# Set listener to our reader
SerialReaderProtocolLine.tk_listener = main_frame
# Initiate serial port
serial_port = Serial("/dev/ttyUSB0")
# Initiate ReaderThread
reader = ReaderThread(serial_port, SerialReaderProtocolLine)
# Start reader
reader.start()
app.mainloop()

Python: Function from different class not running with thread

Im trying to implement a gui for a script I have written.
The script itself runs perfectly fine but as soon as I try to run it using a thread, it will just freeze the whole process without any feedback.
The thread seems to not execute the function.
import Foo
from multiprocessing.pool import ThreadPool
class GUI():
def __init__(self, master):
self.file = "path/file"
self.key = "path/key"
# init GUI
...
def updateDev(self):
fib = Foo.foo()
pool = ThreadPool(processes=1)
async_res = pool.apply_async(fib.update, args=(self.file, self.key))
async_res.wait()
# the code freezes here
res = async_res.get()
...
Is there anything im missing?
fib.update(self.file, self.key) runs like this without any threads perfectly.
EDIT:
I solved it myself yesterday by adding a function that starts the thread.
import Foo
from threading import Thread
class GUI():
def __init__(self, master):
self.file = "path/file"
self.key = "path/key"
# init GUI
...
def startDev(self):
t = Thread(target = self.updateDev)
t.start()
def updateDev(self):
fib = Foo.foo()
fib.update(self.file, self.key)
...
Thanks for the help everyone!

Sorry but I don't understand what is the pourpose of the code. If I correctly understand, the code will never reach the async_res.wait() satement because it is recoursive. This row async_res = pool.apply_async(fib.update, args=(self.file, self.key)) will probably run again the "update" function

Using TCP connection to execute parallel threads over different ports

I am trying to execute this python script for implementing a distributed computing protocol. Currently this executes the functions sequentially one after the other. i want to be able to run all the processes parallel on different ports instead of the ** multiprocessing.Manager().Queue()** as has been mentioned in the statement below but i have no clue how should i go about. Any head start would be appreciated to lead me in the right direction
import multiprocessing
from threading import Thread
class Process(Thread):
def __init__(self, env, id):
super(Process, self).__init__()
self.inbox = multiprocessing.Manager().Queue()
self.env = env
self.id = id
def run(self):
try:
self.body()
self.env.removeProc(self.id)
except EOFError:
print "Exiting.."
def getNextMessage(self):
return self.inbox.get()
def sendMessage(self, dst, msg):
self.env.sendMessage(dst, msg)
def deliver(self, msg):
self.inbox.put(msg)

i was able to run this code in parallel mode by implementing simple socket programming instead of Queues by following the python documentation and then making the communication of messages possible over those sockets.

cannot pass data between thread and Qt object in Python

I have created a GUI, on which i have to pass string data coming from serial COM port. One individual thread is handling the Serial data whereas Qt object needs to take that data and display it via 'setPlainText' method. It gives an error (on line i've marked in comments) is
"QObject: Cannot create children for a parent that is in a different thread.
(Parent is QTextDocument(0x3ebaa68), parent's thread is QThread(0x3dd5c58), current thread is QThread(0x3fbd6a8)"
Heres my code;
import sys
from PyQt4 import QtGui
from My_GUI_code import Ui_Dialog
import serial # import Serial Library
import threading
import time
test=""
arduinoData = serial.Serial('COM2', 9600) #
index=0
incoming_data=""
device_0_V=""
class Serial_read(threading.Thread):
"""
Thread to read data coming from Arduino
"""
def __init__(self):
threading.Thread.__init__(self)
def run(self):
global incoming_data
global device_0_V
global test
while 1:
while (arduinoData.inWaiting()==0): #Wait here until there is data
pass #do nothing
incoming_data = arduinoData.readline() #read the line of text from the serial port
if "V0" in incoming_data:
index = incoming_data.index("V0=")
device_0_V=incoming_data[index+3:index+6]
print device_0_V
#print incoming_data,
class Editor(QtGui.QMainWindow, threading.Thread):
def __init__(self):
threading.Thread.__init__(self)
global device_0_V
super(Editor, self).__init__()
self.ui=Ui_Dialog()
#test=self.ui.Dev_1_V
self.ui.setupUi(self)
self.setWindowIcon(QtGui.QIcon('ICON.png'))
self.ui.Dev_1_V.setPlainText("anum")
self.show()
self.ui.Dev_1_ON.clicked.connect(self.handleButton)
def run(self):
global device_0_V
while 1:
self.ui.Dev_1_V.setPlainText(device_0_V) #<<here it gives ERROR
time.sleep(1)
def handleButton(self):
time = self.ui.time_dev_1.value()
self.ui.Dev_1_V.setPlainText(device_0_V)
print time
#print ('Hello World')
def main():
tx_socket_thread2 = Serial_read()
tx_socket_thread2.start()
app = QtGui.QApplication(sys.argv)
ex = Editor()
ex.start()
sys.exit(app.exec_())
if __name__ == '__main__':
main()
I've seen some relevant questions asked in Stackoverflow, but I am not able to understand the concept still, as I am new in Classes, Qt and OOP things. I know i am doing some basic mistake here... Any help will be highly appreciated.

So after some readings on related asked questions in Stack overflow, I've managed to achieve what I want, Heres the code;
import sys
from PyQt4 import QtGui, QtCore
from My_GUI_code import Ui_Dialog
import serial # import Serial Library
import threading
import time
test=""
arduinoData = serial.Serial('COM2', 9600) #
index=0
incoming_data=""
device_0_V=""
class Serial_read(threading.Thread):
"""
Thread to read data coming from Arduino
"""
def __init__(self):
threading.Thread.__init__(self)
def run(self):
global incoming_data
global device_0_V
global test
while 1:
while (arduinoData.inWaiting()==0): #Wait here until there is data
pass #do nothing
incoming_data = arduinoData.readline() #read the line of text from the serial port
if "V0" in incoming_data:
index = incoming_data.index("V0=")
device_0_V=incoming_data[index+3:index+6]
print device_0_V
#print incoming_data,
class Editor(QtGui.QMainWindow):
def __init__(self):
#threading.Thread.__init__(self)
global device_0_V
super(Editor, self).__init__()
self.ui=Ui_Dialog()
#test=self.ui.Dev_1_V
self.ui.setupUi(self)
self.setWindowIcon(QtGui.QIcon('ICON.png'))
self.ui.Dev_1_V.setPlainText("anum")
self.show()
self.ui.Dev_1_ON.clicked.connect(self.handleButton)
self.worker = Worker(self) # an independent thread that will listen to a signal 'beep' and trigger a function self.update
self.connect(self.worker, QtCore.SIGNAL('beep'), self.update)
self.worker.start() # start the thread
def update(self, Serial_data):
# here, I am getting the Serial data via signaling
if "V0" in incoming_data:
index = incoming_data.index("V0=")
device_0_V=incoming_data[index+3:index+7]
self.ui.Dev_1_V.setPlainText(device_0_V)
def handleButton(self):
time = self.ui.time_dev_1.value()
self.ui.Dev_1_V.setPlainText(device_0_V)
print time
#print ('Hello World')
class Worker(QtCore.QThread):
def __init__(self, host_window):
super(Worker, self).__init__()
self.running = False
def run(self):
self.running = True
global incoming_data #kept the Serial data global
global device_0_V
while self.running:
#sending 'beep' signal to the main Qt object, with string data 'incoming_data'
self.emit(QtCore.SIGNAL('beep'), incoming_data)
time.sleep(0.1)
def stop(self):
self.running = False
def main():
tx_socket_thread2 = Serial_read()
tx_socket_thread2.start()
app = QtGui.QApplication(sys.argv)
ex = Editor()
sys.exit(app.exec_())
if __name__ == '__main__':
main()
Inside the main Qt object, I've created a QThread "Worker" that sends signals to main Qt object with the Serial data. The update function is triggered every time a signal arrives from worker thread, and then further reads the data coming from worker thread.
Got help from this question
Thank you #Andy and #SiHa for your participation

Here's a page describing what Qt objects are and are not thread-safe -- http://doc.qt.io/qt-4.8/threads-reentrancy.html
For the most part, GUI objects are not thread safe and you should avoid modifying them from other threads.
One way of affecting the GUI from other threads is to use the signal and slot system, which is safe to use between threads so long as any objects passed are thread-safe. This usually means creating a thread-safe data structure in the secondary thread and passing it along with a signal to the main thread, which then reads the data structure and updates the GUI.
A more advanced version of that design pattern is to use a 2-way queue. One queue is populated by the main thread, which creates worker threads that process the items in the queue. When finished, the worker threads populate the other queue with thread-safe return values that the main thread then processes. Signals and events are still used to notify the main and worker threads when there are items in the queue to process.
Also, unless absolutely want to directly manage the threads, you can use QRunnable and QThreadPool to kick off threads without the need to directly manage them.

wxpython event not triggering

I'm following the example given in http://www.blog.pythonlibrary.org/2010/05/22/wxpython-and-threads/
I have a thread which is checking an sftp server for new files every 30 seconds. If it finds files, it uploads them to a db, and then it should trigger an update of certain GUI elements which will reload from the db.
The custom event code:
EVT_RESULT_ID = wx.NewId()
def EVT_RESULT(win, func):
"""Define Result Event."""
win.Connect(-1, -1, EVT_RESULT_ID, func)
class ResultEvent(wx.PyEvent):
"""Simple event to carry arbitrary result data."""
def __init__(self, data):
"""Init Result Event."""
wx.PyEvent.__init__(self)
self.SetEventType(EVT_RESULT_ID)
self.data = data
The ftp thread:
class FTPThread(threading.Thread):
def __init__(self,wxObject):
"""Init Worker Thread Class."""
threading.Thread.__init__(self)
self.wxObject = wxObject
self._stop = threading.Event()
self._stop.set()
self.start() # start the thread
def run(self):
while True:
time.sleep(30)
if not self._stop.isSet():
wx.CallAfter(self.parseFTP)
def stop(self):
self._stop.set()
def resume(self):
self._stop.clear()
def parseFTP(self):
#connect to db
...
#connect to sftp site
...
files_found=False
#process each file and delete
for file in dirlist:
files_found=True
...#process into db
sftp.remove(file)
sftp.close()
t.close()
#trigger update event if files found
if files_found==True:
wx.PostEvent(self.wxObject, ResultEvent("Files found"))
One of the GUI elements:
class MyGrid(wx.grid.Grid):
def __init__(self, parent):
wx.grid.Grid.__init__(self, parent,-1,style=wx.EXPAND)
self.parent=parent
...
self.update()
EVT_RESULT(self, self.updateFromEvent)
def updateFromEvent(self,event):
self.update()
def update(self):
...
Following debugging, the wx.PostEvent is being created, but not triggering any response in the grid.
The only difference I can find between the example and my code is that in the example the EVT_RESULT is in the main Frame, and not a GUI element - is this required?

Events don't propagate to its children so if MyGrid is a child of your main frame, events posted in the main won't make it through to MyGrid. What you can do instead is bind the event handler directly to your function within the instance of MyGrid like so:
"""from MainWindow"""
self._workerthread = FtpThread(...)
self._mygrid = MyGrid(...)
# Bind event
EVT_RESULT(self, self._mygrid.updateFromEvent)
I'm not too familiar with this kind of binding as I typically use wx.Bind.

I'm not sure, but that example was based on something in the wiki: http://wiki.wxpython.org/LongRunningTasks
I suspect that since it says "win" as an argument, it is probably referring to a Top Level Window, so the wx.Frame is probably required. You can still update the grid from the frame though.
EDIT: Manny has a good point. That would probably work too. And pubsub rocks!