I am trying to write data in the serial port and then read the same data. But it only returns a blank page when I try to read the data. How can I fix this issue?
My code
ser = serial.Serial('/dev/cu.usbmodem14302', 115200, timeout = 1)
print ("Writing: ")
ser.write(b'somedata\r\n')
time.sleep(2)
read_val = ser.readline().decode()
print(read_val) # returns a blank line
The expected result: "somedata"
The actual result: " "
What is the best approach to process a socket connection where I need var data to end with a line break \n?
I'm using the code below but sometimes the tcp packets get chunked and it takes a long time to match data.endswith("\n").
I've also tried other approaches, like saving the last line if it doesn't end with \n and append it to dataon the next loop. but this also doesn't work because multiple packets get chunked and the 1st and 2nd part don't match.
I've no control over the other end, it basically sends multiple lines that end in \r\n.
Any suggestion will be welcome, as I don't have much knowledge on socket connections.
def receive_bar_updates(s):
global all_bars
data = ''
buffer_size = 4096
while True:
data += s.recv(buffer_size)
if not data.endswith("\n"):
continue
lines = data.split("\n")
lines = filter(None, lines)
for line in lines:
if line.startswith("BH") or line.startswith("BC"):
symbol = str(line.split(",")[1])
all_bars[symbol].append(line)
y = Thread(target=proccess_bars, kwargs={'symbol': symbol})
y.start()
data = ""
Example of "normal" data:
line1\r\n
line2\r\n
line3\r\n
Example of chunked data:
line1\r\n
line2\r\n
lin
If you have a raw input that you want to process as line, the io module is your friend because it will do the low level assembling of packets in lines.
You could use:
class SocketIO(io.RawIOBase):
def __init__(self, sock):
self.sock = sock
def read(self, sz=-1):
if (sz == -1): sz=0x7FFFFFFF
return self.sock.recv(sz)
def seekable(self):
return False
It is more robust than endswith('\n') because if one packet contains an embedded newline ('ab\ncd'), the io module will correctly process it. Your code could become:
def receive_bar_updates(s):
global all_bars
data = ''
buffer_size = 4096
fd = SocketIO(s) # fd can be used as an input file object
for line in fd:
if should_be_rejected_by_filter(line): continue # do not know what filter does...
if line.startswith("BH") or line.startswith("BC"):
symbol = str(line.split(",")[1])
all_bars[symbol].append(line)
y = Thread(target=proccess_bars, kwargs={'symbol': symbol})
y.start()
Use socket.socket.makefile() to wrap the socket in a class that implenents Text I/O. It handles buffering, converting between bytes and strings, and lets you iterate over lines. Remember to flush any writes.
Example:
#!/usr/bin/env python3
import socket, threading, time
def client(addr):
with socket.create_connection(addr) as conn:
conn.sendall(b'aaa')
time.sleep(1)
conn.sendall(b'bbb\n')
time.sleep(1)
conn.sendall(b'cccddd\n')
time.sleep(1)
conn.sendall(b'eeefff')
time.sleep(1)
conn.sendall(b'\n')
conn.shutdown(socket.SHUT_WR)
response = conn.recv(1024)
print('client got %r' % (response,))
def main():
with socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0) as listen_socket:
listen_socket.bind(('localhost', 0))
listen_socket.listen(1)
addr = listen_socket.getsockname()
threading.Thread(target=client, args=(addr,)).start()
conn, _addr = listen_socket.accept()
conn_file = conn.makefile(mode='rw', encoding='utf-8')
for request in conn_file:
print('server got %r' % (request,))
conn_file.write('response1\n')
conn_file.flush()
if __name__ == '__main__':
main()
$ ./example.py
server got 'aaabbb\n'
server got 'cccddd\n'
server got 'eeefff\n'
client got b'response1\n'
$
Are you accepting different connections? Or is it one stream of data, split up by \r\n's?
When accepting multiple connections you'd wait for a connection with s.accept() and then process all its data. When you have all of the packet, process its data, and wait for the next connection.
What you do then depends on what the structure of each packet would be.
(Example: https://wiki.python.org/moin/TcpCommunication)
If instead you are consuming a stream of data, you should probably process each 'line' you find in a separate thread, while you keep consuming on another.
Edit:
So, if I have your situation correct; one connection, the data being a string broken up by \r\n, ending with a \n. The data however does not correspond to what you are expecting, instead looping infinitely waiting for a \n.
The socket interface, as I understand it, ends with an empty data result. So the last buffer might have ended with a \n, but then just continued getting None objects, trying to find another \n.
Instead, try adding this:
if not data:
break
Full code:
def receive_bar_updates(s):
global all_bars
data = ''
buffer_size = 4096
while True:
data += s.recv(buffer_size)
if not data:
break
if not data.endswith("\n"):
continue
lines = data.split("\n")
lines = filter(None, lines)
for line in lines:
if line.startswith("BH") or line.startswith("BC"):
symbol = str(line.split(",")[1])
all_bars[symbol].append(line)
y = Thread(target=proccess_bars, kwargs={'symbol': symbol})
y.start()
data = ""
Edit2: Oops, wrong code
I have not tested this code, but it should work:
def receive_bar_updates(s):
global all_bars
data = ''
buf = ''
buffer_size = 4096
while True:
if not "\r\n" in data: # skip recv if we already have another line buffered.
data += s.recv(buffer_size)
if not "\r\n" in data:
continue
i = data.rfind("\r\n")
data, buf = data[:i+2], data[i+2:]
lines = data.split("\r\n")
lines = filter(None, lines)
for line in lines:
if line.startswith("BH") or line.startswith("BC"):
symbol = str(line.split(",")[1])
all_bars[symbol].append(line)
y = Thread(target=proccess_bars, kwargs={'symbol': symbol})
y.start()
data = buf
Edit: Forgot to mention, i only modified the code for receiving the data, i have no idea what the rest of the function (starting with lines = data.split("\n")) is for.
Edit 2: Now uses "\r\n" for linebreaks instead of "\n".
Edit 3: Fixed an issue.
You basically seem to want to read lines from the socket. Maybe you're better off not using low level recv calls but just use sock.makefile() and treat the result as a regular file where you can read lines from: from line in sfile: ...
That leaves the delay/chunk issue. This is likely to be caused by Nagle's algorithm on the sending side. Try disabling that:
sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
I'm writing a python script that connects to a USB serial device. Whenever a command is sent and executed, the PIC returns with a hashtag. Ie. "Command executed successfully. \n# "
I'd like my python script to wait for the hashtag before outputting the data. How can I do this?
Here's what I have. It doesn't seem to actually print the text received from the PIC. Any help is appreciated
if port.isOpen():
try:
for x in range(0,100):
time.sleep(0.05)
port.write("command 1" + "\r\n")
numLines = 0
// wait for "#" to print output
while True:
response = port.readline()
if "#" in response:
print(response)
numLines = numLines + 1
if(numLines >= 1):
break
time.sleep(0.05)
port.write("command 2" + "\r\n")
numLines = 0
// wait for "#" to print output
while True:
response = port.readline()
if "#" in response:
print(response)
numLines = numLines + 1
if(numLines >= 1):
break
time.sleep(0.05)
port.write("command 3" + "\r\n")
numLines = 0
// wait for "#" to print output
while True:
response = port.readline()
if "#" in response:
print(response)
numLines = numLines + 1
if(numLines >= 1):
break
except Exception, e1:
print("An error occured: " + str(e1))
port.close()
port.readline() will read the serial port till it receives a \n. So, the response will contain the string "Command executed successfully. \n". Since there is no "#" in this string, again the code will encounter the port.readline() statement. This time it will read "#" but since there is no "\n", the code will be stuck there resulting in an infinite loop.
Pyserial provides a method called read():
read(size=1)
Parameters: size – Number of bytes to read. Returns: Bytes read from
the port. Return type: bytes Read size bytes from the serial port. If
a timeout is set it may return less characters as requested. With no
timeout it will block until the requested number of bytes is read.
read() provides a parameter size (with default =1) which specifies the number of bytes to be read. So, you can specify the number of bytes in the string sent by the PIC as a parameter. You can also use the following alternative:
// wait for "#" to print output
while True:
response += port.read()
if "#" in response:
print(response)
numLines = numLines + 1
if(numLines >= 1):
break
If you send some white space to the device, as if it were a terminal command, it will prod it into a response with your "#" in it. I have been successfully using that method. Specifically I send a single space " " plus the terminal line ending (i.e. "\n" or "\r\n" depending on the device).
First of all I am a complete noobie when it comes to python. Actually I started reading about it this morning when I needed to use it, so sorry if the code is a disaster.
I'd like to get this done:
A communication via serial between two devices. The device where the python program is running has to be listening for some data being sent by the other device and storing it in a file. But every 30 seconds of received data it has to send a command to the other device to tell it to stop sending and begin a scan that takes 10 seconds.
This is the code I've written. It's printing continuously Opening connection..
from serial import Serial
from threading import Timer
import time
MOVE_TIME = 30.0
SCAN_TIME = 10.0
DEVICE_ADDRESS = '/dev/ttyACM0'
BAUD_RATE = 9600
while True:
try:
print("Opening connection...")
ser = Serial(DEVICE_ADDRESS, BAUD_RATE
break
except SerialException:
print("No device attached")
def scan():
print("Scanning...")
timeout = time.time() + SCAN_TIME
while True:
#Some code I haven't thought of yet
if time.time() > timeout:
ser.write(b'r') #command to start
break
def send_stop_command():
print("Sending stop command")
ser.write(b's') #command to stop
scan()
t = Timer(MOVE_TIME + SCAN_TIME, send_stop_command)
t.start()
filename = time.strftime("%d-%m-%Y_%H:%M:%S") + ".txt"
while True:
data = ser.readline()
try:
with open(filename, "ab") as outfile:
outfile.write(data)
outfile.close()
except IOError:
print("Data could not be written")
I'm writing a code used to send order to an avr. I send several information but between each write, I have to wait for an answer (I have to wait for the robot to reach a point on the coordinate system). As I read in the documentation, readline() should at least read until the timeout but as soon as I send the first coordinate, the readline() automatically return :
SerialException: device reports readiness to read but returned no data (device disconnected?)
When I put a sleep() between each write() in the for loop, everything works fine. I tried to use inWaiting() but it still does not work. Here is an example of how I used it:
for i in chemin_python:
self.serieInstance.ecrire("goto\n" + str(float(i.x)) + '\n' + str(float(-i.y)) + '\n')
while self.serieInstance.inWaiting():
pass
lu = self.serieInstance.readline()
lu = lu.split("\r\n")[0]
reponse = self.serieInstance.file_attente.get(lu)
if reponse != "FIN_GOTO":
log.logger.debug("Erreur asservissement (goto) : " + reponse)
Here an snipet how to use serial in python
s.write(command);
st = ''
initTime = time.time()
while True:
st += s.readline()
if timeout and (time.time() - initTime > t) : return TIMEOUT
if st != ERROR: return OK
else: return ERROR
This method allows you to separately control the timeout for gathering all the data for each line, and a different timeout for waiting on additional lines.
def serial_com(self, cmd):
'''Serial communications: send a command; get a response'''
# open serial port
try:
serial_port = serial.Serial(com_port, baudrate=115200, timeout=1)
except serial.SerialException as e:
print("could not open serial port '{}': {}".format(com_port, e))
# write to serial port
cmd += '\r'
serial_port.write(cmd.encode('utf-8'))
# read response from serial port
lines = []
while True:
line = serial_port.readline()
lines.append(line.decode('utf-8').rstrip())
# wait for new data after each line
timeout = time.time() + 0.1
while not serial_port.inWaiting() and timeout > time.time():
pass
if not serial_port.inWaiting():
break
#close the serial port
serial_port.close()
return lines