We Keep Coding

PySerial timeout, CPU usage, and barcode scanning

It's worth mentioning up-front that while I have a background in CS, the number of Python scripts I've written in could likely be counted on the number of toes on a sloth's paw. That said, I started playing with PySerial to read from a USB barcode scanner. One problem I'm having is the timeout. If I set it too low, I miss scans. If I set it too high, the processor utilization is huge. Of course, this is mentioned in the documentation for PySerial:
Be careful when using readline(). Do specify a timeout when opening
the serial port otherwise it could block forever if no newline
character is received. Also note that readlines() only works with a
timeout. readlines() depends on having a timeout and interprets that
as EOF (end of file). It raises an exception if the port is not opened
correctly.
Right. So, here's my simple code:
#!/usr/bin/env python
import serial
ser = serial.Serial('/dev/ttyACM0', rtscts=True, dsrdtr=True, timeout=0.05)
ser.baudrate = 115200
while True:
s = ser.readline()
if s:
print(s)
How do I appropriately read from a serial device without risking missed scans? Sure, the odds are incredibly low with that small of a timeout, but I'm wanting to use this for production purposes at my business, so let's assume that this is mission-critical. What's the proper way to approach this problem (again, assuming that my understanding of Python is nil)?
Thanks, everyone!
EDIT: Possible solution?
I came up with the following that doesn't use a timeout and simply reads a single character at a time until it reaches a newline. It seems like this is pretty light on processor utilization (which was the whole issue I was having). Of course, I need to account for other newline possibilities from different scanners, but is there any reason why this wouldn't work?
#!/usr/bin/env python
import serial
ser = serial.Serial('/dev/ttyACM0', rtscts=True, dsrdtr=True)
ser.baudrate = 115200
string = ""
while 1:
char = ser.read(1)
string += char
if char == '\r':
print(string)
string = ""

From what I know about barcode scanners, you can configure them so that they only trigger scanning when you send them a specific write command over serial, you can use that to your advantage.
ser = serial.Serial('/dev/ttyUSBx',timeout=y)
ser.write('<trigger scan>')
value = ser.readline()
ser.close()
For continuous reading, the best way of doing it is to keep reading bytes in a timeout loop like
time_start = datetime.datetime.now()
time_end = time_start + datetime.timedelta(seconds=timeout)
output = []
while datetime.datetime.now() < time_end:
output.append(ser.read(100))

My experience with setting timeout to a high value is the opposite of your assertion. A high timeout ensures that python is not checking the serial buffer every 1/20,000th of a second. That's the point of a serial buffer, to store input until it is read. The timeout is in thousandths of seconds, so 0.05 * 1/1000 = 1/20,000 or 20,000 checks per second. I set it to 10 seconds below. (a minimum of 6 checks per minute) Of course, if python encounters a new line sooner then the readline() does not timeout.
#!/usr/bin/env python
import serial
ser = serial.Serial('/dev/ttyACM0', rtscts=True, dsrdtr=True, timeout=10000)
ser.baudrate = 115200
while True:
s = ser.readline()
if s:
print(s)
However, if your UART has no buffer and discards anything past one character, you could lose input. This depends on your hardware and setup. If the bar code fits in the buffer, you should not encounter any problems.

How to read serial port output in Python using pySerial?

A device is connected to my PC and I need to write a command via COM Port and get the output. Here is my code
from time import sleep
import serial
ser = serial.Serial()
ser.baudrate = 115200
ser.port = 'COM20'
ser.open()
ser.write("system\r")
result = ser.read(100)
print result
ser.close()
This is working perfectly. However, different commands return different strings. so I need to read it without mentioning no of bytes in ser.read(100).
How can I achieve that? Appreciate any help. Thank you

Read one byte at the time in a while loop and break when you find the sequence you are looking for, i.e. the data between your protocols start and end control bytes.

pySerial - Only reading one byte

I am trying to read and write to a sensor via serial using pySerial. I have no software or hardware flow control.
I am able to send a string of hex to the device, but I only receive one byte back instead of the two-to-ten bytes I should see. The sensor is working -- I've verified this using Realterm.
I've tried using ser.readline() (instead of the inWaiting loop), and ser.read(2); this just causes the program to hang. I've also tried increasing the sleep time, and experimented with different baud rates (on both the PC and sensor), but nothing seems to work.
Does anyone have any advice?
import time
import serial
# configure the serial connections
ser = serial.Serial(
port='COM1',
baudrate=115200,
parity=serial.PARITY_EVEN,
stopbits=serial.STOPBITS_ONE,
bytesize=serial.EIGHTBITS
)
ser.isOpen()
print 'Enter your commands below.\r\nInsert "exit" to leave the application.'
while 1 :
# get keyboard input
data_in = raw_input(">> ")
if data_in == 'exit':
ser.close()
exit()
else:
# send the character to the device
ser.write(data_in.decode('hex') + '\r\n')
out = ''
time.sleep(1)
while ser.inWaiting() > 0:
out += ser.read(1)
if out != '':
print ">>" + " ".join(hex(ord(n)) for n in out)
(I slightly modified the code from that found on Full examples of using pySerial package)

Your read statement is explicitly requesting 1 byte:
ser.read(1)
If you know how many bytes to read, you can specify here. If you are unsure, then you can specify a larger number. For example, doing
ser.read(10)
will read up to 10 bytes. If only 8 are available, then it will only return 8 bytes (following the timeout, see below).
It is also worth setting a timeout to prevent the program from hanging. Just add a timeout parameter to your Serial constructor. The following will give you a 2 second timeout:
ser = serial.Serial(
port='COM1',
baudrate=115200,
parity=serial.PARITY_EVEN,
stopbits=serial.STOPBITS_ONE,
bytesize=serial.EIGHTBITS,
timeout=2
)
The documentation states:
read(size=1) Read size bytes from the serial port. If a timeout is set it may return less characters as requested. With no timeout it will block until the requested number of bytes is read.
So if you do not know how many bytes to expect, then set a small timeout (if possible) so your code does not hang.
If your code does not return the full number of bytes expected, then it is likely the device you are connected to is not sending all the bytes you are expecting. Since you have verified it should be working separately, have you verified the data you are sending is correct? Perhaps encode to bytes first using struct.pack(). As an example, to send a byte with a decimal value of 33 (hex 0x21)
import struct
bytes_to_send = struct.pack('B', 33)
I have also never found it necessary to append the end of line chars \r\n to a message before sending

Python serial port read delay

I'm playing around with this serial module in python. I have a little problem with it. I want my script to get a char from the console send it to an AVR board, and read back the response.
Everytime I read from the USB port, and print it out, I see the previous result. Why's that?
For example:
I write 5
I read nothing
I write 6
I read 5
import serial
import sys, time
port=serial.Serial(
port='/dev/ttyUSB0',\
baudrate=9600,\
parity=serial.PARITY_NONE,\
stopbits=serial.STOPBITS_ONE,\
bytesize=serial.EIGHTBITS,\
timeout=0)
i=0
tmp = 0
while True:
tmp=raw_input('send: ')
port.write(tmp)
port.flushOutput()
print port.read(1)
port.flushInput()

From the documentation: "Writes are blocking by default, unless writeTimeout is set. For possible values refer to the list for timeout above." Try setting writeTimeout=0 as well in your constructor.

You are probably receiving a single unexpected byte on startup - either the microcontroller is sending it, or it might be noise from connecting a plug. As you are only reading a single byte for each string transmitted, you will always be off by one.
Instead of port.read(1), try:
while True:
tmp=raw_input('send: ')
port.write(tmp)
port.flushOutput()
print port.read(port.inWaiting())
port.flushInput()
This would also have happened if your typed in more than one character at the input prompt.

reading from sys.stdin without newline or EOF

I want to recieve data from my gps-tracker. It sends data by tcp, so I use xinetd to listen some tcp port and python script to handle data. This is xinetd config:
service gps-gprs
{
disable = no
flags = REUSE
socket_type = stream
protocol = tcp
port = 57003
user = root
wait = no
server = /path/to/gps.py
server_args = 3
}
Config in /etc/services
gps-gprs 57003/tcp # Tracking system
And Python script gps.py
#!/usr/bin/python
import sys
def main():
data = sys.stdin.readline().strip()
#do something with data
print 'ok'
if __name__ =='__main__':
main()
The tracker sends data strings in raw text like
$GPRMC,132017.000,A,8251.5039,N,01040.0065,E,0.00,,010111,0,,A*75+79161234567#
The problem is that sys.stdin in python script doesn't recieve end of line or end of file character and sys.stdin.readline() goes forever. I tried to send data from another pc with a python script
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('', 57003))
s.sendall( u'hello' )
data = s.recv(4024)
s.close()
print 'Received', data
and if the message is 'hello', it fails, but if the message is 'hello\n', it's ok and everything is fine. But I don't know ho to tell tracker or xinetd to add this '\n' at the end of messages. How can I read the data from sys.stdin without EOF or EOL in it?

Simple:
data=sys.stdin.read().splitlines()
for i in data:
print i
No newlines

sys.stdin.readline() waits forever until it receives a newline. Then it considers the current line to be complete and returns it in full. If you want to read data that doesn't contain newlines or you don't want to wait until a newline is received before you process (some of) the data, then you're going to have to use something other than readline. Most likely you should call read, which reads arbitrary data up to a given size.
However, your GPS appears to be sending data in the well-known NEMA format, and that format certainly terminates each line with a newline. Actually, it probably terminates each line with CRLF (\r\n) but it is possible that the \r could be getting munged somewhere before it gets to your TCP socket. Either way there's a \n at the very end of each line.
If your readline call is hanging without returning any lines, most likely it's because the sender is buffering lines until it has a full buffer. If you waited long enough for the sender's buffer to fill up, you'd get a whole bunch of lines at once. If that's what's happening, you'll have to change the sender to that it flushes its send buffer after each NEMA sentence.

It seems you are receiving # instead of <CR><LF>, just read until the # sign.
data = ""
while len(data) == 0 or data[-1] <> '#':
data += sys.stdin.read(1)
#do something with data
print 'ok'

My solution :
var = sys.stdin.readline().replace('\n', '')
It :
find the newline in the entry,
replace it from the entry by '' (none) ~remove,
assigne it to variable.