I did not find a reasonable good example of how to talk to a serial modem using pyserial. I have created a code snippet that should do the following, given an instantiated pyserial object ser:
Send an AT command to the modem
Return the modem answer as quickly as possible
Return e.g. None in the case of a timeout
Handle the communication between the script and the modem most reasonable, robust and easy.
Here is the snippet:
def send(cmd, timeout=2):
# flush all output data
ser.flushOutput()
# initialize the timer for timeout
t0 = time.time()
dt = 0
# send the command to the serial port
ser.write(cmd+'\r')
# wait until answer within the alotted time
while ser.inWaiting()==0 and time.time()-t0<timeout:
pass
n = ser.inWaiting()
if n>0:
return ser.read(n)
else:
return None
My question: Is this good, robust code, or can pieces be changed/simplified? I especially do not like the read(n) method, I would expect pyserial to offer a piece of code that just returns the whole buffer content. Also, do I / should I flush the output at the begin, to avoid having some crap in the output buffer before?
Thanks
Alex
Create the Serial object with the param timeout=2 for read timeout.
Mi recipe is:
def send(data):
try:
ser.write(data)
except Exception as e:
print "Couldn't send data to serial port: %s" % str(e)
else:
try:
data = ser.read(1)
except Exception as e:
print "Couldn't read data from serial port: %s" % str(e)
else:
if data: # If data = None, timeout occurr
n = ser.inWaiting()
if n > 0: data += ser.read(n)
return data
I think that this is a good form of manage the communications with the serial port.
Related
I'm wondering if write_timeout even works or if I have to handle the timeout myself.
I want to send data to my STM32 Nucleo-Board via UART. For that I'm using the library pySerial. But if the write operation fails, because the Nucleo-Board isn't powered, I want to throw an error saying "Nucleo-Board not powered". I assume that when the write operation times out after 1 second, the Nucleo-Board has no power. So, I set the write_timeout of the serial.Serial() object to 1, but it seems that write_timeout doesn't do anything.
I created my own class, to add additional methods.
class Serial:
def __init__(self, baudrate: int, port: str, write_timeout: int = None, read_timeout: int = None):
self.baudrate = baudrate
self.port = port
self.write_timeout = write_timeout
self.read_timeout = read_timeout
self.ser = serial.Serial()
self.ser.baudrate = self.baudrate
self.ser.port = self.port
self.ser.write_timeout = self.write_timeout
self.ser.timeout = self.read_timeout
def _serial_ports():
# ...
def serialWrite(self, string: str, size: int = None):
print(self.write_timeout)
print(self.ser.write_timeout)
# check if COM port exists -> because UART cable is connected, COM6 always exists,
# even when no power is connected.
if self.port not in self._serial_ports():
raise serial.SerialException(f"Make sure this COM Port exists.")
try:
print("opened")
self.ser.open()
except serial.SerialException as e:
print("closed")
self.ser.close()
# error handling ...
else:
encodedString = string.encode()
print("write now")
try:
bytes = self.ser.write(encodedString) # times out here -> program hangs (no exception)
except Exception as e:
print(e)
print("Nucleo-Board not powered")
serialPort = Serial(baudrate=115200, port="COM6", write_timeout=1)
feedback = serialPort.serialWrite(f"ABC", 3)
Output
self.write_timeout=1
self.ser.write_timeout=1
opened
write now
I have to kill the program, because it stops working.
The program works, when I plug in the power of the Nucleo-Board.
Solution
The problem was, that I thought the timeout happens on ser.write(). But a timeout never happens on ser.write(), because (as #jasonharper already commented) the board being unpowered doesn't prevent serial data from being sent - it just means the data isn't going anywhere. So in the end, the timeout was a few lines after the write operation at the read operation ser.read(), to get a feedback to know the Nucleo-Board received something. There it can timeout, because it waits until data are received. But the board isn't powered so data will never received. Because I didn't handled the timeout for read operations, the program started to hang.
Current Code
try:
bytes = self.ser.write(encodedString)
except Exception as e:
# never raise
print(e)
print("Nucleo-Board not powered")
feedback = self.ser.read(size) # here it waits forever, because it doesn't receive something of course, because the Nucleo board isn't powered.
Code Solution
self.ser.timeout = 1 # for self.ser.read()
this is question is really focused on my problem and not relative to any of the other question I could find on this topic.
PSA: When I say "packet" I mean a full string received in a single socket.recv(maxsize)
I developed similar code for the same result on Java (my pref language) and it is ok, now I have to do in python.
I have two processes that run in parallel:
1-Normal client socket connected to a specific IP
2-A "client" Datagram socket binded to "ALL" IPs.
The normal socket is working correctly as I expect, while the datagram not.
I continuosly receive packets from a server (not mine and not opensource) at a rate of more than 5 per second, but I want to process only one of them every 3 seconds. In java I did just a "sleep" and it was ok, I was getting only the last live packet, while in Python with a "time.sleep(3)" the packets are queued (I don't know how and where) and not dropped.
I HAVE to drop them because those are not need and I have to do an HTTP call between one and the other so I can't fire an HTTP post for every set of data received at that rate!
here it is my "code" for the listening socket, some comments are for private code:
def listenPositions():
lsSocket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
lsSocket.bind(("0.0.0.0", 8787))
lsSocket.setblocking(0)
try:
while True:
ready = select.select([lsSocket], [], [], 1)
if ready[0]:
lsSocket.settimeout(1)
recvData = lsSocket.recv(16384)
if len(recvData) != 0:
recv = recvData[0:len(recvData)].decode("utf-8")
#print("LS: Received: " + recv)
strings = filter(None, str(recv).split('\n'))
print("Strings count=" + str(len(strings))+ ": " + str(strings))
for item in strings:
#parse the received strings as json and get the items
jsonPosition = json.loads(item)
strId = jsonPosition["id"]
coordinates = jsonPosition.get("coordinates")
if coordinates is None:
continue
print("coordinates not null:" + str(coordinates))
#DO THE HTTP POST REQUEST
time.sleep(3) #Pause the system for X seconds, but other packets are queued!
else:
print("LS: Received empty")
else:
print("LS: No data, timeout")
except Exception as e:
print(e)
#handle exceptions...
print("Exception, close everything")
When you have an open socket, all correctly addressed packets should be delivered to the application. We want to have our network connections as realiable as possible, don't we? Dropping a packet is an action of last resort.
If you want to get a packet only from time to time, you could create a listening socket, get a packet and close the socket.
However there is nothing easier than ignoring a packet. Just skip its processing and move on. The code below is incomplete, but hopefully expresses what I mean.
TIMEOUT = 1.0
INT = 3.0 # interval in seconds
# create udp_socket
last = time.time() - INT
udp_socket.settimeout(TIMEOUT)
while True:
try:
packet = udp_socket.recv(MAXSIZE)
except socket.timeout:
# handle recv timeout
continue # or break, or return
except OSError:
# handle recv error (Python 3.3+)
break # or continue, or return
now = time.time()
if now - last >= INT:
# process the packet
last = now
Please note that the select is not needed if you read only from one source.
I am connecting with my Arduino through a USB port and sending data to it by using PySerial module. At first I can check if the device is connected by using this code:
try:
ser = serial.Serial("COM3", 9600)
except serial.serialutil.SerialException:
print "Arduino not connected"
Now what I want to do is to check periodically if the Arduino is still connected to the computer. I tried ser.isOpen() but this returns true even if the Arduino is disconnected. I would also like to know how to reconnect the device. I mean once you disconnect the device the program can no longer send any data to Arduino.
Most of the answers propose 2 approaches:
In some point of the code, send some sort of message through serial to check if your device is still alive
Start a separate thread and continuously check if the device is alive by opening a communication
The problem with the first solution is that you are not always checking the connection, but only checking in some specific points: this solution isn't very elegant and if badly written could even be not working.
The second solution solves the problem of the first solution, but introduces a new problem: checking the connection, or worst sending a message, in a threaded loop will cause problem or may even interrupt the connection to the device from other functions.
A solution that allows you to constantly check the connection without monopolizing the communication involves the reading of the existing COM:
import serial.tools.list_ports
myports = [tuple(p) for p in list(serial.tools.list_ports.comports())]
print myports
output:
[(u'COM3', u'Arduino Due Programming Port (COM3)', u'some more data...'),
(u'COM6', u'USB Serial Port (COM6)', u'some more data...'),
(u'COM100', u'com0com - serial port emulator (COM100)', u'some more data...')]
then we save the tuple that contains our port:
arduino_port = [port for port in myports if 'COM3' in port ][0]
then we create a function that checks if this port is still present:
import time
def check_presence(correct_port, interval=0.1):
while True:
myports = [tuple(p) for p in list(serial.tools.list_ports.comports())]
if arduino_port not in myports:
print "Arduino has been disconnected!"
break
time.sleep(interval)
At last, we run this function as a daemon thread:
import threading
port_controller = threading.Thread(target=check_presence, args=(arduino_port, 0.1,))
port_controller.setDaemon(True)
port_controller.start()
in this way, you'll check each 0.1 secs if the arduino is still connected, and the thread will end when arduino is disconnected or all other activities have ended
You can set a timeout.
import serial
ser = serial
try:
ser = serial.Serial("COM3", 9600, timeout=10)
while ser.read():
print 'serial open'
print 'serial closed'
ser.close()
except serial.serialutil.SerialException:
print 'exception'
Unfortunately, the best way I can find to do this is to try some communication and see if it fails. A pretty safe way would be:
try:
ser.inWaiting()
except:
print "Lost connection!"
You'll probably still want to close the connection with a ser.close() after the connection is lost, although you may need to place that in a "try:except" block also.
import serial
import time
ser = serial.Serial()
ser.braudrate = 115200
ser.port = "/dev/ttyUSB0"
ser.open()
print(ser.name)
if ser.isOpen():
print("serial is open!")
ser.close()
For example to detect ttyUSB0:
import os
x=os.system("ls /dev/ttyUSB0")
if x==0:
print "connected"
else:
print "disconnected"
i suggest to use a python thread class to istantiate a serial connection, in the run methos put your while loop , set an var that you use for kill iy at the end, the second public var that you use for store data if have receive and load data in main method.. soon paste an example
class Arduino():
def __init__(self,Port='/dev/ttyUSB0',Boud=9600,connState=0):
self.parent=self
self.port=Port
self.boud=Boud
self.connState=connState
self.timeount=1
self.ser=None
self.connect()
def connect(self):
try:
self.ser=serial.Serial(self.port,self.boud,timeout=0.0001)
self.connState=1
return [1,'connect']
except:
self.connState=0
return [0,'no hardware found']
def loadData(self):
self.buffer=self.ser.read(1)
if (self.buffer!=''):
try:
print self.buffer
except Exception, e:
pass
ard=Arduino()
while True:
if ard.connState:
ard.loadData()
else:
print "Arduino not found"
break
and start with:
import threading
class ThController( threading.Thread ):
# Override Thread's __init__ method to accept the parameters needed:
def __init__( self,parent):
self.parent = parent
threading.Thread.__init__ ( self )
def run ( self ):
while self.parent.ctrlattive:
j=json.loads(data)
self.parent.data=j
I'm trying to read numerical values sent over a bluetooth modem from a serial port using Pyserial. I'm a beginner at Python, and found a good example that I'm trying to make use of.
from threading import Thread
import time
import serial
last_received = ''
def receiving(ser):
global last_received
buffer = ''
while True:
buffer = buffer + ser.read(ser.inWaiting())
if '\n' in buffer:
lines = buffer.split('\n') # Guaranteed to have at least 2 entries
last_received = lines[-2]
#If the modem sends lots of empty lines, you'll lose the
#last filled line, so you could make the above statement conditional
#like so: if lines[-2]: last_received = lines[-2]
buffer = lines[-1]
class SerialData(object):
def __init__(self, init=50):
try:
self.ser = ser = serial.Serial(
port='/dev/tty.FireFly-16CB-SPP',
baudrate=115200,
stopbits=serial.STOPBITS_ONE,
bytesize=serial.EIGHTBITS
)
except serial.serialutil.SerialException:
#no serial connection
self.ser = None
else:
Thread(target=receiving, args=(self.ser,)).start()
def next(self):
if not self.ser:
return 140 #return anything so we can test when Arduino isn't connected
#return a float value or try a few times until we get one
for i in range(40):
raw_line = last_received
try:
return float(raw_line.strip())
except ValueError:
print 'bogus data',raw_line
time.sleep(.005)
return 0.
def __del__(self):
if self.ser:
self.ser.close()
if __name__=='__main__':
s = SerialData()
for i in range(500):
time.sleep(.015)
print s.next()
I can open the port in another program, and can send/receive data from it. However, the code above doesn't seem to open the port, and just repeats "100" to the terminal window 500 times, but I don't know where it comes from or why the port doesn't open correctly. There isn't a delay from opening the port as it is on the other program, so I don't even know if it attempts to open.
I don't know what else to try, or where the error is so I'm asking for help. What am I doing wrong?
except serial.serialutil.SerialException:
You're catching and silencing errors in connecting. Comment out this block, and see if it produces an error message.
How can I make a simple server(simple as in accepting a connection and print to terminal whatever is received) accept connection from multiple ports or a port range?
Do I have to use multiple threads, one for each bind call. Or is there another solution?
The simple server can look something like this.
def server():
import sys, os, socket
port = 11116
host = ''
backlog = 5 # Number of clients on wait.
buf_size = 1024
try:
listening_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
listening_socket.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1)
listening_socket.bind((host, port))
listening_socket.listen(backlog)
except socket.error, (value, message):
if listening_socket:
listening_socket.close()
print 'Could not open socket: ' + message
sys.exit(1)
while True:
accepted_socket, adress = listening_socket.accept()
data = accepted_socket.recv(buf_size)
if data:
accepted_socket.send('Hello, and goodbye.')
accepted_socket.close()
server()
EDIT:
This is an example of how it can be done. Thanks everyone.
import socket, select
def server():
import sys, os, socket
port_wan = 11111
port_mob = 11112
port_sat = 11113
sock_lst = []
host = ''
backlog = 5 # Number of clients on wait.
buf_size = 1024
try:
for item in port_wan, port_mob, port_sat:
sock_lst.append(socket.socket(socket.AF_INET, socket.SOCK_STREAM))
sock_lst[-1].setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1)
sock_lst[-1].bind((host, item))
sock_lst[-1].listen(backlog)
except socket.error, (value, message):
if sock_lst[-1]:
sock_lst[-1].close()
sock_lst = sock_lst[:-1]
print 'Could not open socket: ' + message
sys.exit(1)
while True:
read, write, error = select.select(sock_lst,[],[])
for r in read:
for item in sock_lst:
if r == item:
accepted_socket, adress = item.accept()
print 'We have a connection with ', adress
data = accepted_socket.recv(buf_size)
if data:
print data
accepted_socket.send('Hello, and goodbye.')
accepted_socket.close()
server()
I'm not a python guy, but the function you are interested in is "select". This will allow you to watch multiple sockets and breaks out when activity occurs on any one of them.
Here's a python example that uses select.
Since Python's got so much overhead, multithreaded apps are a big point of debate. Then there's the whole blocking-operation-GIL issue too. Luckily, the Python motto of "If it seems like a big issue, someone's probably already come up with a solution (or several!)" holds true here. My favorite solution tends to be the microthread model, specifically gevent.
Gevent is an event-driven single-thread concurrency library that handles most issues for you out of the box via monkey-patching. gevent.monkey.patch_socket() is a function that replaces the normal socket calls with non-blocking variants, polling and sleeping to allow the switch to other greenlets as need be. If you want more control, or it's not cutting it for you, you can easily manage the switching with select and gevent's cooperative yield.
Here's a simple example.
import gevent
import socket
import gevent.monkey; gevent.monkey.patch_socket()
ALL_PORTS=[i for i in xrange(1024, 2048)]
MY_ADDRESS = "127.0.0.1"
def init_server_sock(port):
try:
s=socket.socket()
s.setblocking(0)
s.bind((MY_ADDRESS, port))
s.listen(5)
return s
except Exception, e:
print "Exception creating socket at port %i: %s" % (port, str(e))
return False
def interact(port, sock):
while 1:
try:
csock, addr = sock.accept()
except:
continue
data = ""
while not data:
try:
data=csock.recv(1024)
print data
except:
gevent.sleep(0) #this is the cooperative yield
csock.send("Port %i got your message!" % port)
csock.close()
gevent.sleep(0)
def main():
socks = {p:init_server_sock(p) for p in ALL_PORTS}
greenlets = []
for k,v in socks.items():
if not v:
socks.pop(k)
else:
greenlets.append(gevent.spawn(interact, k, v))
#now we've got our sockets, let's start accepting
gevent.joinall(greenlets)
That would be a super-simple, completely untested server serving plain text We got your message! on ports 1024-2048. Involving select is a little harder; you'd have to have a manager greenlet which calls select and then starts up the active ones; but that's not massively hard to implement.
Hope this helps! The nice part of the greenlet-based philosophy is that the select call is actually part of their hub module, as I recall, which will allow you to create a much more scalable and complex server more easily. It's pretty efficient too; there are a couple benchmarks floating around.
If you really wanted to be lazy (from a programmer standpoint, not an evaluation standpoint), you could set a timeout on your blocking read and just loop through all your sockets; if a timeout occurs, there wasn't any data available. Functionally, this is similar to what the select is doing, but it is taking that control away from the OS and putting it in your application.
Of course, this implies that as your sleep time gets smaller, your program will approach 100% CPU usage, so you wouldn't use it on a production app. It's fine for a toy though.
It would go something like this: (not tested)
def server():
import sys, os, socket
port = 11116
host = ''
backlog = 5 # Number of clients on wait.
buf_size = 1024
NUM_SOCKETS = 10
START_PORT = 2000
try:
socket.setdefaulttimeout(0.5) # raise a socket.timeout error after a half second
listening_sockets = []
for i in range(NUM_SOCKETS):
listening_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
listening_socket.setsockopt(socket.SOL_SOCKET,socket.SO_REUSEADDR,1)
listening_socket.bind((host, START_PORT + i))
listening_socket.listen(backlog)
listening_sockets.append(listening_socket)
except socket.error, (value, message):
if listening_socket:
listening_socket.close()
print 'Could not open socket: ' + message
sys.exit(1)
while True:
for sock in listening_sockets:
try:
accepted_socket, adress = sock_socket.accept()
data = sock.recv(buf_size)
if data:
sock_socket.send('Hello, and goodbye.')
sock.close()
except socket.timeout:
pass
server()