We Keep Coding

Python serial.readline() not blocking

I'm trying to use hardware serial port devices with Python, but I'm having timing issues. If I send an interrogation command to the device, it should respond with data. If I try to read the incoming data too quickly, it receives nothing.
import serial
device = serial.Serial("/dev/ttyUSB0", 9600, timeout=0)
device.flushInput()
device.write("command")
response = device.readline()
print response
''
The readline() command isn't blocking and waiting for a new line as it should. Is there a simple workaround?

readline() uses the same timeout value you passed to serial.Serial().
If you want readline to be blocking, just delete the timeout argument, the default value is None.
You could also set it to None before calling readline(), if you want to have a timeout for openening the device:
import serial
try:
device = serial.Serial("/dev/ttyUSB0", 9600, timeout=0.5)
except:
#Exception handeling
device.flushInput()
device.write("command")
device.timeout=None
response = device.readline()
print response

I couldn't add a commend so I will just add this as an answer. You can reference this stackoverflow thread. Someone attempted something similar to your question.
Seems they put their data reading in a loop and continuously looped over it while data came in. You have to ask yourself one thing if you will take this approach, when will you stop collecting data and jump out of the loop? You can try and continue to read data, when you are already collecting, if nothing has come in for a few milliseconds, jump out and take that data and do what you want with it.
You can also try something like:
While True:
serial.flushInput()
serial.write(command)
incommingBYTES = serial.inWaiting()
serial.read(incommingBYTES)
#rest of the code down here

Reset an open serial port

I am reading data from a serial port, sent by an arduino.
I have two files, which I use separately to write some code and try differents things. In one of them, I read the data and I draw it using a matplotlib figure. After I finish using it, it remains connected to my computer and sending data. So, what i need to do is to "reset" the port. This is, close the opened port and open it again, and stop it from sending data so I can use the arduino to try some modifications in the code of this file.
So to accomplish this, i mean, to reset the port, i created another file and wrote this code:
import serial
print "Opening port"
try:
serial_port = serial.Serial("com4", 9600)
print "Port is open"
except serial.SerialException:
serial.Serial("com4", 9600).close()
print "Port is closed"
serial_port = serial.Serial("com4",9600)
print "Port is open again"
print "Ready to use"
But this code does not seems to work.The port is still connected and sending data. So, it means that I can not close the port with my code,and then reopen it again.
What am i doing wrong? How can I stop the arduino from sending data? Or how can I reset thw arduino, maybe?
Hope you can help me.
----- EDIT -----
I accomplish to identify the real problem that i am having, and it is not what i thought. The problem was not that the port was open despite that i use the closefunction that Pyserial have. The real thing is that the port is closing as I want, but the device (the arduino) is still sending data. So, i changed the code to reproduce the situation.
This is the code:
print "Abriendo puerto"
ser = serial
try:
ser = serial.Serial("com4", 9600, timeout = 1)
serial_port = "Open"
print "The port %s is available" %ser
except serial.serialutil.SerialException:
print "The port is at use"
ser.close()
ser.open()
while ser.read():
print "Sending data"
ser.setBreak(True)
time.sleep(0.2)
ser.sendBreak(duration = 0.02)
time.sleep(0.2)
ser.close()
time.sleep(0.2)
print "The port is closed"
exit()
With this code, what i do is:
1) I open the serial port
2) If the device is sending data, I print "Sending data"
3) After 1 sec, I try to close the port and stop the device from sending data
I tried these last two thing with the close function to close the port, and reading the docs I tried with setBreak and sendBreak as you can see in the code above (i left them on purpose). But the device is still sending the data, which means that the code does not work.
So, is there a way to tell the arduino "stop sending data", or can i reset the device?

I do a very similar thing, two ways with success.
The first way is to let the Arduino send data continuously. The problem here is when your python code wakes up and starts to read from the serial port, the Arduino might be anywhere in its procedures. The simple solution is to modify the Arduino code to send some kind of "restarting" line. All your python code needs to do in this case is wait for "restart", then read real data until it again sees "restart". I had noisy lines so my code read (and parsed) through multiple cycles to make sure it got good data.
resetCount = 0;
while resetCount < 3:
line = s.readline().rstrip("\r\n")
if string.find(line, "restart") != -1 :
resetCount += 1
elif resetCount > 0 :
fields = string.split(line, " ")
dict[fields[0]] = fields
The second way is to implement a command-response protocol with the Arduino, wherein the Arduino sends data only when requested. In this case your python code sends a command to the Arduino ("RT" in the example below) and then reads data from the Arduino until it sees a "completed" line or it times out.
dict = {}
regex = re.compile('28-[0-9A-Fa-f]{12}') # 28-000005eaa80e
s = serial.Serial('/dev/ttyS0', 9600, timeout=5)
s.write("RT\n");
while True:
line = s.readline().rstrip("\r\n")
print line
if string.find(line, "completed") != -1:
break;
fields = string.split(line)
if (regex.match(fields[0]) != None and len(fields) == 4) :
dict[fields[0]] = fields
s.close()

It is possible that when you close the port, data is still coming from the arduino and being buffered by the operating system. There is a short delay between your script calling close() and the device driver actually shutting stuff down.
An immediate re-open may allow the driver to carry on without resetting its buffer. This is the device driver buffer, not the one seen by the Python serial port instance.
If you wait for at least a couple of seconds after the call to close() before you try to call open() then the behaviour should be as you hope.
I have just spent most of the day working out that this is what had been preventing my code from working properly.

I think you have to do a serial_port.open() immediately after creation to actually open the port.
It also looks like it just opens the port and exits if successful. Maybe I'm missing something here. I've never used pySerial, I'm just going by the docs.

Try using the handle to close the port instead of invoking the constructor again.
If you the port is open and you call serial.Serial("com4", 9600) it will attempt to re-open the port again and fail.
If serial_port was assigned successfully then serial_port.close() should close it.

Clash of connection between serial port read and write codes

This is the code that reads from the z1 mote
while True:
if not ser.isOpen():
try:
ser = serial.Serial(z1port, z1baudrate,timeout=0, parity=serial.PARITY_NONE,
stopbits=serial.STOPBITS_ONE, bytesize=serial.EIGHTBITS)
except:
sys.exit("Error connecting device")
queue = ser.inWaiting()
if queue > 0:
data = ser.read(1000)
print data
time.sleep(0.2)
And this is the code that I have that I use it to write to the mote
# some event-driven code here so that whenever a message is received then do:
print(str(msg.payload))
ser = serial.Serial("/dev/ttyUSB1")
print ser.isOpen()
ser.write(msg.payload)
The output from the second code should be if msg.payload = "hello":
hello
True
But then the read code stops reading from the serial port (the code will run but no input). How do I solve this problem?

You can only create one serial connection to a device. The code in your question creates two connections, one in the main routine and one in the subroutine. In the main routine, you create a connection to establish communication with the device:
ser = serial.Serial(z1port, z1baudrate) # I assume z1port='/dev/ttyUSB1'
Then in your subroutine you also create a connection:
ser = serial.Serial("/dev/ttyUSB1")
So there are now two connections trying to use the same port. This will not work.
Instead, you should use the original connection throughout your program,
and define your subroutines to receive the connection as an input parameter. For example:
ser = serial.Serial(z1port, z1baudrate)
# do whatever to make connection to the device
getMessage(ser) # call subroutine to read data *with the existing connection*
ser.close() # close connection when finished
def getMessage(serConn):
# read data
data = serConn.read(1000)
# send ack
serConn.write(b'OK')
Another option is to open and close serial connections throughout your code, whenever you need to do communication. This is usually much less efficient, and only makes sense if there will only be intermittent communication with the device.

I used the idea by #mhopeng to write a code that implements multithreading programming, where one function handles the reading and the other handles the writing. And before they are both called, I will connect to the serial port and pass it to both of the threads.
I had to use multithreading because I needed a separate thread for writing at any time from the user input.

SerialTimeoutException in Python not working as expected

Sometimes I do not get any data in via the serial interface and I want to catch this case with an exception in the following way:
ser = serial.Serial(3)
ser.baudrate = 115200
timeout = 1
while (some condidion)
try:
dump = ser.read(40)
except ser1.SerialTimeoutException:
print('Data could not be read')
ser.close()
However, when I run this program, the exception is never caught when no
data is in the buffer and I am stuck in an endless loop. Anyone an idea
what I am doing wrong here?

I didn't even know about that exception. After having a look at the API documentation, you'll see that this exception only applies to write().
If you read(), you'll just have a shortened or even empty output.
And this applies only if you have opened the connection with a timeout. Something like serial.Serial(3, timeout=.1).

What is ssh equivalent read_until and read_very_eager methods at telnet?

I will write a SSH communicator class on Python. I have telnet communicator class and I should use functions like at telnet. Telnet communicator have read_until and read_very_eager functions.
read_until : Read until a given string is encountered or until timeout.
read_very_eager : Read everything that's possible without blocking in I/O (eager).
I couldn't find these functions for SSH communicator. Any idea?

You didn't state it in the question, but I am assuming you are using Paramiko as per the tag.
read_until: Read until a given string is encountered or until timeout.
This seems like a very specialized function for a particular high level task. I think you will need to implement this one. You can set a timeout using paramiko.Channel.settimeout and then read in a loop until you get either the string you want or a timeout exception.
read_very_eager: Read everything that's possible without blocking in I/O (eager).
Paramiko doesn't directly provide this, but it does provide primitives for non-blocking I/O and you can easily put this in a loop to slurp in everything that's available on the channel. Have you tried something like this?
channel.setblocking(True)
resultlist = []
while True:
try:
chunk = channel.recv(1024)
except socket.timeout:
break
resultlist.append(chunk)
return ''.join(resultlist)

Hi there even i was searching solution for the same problem.
I think it might help you ....
one observation, tell me if you find solution.
I wont get output if i remove 6th line.
I was actually printing 6th line to know the status, later i found recv_exit_status() should be called for execution of this code.
import paramiko,sys
trans = paramiko.Transport((host, 22))
trans.connect(username = user, password = passwd)
session = trans.open_channel("session")
session.exec_command('grep -rE print .')
session.recv_exit_status()
while session.recv_ready():
temp = session.recv(1024)
print temp
1.Read until > search for the data you are searching for and break the loop
2.Read_very_eager > use the above mentioned code.