We Keep Coding

Is it possible to call the Python serial library as a function in an imported module when communicating with an Arduino?

I am writing a program to communicate between my laptop (Python) and an Arduino. The Arduino code, with which I have no issue, reads the serial data form my laptop and returns a reply. The code below works when I am calling the function which starts the serial communication from within the same file. However, when I import the file as a module in another file, using lal the same commands, it does not work!
To provide more detail, although Python thinks it has connected and even prints out the correct port number, it does not really connect. I know this because in the scenario that does work, when the serial communication is open, the Arduino IDE cannot speak to the Arduino as the Arduino is busy. However in the scenario which is not working, even after Python thinks it has opened serial communication, the Arduino IDE can still communicate with the Arduino.
Is there a way to pass the ser variable when called from a function in an imported module?
def connect():
for n in range(0,21):
try:
ser = serial.Serial('COM'+str(n), 115200 ,timeout=0.1)
status=1
port=n
return ser,port,status
except:
pass
time.sleep(0.05)
return 0, 0, 0
if __name__ == "__main__":
ser,port,status=connect()
n=0
while n<3:
num = input("Enter a word: ") # Taking input from user
ser.write(bytes(num, 'utf-8'))
time.sleep(0.05)
data = ser.readline()
print(data) # printing the value
n+=1
ser.close()
print('closed')

I have found the reason my code was not working! Notice in the code I posted, I use the input function to get a user input which is sent to the Arduino. This effectively results in a delay. In the scenario that was not working, I did not use the input function and so my code went straight from serial.serial to serial.write. The Arduino runs at 16 MHz and just couldn't keep up! All I needed to do was add a delay and now it works!

Python pyserial one write delay

I'm having a weird issue with pyserial, using Python 3.6.9, running under WSL Ubuntu 18.4.2 LTS
I've set up a simple function to send GCODE commands to a serial port:
def gcode_send(data):
print("Sending: " + data.strip())
data = data.strip() + "\n" # Strip all EOL characters for consistency
s.write(data.encode()) # Send g-code block to grbl
grbl_out = s.readline().decode().strip()
print(grbl_out)
It sort of works, but every command I send is 'held' until the next is sent.
e.g.
I send G0 X0 > the device doesn't react
I send G0 X1 > the device reacts to G0 X0
I send G1 X0 > the device reacts to G0 X1
and so on...
My setup code is:
s = serial.Serial(com, 115200)
s.write("\r\n\r\n".encode()) # Wake up grbl
time.sleep(2) # Wait for grbl to initialize
s.flushInput() # Flush startup text in serial input
I can work around the delay for now, but it's quite annoying and I can't find anyone else experiencing the same. Any idea what could be causing this?

There might be a lot of problems here, but rest assured that the pyserial is not causing it. It uses the underlying OS's API to communicate with the UART driver. That being said you first have to test your code with real Linux to see whether WSL is causing it. I.e. whether a Linux and Windows UART buffers are correctly synced.
I am sorry that I cannot tell whether a problem is in your code or not because I do not know the device you are using, so I cannot guess what is happening on its end of communication channel. Have in mind that Windows alone can act weirdly in best of circumstances, so, prepare yourself for some frustrations here. Check your motherboard or USB2Serial converter drivers or whatever hw you are using.
Next thing, you should know that sometimes, communication gets confusing if timeouts aren't set. Why? Nobody really knows. So try setting timeouts. Check whether you need software Xon/Xoff turned on or not, and other RS232 parameters that might be required by the device you are communicating with.
Also, see what is going on with s.readline(), I wouldn't personally use it. Timeouts might help or you can use s.read(1024) with timeouts. I do not remember right now, but see whether pyserial supports asynchronous communication. If it does, you can try using it instead of standard blocking mode.
Also, check whether you have to forcefully flush the serial buffer after s.write() or add a sleep after it. It might happen that the device doesn't get the message but the read request is activated. As the device didn't receive the command it doesn't respond. After you send another command, IO buffer is flushed and the previous one is delivered and so forth. Serial communication is fun, but when it hits a snag it can be a real P in the A, believe me.
Ow, a P.S. Check whether the device sends "\r\n\r\n" or "\r\n" only, or "\r" or "\n" in response. s.readline() might get confused. For a start, try putting there two s.readline()s one after another and print out each output. If the device sends double EOL then the one s.readline() is stopping on the empty line and your program receives an empty response, when you send another command s.readline() goes through the buffer and returns a full line that is already there but not read before.

Here it goes. The code promissed in the comment. Big portions of it removed and error checks too.
It is a typing terminal for using PyS60 Python console on Nokia smartphones in the Symbian series via bluetooth. Works fantastically.
from serial import *
from thread import start_new_thread as thread
from time import sleep
import sys, os
# Original code works on Linux too
# The following code for gettin one character from stdin without echoing it on terminal
# has its Linux complement using tricks from Python's stdlib getpass.py module
# I.e. put the terminal in non-blocking mode, turn off echoing and use sys.stdin.read(1)
# Here is Win code only (for brevity):
import msvcrt
def getchar ():
return msvcrt.getch()
def pause ():
raw_input("\nPress enter to continue . . .")
port = raw_input("Portname: ")
if os.name=="nt":
nport = ""
for x in port:
if x.isdigit(): nport += x
port = int(nport)-1
try:
s = Serial(port, 9600)
except:
print >> sys.stderr, "Cannot open the port!\nThe program will be closed."
pause()
sys.exit(1)
print "Port ready!"
running = 1
def reader():
while running:
try:
msg = s.read()
# If timeout is set
while msg=="":
msg = s.read()
sys.stdout.write(msg)
except: sleep(0.001)
thread(reader,())
while 1:
try: c = getchar()
except Exception, e:
running = 0
print >> sys.stderr, e
s.write('\r\n\x04')
break
if c=='\003' or c=='\x04':
running = 0
s.write('\r\n\x04')
break
s.write(c)
s.close()
pause()

Multitasking Python-Arduino Communication

I am doing a project where I have to acquire data from an Arduino Uno connected to various sensors using Python on Windows (and eventually in Raspberry Pi 2 Model B).
As Python would constantly be listening to Arduino, I was thinking how should I let Python be always ready to read in a user input (any keyboard keys or even better, with a push-button if this is possible) after which Python will tell Arduino Uno to re-acquire data from all the sensors (i.e. refresh data from the sensors) and print it on the Python Console. I am currently using PyCharm with pySerial for communications between Arduino and Python. My code (without considering user input) is as of below:
import sys
import serial
import time
arduino = serial.Serial('COM3', 9600, timeout=1)
time.sleep(3) # wait for Arduino to initialize
def readData(): # reads inputs from Arduino
try:
datastring = arduino.readline()
print datastring
except:
pass
while True:
readData()
strin = 'p'
arduino.write(strin.encode()) # tell arduino a phase shifter setting has been finished
strin = 's'
arduino.write(strin.encode())
arduino.close()
time.sleep(0.5) # waits for 0.5 s
# print('Data to be transfered: %s'%ASCIIdata)
I understand that related topics have been posted before, however, I have tried the solutions online to no avail and I am finding it quite difficult to code as I am very new to Python. (I am reading documentation and examples and trying to code at the moment.)
Currently, I have done some research on this topic and have tried the following methods but to no avail:
Using msvcrt, getch and using nodelay(1) to allow my program to run even when no user input is received. I used "stdscr", and curses, to which I used "stdscr = curses.initscr()", however, in PyCharm, as it is an IDE, it returned an error "Redirection is not supported." My attempt at implementing this is as of below:
def readData(): # reads inputs from Arduino
try:
datastring = arduino.readline()
print datastring
except:
pass
while True:
stdscr = curses.initscr()
stdscr.nodelay(1)
user_input = stdscr.getch()
if user_input == -1:
readData()
I understand there is multithreading in Python and a number of leads lead me to think that it should be the way to go if I want to listen to user inputs continuously while still acquiring data from Arduino. May I ask if this is the correct direction in which I should pursue?
I have heard about select() and poll() though the former isn't supported on Windows. I will try select() when I port my program over to Raspberry Pi which runs on Linux.
May I ask if anyone can point me to a direction on how to implement a program that reads data continuously from Arduino in PyCharm, and but a user input will cause everything to stop (like an interrupt) and have Python (PyCharm) ask Arduino to pass it a set of fresh sensor data?
Thank you! :)

Reset an open serial port

I am reading data from a serial port, sent by an arduino.
I have two files, which I use separately to write some code and try differents things. In one of them, I read the data and I draw it using a matplotlib figure. After I finish using it, it remains connected to my computer and sending data. So, what i need to do is to "reset" the port. This is, close the opened port and open it again, and stop it from sending data so I can use the arduino to try some modifications in the code of this file.
So to accomplish this, i mean, to reset the port, i created another file and wrote this code:
import serial
print "Opening port"
try:
serial_port = serial.Serial("com4", 9600)
print "Port is open"
except serial.SerialException:
serial.Serial("com4", 9600).close()
print "Port is closed"
serial_port = serial.Serial("com4",9600)
print "Port is open again"
print "Ready to use"
But this code does not seems to work.The port is still connected and sending data. So, it means that I can not close the port with my code,and then reopen it again.
What am i doing wrong? How can I stop the arduino from sending data? Or how can I reset thw arduino, maybe?
Hope you can help me.
----- EDIT -----
I accomplish to identify the real problem that i am having, and it is not what i thought. The problem was not that the port was open despite that i use the closefunction that Pyserial have. The real thing is that the port is closing as I want, but the device (the arduino) is still sending data. So, i changed the code to reproduce the situation.
This is the code:
print "Abriendo puerto"
ser = serial
try:
ser = serial.Serial("com4", 9600, timeout = 1)
serial_port = "Open"
print "The port %s is available" %ser
except serial.serialutil.SerialException:
print "The port is at use"
ser.close()
ser.open()
while ser.read():
print "Sending data"
ser.setBreak(True)
time.sleep(0.2)
ser.sendBreak(duration = 0.02)
time.sleep(0.2)
ser.close()
time.sleep(0.2)
print "The port is closed"
exit()
With this code, what i do is:
1) I open the serial port
2) If the device is sending data, I print "Sending data"
3) After 1 sec, I try to close the port and stop the device from sending data
I tried these last two thing with the close function to close the port, and reading the docs I tried with setBreak and sendBreak as you can see in the code above (i left them on purpose). But the device is still sending the data, which means that the code does not work.
So, is there a way to tell the arduino "stop sending data", or can i reset the device?

I do a very similar thing, two ways with success.
The first way is to let the Arduino send data continuously. The problem here is when your python code wakes up and starts to read from the serial port, the Arduino might be anywhere in its procedures. The simple solution is to modify the Arduino code to send some kind of "restarting" line. All your python code needs to do in this case is wait for "restart", then read real data until it again sees "restart". I had noisy lines so my code read (and parsed) through multiple cycles to make sure it got good data.
resetCount = 0;
while resetCount < 3:
line = s.readline().rstrip("\r\n")
if string.find(line, "restart") != -1 :
resetCount += 1
elif resetCount > 0 :
fields = string.split(line, " ")
dict[fields[0]] = fields
The second way is to implement a command-response protocol with the Arduino, wherein the Arduino sends data only when requested. In this case your python code sends a command to the Arduino ("RT" in the example below) and then reads data from the Arduino until it sees a "completed" line or it times out.
dict = {}
regex = re.compile('28-[0-9A-Fa-f]{12}') # 28-000005eaa80e
s = serial.Serial('/dev/ttyS0', 9600, timeout=5)
s.write("RT\n");
while True:
line = s.readline().rstrip("\r\n")
print line
if string.find(line, "completed") != -1:
break;
fields = string.split(line)
if (regex.match(fields[0]) != None and len(fields) == 4) :
dict[fields[0]] = fields
s.close()

It is possible that when you close the port, data is still coming from the arduino and being buffered by the operating system. There is a short delay between your script calling close() and the device driver actually shutting stuff down.
An immediate re-open may allow the driver to carry on without resetting its buffer. This is the device driver buffer, not the one seen by the Python serial port instance.
If you wait for at least a couple of seconds after the call to close() before you try to call open() then the behaviour should be as you hope.
I have just spent most of the day working out that this is what had been preventing my code from working properly.

I think you have to do a serial_port.open() immediately after creation to actually open the port.
It also looks like it just opens the port and exits if successful. Maybe I'm missing something here. I've never used pySerial, I'm just going by the docs.

Try using the handle to close the port instead of invoking the constructor again.
If you the port is open and you call serial.Serial("com4", 9600) it will attempt to re-open the port again and fail.
If serial_port was assigned successfully then serial_port.close() should close it.

Multiple writes to file

I have the following python code that expects data coming from the serial port, and writes it to a file.
import time
import serial
def write_log ( text ):
f = open('logger.log', 'a')
f.write( text )
f.close()
ser = serial.Serial()
ser.port = "/dev/ttyS0"
ser.baudrate = 4800
ser.open()
if ser.isOpen():
while 1:
while ser.inWaiting() <= 0:
time.sleep(1)
response = ser.read(ser.inWaiting())
if len ( response ):
write_log( response )
print response
It works to an extent, as after some time, it starts to hang, bringing the CPU all the way up, and not writing anything (or sometimes writing only pieces of text) to the .log file.
The process here is pretty intensive, as my serial port will be writing an 8 bytes string every second, and this python script is supposed to then receive it, and write its contents to the log file.
I'm thinking the problem here is the fact that I'm opening and closing the file too much, and this is somehow making the whole process slow. I'm no python wizz, so any help or advice on improving this code would be greatly appreciated.
Thanks in advance,

You have an infinite loop in your code, and you don't break out from it when there is a problem - or the serial device is no longer open.
Probably use:
while ser.isOpen():
while ser.inWaiting() <= 0:
time.sleep(1)
response = ser.read(ser.inWaiting())
if len(response):
write_log(response)
print response
or even:
while ser.isOpen() && ser.inWaiting() <= 0:
time.sleep(1)
response = ser.read(ser.inWaiting())
if len(response):
write_log(response)
print response
I'm not sure about the sleep, either; you'd do better just waiting in the read for data to become available.
As I think about it, not knowing the methods available in the serial class, the more I think the main loop should be attempting to read from the serial device, hanging happily if there is nothing currently available, and only terminating when the input method indicates there is no more input to come - the device has been closed on you, or has failed in some way.

I think the problem here is that you are polling for updates.
Python's serial.read() function actually blocks the current thread until something becomes readable on the thread, until timeout. What you should do, therefore, is break out another thread to handle serial IO. Have it loop indefinitely, checking a condition (the master thread wants you to stay listening and the serial port is still available). This thread will do something like:
while ser.isOpen() && thisthread_should_keep_doing_this:
response = ser.read(ser.inWaiting())
if len(response):
write_log(response)
print response
Then, when you want it to exit, your master thread sets thisthread_should_keep_doing_this=False and when your slave thread has finished reading, it kills itself.
Two things:
Do have the read timeout relatively frequently.
Do not "remote kill" the thread. Pass it a message and have it kill itself. Killing threads remotely creates an awful mess.
See http://pyserial.sourceforge.net/examples.html#miniterm