We Keep Coding

Arduino Serial Buffer Flooding When Reading is Delayed using Pyserial

I'm working on a project where an RFID-RC522 reader reads an RFID card, pulls that read number in, and relays in out via serial. The code for that is as follows:
#include <SPI.h>
#include <RFID.h>
#define SS_PIN 10
#define RST_PIN 9
RFID rfid(SS_PIN, RST_PIN);
String rfidCard;
void setup() {
Serial.begin(9600);
//Serial.println("Starting the RFID Reader...");
SPI.begin();
rfid.init();
}
void loop() {
if (rfid.isCard()) {
if (rfid.readCardSerial()) {
rfidCard = (String(rfid.serNum[0]) + String(rfid.serNum[1]) + String(rfid.serNum[2]) + String(rfid.serNum[3]));
Serial.print(rfidCard);
}
rfid.halt();
}
}
That part works great. On the python side, the following code runs on my computer and reads everything in fine:
import serial
import time
connected = False
#This port below is specific to my usb port I think
port = '/dev/cu.usbserial-14620'
baud = 9600
serial_port = serial.Serial(port, baud, timeout=0)
while True:
reading = serial_port.readline().decode()
if reading != '':
print(reading)
So the end result when I plug in the RFID reader to my arduino Nano, and the Nano into my computer, and run the Python code is: When I put an RFID card up to the reader, it continuously prints the number, and when I pull it away, it prints nothing. Looks like this:
786179181
786179181
786179181
786179181
786179181
786179181
Exactly what I want.
Here's where the problems start. I want to introduce a period of time in the python script where it won't read anything. A sort of delay where the RFID reader is essentially not useable. This code is a minimally reproducible example of the effect I'm trying to get:
i = 5
while i > 0:
print(i)
time.sleep(1)
i-=1
When I add this code right before the Serial initialization line, it prints a countdown from 5 to 1, and then opens the port and starts reading and printing. But it looks like if I place the card on the reader before the countdown is up, it adds all that it reads to a buffer, and then spams it all at once, once the port is allowed to print again. What's most strange to me is that it actually begins to spam it in a loop, even when I take the card away. And from here on out, the arduino is now stuck in this state of spamming the data - even if I terminate the python script and rerun it, and dont put any card up to the reader, it's still spamming that old data in an infinite loop. It Looks more like this, stacking into one line at times:
786179181786179181786179181786179181
786179181
786179181786179181786179181786179181786179181
I've tried using commands like flush(), reset_input_buffer(), and reset_output_buffer(), but they don't seem to do what I want, or maybe I don't completely understand how they work. I'm not entirely sure if my problem lies in the arduino code output or the python code input.

Alright, I was able to get this to work using the following methods.
In the arduino, the write code only executes if it's receiving something from python:
#include <SPI.h>
#include <RFID.h>
#define SS_PIN 10
#define RST_PIN 9
RFID rfid(SS_PIN, RST_PIN);
String rfidCard;
void setup() {
Serial.begin(9600);
SPI.begin();
rfid.init();
}
void loop() {
if (rfid.isCard()) {
if (rfid.readCardSerial()) {
if (Serial.available()){
rfidCard = (String(rfid.serNum[0]) + String(rfid.serNum[1]) + String(rfid.serNum[2]) + String(rfid.serNum[3]));
Serial.println(rfidCard);
}
}
rfid.halt();
}
}
In python, it sends data to the arduino only when I want to receive data back, i.e., after the delay has passed:
import serial
import time
port = '/dev/cu.usbserial-14620'
baud = 9600
serial_port = serial.Serial(port, baud, timeout=1)
i = 5
while i > 0:
print(i)
time.sleep(1)
i-=1
while True:
serial_port.write(b'1')
try:
reading = serial_port.readline().decode().rstrip()
print(reading)
except:
pass
This has the desired effect of printing one line of data at a time without any collected buffer data.
Thank you everyone for you assistance!

Python and Arduino Serial Communication: send and receive

What I want to do is actually query Arduino to get data. When I send a command from Python to Arduino, Arduino will send back some data depending on what the query keyword is. I am able to do this but with a very low speed, I wish to have a more efficient way to do this, for example, query at least 100 times per second. Any idea? Thanks!
The code I use:
Arduino:
int temp=1;
char s[2];
int speedIn=0;
void setup() {
Serial.begin(115200);
}
void loop() {
if(Serial.available()>0){
Serial.readBytes(s,1);
speedIn=int(s[0]);
if(speedIn==49){
Serial.println(">Data1Send<");
speedIn=0;
}
else if(speedIn==50){
Serial.print(">");
Serial.print(temp);
Serial.println("<");
speedIn=0;
}
else{
Serial.println(">Wrong Data<");
speedIn=0;
}
}
if(temp<300){
temp=temp+1;
}
//delay(100);
}
Python:
import serial
import time
if __name__=="__main__":
sc = serial.Serial('/dev/tty.usbmodem1421',timeout=None,baudrate=115200)
sc.flush()
while True:
#s=raw_input()
s=str(50)
if len(s) >0:
if s.isdigit():
sc.write(chr(int(s)))
time.sleep(1.0)
if sc.inWaiting()>0:
print(sc.read(sc.inWaiting()))
If I change the time delay in Python from 1.0sec to 0.1sec, it will not work any more (no data can be received and just in waiting state).

It looks like your python code does this:
Send a command
Sleep for some time (presumably to give the arduino time to reply)
Read the reply
If no reply, repeat from step 1
You mentioned that it doesn't work if the sleep period is too short. I wonder if the following could be happening. Imagine there were no sleep period. The python code will immediately check for a reply, which may not exist because the arduino hasn't had time to respond. In that case, it will immediately try to send another command. This could keep happening repeatedly--the python code could essentially flood the line with requests that the arduino can't keep up with. You prevent this from happening using the sleep period, but the sleep period probably takes longer than the arduino actually needs, which would reduce your data rate.
If that's the case, it might work to do this in the python code instead:
Send a command
Wait for the arduino to reply (but not using sleep). Ideally, use a read command that blocks until input is received. A less preferable alternative is to busywait--enter a loop that does nothing but check whether a reply has been received and terminates when input is available.
Read the reply
Repeat from step 1

sending serial commands to arduino

I've been trying to find a way to send serial commands from my PC (Windows 7) to an Arduino Uno R3. I've been working on this simple program, where you're supposed to send a simple "1" through the USB cable, in order to turn on the onboard LED. My Arduino code should be working (what I know of, but I'll make sure to upload it).
I've been trying to send this command using Python and pySerial, but I can't seem to get pySerial to work. Also I've tried using CMD, but it seems like CMD freezes when i type in my command (ECHO 1 > COM3 BAUD:9600).
I'm the adminstrator of my PC.
This is my arduino coding
int var = 0;
int LEDPin = 13;
int val = 0;
void setup() {
Serial.begin(9600);
pinMode(LEDPin, OUTPUT);
}
void loop() {
if (Serial.available()>0){
val = Serial.read();
}
if (val == 1){
digitalWrite(LEDPin, HIGH);
}
digitalRead(LEDPin);
if (LEDPin==HIGH){
var ++;
delay (1000);
if(var==10){
digitalWrite(LEDPin,LOW);
var = 0;
val = 0;
}
}
}
And the short Python program I've got :)
import serial
ser = serial.Serial('COM3', 9600, timeout=0)
while 1:
var = 1
ser.write(var)
delay(12000)
With Kind Regards
Michael Vedel.

just to be sure - is Arduino really connected to COM3?
In order to check serial connection manually you can use for example Putty. With help of it you can connect to the needed serial port and send command manually. And in order to check whether Arduino received it, you can add Serial.println(val); just after val = Serial.read();, with this Arduino should sends back to serial whatever characters it receives.

Does the code work when you type a '1' into the Serial Monitor in the Arduino IDE? If so, check the COM port. On my PC (also Windows 7), COM3 is never the Arduino port, as it is used internally for something else.
By the way everything after the digitalRead() statement will do nothing, since you're comparing LEDpin (which is declared as 13) to HIGH (which is defined as 1). I think you mean to see if the digitalRead output is HIGH, but I'm not sure that will work, either, since the pin is declared as an OUTPUT. You'd be much better off putting that logic in the (val == 1) section.

just save this code as .bat and make sure that you are using com 6.
This code lets you enter and sends your commands to serial port at 9600bpm
mode com6:9600,N,8,1
#echo off
:start
cls
Set /p commands="enter command:"
echo|set /p= %commands% >com6
goto start
Now you have to read serial using this in Arduino
Serial.read();

I think the root problem is in the Python code in the call to write. The argument has to be bytes (or something compatible). You're passing an integer value. You need to explicitly convert that value to a bytes object. I'm not a Python expert, but I think one way to do that is to replace:
var = 1
with
var = b'\x01'
Oh, and you might need to call flush as well.
BTW, your experiment with ECHO doesn't do what you intended. It sends the character value for '1', which is 49. Since 49 != 1, your Arduino code won't respond to it.
I find the easiest way to test serial commands is to use the Arduino Serial Monitor, which will have all the right defaults as long as you match the baud rate. This will enable you to ensure the Arduino code is right before debugging the Python code. (Then again, it may be difficult to send control characters with the Serial Monitor, so consider using printable text in your protocol. For example 'A' for on and 'B' for off. Printable text can be easy to work with among all these tools (Python, C, Serial Monitor, ECHO, etc.).)

do this it works better for me, just change the com4 for the port you have connected to
#echo off
:start
cls
set /p commands="enter command:"
mode com4 baud=115200 parity=n data=8 stop=1 to=on xon=off odsr=off octs=off dtr=off rts=off idsr=off && echo %commands% > com4
cls
goto start

I think Arduino is stable and easy enough to handle the data from serial, the cause of that as I used same library with Pyserial in python 3 was that event of reading the serial get fired once which is way faster than Arduino board and to solve that you should use a thread method that runs in the background and make it always try to read the incoming data from serial.
I've made this function that recieved the opened serial :
def read_from_port(ser):
while True:
try:
reading = ser.read().decode()
handle_data(reading)
except EXCEPTION:
print("error while decoding" + str(EXCEPTION))
after I have got the serial from the list and assigned it with the desired baud rate and send it to the Thread :
serial_me = serial.Serial(serial_port1, 115200, timeout=3)
try:
thread_x = threading.Thread(target=read_from_port, args= (serial_me,))
thread_x.daemon = True
thread_x.start()
except EXCEPTION:
print('couldn\'t start thread')
print(Exception)

Random character at the beginning of serial.readline() in python and arduino

I get random characters when I perform a serial.readline(), sometimes it is other umbers and sometimes it is whole messages. The output should be "1,2" or "2,2"
Here are a couple of screenshots of the output of serial.readline().
I have tried to put a delay at before serial.readline() but it did not make a difference.
There is usually a strange character at the beginning:
I have also received strange messages:
There is a problem later on in the program that causes the program to hand because sometimes I just receive a blank line.
Is there a way to get consistent output from serial?
Here is the arduino code:
void setup() {
// initialize the LED pin as an output:
pinMode(ledPin, OUTPUT);
// initialize the pushbutton pin as an input:
pinMode(buttonPin, INPUT);
Serial.begin(9600);
}
void loop(){
// read the state of the pushbutton value:
buttonState = digitalRead(buttonPin);
// check if the pushbutton is pressed.
// if it is, the buttonState is HIGH:
if (buttonState == HIGH) {
// turn LED on:
digitalWrite(ledPin, HIGH);
Serial.println("1,2");
}
else {
// turn LED off:
digitalWrite(ledPin, LOW);
Serial.println("2,2");
}
}
And here is the python code:
ser = serial.Serial('/dev/ttyUSB0', 9600)
line=ser.readline()
coord= line.strip()
print coord
EDIT:
I tried putting ser.flushInput() after the ser.open() and I get the same output.

Have you flushed the serial buffer
ser.flushInput()

I've been having the same issue when interfacing between pyserial and Arduino. This may be an old post, but in case someone else is having the same trouble, I remedied my problem by inserting:
ser.flushInput()
...right before my ser.readline() call.

How to Read serial data from an Arduino in Linux
Nothing fancy here. I’m getting the current port configuration, setting the input/output speed to 9600 baud, setting the data expectations to be 8 bits per word, with no parity bit and no stop bit, setting canonical mode, and then committing those changes back to the serial port.
If I am not mistaken you have to change the mentioned settings when connecting through serial port.
You do not mention it, but I guess you are using the pySerial library. Regardless you can use it with the correct settings to connect through serial. The constructor of the API allows all the parameters which are noted below:
Pyserial Library
I have not tested this approach as I have had a similar problem in C not Python.

What happens if you break it down to something simpler?
On the Arduino:
void setup()
{
Serial.begin(9600);
}
int i = 0;
void loop()
{
delay(1000);
Serial.print("Hello World ");
Serial.println(i);
i++;
}
And in the Python REPL...
import serial
ser = serial.Serial('/dev/ttyUSB0', 9600)
while(1):
ser.readline()
The first behavior you should observe is that when you initiate the serial connection, the Arduino reboots. This takes a few seconds.
The second behavior you should observe is that (if you're typing this in the REPL slowly) when the while(1) loop begins, you get all of the serial data that had accumulated since you initiated the serial connection. It takes me a few seconds to type all that out, so when I hit Enter after ser.readline() I see:
'Hello World 1\r\n'
'Hello World 2\r\n'
'Hello World 3\r\n'
I only mention this to make sure you're familiar with two things that burned me a bit the last time I messed with serial communication to an Arduino: it needs time to reboot after connecting, and the serial data buffers. readline() will only give you one line in the buffer - not everything that's in the buffer.
Is it possible you're trying to sent/receive data before it's done rebooting? What about button bounce - could a dozen high/low state detections be messing something up?

Control a specific pin on the Arduino Uno board using pyserial

I have a python code that sends in a pattern, in which a light has to blink in. (say eg. 101010. pattern may vary every time the code is run). when it is executing this infinitely i want an interrupt( again sent by the python code )to save the present conditions of the lights (say it is running 1 of the sequence) and perform a specific task like turn off the lights for 10 seconds and then resume the sequence.
one way of doing this is by interrupting the program by making the interrupt pin high. The question is can this making of high/low controlled by the pyserial.
So a simple pseudo code would be :
PYTHON part of the code:
Read the sequence:
Send the sequence to the arduino board using pyserial.
while(1)
{
Run a timer for 15 second.
When the timer overflows interrupt the arduino.
}
ARDUINO part of the code :
Read the sequence
while (1)
{
keep repeating the sequence on the LED.
}
// if interrupted on pin2 // assuming pin2 has the interrupt procedure
// Pyserial has to enable the interrupt WITHOUT using a switch for enabling the pin.
ISR
{
Save the present state of execution.
Turn off the LED.
}
FOR A BETTER UNDERSTANDING :
I built up small codes to show the doubts i had :
CODE FOR THE ARDUINO IS :
int ledpin1 = 13;
int speedy;
int patterns;
void setup()
{
Serial.begin(9600);
Serial.print("Program Initiated: \n");
pinMode(ledpin1,OUTPUT);
//activate the blackout ISR when a interrupt is achieved at a certain pin. In this case pin2 of the arduino
attachInterrupt(0,blackout,CHANGE);
}
void loop()
{
if (Serial.available()>1)
{
Serial.print("starting loop \n");
patterns = Serial.read();
patterns = patterns-48;
speedy = Serial.read();
speedy = (speedy-48)*1000;
while(1)
{
patterns = !(patterns);
Serial.print(patterns);
digitalWrite(ledpin1,patterns);
delay(speedy);
}
}
}
/*
void blackout()
{
// ***Save the present state of the LED(on pin13)***
Serial.print ("The turning off LED's for performing the python code\n");
digitalWrite(ledpin,LOW);
//wait for the Python code to complete the task it wants to perform,
//so got to dealy the completion of the ISR
delay(2000);// delay the whole thing by 2 seconds
//***Continue with the while loop by setting the condition of the light to the saved condition.***
}
*/
==================================================================================
CODE FOR THE PYTHON FRONT IS :
import serial
import time
patterns=1
speedy=1
ser = serial.Serial()
ser.setPort("COM4")
ser.baudrate = 9600
ser.open()
def main():
if (ser.isOpen()):
#while(1):
ser.write(patterns)
ser.write(speedy)
blackoutfunc()
#ser.close()
def blackoutfunc():
while(1):
time.sleep(2)
print "Performing operations as required"
===============================================================================
Now the questions I had :
1) Is there a way to be able to activate the "blackout ISR" depending on the conditions of a pin(in this case pin2 which is the INT0 pin) without using a physical switch present on the pin. Hence the pin state has to be manipulated by the software.
2) Is it possible to perform the operations as mentioned in the comments of the blackout functions?
3) In the python code is it possible to just send in the data(i.e. patterns,speedy) only once and make the arduino perform the pattern in a infinite way without again sending the data by the serial.write command. Hence avoiding the while(1) loop after the ser.isOpen().

Have a look at this:
https://github.com/ajfisher/arduino-command-server
It's something I pulled together on the Arduino side to issue arbitrary commands like switch a pin high / low and set PWM levels etc. It works over both serial and network though it's a touch buggy on the network side at the moment.
To use it, put the code on your arduino then you just write a python script (or any other language that can use a serial connection) to connect over the serial connection and then tell it what you want to do eg DIGW 1 HIGH etc
Also have a look at: https://github.com/ajfisher/arduino-django-visualiser which is where I use a variation of this library to control some LEDs based on some things going on in Django - it's more heavily python based.