How are virtual COM/Serial ports managed by operating system? - python

I'm currenty working on an interface between arduino and raspberry pi. The arduino is connected via usb cable to the Rasbperry Pi. The Arduino has an LED connected to a digital Pin that may be turned on and off by sending specified bytes over the USB serial connection. The Raspberry Pi then runs the following python script:
import serial
import time
arduino = serial.Serial('/dev/ttyACM0', baudrate=115200, timeout=.1)
time.sleep(2)
start = time.time()
for _ in range(10):
arduino.write(bytes([0x58])) //Turns LED on (on Arduino)
print(time.time()-start,'on')
time.sleep(.5)
arduino.write(bytes([0x18])) //Turns LED off (on Arduino)
print(time.time()-start,'off')
time.sleep(.5)
arduino.write(spam_the_COM_port) # will let the arduino start sending
# infinite amout of bytes
print('started spamming')
for _ in range(10):
arduino.write(turn_led_on) #Actual byte is 0x58
print(time.time()-start,'on')
time.sleep(.5)
arduino.write(turn_led_off) #Actual byte is 0x18
print(time.time()-start,'off')
time.sleep(.5)
So I'm switching on and off an LED that is connected to an arduino by sending it the specified bytes: 0x58 and 0x18 The Arduino runs code that looks like this:
bool spamming = false;
void setup()
{
Serial.begin(115200);
pinMode(24, OUTPUT);
}
void loop()
{
while(Serial.available() > 0)
{
unsigned char x = Serial.read();
if (x == 0x58)
{
digitalWrite(24, HIGH); //Turn on LED
}
else if (x == 0x18)
{
digitalWrite(24, LOW); //Turn off LED
}
else if (x == 0x00)
{
spamming = true;
}
}
if (spamming)
{
if (Serial.avialiableForWrite() > 0) Serial.write(0x01);
}
}
I get the expected result (which is just the console printing on and off 10 times (on the raspberry side) and the Arduino actually turning the LED on and off 10 times in a row.
I then send a command that will make the Arduino start sending an infinite amount of bytes to the Raspberry Pi whilst still checking the Serial incoming buffer for new commands. After that I repeat the same process as before and try to send the LED switching command.
Now the python console output stays the same (runs without any error raising), yet the program blocks at each arduino.write() command for a long time (around 10s or longer) even though there is a timeout specified (0.1s). After blocking, the console prints as usual and the LED on the arduino switches as usual. I just added timesamps to the python console output and the results look like this:
2.002582311630249 on
2.502887725830078 off
3.003848075866699 on
3.5050783157348633 off
4.005117177963257 on
4.5058839321136475 off
5.006282806396484 on
5.50661826133728 off
6.008156776428223 on
6.509922504425049 off
7.010948181152344 on
7.512096166610718 off
8.012341737747192 on
8.513236999511719 off
9.013462543487549 on
9.514604091644287 off
10.015231370925903 on
10.515642404556274 off
11.01583743095398 on
11.516374349594116 off
started spamming
12.018024444580078 on
12.518478155136108 off
13.01881217956543 on
13.519550085067749 off
28.621582508087158 on
44.68230104446411 off
57.40802550315857 on
63.064852714538574 off
65.58953928947449 on
68.61984610557556 off
77.50833535194397 on
96.19384908676147 off
102.6581130027771 on
109.2223105430603 off
132.55511474609375 on
155.684588432312 off
171.44103860855103 on
185.78372859954834 off
199.51972341537476 on
216.58949422836304 off
So the problem must be on the raspberry pi. I tried the same code on my windows machine (plugging the arduino in there of course) and the same thing happened, which led to the conclusion that it must have something to do with the virtual COM/Serial port.
So now to the actual question: Why does it take so long for the serial.write() function to write something to the output buffer (on the pi) when the input buffer (on the pi) is beeing spammed by the arduino?
As far as I know, a UART Chip has two physically seperate buffers that are completely independent. Does a virtual COM port not have this independence of the two buffers? (Maybe just some allocated memory for both of them in a stack-heap kind of way?) Or is the problem that USB 1.0 and 2.0 are only half-duplex?
(And then lastly is there a way to fix the fucntion block?)
[update]
I managed to stop the function from blocking by adding
if arduino.inWaiting() > 1000: arduino.flushInput()
before every arduino.write() statement. I'm still interested in how the Serial/COM port is managed by the operating system, be it linux/raspbian or windows.

Related

Interface Arduino/RedBoard vibrator to Python

I am trying to interface my SparkFun Qwiic Haptic Driver - DA7280 with Python3. My current set-up is as follows:
PC -USB to micro-USB-> SparkFun RedBoard Qwiic -Qwiic Cable-> Haptic Driver
I've trialed the accompanying Arduino sketch & managed to get the C++ code up and running fine; modulating the vibrator's intensity & frequency just fine.
Then, what I would like to do is to trigger a vibration pulse in time with some Python code. Such that, when python prints out a word, for example, a vibratory impulse would be triggered.
I have tried using pySerial to interface with the microcontroller, trigger the controller to run a pre-loaded script. This was worked fine with a simple C++ script to repeat an LED blink uploaded to the micro-controller:
/*
Blink
Turns an LED on for one second, then off for one second, repeatedly for 6 seconds
*/
void setup() {
// initialize digital pin LED_BUILTIN as an output.
pinMode(LED_BUILTIN, OUTPUT);
// Open serial connection.
Serial.begin(9600);
Serial.write('1');
}
// the loop function waits until it receives an input from the serial port & will stop again when it receives a stop signal.
void loop() {
if(Serial.available() > 0){ // if data present, blink
digitalWrite(LED_BUILTIN, HIGH); // turn the LED on (HIGH is the voltage level)
delay(1000); // wait for a second
digitalWrite(LED_BUILTIN, LOW); // turn the LED off by making the voltage LOW
delay(1000); // wait for a second
digitalWrite(LED_BUILTIN, HIGH); // turn the LED on (HIGH is the voltage level)
delay(1000); // wait for a second
digitalWrite(LED_BUILTIN, LOW); // turn the LED off by making the voltage LOW
delay(1000); // wait for a second
digitalWrite(LED_BUILTIN, HIGH); // turn the LED on (HIGH is the voltage level)
delay(1000); // wait for a second
digitalWrite(LED_BUILTIN, LOW); // turn the LED off by making the voltage LOW
delay(1000); // wait for a second
Serial.write('0');
}
}
In conjunction with a Python script to trigger & close the controller:
# import necessary modules
import os
import serial
# connect to the arduino
## Boolean variable that will represent whether or not the arduino is connected
connected = False
## open the serial port that your ardiono is connected to.
ser = serial.Serial("COM8", 9600) # you may need to change this pending on how the board connects to the PC
## loop until the arduino tells us it is ready
while not connected:
serin = ser.read()
connected = True
## trigger the arduino to run the uploaded code
ser.write(1)
## Wait until the arduino tells us it is finished
while ser.read() == '1':
ser.read()
print(ser.read())
## trigger the arduino to run the uploaded code
ser.write(0)
## close the port and end the program
ser.close()
However, when I try replacing the LED-specific lines with commands to start and stop vibration (below) there are a few issues:
Although the controller waits for the go signal from Python as expected, Python blasts through the script in no time, not waiting for a stop from the controller.
The controller starts vibrating but there is no delays as expected between vibratory impulses & the vibrator never stops, even after python sends the stop trigger.
/*
Python triggered vibration.
Waits for Python to send a go signal at which point the vibration starts for a given duration of time.
*/
#include <Wire.h>
#include "Haptic_Driver.h" # this module is from the aforementioned Arduino sketch
Haptic_Driver hapDrive;
int event = 0;
void setup() {
// Open serial connection.
Wire.begin();
Serial.begin(9600);
if( !hapDrive.begin())
Serial.println("Could not communicate with Haptic Driver.");
else
Serial.println("Qwiic Haptic Driver DA7280 found!");
if( !hapDrive.defaultMotor() )
Serial.println("Could not set default settings.");
// Frequency tracking is done by the IC to ensure that the motor is hitting
// its resonant frequency. I found that restricting the PCB (squeezing)
// raises an error which stops operation because it can not reach resonance.
// I disable here to avoid this error.
hapDrive.enableFreqTrack(false);
Serial.println("Setting I2C Operation.");
hapDrive.setOperationMode(DRO_MODE);
Serial.println("Ready.");
Serial.write('1');
}
void loop(){
if(Serial.available() > 0){
hapDrive.setVibrate(25);
delay(1500);
hapDrive.setVibrate(0);
delay(1500);
Serial.write(0);
Serial.flush();
}
}
I am a novice when it comes to both micro-controllers & C++ so, forgive me for any major misunderstandings/errors. Also if anything is unclear in the above description please let me know.
Many Thanks,
Liam
I suspect at least part of the problem is that you are not clearing the contents of the read buffer, only checking if something is there. Serial.flush() i think that as of Arduino 1.00 (don't quote me on that) serial flush doesn't do anything to the incoming buffer.
try adding a var = Serial.read() in before your hapDrive.setVibrate(25); and see if that changes the functionality.
I also HEAVILY recommend interrupts for serial. There's a serial event example that's really comprehensive (although i seem to remember that's not actually interrupt driven in the classical microcontroller sense, but it's close enough!)

sending serial commands to arduino

I've been trying to find a way to send serial commands from my PC (Windows 7) to an Arduino Uno R3. I've been working on this simple program, where you're supposed to send a simple "1" through the USB cable, in order to turn on the onboard LED. My Arduino code should be working (what I know of, but I'll make sure to upload it).
I've been trying to send this command using Python and pySerial, but I can't seem to get pySerial to work. Also I've tried using CMD, but it seems like CMD freezes when i type in my command (ECHO 1 > COM3 BAUD:9600).
I'm the adminstrator of my PC.
This is my arduino coding
int var = 0;
int LEDPin = 13;
int val = 0;
void setup() {
Serial.begin(9600);
pinMode(LEDPin, OUTPUT);
}
void loop() {
if (Serial.available()>0){
val = Serial.read();
}
if (val == 1){
digitalWrite(LEDPin, HIGH);
}
digitalRead(LEDPin);
if (LEDPin==HIGH){
var ++;
delay (1000);
if(var==10){
digitalWrite(LEDPin,LOW);
var = 0;
val = 0;
}
}
}
And the short Python program I've got :)
import serial
ser = serial.Serial('COM3', 9600, timeout=0)
while 1:
var = 1
ser.write(var)
delay(12000)
With Kind Regards
Michael Vedel.
just to be sure - is Arduino really connected to COM3?
In order to check serial connection manually you can use for example Putty. With help of it you can connect to the needed serial port and send command manually. And in order to check whether Arduino received it, you can add Serial.println(val); just after val = Serial.read();, with this Arduino should sends back to serial whatever characters it receives.
Does the code work when you type a '1' into the Serial Monitor in the Arduino IDE? If so, check the COM port. On my PC (also Windows 7), COM3 is never the Arduino port, as it is used internally for something else.
By the way everything after the digitalRead() statement will do nothing, since you're comparing LEDpin (which is declared as 13) to HIGH (which is defined as 1). I think you mean to see if the digitalRead output is HIGH, but I'm not sure that will work, either, since the pin is declared as an OUTPUT. You'd be much better off putting that logic in the (val == 1) section.
just save this code as .bat and make sure that you are using com 6.
This code lets you enter and sends your commands to serial port at 9600bpm
mode com6:9600,N,8,1
#echo off
:start
cls
Set /p commands="enter command:"
echo|set /p= %commands% >com6
goto start
Now you have to read serial using this in Arduino
Serial.read();
I think the root problem is in the Python code in the call to write. The argument has to be bytes (or something compatible). You're passing an integer value. You need to explicitly convert that value to a bytes object. I'm not a Python expert, but I think one way to do that is to replace:
var = 1
with
var = b'\x01'
Oh, and you might need to call flush as well.
BTW, your experiment with ECHO doesn't do what you intended. It sends the character value for '1', which is 49. Since 49 != 1, your Arduino code won't respond to it.
I find the easiest way to test serial commands is to use the Arduino Serial Monitor, which will have all the right defaults as long as you match the baud rate. This will enable you to ensure the Arduino code is right before debugging the Python code. (Then again, it may be difficult to send control characters with the Serial Monitor, so consider using printable text in your protocol. For example 'A' for on and 'B' for off. Printable text can be easy to work with among all these tools (Python, C, Serial Monitor, ECHO, etc.).)
do this it works better for me, just change the com4 for the port you have connected to
#echo off
:start
cls
set /p commands="enter command:"
mode com4 baud=115200 parity=n data=8 stop=1 to=on xon=off odsr=off octs=off dtr=off rts=off idsr=off && echo %commands% > com4
cls
goto start
I think Arduino is stable and easy enough to handle the data from serial, the cause of that as I used same library with Pyserial in python 3 was that event of reading the serial get fired once which is way faster than Arduino board and to solve that you should use a thread method that runs in the background and make it always try to read the incoming data from serial.
I've made this function that recieved the opened serial :
def read_from_port(ser):
while True:
try:
reading = ser.read().decode()
handle_data(reading)
except EXCEPTION:
print("error while decoding" + str(EXCEPTION))
after I have got the serial from the list and assigned it with the desired baud rate and send it to the Thread :
serial_me = serial.Serial(serial_port1, 115200, timeout=3)
try:
thread_x = threading.Thread(target=read_from_port, args= (serial_me,))
thread_x.daemon = True
thread_x.start()
except EXCEPTION:
print('couldn\'t start thread')
print(Exception)

Random character at the beginning of serial.readline() in python and arduino

I get random characters when I perform a serial.readline(), sometimes it is other umbers and sometimes it is whole messages. The output should be "1,2" or "2,2"
Here are a couple of screenshots of the output of serial.readline().
I have tried to put a delay at before serial.readline() but it did not make a difference.
There is usually a strange character at the beginning:
I have also received strange messages:
There is a problem later on in the program that causes the program to hand because sometimes I just receive a blank line.
Is there a way to get consistent output from serial?
Here is the arduino code:
void setup() {
// initialize the LED pin as an output:
pinMode(ledPin, OUTPUT);
// initialize the pushbutton pin as an input:
pinMode(buttonPin, INPUT);
Serial.begin(9600);
}
void loop(){
// read the state of the pushbutton value:
buttonState = digitalRead(buttonPin);
// check if the pushbutton is pressed.
// if it is, the buttonState is HIGH:
if (buttonState == HIGH) {
// turn LED on:
digitalWrite(ledPin, HIGH);
Serial.println("1,2");
}
else {
// turn LED off:
digitalWrite(ledPin, LOW);
Serial.println("2,2");
}
}
And here is the python code:
ser = serial.Serial('/dev/ttyUSB0', 9600)
line=ser.readline()
coord= line.strip()
print coord
EDIT:
I tried putting ser.flushInput() after the ser.open() and I get the same output.
Have you flushed the serial buffer
ser.flushInput()
I've been having the same issue when interfacing between pyserial and Arduino. This may be an old post, but in case someone else is having the same trouble, I remedied my problem by inserting:
ser.flushInput()
...right before my ser.readline() call.
How to Read serial data from an Arduino in Linux
Nothing fancy here. I’m getting the current port configuration, setting the input/output speed to 9600 baud, setting the data expectations to be 8 bits per word, with no parity bit and no stop bit, setting canonical mode, and then committing those changes back to the serial port.
If I am not mistaken you have to change the mentioned settings when connecting through serial port.
You do not mention it, but I guess you are using the pySerial library. Regardless you can use it with the correct settings to connect through serial. The constructor of the API allows all the parameters which are noted below:
Pyserial Library
I have not tested this approach as I have had a similar problem in C not Python.
What happens if you break it down to something simpler?
On the Arduino:
void setup()
{
Serial.begin(9600);
}
int i = 0;
void loop()
{
delay(1000);
Serial.print("Hello World ");
Serial.println(i);
i++;
}
And in the Python REPL...
import serial
ser = serial.Serial('/dev/ttyUSB0', 9600)
while(1):
ser.readline()
The first behavior you should observe is that when you initiate the serial connection, the Arduino reboots. This takes a few seconds.
The second behavior you should observe is that (if you're typing this in the REPL slowly) when the while(1) loop begins, you get all of the serial data that had accumulated since you initiated the serial connection. It takes me a few seconds to type all that out, so when I hit Enter after ser.readline() I see:
'Hello World 1\r\n'
'Hello World 2\r\n'
'Hello World 3\r\n'
I only mention this to make sure you're familiar with two things that burned me a bit the last time I messed with serial communication to an Arduino: it needs time to reboot after connecting, and the serial data buffers. readline() will only give you one line in the buffer - not everything that's in the buffer.
Is it possible you're trying to sent/receive data before it's done rebooting? What about button bounce - could a dozen high/low state detections be messing something up?

Changing XBee from 57.6k to 115.2k Causing Problems with TTL Camera and PySerial

I've been working on a simple XBee/Arduino/Python transfer system. Here's how it works: A hex command is sent from python over xbee, which is received by the arduino. This triggers it to take a picture using an Adafruit TTL Serial Camera. The image is then held in memory and sent over the xbee to the computer, 32 bytes at a time. The Python script then adds the bytes to a .jpg, to be viewed when it finishes.
As of now, both run fine, albeit a tad slow (~25 sec round-trip) for my needs. The problem is, switching from 57600 baud to 115200 in the xbee firmware and programs causes them to fail. They will sometimes give a .jpg roughly half the normal size, or not transmit at all, in both cases being unreadable from a viewer. I've tried changing the timeout in the python side, and modifying the xbee's interfacing options, all to no avail.
Here is the Arduino sketch (adapted largely from the Snapshot example sketch in the Adafruit VC0706 examples library):
#include <Adafruit_VC0706.h>
#include <SoftwareSerial.h>
#define chipSelect 10
SoftwareSerial cameraConnection = SoftwareSerial(2,3);
Adafruit_VC0706 cam = Adafruit_VC0706(&cameraConnection);
void setup()
{
Serial.begin(57600);
pinMode(8, OUTPUT);
if (cam.begin()){}
else { return; } //Abort the transfer if camera does not initialize
cam.setImageSize(VC0706_640x480);
}
void loop()
{
if (Serial.read() == 0x01) //Wait for send command
{
snapAndSend();
cam.reset();
}
}
void snapAndSend()
{
cam.takePicture();
uint16_t jpgLen = cam.frameLength();
while (jpgLen > 0)
{ //Send off 32 bytes of data at a time
uint8_t *buffer;
uint8_t bytesToRead = min(32, jpgLen);
buffer = cam.readPicture(bytesToRead);
Serial.write(buffer, bytesToRead);
jpgLen -= bytesToRead;
}
}
And the Python script:
import serial
import time
link = serial.Serial('COM12', 57600)
print("Engage!")
while True:
answer = input("Press enter to take a picture (type ' to exit): ")
if answer == "'":
break
file = open('imageTest.jpg', 'wb')
link.write(b"\x01")
time1 = time.time()
while True:
if link.inWaiting() > 0:
file.write(link.read())
time1 = time.time()
else:
time2 = time.time()
if (time2 - time1) > .5:
break
print ("Complete! Closing file now...")
file.close()
I'm still a bit new to serial communication and xbees, so I may be overlooking something here. Anyone more experienced have any thoughts on why a switch in baudrate breaks it?
I'm fairly certain that this is going to be related to flow control.
The XBee module can't keep up with a steady stream of data at 115200, so you need to configure it to use hardware flow control. In your case, you need to watch the CTS (clear to send) line from the XBee module, and have your program hold off on sending more data whenever it's not asserted (held low).
You can also configure the XBee to use one of its inputs for an RTS (request to send) signal -- the line the host uses to tell the XBee module when it's ready to receive data.
You may need to modify your Arduino code to receive the entire image from the camera into a buffer, and then slowly send it back out to the XBee module. If you were to check the CTS line before sending each byte, you could hold off whenever the XBee module's buffers were full. You'd need to call a function in your main loop to check the status of your buffered data, and the status of the pin to see if it was possible to send another byte.

Control a specific pin on the Arduino Uno board using pyserial

I have a python code that sends in a pattern, in which a light has to blink in. (say eg. 101010. pattern may vary every time the code is run). when it is executing this infinitely i want an interrupt( again sent by the python code )to save the present conditions of the lights (say it is running 1 of the sequence) and perform a specific task like turn off the lights for 10 seconds and then resume the sequence.
one way of doing this is by interrupting the program by making the interrupt pin high. The question is can this making of high/low controlled by the pyserial.
So a simple pseudo code would be :
PYTHON part of the code:
Read the sequence:
Send the sequence to the arduino board using pyserial.
while(1)
{
Run a timer for 15 second.
When the timer overflows interrupt the arduino.
}
ARDUINO part of the code :
Read the sequence
while (1)
{
keep repeating the sequence on the LED.
}
// if interrupted on pin2 // assuming pin2 has the interrupt procedure
// Pyserial has to enable the interrupt WITHOUT using a switch for enabling the pin.
ISR
{
Save the present state of execution.
Turn off the LED.
}
FOR A BETTER UNDERSTANDING :
I built up small codes to show the doubts i had :
CODE FOR THE ARDUINO IS :
int ledpin1 = 13;
int speedy;
int patterns;
void setup()
{
Serial.begin(9600);
Serial.print("Program Initiated: \n");
pinMode(ledpin1,OUTPUT);
//activate the blackout ISR when a interrupt is achieved at a certain pin. In this case pin2 of the arduino
attachInterrupt(0,blackout,CHANGE);
}
void loop()
{
if (Serial.available()>1)
{
Serial.print("starting loop \n");
patterns = Serial.read();
patterns = patterns-48;
speedy = Serial.read();
speedy = (speedy-48)*1000;
while(1)
{
patterns = !(patterns);
Serial.print(patterns);
digitalWrite(ledpin1,patterns);
delay(speedy);
}
}
}
/*
void blackout()
{
// ***Save the present state of the LED(on pin13)***
Serial.print ("The turning off LED's for performing the python code\n");
digitalWrite(ledpin,LOW);
//wait for the Python code to complete the task it wants to perform,
//so got to dealy the completion of the ISR
delay(2000);// delay the whole thing by 2 seconds
//***Continue with the while loop by setting the condition of the light to the saved condition.***
}
*/
==================================================================================
CODE FOR THE PYTHON FRONT IS :
import serial
import time
patterns=1
speedy=1
ser = serial.Serial()
ser.setPort("COM4")
ser.baudrate = 9600
ser.open()
def main():
if (ser.isOpen()):
#while(1):
ser.write(patterns)
ser.write(speedy)
blackoutfunc()
#ser.close()
def blackoutfunc():
while(1):
time.sleep(2)
print "Performing operations as required"
===============================================================================
Now the questions I had :
1) Is there a way to be able to activate the "blackout ISR" depending on the conditions of a pin(in this case pin2 which is the INT0 pin) without using a physical switch present on the pin. Hence the pin state has to be manipulated by the software.
2) Is it possible to perform the operations as mentioned in the comments of the blackout functions?
3) In the python code is it possible to just send in the data(i.e. patterns,speedy) only once and make the arduino perform the pattern in a infinite way without again sending the data by the serial.write command. Hence avoiding the while(1) loop after the ser.isOpen().
Have a look at this:
https://github.com/ajfisher/arduino-command-server
It's something I pulled together on the Arduino side to issue arbitrary commands like switch a pin high / low and set PWM levels etc. It works over both serial and network though it's a touch buggy on the network side at the moment.
To use it, put the code on your arduino then you just write a python script (or any other language that can use a serial connection) to connect over the serial connection and then tell it what you want to do eg DIGW 1 HIGH etc
Also have a look at: https://github.com/ajfisher/arduino-django-visualiser which is where I use a variation of this library to control some LEDs based on some things going on in Django - it's more heavily python based.

Categories