Say I have the following python script to read in serial data from my Arduino:
import serial
ser = serial.Serial("dev/ttyACM1", 9600)
ser.timeout = 2
ser.readlines()
On the other end I've flashed my Arduino with a program that sends 20 voltage readings every 0.5 seconds. The Arduino starts sending those readings from the moment it's hooked up, then after 20 seconds it stops and sends nothing.
Now what I've noticed is that I can read those 20 voltage values using the first script whenever I want. That is, I can hook up the Arduino, wait a couple of minutes then read in the values. This makes me think that the data is getting stored somewhere. I'm inclined to think that it's not being stored on the Arduino but on my laptop somewhere.
I've come up with a few questions that I hope the community could help me with:
Where is PySerial getting the data from (the Arduino or some buffer on my laptop)?
For how long is the data stored in this place?
Is there a way to clear this space before reading in values?
How much storage space is there?
When you set the baud rate in PySerial (second line in script), is this the rate that
PySerial reads data from the storage area (not the Arduino)?
I've noticed that if I set the baud rate in PySerial too high the first few lines of data are fragmented and sometimes completely wrong, why?
Not exactly related but when you set serial.Serial.timeout are the
units in seconds?
I appreciate your time.
Have you tried using a terminal program like TerraTerm (windows) or GTKTerm (linux) to open the same port to the arduino? I think this would be helpful to answer some of your questions.
Some quick answers to your questions that I can dump off the top of my head.
From the port specified, I'm guessing you are asking something deeper than this?
If you do a
x = ser.readlines()
then the data will be in x as long as you'd like.
There is a flush function defined in PySerial
Not sure. You can specify how many characters you would like to read though
example:
x = ser.read(number)
The pyserial documentation states the following
Read size bytes from the serial port. If a timeout is set it may return less characters as requested. With no timeout it will block until the requested number of bytes is read.
http://pyserial.sourceforge.net/pyserial_api.html
This is the clock rate of the port you are opening, ie /dev/ttyACM1, most serial comms are at 9600, if you happen to be using a USB to serial you'll need 115200
Clock rate mismatch. You're computer is sampling data at a rate higher than the arduino is providing it, causing it to be incorrectly displayed.
Seconds, quote from Pyserial documentation : "timeout = x: set timeout to x seconds (float allowed)" Same link as number 4
Hope that helps some!
Related
I have a serial connection to a solar inverter using a USB. Using a free serial device monitor I can spy on the connection and watch it read/write information.
The info my computer gets from the inverter can only be read and exported (to .xlsx) on their application. Their app has no automatic export functionality and no api. The end goal is to be able to export automatically without me touching the computer at designated time intervals. I'm trying to circumvent their app by reading the data myself, and saving it to .csv so I can do anything else that I want with it once I have it.
What the serial monitor shows, top is read, bottom is write
I can't seem to decode what its sending/recieving. And I also don't recieve anything when I send those same write messages. And when I'm just looping and trying to readline() I get empty bytes.
My code looks like this, though I have a few different iterations of it doing similar things like looping through every baudrate, stopbits, and bitsize option. As well as looping through the 5 different unique write messages I see in the monitor.
# Construct serial connection using baud and stopbit
serialPort = serial.Serial(port= "COM3",
baudrate=115200,
bytesize=serial.EIGHTBITS,
timeout=0,
stopbits=serial.STOPBITS_ONE)
while(1):
serialPort.write('01 03 00 2D 00 2D 15 DE'.encode())
serialString = serialPort.readline()
if serialString.decode() != '':
print(serialString.decode())
Any help or advice for what I'm doing is appreciated.
Is there a way to capture and write very fast serial data to a file?
I'm using a 32kSPS external ADC and a baud rate of 2000000 while printing in the following format: adc_value (32bits) \t millis()
This results in ~15 prints every 1 ms. Unfortunately every single soulution I have tried fails to capture and store real time data to a file. This includes: Processing sketches, TeraTerm, Serial Port Monitor, puTTY and some Python scripts. All of them are unable to log the data in real time.
Arduino Serial Monitor on the other hand is able to display real time serial data, but it's unable to log it in a file, as it lacks this function.
Here's a printscreen of the serial monitor in Arduino with the incoming data:
One problematic thing is probably that you try to do a write each time you receive a new record. That will waste a lot of time writing data.
Instead try to collect the data into buffers, and as a buffer is about to overflow write the whole buffer in a single and as low-level as possible write call.
And to not stop the receiving of the data to much, you could use threads and double-buffering: Receive data in one thread, write to a buffer. When the buffer is about to overflow signal a second thread and switch to a second buffer. The other thread takes the full buffer and writes it to disk, and waits for the next buffer to become full.
After trying more than 10 possible solutions for this problem including dedicated serial capture software, python scripts, Matlab scripts, and some C projects alternatives, the only one that kinda worked for me proved to be MegunoLink Pro.
It does not achieve the full 32kSPS potential of the ADC, rather around 12-15kSPS, but it is still much better than anything I've tried.
Not achieving the full 32kSPS might also be limited by the Serial.print() method that I'm using for printing values to the serial console. By the way, the platform I've been using is ESP32.
Later edit: don't forget to edit MegunoLinkPro.exe.config file in the MegunoLink Pro install directory in order to add further baud rates, like 1000000 or 2000000. By default it is limited to 500000.
I have a simple Python script to communicate with a micro-controller (STM32F103C8T6) using the serial port. I'm using pySerial to write a couple of 44-bytes messages at a time.
[...]
serial = serial.Serial(serial.tools.list_ports.comports()[0].device, 115200)
packet0 = bytearray(INSERT_RELEVANT_44-BYTES)
packet1 = bytearray(INSERT_RELEVANT_44-BYTES)
serial.write(packet0)
time.sleep(0.1) # Delay between communications
serial.write(packet1)
[...]
I had to insert a delay between the communications, otherwise it wouldn't work. My reasoning is that for a baud rate of 115200 bps the messages should take 44*8/115200 = ~0,003 seconds to be sent, thus this should be the minimum ideal interval between sending the packets. The code, however, doesn't work for anything smaller than 0,1.
Why? Am I missing something? I suppose there is some delay due to the operating system and the USB, but it shouldn't account for ~0,7 seconds. How can I optimize this to use the minimum possible delay?
Rather then calculating a nominal delay based on the UART link, you could simply poll the serial driver to determine whether the Tx buffer is empty:
serial.write(packet0)
while serial.outWaiting() > 0 :
pass
serial.write(packet1)
This has the advantage of accounting automatically for any latency, software overhead and buffer limitations anywhere in the chain of application code, library, driver, USB-Serial bridge. It will not however solve any problem with the STM32 serial I/O implementation, you should probably address the root problem of why the data cannot be streamed, which is most likely down to poor implementation of the at the STM32 device end.
I am trying to establish communications via serial port between a PC with Ubuntu 14.04LTS and my RoMeo Arduino Board (Atmega 328). The used serial interface are 2 Xbee modules, one at PC and the other at the board.
Firstly, I am trying to develop a simple program to send messages to the board and receive them back. The code I use for the Arduino board is the following:
void loop(void)
{
char msg;
if (Serial.available()){
msg = Serial.read();
msg = Serial.print(msg);
}
}
When I send a unique character, the PC receives it back correctly. However, the problem I am facing is that for longer strings, the following characters are misspelled, as I obtain back strange hex numbers, as follows:
>>> import serial
>>> ser = serial.Serial(port='/dev/ttyUSB0', baudrate=57600, timeout=0.1)
>>> ser.write('H')
>>> ser.read(1)
'H'
>>> ser.write('Hello')
>>> ser.read(5)
'H\x8b\xbd'
Thanks in advance.
EDIT: Seems like there is an overflow problem with the XBee modules, but I can not figure it out completely: The problem is solved if I wait 0.01 seconds or more between sent characters, which is a huge amount of time. Namely, the code I use now for sending a word is:
for letter in word:
serial.write(letter)
time.sleep(0.01)
However, this waiting time is only needed when sending data from the PC to the Arduino. When the communication goes the other way (Arduino sends data to PC), I do not need to sleep and bytes are correctly sent all together at 57600 bauds.
The reason why the PC could not send more than 1 character to the Arduino board was that there was an XBee module configured with different port parameters than both the other module and the pyserial instance. In this case, the communication was established in Python with the following main parameters:
Baud rate: 57600
Bytesize: 8
Parity: None
Stop bits: 1
If one of this parameters is different in one of the XBee modules, the communication will be faulty (like this case) or even impossible.
To check the XBee configuration, the Digi XCTU application can be used: With the RF modules connected to the PC, open the program and read their configuration. Once opened, it has to be made sure that the 'Serial interfacing' options are equal to the previously listed.
At the image, it is shown the menu where these options can be changed. Note that the Stop bits and the Bytesize can not be configured. The first parameter was not adjustable until the XB24-ZB versions, and the last one seems to not be possible to change.
In the case of this question, the wrong parameter was the parity, as it was set to space parity in one of the modules, instead of no parity. Thus, the first byte was sent correctly, but after it the data was not coherent. After changing this parameter, the connection run OK.
I am working on a project for work and I am stuck on a part where I need to monitor a serial line and listen for certain words using python
so the setup is that we have a automated RAM testing machine that tests RAM one module at a time and interacts with software that came with the machine via serial. The software that came with the RAM tester is for monitoring/configuring the testing process, it also displays all of the information from the SPD chip from each module. while the RAM tester was running I ran a serial port monitoring program and I was able to see the same information that it displays in the software. The data I'm interested in is the speed of the RAM and the pass/fail result, both of which I was able to find in the data I monitored coming over the serial line. There are only 5 different speeds of RAM that we test, so I was hoping to have python monitor the serial line and wait for the speed of the RAM and the pass/fail results to come across. once python detects the speed of the RAM, and if it passes, I will have python write to an Arduino, and the Arduino will control a conveyor belt that will sort the ram by speed.
my idea is to have a variable for each of the RAM speeds and set the variables to 0. when python detects the RAM speed from the serial line it will set the corresponding variable to 1. then when the test is over the results, either pass or fail, will come over the serial line. this is where I am going to try to use a if statement. I imagine it would look something like this:
if PC-6400 == 1 and ser.read() == pass
ser.write(PC-6400) #serial write to the arduino
I know the use of the ser.read() == pass is incorrect and that's where I'm stuck. I do not know how to use a ser.read() function to look for certain words. I need it to look for the ram speed (in this case its PC-6400) and the word pass but I have not been successful in getting it to find either. I am currently suck in is getting python to detect the RAM speed so it can change the value of the variable. would it be something close to this?
if ser.read() == PC-6400
PC-6400 = 1
This whole thing is a bit difficult for me to explain and I hope it all makes sense, I thank you in advance if anyone can give me some advice on how to get this going. I am pretty new to python and this is the most adventurous project I have worked on using python so far.
I'm still a bit confused, but here's a very basic example to hopefully get you started.
This will read from a serial port. It will attempt to read 512 bytes (which just means 512 characters from a string). If 512 bytes aren't available then it will wait forever, so make sure you set a timeout when you made the serial connection.
return_str = ser.read(size = 512)
You can also see how many bytes are waiting to be read:
print "num bytes available = ", ser.inWaiting()
Once you have a string, you can check words within the string:
if "PASS" in return_str:
print "the module passed!"
if "PC-6400" in return_str:
print "module type is PC-6400"
To do something similar, but with variables:
# the name pass is reserved
pass_flag = "PASS"
PC6400 = 0
if pass_flag in return_str and "PC-6400" in return_str:
PC6400 = 1
Keep in mind that you it is possible to read part of a line if you are too quick. You can add delays by using timeouts or the time.sleep() function. You might also find you need to wait for a couple of seconds after initiating the connection as the arduino resets when you connect. This will give it time to reset before you try and read/write.