I am reading values from a pressure sensing mat which has 32x32 individual pressure points. It outputs the readings on serial as 1024 bytes between 1 and 250 + 1 'end token' byte which is always 255 (or xFF).
I thought the function bellow would flush/reset the input buffer and then take a 'fresh' reading and return the max pressure value from that reading whenever I call it.
However, none of the ser.reset_input_buffer() and similar methods seem to actually empty the buffer. When I press down on the mat, run the program and immediately release the pressure, I don't see the max value drop immediately. Instead, it seems to be going through the buffer one by one.
import serial
import numpy as np
import time
def read_serial():
ser_bytes = bytearray([0])
# none of these seem to make a differece
ser.reset_input_buffer()
ser.flushInput()
ser.flush()
# 2050 bytes should always contain a whole chunk of 1025 bytes ending with 255 (xFF)
while len(ser_bytes) <= 2050:
ser_bytes = ser_bytes + ser.read_until(b'\xFF')
ser_ints = np.array(ser_bytes, dtype='int32') #bytes to ints
last_end_byte_index = np.max( np.where(ser_ints == 255) ) #find the last end byte
# get only the 1024 sensor readings as 32x32 np array
mat_reading = np.array( ser_ints[last_end_byte_index-1024: last_end_byte_index]).reshape(32,32)
return np.amax(mat_reading)
ser = serial.Serial('/dev/tty.usbmodem14201', 115200, timeout=1)
while True:
print(read_serial())
time.sleep(1)
The best solution I found so far is having a designated thread which keeps reading the buffer and updating a global variable. It works but seems a bit unresourceful if I only want to read the value about every 60 seconds. Is there a better way?
Also... is there a better way to read the 1025-byte chunk representing the entire mat? There are no line breaks, so ser.readline() won't work.
Thanks! Not sure how to make an MWE with serial, sorry about that ;)
Related
I have a script that reads some data from a binary stream of "packets" containing "parameters". The parameters read are stored in a dictionary for each packet, which is appended to an array representing the packet stream.
At the end this array of dict is written to an output CSV file.
Among the read data is a CUC7 datetime, stored as coarse/fine integer parts of a GPS time, which I also want to convert to a UTC ISO string.
from astropy.time import Time
def cuc2gps_time(coarse, fine):
return Time(coarse + fine / (2**24), format='gps')
def gps2utc_time(gps):
return Time(gps, format='isot', scale='utc')
The issue is that I realized that these two time conversions make up 90% of the total run time of my script, most of the work of my script being done in the 10% remaining (read binary file, decode 15 other parameters, write to CSV).
I somehow improved the situation by making the conversions in batches on Numpy arrays, instead of on each packet. This reduces the total runtime by about half.
import numpy as np
while end_not_reached:
# Read 1 packet
# (...)
nb_packets += 1
end_not_reached = ... # boolean
# Process in batches for better performance
if not nb_packets%1000 or not end_not_reached:
# Convert CUC7 time to GPS and UTC times
all_coarse = np.array([packet['lobt_coarse'] for packet in packets])
all_fine = np.array([packet['lobt_fine'] for packet in packets])
all_gps = cuc2gps_time(all_coarse, all_fine)
all_utc = gps2utc_time(all_gps)
# Add times to each packet
for packet, gps_time, utc_time in zip(packets, all_gps, all_utc):
packet.update({'gps_time': gps_time, 'utc_time': utc_time})
But my script is still absurdly slow. Reading 60000 packets from a 1.2GB file and writing it as a CSV takes 12s, against only 2.5s if I remove the time conversion.
So:
Is it expected that Astropy time conversions are so slow? Am I using it wrong? Is there a better library?
Is there a way to improve my current implementation? I suspect that the remaining "for" loop in there is very costly, but could not find a good way to replace it.
I think the problem is that you're doing multiple loops over your sequence of packets. In Python I would recommend having arrays representing each parameter, instead of having a list of objects, each with a bunch of scalar parameters.
If you can read all the packets in at once, I would recommend something like:
num_bytes = ...
num_bytes_per_packet = ...
num_packets = num_bytes / num_bytes_per_packet
param1 = np.empty(num_packets)
param2 = np.empty(num_packets)
...
time_coarse = np.empty(num_packets)
time_fine = np.empty(num_packets)
...
param_N = np.empty(num_packets)
for i in range(num_packets):
param_1[i], param_2[i], ..., time_coarse[i], time_fine[i], ... param_N[i] = decode_packet(...)
time_gps = Time(time_coarse + time_fine / (2**24), format='gps')
time_utc = time_gps.utc.isot
I have an industrial sensor which provides me information via telnet over port 10001.
It has a Data Format as follows:
Also the manual:
All the measuring values are transmitted int32 or uint32 or float depending on the sensors
Code
import telnetlib
import struct
import time
# IP Address, Port, timeout for Telnet
tn = telnetlib.Telnet("169.254.168.150", 10001, 10)
while True:
op = tn.read_eager() # currently read information limit this till preamble
print(op[::-1]) # make little-endian
if not len(op[::-1]) == 0: # initially an empty bit starts (b'')
data = struct.unpack('!4c', op[::-1]) # unpacking `MEAS`
time.sleep(0.1)
my initial attempt:
Connect to the sensor
read data
make it to little-endian
OUTPUT
b''
b'MEAS\x85\x8c\x8c\x07\xa7\x9d\x01\x0c\x15\x04\xf6MEAS'
b'\x04\xf6MEAS\x86\x8c\x8c\x07\xa7\x9e\x01\x0c\x15\x04\xf6'
b'\x15\x04\xf6MEAS\x85\x8c\x8c\x07\xa7\x9f\x01\x0c\x15'
b'\x15\x04\xf6MEAS\x87\x8c\x8c\x07\xa7\xa0\x01\x0c'
b'\xa7\xa2\x01\x0c\x15\x04\xf6MEAS\x87\x8c\x8c\x07\xa7\xa1\x01\x0c'
b'\x8c\x07\xa7\xa3\x01\x0c\x15\x04\xf6MEAS\x87\x8c\x8c\x07'
b'\x88\x8c\x8c\x07\xa7\xa4\x01\x0c\x15\x04\xf6MEAS\x88\x8c'
b'MEAS\x8b\x8c\x8c\x07\xa7\xa5\x01\x0c\x15\x04\xf6MEAS'
b'\x04\xf6MEAS\x8b\x8c\x8c\x07\xa7\xa6\x01\x0c\x15\x04\xf6'
b'\x15\x04\xf6MEAS\x8a\x8c\x8c\x07\xa7\xa7\x01\x0c\x15'
b'\x15\x04\xf6MEAS\x88\x8c\x8c\x07\xa7\xa8\x01\x0c'
b'\x01\x0c\x15\x04\xf6MEAS\x88\x8c\x8c\x07\xa7\xa9\x01\x0c'
b'\x8c\x07\xa7\xab\x01\x0c\x15\x04\xf6MEAS\x8b\x8c\x8c\x07\xa7\xaa'
b'\x8c\x8c\x07\xa7\xac\x01\x0c\x15\x04\xf6MEAS\x8c\x8c'
b'AS\x89\x8c\x8c\x07\xa7\xad\x01\x0c\x15\x04\xf6MEAS\x8a'
b'MEAS\x88\x8c\x8c\x07\xa7\xae\x01\x0c\x15\x04\xf6ME'
b'\x15\x04\xf6MEAS\x87\x8c\x8c\x07\xa7\xaf\x01\x0c\x15\x04\xf6'
b'\x15\x04\xf6MEAS\x8a\x8c\x8c\x07\xa7\xb0\x01\x0c'
b'\x0c\x15\x04\xf6MEAS\x8a\x8c\x8c\x07\xa7\xb1\x01\x0c'
b'\x07\xa7\xb3\x01\x0c\x15\x04\xf6MEAS\x89\x8c\x8c\x07\xa7\xb2\x01'
b'\x8c\x8c\x07\xa7\xb4\x01\x0c\x15\x04\xf6MEAS\x89\x8c\x8c'
b'\x85\x8c\x8c\x07\xa7\xb5\x01\x0c\x15\x04\xf6MEAS\x84'
b'MEAS\x87\x8c\x8c\x07\xa7\xb6\x01\x0c\x15\x04\xf6MEAS'
b'\x04\xf6MEAS\x8b\x8c\x8c\x07\xa7\xb7\x01\x0c\x15\x04\xf6'
b'\x15\x04\xf6MEAS\x8b\x8c\x8c\x07\xa7\xb8\x01\x0c\x15'
b'\x15\x04\xf6MEAS\x8a\x8c\x8c\x07\xa7\xb9\x01\x0c'
b'\xa7\xbb\x01\x0c\x15\x04\xf6MEAS\x87\x8c\x8c\x07\xa7\xba\x01\x0c'
try to unpack the preamble !?
How do I read information like Article number, Serial number, Channel, Status, Measuring Value between the preamble?
The payload size seems to be fixed here for 22 Bytes (via Wireshark)
Parsing the reversed buffer is just weird; please use struct's support for endianess. Using big-endian '!' in a little-endian context is also odd.
The first four bytes are a text constant. Ok, fine perhaps you'll need to reverse those. But just those, please.
After that, use struct.unpack to parse out 'IIQI'. So far, that was kind of working OK with your approach, since all fields consume 4 bytes or a pair of 4 bytes. But finding frame M's length is the fly in the ointment since it is just 2 bytes, so parse it with 'H', giving you a combined 'IIQIH'. After that, you'll need to advance by only that many bytes, and then expect another 'MEAS' text constant once you've exhausted that set of measurements.
I managed to avoid TelnetLib altogether and created a tcp client using python3. I had the payload size already from my wireshark dump (22 Bytes) hence I keep receiving 22 bytes of Information. Apparently the module sends two distinct 22 Bytes payload
First (frame) payload has the preamble, serial, article, channel information
Second (frame) payload has the information like bytes per frame, measuring value counter, measuring value Channel 1, measuring value Channel 2, measuring value Channel 3
The information is in int32 and thus needs a formula to be converted to real readings (mentioned in the instruction manual)
(as mentioned by #J_H the unpacking was as He mentioned in his answer with small changes)
Code
import socket
import time
import struct
DRANGEMIN = 3261
DRANGEMAX = 15853
MEASRANGE = 50
OFFSET = 35
# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_address = ('169.254.168.150', 10001)
print('connecting to %s port %s' % server_address)
sock.connect(server_address)
def value_mm(raw_val):
return (((raw_val - DRANGEMIN) * MEASRANGE) / (DRANGEMAX - DRANGEMIN) + OFFSET)
if __name__ == '__main__':
while True:
Laser_Value = 0
data = sock.recv(22)
preamble, article, serial, x1, x2 = struct.unpack('<4sIIQH', data)
if not preamble == b'SAEM':
status, bpf, mValCounter, CH1, CH2, CH3 = struct.unpack('<hIIIII',data)
#print(CH1, CH2, CH3)
Laser_Value = CH3
print(str(value_mm(Laser_Value)) + " mm")
#print('RAW: ' + str(len(data)))
print('\n')
#time.sleep(0.1)
Sure enough, this provides me the information that is needed and I compared the information via the propreitary software which the company provides.
I'm planning to implement a "DSP-like" signal processor in Python. It should capture small fragments of audio via ALSA, process them, then play them back via ALSA.
To get things started, I wrote the following (very simple) code.
import alsaaudio
inp = alsaaudio.PCM(alsaaudio.PCM_CAPTURE, alsaaudio.PCM_NORMAL)
inp.setchannels(1)
inp.setrate(96000)
inp.setformat(alsaaudio.PCM_FORMAT_U32_LE)
inp.setperiodsize(1920)
outp = alsaaudio.PCM(alsaaudio.PCM_PLAYBACK, alsaaudio.PCM_NORMAL)
outp.setchannels(1)
outp.setrate(96000)
outp.setformat(alsaaudio.PCM_FORMAT_U32_LE)
outp.setperiodsize(1920)
while True:
l, data = inp.read()
# TODO: Perform some processing.
outp.write(data)
The problem is, that the audio "stutters" and is not gapless. I tried experimenting with the PCM mode, setting it to either PCM_ASYNC or PCM_NONBLOCK, but the problem remains. I think the problem is that samples "between" two subsequent calls to "inp.read()" are lost.
Is there a way to capture audio "continuously" in Python (preferably without the need for too "specific"/"non-standard" libraries)? I'd like the signal to always get captured "in the background" into some buffer, from which I can read some "momentary state", while audio is further being captured into the buffer even during the time, when I perform my read operations. How can I achieve this?
Even if I use a dedicated process/thread to capture the audio, this process/thread will always at least have to (1) read audio from the source, (2) then put it into some buffer (from which the "signal processing" process/thread then reads). These two operations will therefore still be sequential in time and thus samples will get lost. How do I avoid this?
Thanks a lot for your advice!
EDIT 2: Now I have it running.
import alsaaudio
from multiprocessing import Process, Queue
import numpy as np
import struct
"""
A class implementing buffered audio I/O.
"""
class Audio:
"""
Initialize the audio buffer.
"""
def __init__(self):
#self.__rate = 96000
self.__rate = 8000
self.__stride = 4
self.__pre_post = 4
self.__read_queue = Queue()
self.__write_queue = Queue()
"""
Reads audio from an ALSA audio device into the read queue.
Supposed to run in its own process.
"""
def __read(self):
inp = alsaaudio.PCM(alsaaudio.PCM_CAPTURE, alsaaudio.PCM_NORMAL)
inp.setchannels(1)
inp.setrate(self.__rate)
inp.setformat(alsaaudio.PCM_FORMAT_U32_BE)
inp.setperiodsize(self.__rate / 50)
while True:
_, data = inp.read()
self.__read_queue.put(data)
"""
Writes audio to an ALSA audio device from the write queue.
Supposed to run in its own process.
"""
def __write(self):
outp = alsaaudio.PCM(alsaaudio.PCM_PLAYBACK, alsaaudio.PCM_NORMAL)
outp.setchannels(1)
outp.setrate(self.__rate)
outp.setformat(alsaaudio.PCM_FORMAT_U32_BE)
outp.setperiodsize(self.__rate / 50)
while True:
data = self.__write_queue.get()
outp.write(data)
"""
Pre-post data into the output buffer to avoid buffer underrun.
"""
def __pre_post_data(self):
zeros = np.zeros(self.__rate / 50, dtype = np.uint32)
for i in range(0, self.__pre_post):
self.__write_queue.put(zeros)
"""
Runs the read and write processes.
"""
def run(self):
self.__pre_post_data()
read_process = Process(target = self.__read)
write_process = Process(target = self.__write)
read_process.start()
write_process.start()
"""
Reads audio samples from the queue captured from the reading thread.
"""
def read(self):
return self.__read_queue.get()
"""
Writes audio samples to the queue to be played by the writing thread.
"""
def write(self, data):
self.__write_queue.put(data)
"""
Pseudonymize the audio samples from a binary string into an array of integers.
"""
def pseudonymize(self, s):
return struct.unpack(">" + ("I" * (len(s) / self.__stride)), s)
"""
Depseudonymize the audio samples from an array of integers into a binary string.
"""
def depseudonymize(self, a):
s = ""
for elem in a:
s += struct.pack(">I", elem)
return s
"""
Normalize the audio samples from an array of integers into an array of floats with unity level.
"""
def normalize(self, data, max_val):
data = np.array(data)
bias = int(0.5 * max_val)
fac = 1.0 / (0.5 * max_val)
data = fac * (data - bias)
return data
"""
Denormalize the data from an array of floats with unity level into an array of integers.
"""
def denormalize(self, data, max_val):
bias = int(0.5 * max_val)
fac = 0.5 * max_val
data = np.array(data)
data = (fac * data).astype(np.int64) + bias
return data
debug = True
audio = Audio()
audio.run()
while True:
data = audio.read()
pdata = audio.pseudonymize(data)
if debug:
print "[PRE-PSEUDONYMIZED] Min: " + str(np.min(pdata)) + ", Max: " + str(np.max(pdata))
ndata = audio.normalize(pdata, 0xffffffff)
if debug:
print "[PRE-NORMALIZED] Min: " + str(np.min(ndata)) + ", Max: " + str(np.max(ndata))
print "[PRE-NORMALIZED] Level: " + str(int(10.0 * np.log10(np.max(np.absolute(ndata)))))
#ndata += 0.01 # When I comment in this line, it wreaks complete havoc!
if debug:
print "[POST-NORMALIZED] Level: " + str(int(10.0 * np.log10(np.max(np.absolute(ndata)))))
print "[POST-NORMALIZED] Min: " + str(np.min(ndata)) + ", Max: " + str(np.max(ndata))
pdata = audio.denormalize(ndata, 0xffffffff)
if debug:
print "[POST-PSEUDONYMIZED] Min: " + str(np.min(pdata)) + ", Max: " + str(np.max(pdata))
print ""
data = audio.depseudonymize(pdata)
audio.write(data)
However, when I even perform the slightest modification to the audio data (e. g. comment that line in), I get a lot of noise and extreme distortion at the output. It seems like I don't handle the PCM data correctly. The strange thing is that the output of the "level meter", etc. all appears to make sense. However, the output is completely distorted (but continuous) when I offset it just slightly.
EDIT 3: I just found out that my algorithms (not included here) work when I apply them to wave files. So the problem really appears to actually boil down to the ALSA API.
EDIT 4: I finally found the problems. They were the following.
1st - ALSA quietly "fell back" to PCM_FORMAT_U8_LE upon requesting PCM_FORMAT_U32_LE, thus I interpreted the data incorrectly by assuming that each sample was 4 bytes wide. It works when I request PCM_FORMAT_S32_LE.
2nd - The ALSA output seems to expect period size in bytes, even though they explicitely state that it is expected in frames in the specification. So you have to set the period size four times as high for output if you use 32 bit sample depth.
3rd - Even in Python (where there is a "global interpreter lock"), processes are slow compared to Threads. You can get latency down a lot by changing to threads, since the I/O threads basically don't do anything that's computationally intensive.
When you
read one chunk of data,
write one chunk of data,
then wait for the second chunk of data to be read,
then the buffer of the output device will become empty if the second chunk is not shorter than the first chunk.
You should fill up the output device's buffer with silence before starting the actual processing. Then small delays in either the input or output processing will not matter.
You can do that all manually, as #CL recommend in his/her answer, but I'd recommend just using
GNU Radio instead:
It's a framework that takes care of doing all the "getting small chunks of samples in and out your algorithm"; it scales very well, and you can write your signal processing either in Python or C++.
In fact, it comes with an Audio Source and an Audio Sink that directly talk to ALSA and just give/take continuous samples. I'd recommend reading through GNU Radio's Guided Tutorials; they explain exactly what is necessary to do your signal processing for an audio application.
A really minimal flow graph would look like:
You can substitute the high pass filter for your own signal processing block, or use any combination of the existing blocks.
There's helpful things like file and wav file sinks and sources, filters, resamplers, amplifiers (ok, multipliers), …
I finally found the problems. They were the following.
1st - ALSA quietly "fell back" to PCM_FORMAT_U8_LE upon requesting PCM_FORMAT_U32_LE, thus I interpreted the data incorrectly by assuming that each sample was 4 bytes wide. It works when I request PCM_FORMAT_S32_LE.
2nd - The ALSA output seems to expect period size in bytes, even though they explicitely state that it is expected in frames in the specification. So you have to set the period size four times as high for output if you use 32 bit sample depth.
3rd - Even in Python (where there is a "global interpreter lock"), processes are slow compared to Threads. You can get latency down a lot by changing to threads, since the I/O threads basically don't do anything that's computationally intensive.
Audio is gapless and undistorted now, but latency is far too high.
I'm trying to do a TCP ACK Spoofing. I sniff one ACK packet from a pcap file and send it in a loop incrementing its ACK number as well as another option field.
Sniffing Part: (Prespoofing)
from scapy.all import *
from struct import unpack, pack
pkt = sniff(offline="mptcpdemo.pcap", filter="tcp", count=15)
i=6
while True:
ack_pkt = pkt[i]
if ack_pkt.sprintf('%TCP.flags%') == 'A':
break
i+=1
del ack_pkt.chksum
del ack_pkt[TCP].chksum
print ack_pkt.chksum, ack_pkt[TCP].chksum
hex2pkt = ack_pkt.__class__
Spoofing Part: (Non Optimized)
count=1
while count<5:
ack_pkt[TCP].ack += 1
pkt_hex = str(ack_pkt)
rest = pkt_hex[:-4]
last_4_bit = unpack('!I',pkt_hex[-4:])[0]
new_hex_pkt = rest + pack('>I',(last_4_bit+1))
new_pkt=hex2pkt(new_hex_pkt)
#sendp(new_pkt, verbose=0)
print new_pkt.chksum, new_pkt[TCP].chksum
count+=1
The output comes like this: (Which is changing)
None None
27441 60323
27441 58895
27441 57467
27441 56039
After sending, The average time gap between two packets is around 15 ms. (For 1000 Packets)
When I check it with Wireshark, it shows "checksum is correct" for the 1st packet and "incorrect" for others.
Spoofing Part: (Little bit Optimized)
pkt_hex=str(ack_pkt)
rest1=pkt_hex[:42]
tcp_ack=unpack('!I',pkt_hex[42:46])[0]
rest2=pkt_hex[46:-4]
last_4_bit = unpack('!I',pkt_hex[-4:])[0]
count=1
while count<5:
new_hex_pkt = rest1 + pack('>I',(tcp_ack+1)) + rest2 + pack('>I',(last_4_bit+1))
new_pkt = hex2pkt(new_hex_pkt)
#sendp(new_pkt, verbose=0)
print new_pkt.chksum, new_pkt[TCP].chksum
count+=1
The output comes like this: (Which is not changing)
None None
27441 61751
27441 61751
27441 61751
27441 61751
After sending, The average time gap between two packets is around 10 ms. (For 1000 Packets)
The Checksum is not changing for the 2nd case. The process is quite same. Then what is the problem in the 2nd optimized case? And why the TCP checksum calculated in a loop are wrong for subsequent packets?
Note:
last_4_bit is not the checksum field.
I'm able to see the ack number of the packets being incremented in tcpdump.
After a extended testing, I saw that, del ack_pkt[TCP].checksum deletes the checksum. But while converting to hex string with str(ack_pkt), I guess, it recalculates the checksum. After trying:
ack_pkt = sniff(offline="mptcpdemo.pcap", filter="tcp", count=15)[14]
del ack_pkt[TCP].chksum
print ack_pkt[TCP].chksum
print str(ack_pkt)
It 1st prints the checksum as None. But while printing the hex string, I'm able to see that the checksum field is non zero and contains the actual recalculated checksum.
In the non-optimized code, inside the loop, the packet is converted to hex-string and hence it's re-calculating the checksum each time. But in the optimized version, conversion is outside the loop and hence it carries one value only.
I have an Arduino board that puts out a string like this "287, 612, 109, 1134" every second. My code tries to read this string and convert it to wind speed and wind direction. Initially it works fine but eventually on one of the readines it reads only the first number in the string which causes an error. How can I have the code get the full string and not just the first number?
import serial, time, csv, decimal
#for Mac
port = '/dev/cu.usbserial-A401316V'
ser = serial.Serial(port , baudrate=57600, bytesize=8, parity=serial.PARITY_NONE, stopbits=1, timeout=2)
while True:
time.sleep(2)
data = ser.readline( eol="\r\n").strip()
data = data.split(',')
if len(data) > 0:
if data[0] != '2501':
s = float( 1000 / float(data[0]) )
print 'wind speed %.1f' %(round(float((2.5 * s)), 1))
print 'wind direction', round((int(data[1]) - (50)) * .39, 0)
print 'software version', data[2], '\n'
else:
print 'wind speed is zero'
print 'wind direction', round((int(data[1]) - (50)) * .39, 0)
print 'software version', data[2]
ser.close()
Without seeing more of the error data from the serial port, I can say that the most common reason you would see single numbers on lines (or even blank lines) is due to the non-synchronization between the Arduino program sending data to the port and your program trying to read that data. I'd wager if you print data after your data=ser.readline() and comment out the rest of the processing, you'd find lines like some of the following in the output:
252, 236, 218, 1136
251, 202, 215
2, 1353
199, 303, 200, 1000
259, 231, 245, 993
28
4, 144, 142, 1112
245, 199, 143, 1403
251, 19
2, 187, 1639
246, 235, 356, 1323
The reason for the data split across lines, or even blank lines, is that the program is trying to read data from the serial connection during or between writes. When this happens, the readline() gets what's available (even if partially/not written) and throws a \n on the end of what it found.
The fix for this is to ensure that the Arduino has finished sending data before we read, and that there is data there for us to read. A good way to handle this is with a generator that only yields lines to the main loop when they are complete (e.g. they end with a \r\n signifying the end of a write to the port).
def get_data():
ibuffer = "" # Buffer for raw data waiting to be processed
while True:
time.sleep(.1) # Best between .1 and 1
data = ser.read(4096) # May need to adjust read size
ibuffer += data # Concat data sets that were possibly split during reading
if '\r\n' in ibuffer: # If complete data set
line, ibuffer = ibuffer.split('\r\n', 1) # Split off first completed set
yield line.strip('\r\n') # Sanitize and Yield data
The yield makes this function a generator you can invoke to grab the next complete set of data while keeping any thing that was split by the read() in a buffer to await the next read() where the pieces of the data set can be concatenated. Think of yield like a return, but rather than passing a value and leaving the function/loop, it passes the value and waits for next() to be called where it will pick up where it left off for the next pass through the loop. With this function, the rest of your program will look something like this:
import serial, time, csv, decimal
port = '/dev/cu.usbserial-A401316V'
ser = serial.Serial(port , baudrate=57600, bytesize=8, parity=serial.PARITY_NONE, stopbits=1, timeout=2)
def get_data():
"""
The above function here
"""
ser_data = get_data()
while True:
data = ser_data.next().replace(' ','').split(',')
if len(data) > 0:
"""
data processing down here
"""
ser.close()
Depending on your set up, you may want to throw a conditional in get_data() to break, if the serial connection is lost for example, or a try/except around the data declaration.
It's worth noting that one thing I did change aside from the aforementioned is your ser.readline(eol) to a byte sized ser.read(4096). PySerial's readline() with eol is actually deprecated with the latest versions of Python.
Hopefully this helps; or if you have more problems, hopefully it at least gives you some ideas to get on the right track.