I wonder if someone can help me out?
I am reading one line at a time from the serial data on a pi in python.
while True:
try:
ser_bytes= ser.readline()
event = (ser_bytes[0:len(ser_bytes)-2].decode("utf-8"))
print(event)
except Exception as ex:
print(ex)
The data comes in the following format.
Initialize (one time event)
Name
Location
Type
Time/date
Name
Location
Type
Time/date
Name
Location
Type
Time/date
I would like to combine all 4 into a single string (Name-Location-Type-Time/date).
What would be the neatest way of achieving this?
Can't test this since there's no minimal reproducible example, but it should work fine:
ser.readline() # toss out the "initialize" event
lines = []
while True:
for _ in range(4):
new_event = []
try:
ser_bytes = ser.readline()
new_event.append(ser_bytes[0:len(ser_bytes)-2].decode("utf-8"))
except Exception as ex:
print(ex)
lines.append("-".join(new_event))
Basically just running a for loop which reads every four lines, joins them in the Name-Location-Type-Time/date format specified in the question, then adds that new string to an accumulator.
Related
I am trying to extract sensor data from Arduino. Arduino code sends sensor data using serial communication every 1 second. My goal is to display this data and plot it in real-time in a GUI using Tkinter. But where I seem to be stuck is retrieving data from Arduino. In my code, I tried to extract the data and save them in multiple variables. But when Arduino is sending only 5 seconds worth of data which is 5 sensor measurements, my python code is saving more than 60 measurements in the variable. I don't have any idea how to solve this. I have been stuck here for some time. Here is my python code.
In the given pictures I ran the code for 5 seconds. That is 5 measurements will be sent from Arduino. I am recieving the data correctly, but when I process them and seperate them in individual variables I have analog_voltage containing:80 , current:70, voltage:75 and temperature:65 measurements as they should only contain 5. What should I do, is there any way to go around this problem?
This is the same issue when I am trying to save the file. For the 5 seconds of data the program saves a million rows of data in csv. Any help is much appreciated. Thank you for reading.
The image for arduino data is just for reference as it is not possible to run both the programs in a same time due to the serial communication protocols hence the data looks different.
data_to_sort =[]
analog_voltage = []
Voltage= []
Current=[]
Temperature=[]
def arduino_read_serialdata():
global ser
ser = serial.Serial('COM4', 9600, timeout=1)
time.sleep(1.0)
ser.flushInput()
while True:
global analog_voltage, Voltage, Temperature, Current, yp, cleaned_data, anvol
incoming_data = ser.inWaiting()
incoming_data = ser.readline()
incoming_data = incoming_data.decode()
cleaned_data = incoming_data.strip("/n")
data_to_sort.append(cleaned_data)
for everyrow in data_to_sort:
if everyrow.startswith("Measuring voltage and current with INA219 ..."):
pass
elif everyrow.startswith("MAX31855 test"):
pass
elif everyrow.startswith("Initializing sensor...DONE checking reading the values now."):
pass
elif everyrow.startswith("The Analog Voltage is:"):
anvol = everyrow[-8:]
anvol = float(anvol)
analog_voltage.append(anvol)
elif everyrow.startswith("Bus Voltage............:"):
vol = everyrow[-8:]
Voltage.append(float(vol))
elif everyrow.startswith("Current..............:"):
cur = everyrow[-8:]
Current.append(float(cur))
elif everyrow.startswith("Temperature in celcius:"):
temp = everyrow[-7:]
Temperature.append(float(temp))
else:
pass
with open("Sensor_data.csv", 'w+', newline='') as output_file:
output_file.flush()
output_file = csv.writer(output_file)
fields = ['Analog Voltage V', 'Voltage V', 'Current mA', 'Temperature °C']
output_file.writerow(fields)
for idx in range(0, len(analog_voltage)):
output_file.writerow([analog_voltage[idx]])
if keyboard.is_pressed("q"):
print("Manually interrupted by user")
ser.close()
# output_file.close()
print("Serial port closed")
file.close()
print("File closed, Writing done")
You have an array
data_to_sort =[]
and you append each incoming line to it
data_to_sort.append(cleaned_data)
And with each incoming data, you process the whole data
for everyrow in data_to_sort:
and add the values to these four lists:
analog_voltage = []
Voltage= []
Current=[]
Temperature=[]
and these lists are never cleared.
What should be done instead:
either clear these four lists with each incoming data or
process only the incoming line
My personal preference would be the latter. Instead of always writing the whole output file, append only the last line to the output file using "a" instead of "w+".
I am repeating a sequence of experiments (with random initialization) like this in Python:
num_rounds = 1e3
result_list = []
for i in range(num_rounds):
result = run_experiment() # with random initialization inside the call of `run_experiment`
result_list.append(result)
However, my code can be buggy and exceptions might be thrown halfway, which will then interrupt the whole loop.
I'm wondering if there a way to:
let go the exception and continue the loop
meanwhile, store the condition (input parameters) that causes the exception(s) somewhere
gather basic statistics of different exceptions, for example the frequency of each exception type.
Or is there already a tool to do that?
1) Let go exception and continue loop:
try:
<whatever code can cause exception>
except <Exceptions you want to catch>:
pass
that will catch the exception and let it slip. Alternatively you can put a continue in the except block:
try:
<whatever code can cause exception>
except <Exceptions you want to catch>:
pass
2) and 3) With the code from 1. you can collect any data you want in the except block and then do a continue:
statistics_list = []
for <something>:
try:
<whatever code can cause exception>
except <Exceptions you want to catch>:
statistics_list.append((exception_type, input params....))
continue
after your loop you can process the data in your statistics list to your liking
There is try..except..else:
num_rounds = 1e3
result_list = []
exceptions = {}
for i in range(num_rounds):
values = 1,2,3, ... # with random initialization outside the call of `run_experiment`
try:
result = run_experiment(*values)
except Exception as e:
exceptions[values] = e
else:
result_list.append(result)
I think you will be able to do your statistics if modify your code like this
num_rounds = 1e3
result_list = []
for i in range(num_rounds):
try:
result = run_experiment()
result_list.append(result)
except Exception as ex:
# do statistics
That way you have the exception as ex, you can check and store the type.
If you also want to store the random input parameters than you should put the same try-except in your run_experiment function
So I'm sending a bunch of serial data from my Arduino 2560 Mega to my Python program, where I will be acting on integer data only. Initially, my Arduino calibrates a bunch of things, serially-printing confirmation information...then it starts to take temperature values from an LM35. Those temperature values then get serially printed.
I either:
a) Want to know how to use Python's readline() function when an integer is received starting when the temperature readings start getting printed serially.
OR
b) Test the incoming string from readline() from the start, determining when the numbers I care about start getting received.
And yes, treat these temperature values as integers not floats.
Right now, here's what I'm doing:
while(1==1):
if (s.inWaiting() > 0):
myData = s.readline()
time = myData
value = int(time.strip('\0'))
if (time.isDigit()):
# do stuff
I receive the error:
value = int(time.strip('\0'))
ValueError: invalid literal for int() with base 10: 'Obtaining color values for: Saline/Air\r\n'
Which makes sense because the string literal 'Obtaining color values for:Saline/Air\r\n', even after stripping, would never convert via the int() function.
Also let me know if .isDigit() is even necessary (or for that matter, being used correctly). I just started working with Python a couple weeks ago so it's all up in the air from my perspective.
You can do the following to convert a string to an integer:
while(1==1):
if (s.inWaiting() > 0):
myData = s.readline()
try:
# This will make it an integer, if the string is not an integer it will throw an error
myData = int(myData)
except ValueError: # this deals will the error
pass # if we don't change the value of myData it stays a string
Example
Here is an example you can try.
In Python:
import serial
# Converts to an integer if it is an integer, or it returns it as a string
def try_parse_int(s):
try:
return int(s)
except ValueError:
return s
ser = serial.Serial('/dev/ttyACM0', 115200)
while True:
data = ser.readline().decode("utf-8").strip('\n').strip('\r') # remove newline and carriage return characters
print("We got: '{}'".format(data))
data = try_parse_int(data)
print(type(data))
On my Arduino:
void setup() {
// put your setup code here, to run once:
Serial.begin(115200);
}
void loop() {
// put your main code here, to run repeatedly:
Serial.println(1);
delay(1000);
Serial.println("test");
delay(1000);
}
This will produce:
We got: 'test'
<class 'str'>
We got: '1'
<class 'int'>
You should catch the exception.
while True:
if s.inWaiting() > 0:
myData = s.readline()
try:
value = int(mydata.strip('\0'))
except ValueError:
# handle string data (in mydata)
else:
# handle temperature reading (in value)
So if the value can be converted to an int, the else block runs. If the value is a string, the except block runs.
I am writing a python script that will communicate to a Fluke meter over a COM port. I am able to receive the data but want to parse it into a usable float. The code looks like this:
import serial
ser = serial.Serial('COM3', 115200, timeout=1)
#Decalring some variables
FlukeID = b'ID\r'
FlukeQM = b'QM\r'
#Requesting the meters ID to verify connection on terminal
ser.writelines(FlukeID)
line = ser.readline()
print(line)
#Declaring variables for my while loop
thermdata = 0
t=1
ser.writelines(FlukeQM)
thermdata = ser.readline()
while(t < 5):
ser.writelines(FlukeQM)
#thermdata = (thermdata + ser.readline()) /2
thermdata = ser.readline()
print(thermdata)
t+=1
The data returned by the device looks like this on the console:
8.597E3,OHM,NORMAL,NONE INCORRECT
EDIT: The data actually appears like this over the terminal:
b'0\r8.597E3,OHM,NORMAL,NONE\r'
I just want to be able to use the numerical value at the beginning so I can do some calculations over time. I also need to be able to use the scientific notion portion in my number as I will not know the range of my measurements before hand. I know there must be a simple way to do this and would greatly appreciate any help.
On a side note, I would also like to be able to graph these values or place them into some kind of .csv file. If you have any comments on where to look to learn how to do this also that would be great, but I am mostly concerned with the handling of the bytearray.
Use split() to break your string into the comma separated parts. Then the first part is the string '8.597E3', which you convert using the float() function.
s = '8.597E3,OHM,NORMAL,NONE'.split(',')
value = float(s[0])
How about something like:
def atof(text):
try:
return float(text)
except ValueError:
return text
thermdata = b'0\r8.597E3,OHM,NORMAL,NONE\r'
for line in thermdata.strip().split(b'\r'):
print(list(map(atof, line.split(b','))))
# [0.0]
# [8597.0, b'OHM', b'NORMAL', b'NONE']
I would like to be able to have a series of nested loops that use the same pickle file. See below:
def pickleRead(self):
try:
with open(r'myfile', 'rb') as file:
print 'Reading File...'
while True:
try:
main = pickle.load(file)
id = main[0]
text = main[1]
while True:
try:
data = pickle.load(file)
data_id = data[0]
data_text = data[1]
coefficient = Similarity().jaccard(text.split(),data_text.split())
if coefficient > 0 and data_text is not None:
print str(id) + '\t' + str(data_id) + '\t' + str(coefficient)
except EOFError:
break
except Exception as err:
print err
except EOFError:
break
print 'Done Reading File...'
file.close()
except Exception as err:
print err
The second (inner) loop runs without any problems but the first one just does a single iteration and then stops. I am trying to grab a single row at a time then compare it against every other row in the file. There are several thousand rows and I have found that the cPickle module out performs anything similar. The problem is that it is limited in what is exposed. Can anyone point me in the right direction?
The inner loop only stops when it hits an EOFError while reading the file, so by the time you get to what would have been the second iteration of the outer loop, you've read the entire file. So trying to read more just gives you another EOFError, and you're out.
First, I should say ben w's answer does explain the behavior you're experiencing.
As for your broader question of "how do I accomplish my task using Python?" I recommend just using a single loop through the file to load all the pickled objects into a data structure in memory (a dictionary with IDs as keys and text as values seems like a natural choice). Once all the objects are loaded, you don't mess with the file at all; just use the in-memory data structure. You can use your existing nested-loop logic, if you like. It might look something like (pseudocode)
for k1 in mydict:
for k2 in mydict:
if k1 != k2:
do_comparison(mydict[k1], mydict[k2])