Python newbie here.
Currently I'm making a heartbeat signal detector with an Arduino for my school project.
I want to draw a plotter graph with a signal value that I sent from Arduino to Python, but I'm still not sure exact way to make a correct array and plot it in Python.
My arduino is connected with bluetooth module, and it is continuously send double integer value around 30~40 times/second.
In Python,I want to write a script that will receive 20 seconds of data, store it in a file, and plot it.
I built my code based on my basic C knowledge.
import serial, sys
from time import sleep
import time
import matplotlib.pyplot as plt
def read_BLE( ser ):
msg = ""
if( ser.in_waiting > 0 ):
msg = ser.readline( ser.in_waiting ).decode('utf-8')
return msg
with serial.Serial(port='COM7', baudrate=9600, timeout=1) as ser:
while(True) :
max_time = int(input('20')) #limit time : 20seconds
start_time = time.time() # remember when we started
while (time.time() - start_time) < max_time:
values = []
currenttime = []
i,j=0
currenttime[i] = (time.time() - start_time)
values[j] = read_BLE( ser )
i += 1
j += 1
plt.plot(currenttime, values)
Restructured part of your code, fixed a couple of breaking bugs, now it should work assuming your other parts of code is correct:
with serial.Serial(port='COM7', baudrate=9600, timeout=1) as ser:
values = [] # Initiate your list before the loop!
currenttime = []
max_time = 20 #limit time : 20seconds
start_time = time.time() # remember when we started
while True:
if (time.time() - start_time) > max_time:
break # if time elapsed larger than max_time, break out of loop
currenttime.append(time.time() - start_time)
values.append(read_BLE(ser)) # python list has no set length. just append to it.
plt.plot(currenttime, values)
Python list has no fixed memory allocation, .append will always add a element without causing memory issue -> It's just reference passing really. You also want to initiate your list before loop other wise each loop will overwrite and have a new list
Related
I'm trying to use pyfirmata to use an Arduino Ultrasonic sensor. I used Arduino Uno board and HC-SR04 Ultrasonic sensor. Here is the code I'm using. The code ran smoothly, it's just that it seems the echo pin failed to get an impulse from the trigger ultrasonic sound, so it keeps on getting False (LOW reading) and thus giving me false distance reading. Does anyone have a solution for this problem?
import pyfirmata
import time
board = pyfirmata.Arduino('COM16')
start = 0
end = 0
echo = board.get_pin('d:11:i')
trig = board.get_pin('d:12:o')
LED = board.get_pin('d:13:o')
it = pyfirmata.util.Iterator(board)
it.start()
trig.write(0)
time.sleep(2)
while True:
time.sleep(0.5)
trig.write(1)
time.sleep(0.00001)
trig.write(0)
print(echo.read())
while echo.read() == False:
start = time.time()
while echo.read() == True:
end = time.time()
TimeElapsed = end - start
distance = (TimeElapsed * 34300) / 2
print("Measured Distance = {} cm".format(distance) )
I've tried changing the time.sleep() to several value and it still doesn't work. It works just fine when I'm using Arduino code dirrectly from Arduino IDE.
I haven't done the exact math but given a range of 50cm you're at about 3ms travel time. That would mean you need to turn off the pulse and poll the pin state within that time.
That's not going to happen. The echo probably arrives befor you have turned off the emitter through PyFirmata. You should do the delay measurement on the Arduino.
I solve this false data problem by counting. I observe that false data comes after 2 or 3 sec. So if it takes More than 2 or 3 sec I clear count and restarts it from 0;
Sudo code:
cnt = 0;
if sensorvalue <= 20 && sensorvalue <= 30:
cnt++;
if cnt>=5:
detected = true;
cnt =0;
if cnt<5 && lastDecttime>2 (2 sec):
cnt = 0; // Here we handle the false value and clear the data
I'm currently trying to work this exact problem out. I can get the sensor to work using the Arduino IDE directly, but not with python and pyfirmata. I am getting some output, but its mostly non-sensical.
Here's an example output I'm getting, while keeping the sensor at the same distance from my object:
817.1010613441467
536.828875541687
0.0
546.0820078849792
0.0
0.0
1060.0213408470154
Regarding your code, the only thing I can see that you could do differently is to use the board.pass_time function instead of time.sleep(). Let me know if you get anywhere!
import pyfirmata as pyf
import time
def ultra_test():
board = pyf.Arduino("COM10")
it = pyf.util.Iterator(board)
it.start()
trigpin = board.get_pin("d:7:o")
echopin = board.get_pin("d:8:i")
while True:
trigpin.write(0)
board.pass_time(0.5)
trigpin.write(1)
board.pass_time(0.00001)
trigpin.write(0)
limit_start = time.time()
while echopin.read() != 1:
if time.time() - limit_start > 1:
break
pass
start = time.time()
while echopin.read() != 0:
pass
stop = time.time()
time_elapsed = stop - start
print((time_elapsed) * 34300 / 2)
board.pass_time(1)
i need your help.
I need a non-bloking timer, that allows me, in the period it's still counting, doing other tasks.
I need this function for my bot and obviously I don't want to block it all times I call this type of function which requires these timers.
So, in the past i used to use in Arduino (c++) the function millis() that in the same configuration seems not working well like
int t0 =0
int t1
void loop(){
t1= millis()
while (t1-t0 < 6000){
Serial.print(Timer!);
t0 = millis();}}
Do you have any advice for me? A code where I can start from?
Thanks!
The following will print "Timer" for 6 seconds:
import time
start_time = time.time() # returns number of seconds passed since epoch
current_time = time.time()
max_loop_time = 6 # 6 seconds
while (current_time - start_time) <= max_loop_time:
# do stuff
print("Timer")
current_time = time.time()
Okay, i found the solution by myself, trying to remember what i previously did on Arduino.
I based this answer from Adam Minas's one, but mine is quite different
So the expected behavior had to be:
print(something) every 5 seconds so:
import time
start_time = time.time() # returns number of seconds passed since epoch
#current_time = time.time()
print(start_time)
max_loop_time = 20 # 6 seconds
while True:
while (time.time() - start_time) > max_loop_time:
print("Timer")
start_time = time.time()
Then you can stop your while loop with break and other functions like
if == smth :
break
I have raspberry pi model 3b+ with a HC-SR04 ultrasonic distance sensor (there is also a couple of ds18b20 and a DHT21 but I think they're unrelated to my problem).
I have found a python script to make measurements from it and modified it to my needs - mostly to take a couple of reading spanned in time, take an average from it and map the value to range from 0 to 100, as the percentage and commit it to the influx database for grafana and domoticz.
The code:
#source: https://tutorials-raspberrypi.com/raspberry-pi-ultrasonic-sensor-hc-sr04/
#Libraries
import RPi.GPIO as GPIO
import time
from influxdb import InfluxDBClient
import sys
# https://www.domoticz.com/wiki/Domoticz_API/JSON_URL%27s#Python
import requests
client = InfluxDBClient(database='pellet')
series = []
#GPIO Mode (BOARD / BCM)
GPIO.setmode(GPIO.BCM)
#set GPIO Pins
GPIO_TRIGGER = 23
GPIO_ECHO = 22
#set GPIO direction (IN / OUT)
GPIO.setup(GPIO_TRIGGER, GPIO.OUT)
GPIO.setup(GPIO_ECHO, GPIO.IN)
def distance():
# set Trigger to HIGH
GPIO.output(GPIO_TRIGGER, True)
# set Trigger after 0.01ms to LOW
time.sleep(0.00001)
GPIO.output(GPIO_TRIGGER, False)
StartTime = time.time()
StopTime = time.time()
# save StartTime
while GPIO.input(GPIO_ECHO) == 0:
StartTime = time.time()
# save time of arrival
while GPIO.input(GPIO_ECHO) == 1:
StopTime = time.time()
# time difference between start and arrival
TimeElapsed = StopTime - StartTime
# multiply with the sonic speed (34300 cm/s)
# and divide by 2, because there and back
distance = (TimeElapsed * 34300) / 2
return distance
def pellet(dist):
# zmierzona odleglosc
# dist = distance()
# do zmierzenia poziom maksymalny
# 63 - do pokrywy
in_min = 63
# do zmierzenia poziom minimalny
in_max = in_min + 100
#wyjscie jako procent, od 0 do 100
out_min = 100
out_max = 0
# map z arduino: https://www.arduino.cc/reference/en/language/functions/math/map/
return (dist - in_min) * (out_max - out_min) / (in_max - in_min) + out_min;
def loop():
# nie wiecej jak 200 iteracji
loop = 200
# suma
total = 0
# tabelka z pojedynczmi wynikami
measurements = []
# liczba pomiarow do zrobienia
counter = 10
counter1 = 0
# czas pomiedzy pomiarami
sleep =30
#
while loop > 0:
loop -= 1
time.sleep(sleep)
# koniec, jesli wykonano liczbe pomiarow
if counter == 0:
#print(total/10)
return pellet(total/10), measurements
break
if loop == 0 and counter1 != 0:
return pellet(total/counter1), measurements
break
if loop == 0 and (counter1 == 0 or total == 0):
GPIO.cleanup()
sys.exit()
dist = distance()
# jesli wynik jest zly
if dist < 63 or dist > 163:
print("nie ok")
continue
counter -= 1
measurements.append(dist)
counter1 += 1
total += dist
print("total po ",counter1 , "sek: ", total, "dist: ", dist)
print(total/10)
#return total/10
if __name__ == '__main__':
try:
#dist = distance()
#print ("Measured Distance = %.1f cm" % dist)
#print (pellet(dist))
loop=loop()
print("avg :", loop[0])
#print("measurs :", loop[1])
#print("test :", loop[1][2])
if (1):
point = {
"measurement": "pellet",
"tags": {
"location": "piwnica",
"type": "hc-sr04"
},
"fields": {
"value": loop[0],
"raw_measurement1": loop[1][0],
"raw_measurement2": loop[1][1],
"raw_measurement3": loop[1][2],
"raw_measurement4": loop[1][3],
"raw_measurement5": loop[1][4],
"raw_measurement6": loop[1][5],
"raw_measurement7": loop[1][6],
"raw_measurement8": loop[1][7],
"raw_measurement9": loop[1][8],
"raw_measurement10": loop[1][9]
}
}
series.append(point)
client.write_points(series)
url = 'http://localhost:8080/json.htm?type=command¶m=udevice&nvalue=0&idx=13&svalue='+str(loop[0])
r = requests.get(url)
GPIO.cleanup()
# Reset by pressing CTRL + C
except KeyboardInterrupt:
print("Measurement stopped by User")
GPIO.cleanup()
The problem is I noticed that the CPU temperature graph was elevated, with many short valleys to the about correct temperature.
When I ssh'd to the pi and run htop I saw that it was this script that is using 100% cpu.
But the weirdest thing is that the script is running in crontab every 15 minutes since yesterday, from about 14:30 and raise CPU temp started today at around 11:00.
I'm not a developer or a programmer and I just mostly copied the code from around the web so I don't know if this is some part of the code that did this (but why after 21 hours?) or what and why, and how to debug and fix it.
so it isn't just enviromental thing as the pi is in the attic where is about 5C to 10C.
Thank you for your help.
Here:
while GPIO.input(GPIO_ECHO) == 0:
StartTime = time.time()
this says "if the pin is 0, save the time, if the pin is zero, save the time, if the pin...." incessantly. You'll want to wait a little time after each check
while GPIO.input(GPIO_ECHO) == 0:
time.sleep(0.001) # 1 ms
StartTime = time.time()
The check itself probably takes ~us, so this will reduce CPU usage by 99%. You might want to do the same for the pin==1 case, depending on how accurate you need the times to be.
While it is impossible to know for sure where the issue lies without debugging directly on your system, and a glance at the code reveals several possible bugs in the logic, the one place that is most likely to cause the issue is the distance function.
As #mdurant already pointed out, your read loops will jump the CPU usage to 100%, but I suspect there is also another issue:
The trigger code and the read code are time sensitive!
The problem is, we don't know how much time actually passes between
# set Trigger to HIGH
GPIO.output(GPIO_TRIGGER, True)
# set Trigger after 0.01ms to LOW
time.sleep(0.00001)
GPIO.output(GPIO_TRIGGER, False)
and:
# save StartTime
while GPIO.input(GPIO_ECHO) == 0:
StartTime = time.time()
# save time of arrival
while GPIO.input(GPIO_ECHO) == 1:
StopTime = time.time()
While this simple algorithm - pulse trigger, count return interval will work on a microcontroller like Arduino, it is not reliable on a full blown computer like Raspberry Pi.
Microcontrollers run a single thread, with no OS or task scheduling, so they run code in real time or as close to it as possible (borrowing a few interrupts here and there).
But in your case you are running an interpreted language on a multitasking operating system, without explicitly giving it any high priority.
This means, your process could be suspended just enough time to miss the return "ping" and get stuck in the first loop.
This may only happen rarely when something else puts a load on the Pi, which would explain why you only noticed the issue after 21 hours running.
You should implement some form of timeout for GPIO reading loops and return an error value from the distance function if that timeout is reached to ensure you do not have an infinite loop when something goes wrong with the hardware, or you miss the return ping due to scheduling issues.
I suggest something like this:
def distance():
MAX_READ_ATTEMPTS = 10000 #this is a random value I chose. You will need to fine-tune it based on actual hardware performance
# set Trigger to HIGH
GPIO.output(GPIO_TRIGGER, True)
# set Trigger after 0.01ms to LOW
time.sleep(0.00001)
GPIO.output(GPIO_TRIGGER, False)
# lets not have any unneeded code here, just in case
# save StartTime
while GPIO.input(GPIO_ECHO) == 0 and retry_counter < MAX_READ_ATTEMPTS:
retry_counter += 1
StartTime = time.time()
# maximum number of retries reached, returning with error condition
if retry_counter == MAX_READ_ATTEMPTS:
return -1
reatry_counter = 0
# save time of arrival
while GPIO.input(GPIO_ECHO) == 1 and retry_counter < MAX_READ_ATTEMPTS:
retry_counter += 1
StopTime = time.time()
# maximum number of retries reached, returning with error condition
if retry_counter == MAX_READ_ATTEMPTS:
return -1
# time difference between start and arrival
TimeElapsed = StopTime - StartTime
# multiply with the sonic speed (34300 cm/s)
# and divide by 2, because there and back
distance = (TimeElapsed * 34300) / 2
return distance
I am intentionally not adding a delay between reads, because I am not sure what the tolerance for this measurement is in terms of timing, and sleep functions can't guarantee an exact delay (again, due to OS / CPU scheduling).
A brief 100% CPU load should be worth it to ensure accurate and valid measurements, as long as it is not kept up for too long, which our retry counting method should prevent.
Note that my only experience with ultrasonic sensors is using Arduino where a special pulseIn function is used that takes care of the implementation details of measurement so this solution is mostly an educated guess and I have no way of testing it.
How would i implement processing of an array with millisecond precision using python under linux (running on a single core Raspberry Pi).
I am trying to parse information from a MIDI file, which has been preprocessed to an array where each millisecond i check if the array has entries at the current timestamp and trigger some functions if it does.
Currently i am using time.time() and employ busy waiting (as concluded here). This eats up all the CPU, therefor i opt for a better solution.
# iterate through all milliseconds
for current_ms in xrange(0, last+1):
start = time()
# check if events are to be processed
try:
events = allEvents[current_ms]
# iterate over all events for this millisecond
for event in events:
# check if event contains note information
if 'note' in event:
# check if mapping to pin exists
if event['note'] in mapping:
pin = mapping[event['note']]
# check if event contains on/off information
if 'mode' in event:
if event['mode'] == 0:
pin_off(pin)
elif event['mode'] == 1:
pin_on(pin)
else:
debug("unknown mode in event:"+event)
else:
debug("no mapping for note:" + event['note'])
except:
pass
end = time()
# fill the rest of the millisecond
while (end-start) < (1.0/(1000.0)):
end = time()
where last is the millisecond of the last event (known from preprocessing)
This is not a question about time() vs clock() more about sleep vs busy wait.
I cant really sleep in the "fill rest of millisecond" loop, because of the too low accuracy of sleep(). If i were to use ctypes, how would i go about it correctly?
Is there some Timer library which would call a callback each millisecond reliably?
My current implementation is on GitHub. With this approach i get a skew of around 4 or 5ms on the drum_sample, which is 3.7s total (with mock, so no real hardware attached). On a 30.7s sample, the skew is around 32ms (so its at least not linear!).
I have tried using time.sleep() and nanosleep() via ctypes with the following code
import time
import timeit
import ctypes
libc = ctypes.CDLL('libc.so.6')
class Timespec(ctypes.Structure):
""" timespec struct for nanosleep, see:
http://linux.die.net/man/2/nanosleep """
_fields_ = [('tv_sec', ctypes.c_long),
('tv_nsec', ctypes.c_long)]
libc.nanosleep.argtypes = [ctypes.POINTER(Timespec),
ctypes.POINTER(Timespec)]
nanosleep_req = Timespec()
nanosleep_rem = Timespec()
def nsleep(us):
#print('nsleep: {0:.9f}'.format(us))
""" Delay microseconds with libc nanosleep() using ctypes. """
if (us >= 1000000):
sec = us/1000000
us %= 1000000
else: sec = 0
nanosleep_req.tv_sec = sec
nanosleep_req.tv_nsec = int(us * 1000)
libc.nanosleep(nanosleep_req, nanosleep_rem)
LOOPS = 10000
def do_sleep(min_sleep):
#print('try: {0:.9f}'.format(min_sleep))
total = 0.0
for i in xrange(0, LOOPS):
start = timeit.default_timer()
nsleep(min_sleep*1000*1000)
#time.sleep(min_sleep)
end = timeit.default_timer()
total += end - start
return (total / LOOPS)
iterations = 5
iteration = 1
min_sleep = 0.001
result = None
while True:
result = do_sleep(min_sleep)
#print('res: {0:.9f}'.format(result))
if result > 1.5 * min_sleep:
if iteration > iterations:
break
else:
min_sleep = result
iteration += 1
else:
min_sleep /= 2.0
print('FIN: {0:.9f}'.format(result))
The result on my i5 is
FIN: 0.000165443
while on the RPi it is
FIN: 0.000578617
which suggest a sleep period of about 0.1 or 0.5 milliseconds, with the given jitter (tend to sleep longer) that at most helps me reduce the load a little bit.
One possible solution, using the sched module:
import sched
import time
def f(t0):
print 'Time elapsed since t0:', time.time() - t0
s = sched.scheduler(time.time, time.sleep)
for i in range(10):
s.enterabs(t0 + 10 + i, 0, f, (t0,))
s.run()
Result:
Time elapsed since t0: 10.0058200359
Time elapsed since t0: 11.0022959709
Time elapsed since t0: 12.0017120838
Time elapsed since t0: 13.0022599697
Time elapsed since t0: 14.0022521019
Time elapsed since t0: 15.0015859604
Time elapsed since t0: 16.0023040771
Time elapsed since t0: 17.0023028851
Time elapsed since t0: 18.0023078918
Time elapsed since t0: 19.002286911
Apart from some constant offset of about 2 millisecond (which you could calibrate), the jitter seems to be on the order of 1 or 2 millisecond (as reported by time.time itself). Not sure if that is good enough for your application.
If you need to do some useful work in the meantime, you should look into multi-threading or multi-processing.
Note: a standard Linux distribution that runs on a RPi is not a hard real-time operating system. Also Python can show non-deterministic timing, e.g. when it starts a garbage collection. So your code might run fine with low jitter most of the time, but you might have occasional 'hickups', where there is a bit of delay.
I wrote a system to exchange crc-checked struct data between an arduino nano and my python script. This is working pretty well but when i let the system run i get unexpected output on my python monitor (using pycharm)
print "Took ", (time.time() - timeout), " s" sometimes prints out Took 0.0 s.
Usually it prints Took 0.0160000324249 s.
Using win7-64bit professional.
From time doc : Return the time in seconds since the epoch as a floating point number. Note that even though the time is always returned as a floating point number, not all systems provide time with a better precision than 1 second.
I´m looking for something like millis() thats enough precision for my case
Code Python :
import serial
import time
import binascii
import struct
from ctypes import *
arduino = serial.Serial()
def receive_struct2():
start = 0x85
detected_start = False
arduino.baudrate = 57600
arduino.timeout = 0
arduino.port = 'COM8'
try:
arduino.open()
except serial.SerialException, e:
print e
while True:
if(arduino.inWaiting() >= 1 and detected_start == False):
data = ord(arduino.read())
if data == start:
print "Detected begin"
detected_start = True
else: print chr(data),
if arduino.inWaiting() >= 1 and detected_start == True:
message_length = ord(arduino.read())
#print "Got message length ", message_length
timeout = time.time()
while time.time() - timeout <= 0.3 and arduino.inWaiting() < message_length-1:pass
print "Took ", (time.time() - timeout), " s"
....
Summary: Use timeit.default_timer() instead of time.time(), to measure a short duration.
16ms error for time.time() is not surprising on Windows.
Currently, Python uses GetSystemTimeAsFileTime() to implement time.time() on Windows that has the resolution (precision) of 0.1ms (instead of ftime()'s 1ms) and the accuracy is between 0.5 ms and 15 ms (you could change it system-wide using NtSetTimerResolution()). See Python bug: Use GetSystemTimeAsFileTime() to get a resolution of 100 ns on Windows and another SO question: How does python's time.time() method work?
A better alternative to measure short time intervals on Windows is to use time.clock() that is implemented using QueryPerformanceCounter() on Windows. For portability, you could use timeit.default_timer that is assigned to time.clock() on Windows, time.time() on other systems, and it is time.perf_counter() since Python3.3. See Python - time.clock() vs. time.time() - accuracy?