We Keep Coding

Simple python script using 100% CPU on rPi3B+

I have raspberry pi model 3b+ with a HC-SR04 ultrasonic distance sensor (there is also a couple of ds18b20 and a DHT21 but I think they're unrelated to my problem).
I have found a python script to make measurements from it and modified it to my needs - mostly to take a couple of reading spanned in time, take an average from it and map the value to range from 0 to 100, as the percentage and commit it to the influx database for grafana and domoticz.
The code:
#source: https://tutorials-raspberrypi.com/raspberry-pi-ultrasonic-sensor-hc-sr04/
#Libraries
import RPi.GPIO as GPIO
import time
from influxdb import InfluxDBClient
import sys
# https://www.domoticz.com/wiki/Domoticz_API/JSON_URL%27s#Python
import requests
client = InfluxDBClient(database='pellet')
series = []
#GPIO Mode (BOARD / BCM)
GPIO.setmode(GPIO.BCM)
#set GPIO Pins
GPIO_TRIGGER = 23
GPIO_ECHO = 22
#set GPIO direction (IN / OUT)
GPIO.setup(GPIO_TRIGGER, GPIO.OUT)
GPIO.setup(GPIO_ECHO, GPIO.IN)
def distance():
# set Trigger to HIGH
GPIO.output(GPIO_TRIGGER, True)
# set Trigger after 0.01ms to LOW
time.sleep(0.00001)
GPIO.output(GPIO_TRIGGER, False)
StartTime = time.time()
StopTime = time.time()
# save StartTime
while GPIO.input(GPIO_ECHO) == 0:
StartTime = time.time()
# save time of arrival
while GPIO.input(GPIO_ECHO) == 1:
StopTime = time.time()
# time difference between start and arrival
TimeElapsed = StopTime - StartTime
# multiply with the sonic speed (34300 cm/s)
# and divide by 2, because there and back
distance = (TimeElapsed * 34300) / 2
return distance
def pellet(dist):
# zmierzona odleglosc
# dist = distance()
# do zmierzenia poziom maksymalny
# 63 - do pokrywy
in_min = 63
# do zmierzenia poziom minimalny
in_max = in_min + 100
#wyjscie jako procent, od 0 do 100
out_min = 100
out_max = 0
# map z arduino: https://www.arduino.cc/reference/en/language/functions/math/map/
return (dist - in_min) * (out_max - out_min) / (in_max - in_min) + out_min;
def loop():
# nie wiecej jak 200 iteracji
loop = 200
# suma
total = 0
# tabelka z pojedynczmi wynikami
measurements = []
# liczba pomiarow do zrobienia
counter = 10
counter1 = 0
# czas pomiedzy pomiarami
sleep =30
#
while loop > 0:
loop -= 1
time.sleep(sleep)
# koniec, jesli wykonano liczbe pomiarow
if counter == 0:
#print(total/10)
return pellet(total/10), measurements
break
if loop == 0 and counter1 != 0:
return pellet(total/counter1), measurements
break
if loop == 0 and (counter1 == 0 or total == 0):
GPIO.cleanup()
sys.exit()
dist = distance()
# jesli wynik jest zly
if dist < 63 or dist > 163:
print("nie ok")
continue
counter -= 1
measurements.append(dist)
counter1 += 1
total += dist
print("total po ",counter1 , "sek: ", total, "dist: ", dist)
print(total/10)
#return total/10
if __name__ == '__main__':
try:
#dist = distance()
#print ("Measured Distance = %.1f cm" % dist)
#print (pellet(dist))
loop=loop()
print("avg :", loop[0])
#print("measurs :", loop[1])
#print("test :", loop[1][2])
if (1):
point = {
"measurement": "pellet",
"tags": {
"location": "piwnica",
"type": "hc-sr04"
},
"fields": {
"value": loop[0],
"raw_measurement1": loop[1][0],
"raw_measurement2": loop[1][1],
"raw_measurement3": loop[1][2],
"raw_measurement4": loop[1][3],
"raw_measurement5": loop[1][4],
"raw_measurement6": loop[1][5],
"raw_measurement7": loop[1][6],
"raw_measurement8": loop[1][7],
"raw_measurement9": loop[1][8],
"raw_measurement10": loop[1][9]
}
}
series.append(point)
client.write_points(series)
url = 'http://localhost:8080/json.htm?type=command&param=udevice&nvalue=0&idx=13&svalue='+str(loop[0])
r = requests.get(url)
GPIO.cleanup()
# Reset by pressing CTRL + C
except KeyboardInterrupt:
print("Measurement stopped by User")
GPIO.cleanup()
The problem is I noticed that the CPU temperature graph was elevated, with many short valleys to the about correct temperature.
When I ssh'd to the pi and run htop I saw that it was this script that is using 100% cpu.
But the weirdest thing is that the script is running in crontab every 15 minutes since yesterday, from about 14:30 and raise CPU temp started today at around 11:00.
I'm not a developer or a programmer and I just mostly copied the code from around the web so I don't know if this is some part of the code that did this (but why after 21 hours?) or what and why, and how to debug and fix it.
so it isn't just enviromental thing as the pi is in the attic where is about 5C to 10C.
Thank you for your help.

Here:
while GPIO.input(GPIO_ECHO) == 0:
StartTime = time.time()
this says "if the pin is 0, save the time, if the pin is zero, save the time, if the pin...." incessantly. You'll want to wait a little time after each check
while GPIO.input(GPIO_ECHO) == 0:
time.sleep(0.001) # 1 ms
StartTime = time.time()
The check itself probably takes ~us, so this will reduce CPU usage by 99%. You might want to do the same for the pin==1 case, depending on how accurate you need the times to be.

While it is impossible to know for sure where the issue lies without debugging directly on your system, and a glance at the code reveals several possible bugs in the logic, the one place that is most likely to cause the issue is the distance function.
As #mdurant already pointed out, your read loops will jump the CPU usage to 100%, but I suspect there is also another issue:
The trigger code and the read code are time sensitive!
The problem is, we don't know how much time actually passes between
# set Trigger to HIGH
GPIO.output(GPIO_TRIGGER, True)
# set Trigger after 0.01ms to LOW
time.sleep(0.00001)
GPIO.output(GPIO_TRIGGER, False)
and:
# save StartTime
while GPIO.input(GPIO_ECHO) == 0:
StartTime = time.time()
# save time of arrival
while GPIO.input(GPIO_ECHO) == 1:
StopTime = time.time()
While this simple algorithm - pulse trigger, count return interval will work on a microcontroller like Arduino, it is not reliable on a full blown computer like Raspberry Pi.
Microcontrollers run a single thread, with no OS or task scheduling, so they run code in real time or as close to it as possible (borrowing a few interrupts here and there).
But in your case you are running an interpreted language on a multitasking operating system, without explicitly giving it any high priority.
This means, your process could be suspended just enough time to miss the return "ping" and get stuck in the first loop.
This may only happen rarely when something else puts a load on the Pi, which would explain why you only noticed the issue after 21 hours running.
You should implement some form of timeout for GPIO reading loops and return an error value from the distance function if that timeout is reached to ensure you do not have an infinite loop when something goes wrong with the hardware, or you miss the return ping due to scheduling issues.
I suggest something like this:
def distance():
MAX_READ_ATTEMPTS = 10000 #this is a random value I chose. You will need to fine-tune it based on actual hardware performance
# set Trigger to HIGH
GPIO.output(GPIO_TRIGGER, True)
# set Trigger after 0.01ms to LOW
time.sleep(0.00001)
GPIO.output(GPIO_TRIGGER, False)
# lets not have any unneeded code here, just in case
# save StartTime
while GPIO.input(GPIO_ECHO) == 0 and retry_counter < MAX_READ_ATTEMPTS:
retry_counter += 1
StartTime = time.time()
# maximum number of retries reached, returning with error condition
if retry_counter == MAX_READ_ATTEMPTS:
return -1
reatry_counter = 0
# save time of arrival
while GPIO.input(GPIO_ECHO) == 1 and retry_counter < MAX_READ_ATTEMPTS:
retry_counter += 1
StopTime = time.time()
# maximum number of retries reached, returning with error condition
if retry_counter == MAX_READ_ATTEMPTS:
return -1
# time difference between start and arrival
TimeElapsed = StopTime - StartTime
# multiply with the sonic speed (34300 cm/s)
# and divide by 2, because there and back
distance = (TimeElapsed * 34300) / 2
return distance
I am intentionally not adding a delay between reads, because I am not sure what the tolerance for this measurement is in terms of timing, and sleep functions can't guarantee an exact delay (again, due to OS / CPU scheduling).
A brief 100% CPU load should be worth it to ensure accurate and valid measurements, as long as it is not kept up for too long, which our retry counting method should prevent.
Note that my only experience with ultrasonic sensors is using Arduino where a special pulseIn function is used that takes care of the implementation details of measurement so this solution is mostly an educated guess and I have no way of testing it.

Array and matplotlib in Python

Python newbie here.
Currently I'm making a heartbeat signal detector with an Arduino for my school project.
I want to draw a plotter graph with a signal value that I sent from Arduino to Python, but I'm still not sure exact way to make a correct array and plot it in Python.
My arduino is connected with bluetooth module, and it is continuously send double integer value around 30~40 times/second.
In Python,I want to write a script that will receive 20 seconds of data, store it in a file, and plot it.
I built my code based on my basic C knowledge.
import serial, sys
from time import sleep
import time
import matplotlib.pyplot as plt
def read_BLE( ser ):
msg = ""
if( ser.in_waiting > 0 ):
msg = ser.readline( ser.in_waiting ).decode('utf-8')
return msg
with serial.Serial(port='COM7', baudrate=9600, timeout=1) as ser:
while(True) :
max_time = int(input('20')) #limit time : 20seconds
start_time = time.time() # remember when we started
while (time.time() - start_time) < max_time:
values = []
currenttime = []
i,j=0
currenttime[i] = (time.time() - start_time)
values[j] = read_BLE( ser )
i += 1
j += 1
plt.plot(currenttime, values)

Restructured part of your code, fixed a couple of breaking bugs, now it should work assuming your other parts of code is correct:
with serial.Serial(port='COM7', baudrate=9600, timeout=1) as ser:
values = [] # Initiate your list before the loop!
currenttime = []
max_time = 20 #limit time : 20seconds
start_time = time.time() # remember when we started
while True:
if (time.time() - start_time) > max_time:
break # if time elapsed larger than max_time, break out of loop
currenttime.append(time.time() - start_time)
values.append(read_BLE(ser)) # python list has no set length. just append to it.
plt.plot(currenttime, values)
Python list has no fixed memory allocation, .append will always add a element without causing memory issue -> It's just reference passing really. You also want to initiate your list before loop other wise each loop will overwrite and have a new list

Python Serial communition with device

enter image description here
I am making serial port communication device from a genious guy's work instructables.com.
It will measure the distance of the hamster's running in a day or month.
Using 4, 6 pin of the serial port cable, if the hamster runs, the device can count the numbers how many time did she run.
When I run the py file with Python27 like below, some errors occure.
"python hamster-serial.py progress.txt"
I cannot understand what's going on.
I am using windows8 and Python2.7 version.
Could you check my source, please?
import datetime
import serial
import sys
# Check for commandline argument. The first argument is the the name of the program.
if len(sys.argv) < 2:
print "Usage: python %s [Out File]" % sys.argv[0]
exit()
# Open the serial port we'll use the pins on and the file we'll write to.
ser = serial.Serial("/dev/ttyS1")
# Open the file we're going to write the results to.
f = open(sys.argv[1], 'a')
# Bring DTR to 1. This will be shorted to DSR when the switch is activated as the wheel turns.
ser.setDTR(1)
# The circumferance of the wheel.
circ = 0.000396 # miles
# Total distance traveled in this run of the program.
distance = 0.0
print "%s] Starting logging." % datetime.datetime.now()
start = datetime.datetime.now()
# This function a period of the wheel to a speed of the hamster.
def toSpeed(period):
global circ
seconds = period.days * 24 * 60 * 60 + period.seconds + period.microseconds / 1000000.
return circ / (seconds / 60. / 60.)
# Waits for the DSR pin on the serial port to turn off. This indicates that the
# switch has turned off and the magnet is no longer over the switch.
def waitForPinOff():
while ser.getDSR() == 1:
1 # Don't do anything while we wait.
# Waits for the DSR pin on the serial port to turn on. This indicates that the
# switch has turned on and the magnet is current over the switch.
def waitForPinOn():
while ser.getDSR() == 0:
1 # Don't do anything while we wait.
# The main loop of the program.
while 1:
waitForPinOn()
# Calculate the speed.
end = datetime.datetime.now()
period = end - start
start = end
speed = toSpeed(period)
# Increment the distance.
distance = distance + circ
waitForPinOff()
# We'll calculate the time the switch was held on too so but this isn't too useful.
hold = datetime.datetime.now() - start
# If the switch bounces or the hamster doesn't make a full revolution then
# it might seem like the hamster is running really fast. If the speed is
# more than 4 mph then ignore it, because the hamster can't run that fast.
if speed < 4.0:
# Print out our speed and distance for this session.
print "%s] Distance: %.4f miles Speed: %.2f mph" % (datetime.datetime.now(), distance, speed)
# Log it to and flush the file so it actually gets written.
f.write("%s\t%.2f\n" % (datetime.datetime.now().strftime("%D %T"), speed))
f.flush()

Well, ser = serial.Serial("/dev/ttyS1") is for a linux machine, on windows you'll need something like ser = serial.Serial("COM1") (you can check what COM do you need in the device manager).
As a side note,
def waitForPinOff():
while ser.getDSR() == 1:
1 # Don't do anything while we wait.
Will eat you CPU. You are better of with:
def waitForPinOff():
while ser.getDSR() == 1:
time.sleep(1) # Don't do anything while we wait.

Serial data transmission input-output delay with Raspberry Pi

The Goal:
Drive several servo/RC motors wireless from one pi to another pi.
In essence, I want to build a RC remote control using a pi, with a second pion the receiver end.
Now I have a serial transmission over a RF link module that is stable and has very few corrupt entries. The serial transmission has a max baud rate of 4800 due to the RF link module.
Problem:
It seems there is a 2-4 seconds difference between the transmitter pi printing values and the receiver pi printing values. I cannot figure out where and why this delay comes from and why is it so large. Please note that the signal on the receiving pi is exactly the same data that is sent by the transmitter pi, but 2-4 seconds later. EVEN when I bypassed the the transmitter/receiver module and connected the Tx and Rx pins with a jumper wire the same delay was seen.
What is causing the data on the receiving Pi to be decoded so much later? I have pasted in the code below.
---------- Tx Pi -----------------
import serial
import struct
ser = serial.Serial("/dev/ttyAMA0")
ser.baudrate = 4800
iCount = 0
bProgramLoop = True
while (bProgramLoop == True):
iOne = iCount
if (iOne == 100):
iOne = 0
iTwo += 1
iCount = 0
if (iTwo == 100):
iTwo = 0
iThree += 1
if (iThree == 100):
iThree = 0
iFour += 1
if (iFour == 100):
iFour = 0
iFive += 1
if (iFive == 100):
iFive = 0
lData = [iOne,iTwo,iThree,iFour,iFive] # i have done it like this so that I can track in real time where the transmitter is and receiver is with respect to real time changes on the transmitter side
sData = struct.pack('5B',*lData)
ser.write(sData)
print sData
iCount += 1
-------------- Rx Pi -----------------
import serial
import struct
ser = serial.Serial("/dev/ttyAMA0")
ser.baudrate = 4800
bProgramLoop = True
while (bProgramLoop == True):
sSerialRead = ser.read(5)
tData = struct.unpack('5B',sSerialRead)
print tData
The time difference between when the Tx Pi prints the string sData and the Rx Pi prints the touple tData is somewhere between 2-4 seconds. Is the struct.unpack function slow?
I need this time difference to be absolutely minimal. Any ideas?

First, I'm not sure the ser.write() is an async call. If this is using the pySerial library, the docs say that it is a blocking call.
Perhaps try:
...
ser.write(sData)
ser.flush() # Force the 'send'
print sData
...
Also, your ldata might be easier to populate like so:
lData = [iCount % 100, iCount % 10000, ...etc]
Also, setting a write timeout might help (but I don't think so).

(Posted on behalf of the OP).
As suggested by Doddie use the ser.flush command just after ser.write. This results in a near instant response on the Rx side. The overall mainloop sample rate has dropped a bit, but for me at least that is not a deal breaker.

Single Ultrasonic Sensor for Raspberry pi 2B+ not functioning from Pi Terminal

I have been working with 4-pin HC-SRO4 ultrasonic sensors, up to four at a time. I have been developing code to make 4 of these sensors work simultaneously and after reorganizing wires for installation on a project and using the basic code to run one, I cannot make the sensor function. Code follows:
import RPi.GPIO as GPIO
import time
TRIG1 = 15
ECHO1 = 13
start1 = 0
stop1 = 0
GPIO.setmode(GPIO.BOARD)
GPIO.setwarnings(False)
GPIO.setup(TRIG1, GPIO.OUT)
GPIO.output(TRIG1, 0)
GPIO.setup(ECHO1, GPIO.IN)
while True:
time.sleep(0.1)
GPIO.output(TRIG1, 1)
time.sleep(0.00001)
GPIO.output(TRIG1, 0)
while GPIO.input(ECHO1) == 0:
start1 = time.time()
print("here")
while GPIO.input(ECHO1) == 1:
stop1 = time.time()
print("also here")
print("sensor 1:")
print (stop1-start1) * 17000
GPIO.cleanup()
After changing wires, sensors, and other components within the circuit (including the GPIO pins) I have looked at the code, and added print statements to the terminal to see which parts of the code are running. The 1st print statement
print("here")
executes consistently, but the second print statementprint("also here") does not, and I am at a loss for an explanation. In other words, why is the second while loop not being executed? Other questions asked here have not worked for my problem. Any help would be greatly appreciated.
Thanks,
H.

Here is a tutorial by Gaven MacDonald that might help with this : https://www.youtube.com/watch?v=xACy8l3LsXI
First of all, the while block with ECHO1 == 0 would loop forever until the ECHO1 becomes 1. In this period of time, the code inside would be executed again and again. You wouldn't want to set the time again and again, so you can do something like this:
while GPIO.input(ECHO1) == 0:
pass #This is here to make the while loop do nothing and check again.
start = time.time() #Here you set the start time.
while GPIO.input(ECHO1) == 1:
pass #Doing the same thing, looping until the condition is true.
stop = time.time()
print (stop - start) * 170 #Note that since both values are integers, python would multiply the value with 170. If our values were string, python would write the same string again and again: for 170 times.
Also, just as a best practice, you should use try except blocks to safely exit the code. Such as:
try:
while True:
#Code code code...
except KeyboardInterrupt: #This would check if you have pressed Ctrl+C
GPIO.cleanup()