We Keep Coding

Read multiple DS18B20 temperature sensors faster using Raspberry Pi

My custom sensor dashboard requests new readings every second.
This worked well, until I hooked up 3 DS18B20 temperature sensors (1-wire protocol, so all on 1 pin), which each take 750ms to provide new data.
This is the class I currently use to read the temperature of each sensor:
# ds18b20.py
# written by Roger Woollett
import os
import glob
import time
class DS18B20:
# much of this code is lifted from Adafruit web site
# This class can be used to access one or more DS18B20 temperature sensors
# It uses OS supplied drivers and one wire support must be enabled
# To do this add the line
# dtoverlay=w1-gpio
# to the end of /boot/config.txt
#
# The DS18B20 has three pins, looking at the flat side with the pins pointing
# down pin 1 is on the left
# connect pin 1 to GPIO ground
# connect pin 2 to GPIO 4 *and* GPIO 3.3V via a 4k8 (4800 ohm) pullup resistor
# connect pin 3 to GPIO 3.3V
# You can connect more than one sensor to the same set of pins
# Only one pullup resistor is required
def __init__(self):
# Load required kernel modules
os.system('modprobe w1-gpio')
os.system('modprobe w1-therm')
# Find file names for the sensor(s)
base_dir = '/sys/bus/w1/devices/'
device_folder = glob.glob(base_dir + '28*')
self._num_devices = len(device_folder)
self._device_file = list()
i = 0
while i < self._num_devices:
self._device_file.append(device_folder[i] + '/w1_slave')
i += 1
def _read_temp(self, index):
# Issue one read to one sensor
# You should not call this directly
# First check if this index exists
if index >= len(self._device_file):
return False
f = open(self._device_file[index], 'r')
data = f.read()
f.close()
return data
def tempC(self, index=0):
# Call this to get the temperature in degrees C
# detected by a sensor
data = self._read_temp(index)
retries = 0
# Check for error
if data == False:
return None
while (not "YES" in data) and (retries > 0):
# Read failed so try again
time.sleep(0.1)
#print('Read Failed', retries)
data = self._read_temp(index)
retries -= 1
if (retries == 0) and (not "YES" in data):
return None
(discard, sep, reading) = data.partition(' t=')
if reading == 85000:
# 85ºC is the boot temperature of the sensor, so ignore that value
return None
temperature = float(reading) / 1000.0
return temperature
def device_count(self):
# Call this to see how many sensors have been detected
return self._num_devices
I already tried to return the previous temperature reading if the current one isn't finished yet, however this didn't reduce the time it took to read a sensor, so I guess the only way is to do things asynchronously.
I could reduce the precision to reduce the time it takes per reading, but ideally I would read all of the sensors simultaneously on separate threads.
How can I best implement this? Or are there other ways to improve the reading speed of multiple DS18B20 sensors?
Thanks for any insights!

You're facing some limitations introduced by the Linux kernel driver. If you were interacting with the OneWire protocol directly, you would only have a single 750ms read cycle for all three sensors, rather than (3 * 750ms). When speaking the 1-wire protocol directly, you can issue a single "convert temperature" command to all devices on the bus, as described here, and then read all the sensors.
The Linux driver explicitly doesn't support this mode of operation:
If none of the devices are parasite powered it would be possible to convert all the devices at the same time and then go back to read individual sensors. That isn’t currently supported. The driver also doesn’t support reduced precision (which would also reduce the conversion time) when reading values.
That means you're stuck with a 750ms per device read cycle. Your best option is probably placing the sensor reading code in a separate thread, e.g.:
import glob
import threading
import time
# Note that we're inheriting from threading.Thread here;
# see https://docs.python.org/3/library/threading.html
# for more information.
class DS18B20(threading.Thread):
default_base_dir = "/sys/bus/w1/devices/"
def __init__(self, base_dir=None):
super().__init__()
self._base_dir = base_dir if base_dir else self.default_base_dir
self.daemon = True
self.discover()
def discover(self):
device_folder = glob.glob(self._base_dir + "28*")
self._num_devices = len(device_folder)
self._device_file: list[str] = []
for i in range(self._num_devices):
self._device_file.append(device_folder[i] + "/w1_slave")
self._values: list[float | None] = [None] * self._num_devices
self._times: list[float] = [0.0] * self._num_devices
def run(self):
"""Thread entrypoint: read sensors in a loop.
Calling DS18B20.start() will cause this method to run in
a separate thread.
"""
while True:
for dev in range(self._num_devices):
self._read_temp(dev)
# Adjust this value as you see fit, noting that you will never
# read actual sensor values more often than 750ms * self._num_devices.
time.sleep(1)
def _read_temp(self, index):
for i in range(3):
with open(self._device_file[index], "r") as f:
data = f.read()
if "YES" not in data:
time.sleep(0.1)
continue
disacard, sep, reading = data.partition(" t=")
temp = float(reading) / 1000.0
self._values[index] = temp
self._times[index] = time.time()
break
else:
print(f"failed to read device {index}")
def tempC(self, index=0):
return self._values[index]
def device_count(self):
"""Return the number of discovered devices"""
return self._num_devices
Because this is a thread, you need to .start() it first, so your
code would look something like:
d = DS18B20()
d.start()
while True:
for i in range(d.device_count()):
print(f'dev {i}: {d.tempC(i)}')
time.sleep(0.5)
You can call the tempC method as often as you want, because it's
just return a value from the _values array. The actual update
frequency is controlled by the loop in the run method (and the
minimum cycle time imposed by the sensors).

Amending Apogee sensor open-source Python 2 code to operate on Raspberry pi 4 through USB port

Working on a project to connect several sensors to Raspberry Pi 4 Model B and stream collected data to Google Cloud Platform for storage/reporting/analysis
I have an Apogee sq520 PAR sensor which connects through USB and is designed to operate on Windows and Mac (but not Linux) with an accompanying GUI to view the data. I've connected the sensor to the Raspberry PI through the USB port (since IO pins are out of the picture) and now need to write a program for the Pi to read it. After reaching out to the Apogee support team, they've shared with me the source-code (below). However, as I am a complete beginner I am having a hard time amending the code to be read by the Raspberry Pi. Need all the help and support I can get.
Notes:
Code utilizes Pyserial library
Code is written on Python 2.7 (according to website). - I am using Python 3
When compiling the code as-is I receive a lot of syntax errors, regardless whether compiling in Python 2 or 3
Sensor connects through 5V USB
More info on sensor: https://www.apogeeinstruments.com/sq-520-full-spectrum-smart-quantum-sensor-usb/#product-tab-description
Code:
from serial import Serial
from time import sleep
import struct
GET_VOLT = '\x55!'
READ_CALIBRATION = '\x83!'
SET_CALIBRATION = '\x84%s%s!'
READ_SERIAL_NUM = '\x87!'
GET_LOGGING_COUNT = '\xf3!'
GET_LOGGED_ENTRY = '\xf2%s!'
ERASE_LOGGED_DATA = '\xf4!'
class Quantum(object):
def __init__(self):
"""Initializes class variables, and attempts to connect to device"""
self.quantum = None
self.offset = 0.0
self.multiplier = 0.0
self.connect_to_device()
def connect_to_device(self):
"""This function creates a Serial connection with the defined comport
and attempts to read the calibration values"""
port = 'COM1' # you'll have to check your device manager and put the actual com port here
self.quantum = Serial(port, 115200, timeout=0.5)
try:
self.quantum.write(READ_CALIBRATION)
multiplier = self.quantum.read(5)[1:]
offset = self.quantum.read(4)
self.multiplier = struct.unpack('<f', multiplier)[0]
self.offset = struct.unpack('<f', offset)[0]
except (IOError, struct.Error), data:
print data
self.quantum = None
def get_micromoles(self):
"""This function converts the voltage to micromoles"""
voltage = self.read_voltage()
if voltage == 9999:
# you could raise some sort of Exception here if you wanted to
return
# this next line converts volts to micromoles
micromoles = (voltage - self.offset) * self.multiplier * 1000
if micromoles < 0:
micromoles = 0
return micromoles
def read_voltage(self):
"""This function averages 5 readings over 1 second and returns
the result."""
if self.quantum == None:
try:
self.connect_to_device()
except IOError:
# you can raise some sort of exception here if you need to
return
# store the responses to average
response_list = []
# change to average more or less samples over the given time period
number_to_average = 5
# change to shorten or extend the time duration for each measurement
# be sure to leave as floating point to avoid truncation
number_of_seconds = 1.0
for i in range(number_to_average):
try:
self.quantum.write(GET_VOLT)
response = self.quantum.read(5)[1:]
except IOError, data:
print data
# dummy value to know something went wrong. could raise an
# exception here alternatively
return 9999
else:
if not response:
continue
# if the response is not 4 bytes long, this line will raise
# an exception
voltage = struct.unpack('<f', response)[0]
response_list.append(voltage)
sleep(number_of_seconds/number_to_average)
if response_list:
return sum(response_list)/len(response_list)
return 0.0

Pyserial package doesnot read all the data from the COM port ( reads only 6000 to 6150 bytes always)

I wrote a small pyserial interface to read the data from the COM port after issuing a command. For eg : in my case my system has a lot of network interface so i need to validate whether all the interfaces are up using ifconfig command. But when i gave this command , the output of the command is getting truncated at the last few lines. The approximate size of the output in bytes would be 6500-7000 bytes but i am receiving only around 6000-6150 bytes all the time. Please find my code below
'''
import serial
import time
com_serial = serial.Serial("COM6", 115200, timeout = 10)
com_serial.reset_input_buffer()
com_serial.write(b"ifconfig\n")
data_all = b" "
time.sleep(5)
while True:
bytetoread = com_serial.inWaiting()
time.sleep(2)
print ("Bytetoread: " , bytetoread)
data = com_serial.read(bytetoread)
data_all += data
if bytetoread < 1:
break
print ("Data:", data_all)
com_serial.close()
'''
**Output:
Bytetoread: 3967
Bytetoread: 179
Bytetoread: 2049
Bytetoread: 0
**
Data: *********with missing data at the end.
I am not sure why the logs are missing?
I have tried another approach.
'''
import serial
import time
com_serial = serial.Serial("COM6", 115200, timeout = 10)
com_serial.reset_input_buffer()
com_serial.write(b"ifconfig\n")
time.sleep(5)
data_all = b" "
data_all = com_serial.read(100000000)
print (data_all)
com_serial.close()
'''
Here also the last few logs are getting truncated.

The root cause seems to be inadequate buffer size of the Tx and Rx serial buffer. By increasing the buffer size using .set_buffer_size() resolved the issue.
'''
import serial
import time
com_serial = serial.Serial("COM6", 115200, timeout = 10)
com_serial.set_buffer_size(rx_size = 12800, tx_size = 12800)
com_serial.reset_input_buffer()
com_serial.write(b"ifconfig\n")
data_all = b" "
data_all = com_serial.read(100000000)
print (data_all)
com_serial.close()
'''

How to run 2 differents loops in 2 differents threads?

I'm doing a telemetry application using Azure IoT Hub, Azure IoT SDK in Python and a raspberry pi with temperature and humidity sensors.
Humidity + Temperature sensors => Rasperry Pi => Azure IoT Hub
In my first implementation thanks azure examples, I used one loop that collect data from the temperature sensor and the humidity sensor, and send them to Azure IoT Hub in the same time every 60 second.
>>> 1 Loop every 60s = Collect data & send data of temperature and humidity
Now I would like to send them with different frequencies, I mean :
One loop will collect the data of the temperature sensor and send it to Azure IoT Hub every 60 seconds;
Whereas a second loop will collect the data of the humidity sensor and send it to Azure IoT Hub every 600 seconds.
>>> 1 Loop every 60s= Collect data & send data of temperature
>>> 2 Loop every 600s= Collect data & send data of humidity
I think the tool I need is multi-threading, but I don't understand which library or structure I have to implement in my case.
Here is the code provided by Azure, including one loop that handles temperature and humidity at the same time. Reading the data and sending to Azure every 60 seconds.
import random
import time
import sys
# Using the Python Device SDK for IoT Hub:
from iothub_client import IoTHubClient, IoTHubClientError,
IoTHubTransportProvider, IoTHubClientResult
from iothub_client import IoTHubMessage, IoTHubMessageDispositionResult,
IoTHubError, DeviceMethodReturnValue
# The device connection string to authenticate the device with your IoT hub.
CONNECTION_STRING = "{Your IoT hub device connection string}"
# Using the MQTT protocol.
PROTOCOL = IoTHubTransportProvider.MQTT
MESSAGE_TIMEOUT = 10000
# Define the JSON message to send to IoT Hub.
TEMPERATURE = 20.0
HUMIDITY = 60
MSG_TXT = "{\"temperature\": %.2f,\"humidity\": %.2f}"
def send_confirmation_callback(message, result, user_context):
print ( "IoT Hub responded to message with status: %s" % (result) )
def iothub_client_init():
# Create an IoT Hub client
client = IoTHubClient(CONNECTION_STRING, PROTOCOL)
return client
def iothub_client_telemetry_sample_run():
try:
client = iothub_client_init()
print ( "IoT Hub device sending periodic messages, press Ctrl-C to exit" )
#******************LOOP*******************************
while True:
# Build the message with simulated telemetry values.
temperature = TEMPERATURE + (random.random() * 15)
humidity = HUMIDITY + (random.random() * 20)
msg_txt_formatted = MSG_TXT % (temperature, humidity)
message = IoTHubMessage(msg_txt_formatted)
# Send the message.
print( "Sending message: %s" % message.get_string() )
client.send_event_async(message, send_confirmation_callback, None)
time.sleep(60)
except IoTHubError as iothub_error:
print ( "Unexpected error %s from IoTHub" % iothub_error )
return
except KeyboardInterrupt:
print ( "IoTHubClient sample stopped" )
if __name__ == '__main__':
print ( "IoT Hub Quickstart #1 - Simulated device" )
print ( "Press Ctrl-C to exit" )
iothub_client_telemetry_sample_run()
I would like to use the same structure of functions, including two loops that handles temperature and humidity, one every 60s and one every 600s.
while True:
# Build the message with simulated telemetry values.
temperature = TEMPERATURE + (random.random() * 15)
msg_txt_formatted1 = MSG_TXT1 % (temperature)
message1 = IoTHubMessage(msg_txt_formatted1)
# Send the message.
print( "Sending message: %s" % message1.get_string() )
client.send_event_async(message1, send_confirmation_callback, None)
time.sleep(60)
while True:
# Build the message with simulated telemetry values.
humidity = HUMIDITY + (random.random() * 20)
msg_txt_formatted2 = MSG_TXT2 % (humidity)
message2 = IoTHubMessage(msg_txt_formatted2)
# Send the message.
print( "Sending message: %s" % message2.get_string() )
client.send_event_async(message2, send_confirmation_callback, None)
time.sleep(600)
How can I do that? How to call those loops with multi-threading or another method?

It may be simpler to do something like
while True:
loop_b()
for _ in range(10):
loop_a()
time.sleep(60)
or even
while True:
time.sleep(1)
now = time.time()
if now % 60 == 0:
loop_a()
if now % 600 == 0:
loop_b()
But if you really want to use threads, then:
import threading
class LoopAThread(threading.Thread):
def run(self):
loop_a()
class LoopBThread(threading.Thread):
def run(self):
loop_b()
...
thread_a = LoopAThread()
thread_b = LoopBThread()
thread_a.start()
thread_b.start()
thread_a.join()
thread_b.join()

Here are two competing approaches to consider
Don't bother with threads at all. Just have one loop that sleeps every 60 seconds like you have now. Keep track of the last time you sent humidity data. If 600 seconds has passed, then send it. Otherwise, skip it and go to sleep for 60 seconds. Something like this:
from datetime import datetime, timedelta
def iothub_client_telemetry_sample_run():
last_humidity_run = None
humidity_period = timedelta(seconds=600)
client = iothub_client_init()
while True:
now = datetime.now()
send_temperature_data(client)
if not last_humidity_run or now - last_humidity_run >= humidity_period:
send_humidity_data(client)
last_humidity_run = now
time.sleep(60)
Rename iothub_client_telemetry_sample_run to temperature_thread_func or something like it. Create a separate function that looks just like it for humidity. Spawn two threads from the main function of your program. Set them to daemon mode so they shutdown when the user exits
from threading import Thread
def temperature_thread_func():
client = iothub_client_init()
while True:
send_temperature_data(client)
time.sleep(60)
def humidity_thread_func():
client = iothub_client_init()
while True:
send_humidity_data(client)
time.sleep(600)
if __name__ == '__main__':
temp_thread = Thread(target=temperature_thread_func)
temp_thread.daemon = True
humidity_thread = Thread(target=humidity_thread_func)
humidity_thread.daemon = True
input('Polling for data. Press a key to exit')
Notes:
If you decide to use threads, consider using an
event
to terminate them cleanly.
time.sleep is not a precise way to keep
time. You might need a different timing mechanism if the samples need
to be taken at precise moments.

Storing xbee source addr into database in Python

I have raspberry pi connected with xbees and a motion sensor, receiving data from the motion sensor connected to one of the xbee. It would then send the data to my raspberry pi. Is there any way I could manipulate or split the output such as the status being only True/False and the address =\x00\x13\xa2\x00#\xbbJ as I wanted to store the address into database if status=="True".
So if I do
if status[0]['dio-0'] == True :
print "Yes"
cur = con.cursor()
cur.execute("INSERT ignore into sensor(sensor_id, status) VALUES(%s,True)",(add[0]))
con.commit()
But the address stored into the database is weird characters instead of \x00\x13\xa2\x00#\xbbJ. or should I do any other ways?
This is the codes.
from xbee import XBee
import serial
PORT = '/dev/ttyUSBXBEE'
BAUD_RATE = 9600
# Open serial port
ser = serial.Serial(PORT, BAUD_RATE)
# Create API object
xbee = XBee(ser)
def decodeReceivedFrame(response):
add = str(response['source_addr_long'])
status = response['samples']
return [add, status]
# Continuously read and print packets
while True:
try:
response = xbee.wait_read_frame()
decodedData = decodeReceivedFrame(response)
status = decodeReceivedFrame(response)[1]
print status
print decodedData
add= decodedData[0]
except KeyboardInterrupt:
break
ser.close()
And this is the output.
[{'dio-0': True}]
['\x00\x13\xa2\x00#\xbbJ}', [{'dio-0': True}]]
[{'dio-0': False}]
['\x00\x13\xa2\x00#\xbbJ}', [{'dio-0': False}]]
In the database
+------------+--------+
| sensor_id | status |
+------------+--------+
| ¢ #»J} | 1 |
+------------+--------+

The variable sensor_id is an array of bytes, and it sounds like you want to store it in a human-readable format.
One way is to convert it to a formatted string before storing it in the database.
sensor_id = ':'.join("%02X" % ord(b) for b in add)
That statement loops through the bytes in the address (for b in add), formats each as a two-character hex string ("%02X" % ord(b)), and then joins each of those strings together with a colon in between (':'.join()).