We Keep Coding

Sending Float Values from Python to Arduino - Data in buffer but no values read

I am trying to send float values from Python on Windows to an Arduino. The first problem I encounter is that I cannot view the serial monitor when I think I am sending data from my Python script. I have read online that this is becuase only one application can manage the port at once: https://forum.arduino.cc/t/serial-communication-only-working-when-serial-monitor-is-opened/601107/12
However I have seen examples where the user is viewing the serial monitor to see data coming in over serial from Python and serial.print outs from the Arduino. So I am unsure what is the case... not being able to view the serial monitor sure does make debugging this senario hard.
My Python code:
import struct
import serial
import time
print('test123')
x=0.8
y=0.2
ser = serial.Serial('COM5', 9600, timeout=1)
#print(ser)
time.sleep(3)
def sendmess():
bin = struct.pack('ff',x,y) #Pack float value into 4 bytes
data = ser.write(bin)
#print(bin)
#print(len(bin))
#print([ "0x%02x" % b for b in bin])
#ser.close()
while True:
sendmess()
My Arduino Code:
int d = 250;
float incomingByte = 1;
void setup() {
// initialize the serial communication:
Serial.begin(9600);
pinMode(2, OUTPUT);
}
void loop() {
digitalWrite(2, HIGH);
delay(d);
digitalWrite(2, LOW);
delay(d);
// reply only when you receive data:
if (Serial.available() > 0) {
// read the incoming byte:
d = 1000;
float incomingByte = Serial.read();
// say what you got:
Serial.println(incomingByte);
}
else{
Serial.println(incomingByte);
d = 20;
}
}
I see the LED flash every second so I know the serial buffer is >0, but I cannot get any data out :(.
Thanks in advance!
Mark
I have tried examples online, but I never get any data to display in the serial monitor. Nor can I say have a pin turn HIGH when I think I am getting the data I think I have sent. But without the Serial Monitor how can I debug this?

Unfortunately, you aren't able to connect more than two devices to the serial connection. However, using Python you should be able to read a response from your arduino and then print that to the python Terminal (or a file) and take a look at that.
The easiest way to do that would be to use pySerial and either ser.read(4) to read back 4 bytes of your float or ser.readline() which will look for either a '\n' which you can add to your arduino code, or a timeout. Then just print it to the python terminal with print().
As for actually reading the float, Serial.read() will only read in the next byte in the buffer. Since a float is 4 bytes long, you need to either read the 4 bytes into an array or use Serial.parseFloat()
Python Code
import struct
import serial
import time
x=0.8
y=0.2
ser = serial.Serial('COM5', 9600, timeout=1)
#print(ser)
time.sleep(3)
def sendmess():
bin = str(x) + str(y) #Pack float value into 4 bytes
data = ser.write(bin.encode())
echo = ser.readline()
print("Echo: " + echo)
#ser.close()
while True:
sendmess()
Arduino Code:
int d = 250;
float X;
float Y;
char buffer[40];
void setup() {
// initialize the serial communication:
Serial.begin(9600);
pinMode(2, OUTPUT);
}
void loop() {
digitalWrite(2, HIGH);
delay(d);
digitalWrite(2, LOW);
delay(d);
// reply only when you receive data:
if (Serial.available() > 0) {
// read the incoming byte:
d = 1000;
X = Serial.parseFloat();
Y = Serial.parseFloat();
// say what you got:
sprintf(buffer, "X: %f Y: %f", X, Y);
Serial.println(buffer);
}
else{
Serial.println(buffer);
d = 20;
}
}

Sending Real Time images captured using unity camera

server
private void SendImageByte()
{
image_bytes = cm.Capture();
print(image_bytes.Length);
if (connectedTcpClient == null)
{
return;
}
try
{
// Get a stream object for writing.
NetworkStream stream = connectedTcpClient.GetStream();
if (stream.CanWrite)
{
// string serverMessage = "This is a message from your server.";
// Convert string message to byte array.
byte[] serverMessageAsByteArray = Encoding.ASCII.GetBytes(image_bytes.ToString());
// Write byte array to socketConnection stream.
stream.Write(serverMessageAsByteArray, 0, serverMessageAsByteArray.Length);
Debug.Log("Server sent his message - should be received by client");
}
}
catch (SocketException socketException)
{
Debug.Log("Socket exception: " + socketException);
}
}
client
import socket
host = "127.0.0.1"
port = 1755
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((host, port))
def receive_image():
data = sock.recv(999999).decode('utf-8')
print(len(data))
while True:
receive_image()
here script capture images from unity camera
public byte[] Capture()
{
if(renderTexture == null)
{
// creates off-screen render texture that can rendered into
rect = new Rect(0, 0, captureWidth, captureHeight);
renderTexture = new RenderTexture(captureWidth, captureHeight, 24);
screenShot = new Texture2D(captureWidth, captureHeight, TextureFormat.RGB24, false);
}
// _camera = GetComponent<Camera>();
_camera.targetTexture = renderTexture;
_camera.Render();
// reset active camera texture and render texture
_camera.targetTexture = null;
RenderTexture.active = null;
// read pixels will read from the currently active render texture so make our offscreen
// render texture active and then read the pixels
RenderTexture.active = renderTexture;
screenShot.ReadPixels(rect, 0, 0);
screenShot.Apply();
byte[] imageBytes = screenShot.EncodeToPNG();
//Object.Destroy(screenShot);
//File.WriteAllBytes(Application.dataPath + "/../"+ imagePath + "/img{counter}.png", bytes);
//counter = counter + 1;
return imageBytes;
}
Am trying to send real-time images on Unity3D from C# to python using socket communication to be processed and return back values to unity, but the problem even the bytes length received on the client is not the same as the server. I send about 400K bytes but I receive only 13
C# is the server and python is the client
or am doing it wrong but the main goal I want to create simulator work as udacity self-driving

Are you sure that image_bytes.ToString() returns what you expect and not maybe just something like "System.Byte[]" => 13 chars => 13 bytes.
In general why would you convert an already byte[] into a string just in order to convert it back into byte[] to send? I'm pretty sure you do not want to transmit binary image data using UTF-8 ... one option might be Base64 string, but still that would be quite inefficient.
Just send the raw bytes like e.g.
stream.Write(image_bytes, 0, image_bytes.Length);
And then receive until you receive that length.
A typical solution is to prepend the length of the message to send and on the receiver side actually wait until you received exactly that amount of bytes like e.g.
var lengthBytes = BitConverter.GetBytes(image_bytes.Length);
stream.Write(lengthBytes, 0, lengthBytes.Length);
stream.Write(image_bytes, 0, image_bytes.Length);
Now you know that on the receiver side you first have to receive exactly 4 bytes (== one int) which will tell you exactly how many bytes to receive for the actual payload.
Now I'm no python expert but after googling around a bit I think something like
def receive_image()
lengthBytes = sock.recv(4)
length = struct.unpack("!i", lengthBytes)[0]
data = sock.recv(length)
Note: After reading John Gordon's comment on the question I guess this still doesn't fully solve the waiting until according buffers are actually filled - as said no python expert - but I hope it gives you a idea where to go ;)

How to send an int from Python to an Arduino in order to use it as an argument for the neopixel function setPixelcolor()?

I am trying to send an int number from Python to an Arduino using PySerial, using .write([data]) to send with Python and Serial.read() or Serial.readString() to recieve on the Arduino, then .setPixelColor() and .show() to light a LED on a matrix which position corresponds to the int sent by the arduino (I am using the Duinofun Neopixel Shield).
But It does not seem to work properly, and I can't use the Serial Monitor as I am sending my data as the port would be busy.
I have tried to input a number myself using Serial.readString() then converting the string to an int and finally putting in into my function that displays the LED.
It does work properly when I do this, but when I send some data over, all the previously lit LEDs suddenly switch off which can only be caused by a reset of the Arduino board as far as I know.
This is the python code, it simply sends an int chosen by the user
import serial
a = int(input('Enter pixel position : '))
ser = serial.Serial("COM16", 9600)
ser.write([a])
And this is the part of the Arduino program that reads the incoming data.
Adafruit_NeoPixel pixels = Adafruit_NeoPixel(40, 6, NEO_GRB + NEO_KHZ800);
void setup() {
// put your setup code here, to run once:
pixels.begin();
Serial.begin(9600);
}
void loop() {
String a = Serial.readString();
int b = a.toInt();
pixels.setPixelColor(b, 30,30,30);
pixels.show();
Serial.println(a);
delay(1000);
}
All the LED switch off when I send some data, except the first LED which position corresponds to a 0 used in the .setPixelColor() function.
Problem is, the LED should light to the corresponding int sent by Python (e.g light the fifth LED for an int of 4).

You don't need to send an int from your Python script. Just send a string and then convert it back to int on your Arduino. Also, you can verify the number simply on your Arduino code if the received value is valid.
Another problem with your Arduino code is you are not checking the Serial port availability which would return an empty string by Serial.readString().
A simple approach is shown below but you can extend it for other pixels.
Python script:
import serial
ser = serial.Serial("COM16", 9600)
while True:
input_value = input('Enter pixel position: ')
ser.write(input_value.encode())
Arduino code:
#define MIN_PIXEL_RANGE 0
#define MAX_PIXEL_RANGE 100
Adafruit_NeoPixel pixels = Adafruit_NeoPixel(40, 6, NEO_GRB + NEO_KHZ800);
void setup()
{
// put your setup code here, to run once:
pixels.begin();
Serial.begin(9600);
}
void loop()
{
if (Serial.available())
{
String a = Serial.readString();
Serial.print("Received Value: ");
Serial.println(a);
int b = a.toInt();
if ((b >= MIN_PIXEL_RANGE) && (b <= MAX_PIXEL_RANGE))
{
pixels.setPixelColor(b, 30, 30, 30);
pixels.show();
delay(1000);
}
}
}

You can communicate between Ardinos and Python really easily and reliably if you use the pip-installable package pySerialTransfer. The package is non-blocking, easy to use, supports variable length packets, automatically parses packets, and uses CRC-8 for packet corruption detection.
Here's an example Python script:
from pySerialTransfer import pySerialTransfer as txfer
if __name__ == '__main__':
try:
link = txfer.SerialTransfer('COM13')
link.txBuff[0] = 'h'
link.txBuff[1] = 'i'
link.txBuff[2] = '\n'
link.send(3)
while not link.available():
if link.status < 0:
print('ERROR: {}'.format(link.status))
print('Response received:')
response = ''
for index in range(link.bytesRead):
response += chr(link.rxBuff[index])
print(response)
link.close()
except KeyboardInterrupt:
link.close()
Note that the Arduino will need to use the library SerialTransfer.h. You can install SerialTransfer.h using the Arduino IDE's Libraries Manager.
Here's an example Arduino sketch:
#include "SerialTransfer.h"
SerialTransfer myTransfer;
void setup()
{
Serial.begin(115200);
Serial1.begin(115200);
myTransfer.begin(Serial1);
}
void loop()
{
myTransfer.txBuff[0] = 'h';
myTransfer.txBuff[1] = 'i';
myTransfer.txBuff[2] = '\n';
myTransfer.sendData(3);
delay(100);
if(myTransfer.available())
{
Serial.println("New Data");
for(byte i = 0; i < myTransfer.bytesRead; i++)
Serial.write(myTransfer.rxBuff[i]);
Serial.println();
}
else if(myTransfer.status < 0)
{
Serial.print("ERROR: ");
Serial.println(myTransfer.status);
}
}
Lastly, note that you can transmit ints, floats, chars, etc. using the combination of these libraries!

Python Tkinter Slider Widget write Value over I2C

Ok I have been searching for an example for a while now can not seem to find any examples of relaying the tkinter slider values over I2C to Arduino. So far I have not tried to communicate with the Arduino yet. Ill cross that bridge next; for now I just want to figure out how to write the slide widget values and send over I2C.
Here is a simple GUI slider widget in Python 2 with what I believe is the correct setup for I2C communications. Ive updated the Rpi to set up the I2C as well. What I want to do in Arduino is simply read the values 0 to 180 for a servo control. It is important that it just writes values or in some way that can be used for an input. I have other code in the arduino that drives the same servo, if other conditions are met and this would then be ignored.
from Tkinter import*
import RPi.GPIO as GPIO
import time
import smbus
bus = smbus.SMBus=(1)
SLAVE_ADDRESS = 0x04
class App:
def __init__(self, master):
def SendScaleReading(self):
S = scale.get()# Now how do we write this and get the Scale Value and send it??
bus(SLAVE_ADDRESS, ord('S'))#According to an example this should be
#"bus.write_byte(SLAVE_ADDRESS, ord('S'))"
frame = Frame(master)
frame.pack()
scale = Scale(frame, from_=0, to=180, orient=HORIZONTAL, command=SendScaleReading)
scale.grid(row=1, column=1)
root = Tk()
root.wm_title('I2C servo control')
app = App(root)
root.geometry("200x50+0+0")
root.mainloop()

Ok a friend helped me out with and I wasnt too far off. Once I got this far I only a small issue with an IO error. I was getting [Errno 5] IO errors. Make sure you have a ground connected between the Arduino and Pi. When I searched this seemed to be overlooked by the many fixes out there offered. You need SDA, SLA and Gnd all connected.
So anyways here is the code that is working running an arduino i2c blink sketch. This will blink the led on pin 13 faster or slower based on the slider input in from the Rpi over I2C. I am going to try and write this code to control a servo next and if successful I will also post that code.
Rpi / python2 code below:
from Tkinter import*
import RPi.GPIO as GPIO
import time
import smbus
bus = smbus.SMBus(1)
SLAVE_ADDRESS = 0x28
class App:
def __init__(self, master):
def SendScaleReading(self):
S = scale.get()
print("we have" );
print( S )
bus.write_byte_data(SLAVE_ADDRESS, S, S )
frame = Frame(master)
frame.pack()
scale = Scale(frame, from_=0, to=180, orient=HORIZONTAL, command=SendScaleReading)
scale.grid(row=1, column=1)
root = Tk()
root.wm_title('I2C servo control')
app = App(root)
root.geometry("200x50+0+0")
root.mainloop()
Arduino i2c wire Blink sketch below:
#include <Wire.h>
// unique address for this I2C slave device
#define ADDRESS 0x28
// constants won't change. Used here to
// set pin numbers:
const int ledPin = 13; // the number of the LED pin
// Variables will change:
int ledState = LOW; // ledState used to set the LED
long previousMillis = 0; // will store last time LED was updated
// the follow variables is a long because the time, measured in miliseconds,
// will quickly become a bigger number than can be stored in an int.
volatile long interval = 1000; // interval at which to blink (milliseconds)
void setup() {
// set the digital pin as output:
pinMode(ledPin, OUTPUT);
Wire.begin(ADDRESS); // join i2c bus with address #4
Wire.onReceive(receiveEvent); // register event
Wire.onRequest(requestEvent); // register the request handler
}
//gets called when I2C read occurs
void requestEvent() {
//any request for data will return 0x14 (random number i picked for testing)
Wire.write( 0x14 );
}
// get called when I2C write occurs
void receiveEvent(int howMany) {
//just going to support 1 byte commands for now
if (howMany >0 ) {
int c = Wire.read();
interval = c * 10;
}
while (Wire.available() > 0 ) {
Wire.read();
}
}
void loop()
{
// here is where you'd put code that needs to be running all the time.
// check to see if it's time to blink the LED; that is, if the
// difference between the current time and last time you blinked
// the LED is bigger than the interval at which you want to
// blink the LED.
unsigned long currentMillis = millis();
if(currentMillis - previousMillis > interval) {
// save the last time you blinked the LED
previousMillis = currentMillis;
// if the LED is off turn it on and vice-versa:
if (ledState == LOW)
ledState = HIGH;
else
ledState = LOW;
// set the LED with the ledState of the variable:
digitalWrite(ledPin, ledState);
}
}

Ok well the servo code ended up being rather simple. Im sure it can be cleaned up so anyone with the know how please feel free to make improvements.
Arduino I2C Driven Servo Sketch is below:
#include <Wire.h>
#include <Servo.h>
// unique address for this I2C slave device
#define ADDRESS 0x28
Servo myservo;
int pos = 0;
void setup() {
myservo.attach(9);
Wire.begin(ADDRESS); // join i2c bus with address #4
Wire.onReceive(receiveEvent); // register event
Wire.onRequest(requestEvent); // register the request handler
}
//gets called when I2C read occurs
void requestEvent() {
//any request for data will return 0x14 (random number i picked for testing)
Wire.write( 0x14 );
}
// get called when I2C write occurs
void receiveEvent(int Pos) {
int val = Wire.read();
pos = val;
while (Wire.available() > 0 ) {
Wire.read();
}
}
void loop()
{
myservo.write(pos);
delay(15);
}

Logging from serial is not real-time. How to make it not drift off?

I'm getting a value of my moisture sensor over serial from Arduino to Raspberry PI. My Python script is supposed to log it. And it does, but the timing is getting delayed more and more exponentially. My guess (after 5 hours of Google) is that the problem is in the buffer somewhere and I am reading old data and "catching up". How do I adjust my code to get the latest serial info from my Arduino? Please keep in mind that I am a BIG noob and if you can explain as simple as possible. I've been up all night figuring it out but it's just that I know so very little about programming. Also adding the graph, there you can see the drift-off, I put the sensor out of water and put it in when I saw the graph drop.
Arduino code:
/*
Chirp - arduino example
Connection
Chirp pin 1 - no connection
Chirp pin 2 - Arduino VCC
Chirp pin 3 - Arduino A5
Chirp pin 4 - Arduino A4
Chirp pin 5 - Arduino pin 2
Chirp pin 6 - Arduino GND
*/
#include <Wire.h>
#define RELAY1 7
void writeI2CRegister8bit(int addr, int value) {
Wire.beginTransmission(addr);
Wire.write(value);
Wire.endTransmission();
}
unsigned int readI2CRegister16bit(int addr, int reg) {
Wire.beginTransmission(addr);
Wire.write(reg);
Wire.endTransmission();
delay(1100);
Wire.requestFrom(addr, 2);
unsigned int t = Wire.read() << 8;
t = t | Wire.read();
return t;
}
void setup() {
Wire.begin();
Serial.begin(9600);
pinMode(RELAY1, OUTPUT);
digitalWrite(RELAY1,HIGH);
pinMode(2, OUTPUT);
digitalWrite(2, LOW); //Reset the Chirp
delay(1); //maybe allow some time for chirp to reset
digitalWrite(2, HIGH); //Go out from reset
writeI2CRegister8bit(0x20, 3); //send something on the I2C bus
delay(1000); //allow chirp to boot
}
void loop() {
Serial.println(readI2CRegister16bit(0x20, 0)); //read capacitance register
//writeI2CRegister8bit(0x20, 3); //request light measurement
//delay(9000); //this can take a while
//Serial.print(", ");
//Serial.println(readI2CRegister16bit(0x20, 4)); //read light register
if (readI2CRegister16bit(0x20, 0) < 420){
//Serial.println ("watering");
digitalWrite(RELAY1,LOW);
delay(2000);
digitalWrite(RELAY1,HIGH);
delay(2000);
}
else{
digitalWrite(RELAY1,HIGH);
//Serial.println ("moist");
delay(2000);
}
}
Python code:
import serial
import time
import csv
import os
import plotly.plotly as py
from plotly.graph_objs import Scatter, Layout, Figure
os.chdir('/home/pi/csvdata')
username = '------------'
api_key = '------------'
stream_token = '------------'
py.sign_in(username, api_key)
trace1 = Scatter(
x=[],
y=[],
stream=dict(
token=stream_token,
maxpoints=200000000000
)
)
layout = Layout(
title='------------'
)
fig = Figure(data=[trace1], layout=layout)
print py.plot(fig, filename='------------')
stream = py.Stream(stream_token)
stream.open()
def mainloop():
name=time.strftime('%b %d %Y %H:%M:%S')
f=open(name+'.csv', 'wt')
writer = csv.writer(f,delimiter=',')
ser = serial.Serial('/dev/ttyUSB0',9600,timeout=1)
for t in range(500):
#time.sleep(5)
kk=ser.readline()
kk2=kk.split('\r',1)
s=kk2[0]
text=time.strftime('%X %x'),s
streamtext=({'x': time.strftime('%X %x'), 'y': kk})
writer.writerow (text)
stream.write (streamtext)
f.flush()
print (text)
time.sleep(5);
ser.flush()
#ser.flushInput()
#ser.flushOutput()
if t == 499:
print ("why")
f.close()
ser.close()
mainloop()
else:
time.sleep(5);
mainloop()

You haven't posted your code from the Arduino and Python, so here is an example that demonstrates what you want to do. copy the relevant parts into your code.
Arduino:
void setup() {
// put your setup code here, to run once:
Serial.begin(115200);
randomSeed(analogRead(0));
}
void loop() {
// put your main code here, to run repeatedly:
int moisture_value = random(300);
// millis() is the time in milliseconds since the arduino started running
Serial.println(String(millis()) + ":" + String(moisture_value)); // We send two pieces of data, i.e. time since arduino started and the sensor value
delay(1000);
}
Python:
import serial
import datetime
# Converts to an integer if it is an integer, or it returns it as a string
def try_parse_int(s):
try:
return int(s)
except ValueError:
return s
ser = serial.Serial('/dev/ttyACM0', 115200)
time_start = datetime.datetime.now() # The time that we started receiving data
while True:
data = ser.readline().decode("utf-8").strip('\n').strip('\r') # remove newline and carriage return characters
[time, moisture_value] = data.split(':')
print("Received: '{}'".format(data))
time_received = time_start + datetime.timedelta(milliseconds=try_parse_int(time)) # Add delta time to start time
print("Moisture value: {} at {}".format(try_parse_int(moisture_value), time_received))
This works by sending the time since the Arduino started along with the sensor reading. If we add this to the know starting time in Python, then we know the time that the reading was taken. Even if the sending is delayed for some reason, it doesn't matter.
Example output:
Received: '0:116'
Moisture value: 116 at 2017-03-15 11:26:43.024711
Received: '1000:4'
Moisture value: 4 at 2017-03-15 11:26:44.024711
Received: '2000:128'
Moisture value: 128 at 2017-03-15 11:26:45.024711
Received: '3001:123'
Moisture value: 123 at 2017-03-15 11:26:46.025711
Time formatting:
You can also format the time in a way that you prefer;
time_received = time_start + datetime.timedelta(milliseconds=try_parse_int(time)) # Add delta time to start time
time_formatted = '{0:%H}:{0:%M}:{0:%S} on {0:%d}/{0:%m}/{0:%y}'.format(time_received)
print("Moisture value: {} at {}".format(try_parse_int(moisture_value), time_formatted))
Output:
Moisture value: 46 at 11:42:10 on 15/03/17
Received: '117050:174'
Moisture value: 174 at 11:42:11 on 15/03/17
Received: '118050:298'