server
private void SendImageByte()
{
image_bytes = cm.Capture();
print(image_bytes.Length);
if (connectedTcpClient == null)
{
return;
}
try
{
// Get a stream object for writing.
NetworkStream stream = connectedTcpClient.GetStream();
if (stream.CanWrite)
{
// string serverMessage = "This is a message from your server.";
// Convert string message to byte array.
byte[] serverMessageAsByteArray = Encoding.ASCII.GetBytes(image_bytes.ToString());
// Write byte array to socketConnection stream.
stream.Write(serverMessageAsByteArray, 0, serverMessageAsByteArray.Length);
Debug.Log("Server sent his message - should be received by client");
}
}
catch (SocketException socketException)
{
Debug.Log("Socket exception: " + socketException);
}
}
client
import socket
host = "127.0.0.1"
port = 1755
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((host, port))
def receive_image():
data = sock.recv(999999).decode('utf-8')
print(len(data))
while True:
receive_image()
here script capture images from unity camera
public byte[] Capture()
{
if(renderTexture == null)
{
// creates off-screen render texture that can rendered into
rect = new Rect(0, 0, captureWidth, captureHeight);
renderTexture = new RenderTexture(captureWidth, captureHeight, 24);
screenShot = new Texture2D(captureWidth, captureHeight, TextureFormat.RGB24, false);
}
// _camera = GetComponent<Camera>();
_camera.targetTexture = renderTexture;
_camera.Render();
// reset active camera texture and render texture
_camera.targetTexture = null;
RenderTexture.active = null;
// read pixels will read from the currently active render texture so make our offscreen
// render texture active and then read the pixels
RenderTexture.active = renderTexture;
screenShot.ReadPixels(rect, 0, 0);
screenShot.Apply();
byte[] imageBytes = screenShot.EncodeToPNG();
//Object.Destroy(screenShot);
//File.WriteAllBytes(Application.dataPath + "/../"+ imagePath + "/img{counter}.png", bytes);
//counter = counter + 1;
return imageBytes;
}
Am trying to send real-time images on Unity3D from C# to python using socket communication to be processed and return back values to unity, but the problem even the bytes length received on the client is not the same as the server. I send about 400K bytes but I receive only 13
C# is the server and python is the client
or am doing it wrong but the main goal I want to create simulator work as udacity self-driving
Are you sure that image_bytes.ToString() returns what you expect and not maybe just something like "System.Byte[]" => 13 chars => 13 bytes.
In general why would you convert an already byte[] into a string just in order to convert it back into byte[] to send? I'm pretty sure you do not want to transmit binary image data using UTF-8 ... one option might be Base64 string, but still that would be quite inefficient.
Just send the raw bytes like e.g.
stream.Write(image_bytes, 0, image_bytes.Length);
And then receive until you receive that length.
A typical solution is to prepend the length of the message to send and on the receiver side actually wait until you received exactly that amount of bytes like e.g.
var lengthBytes = BitConverter.GetBytes(image_bytes.Length);
stream.Write(lengthBytes, 0, lengthBytes.Length);
stream.Write(image_bytes, 0, image_bytes.Length);
Now you know that on the receiver side you first have to receive exactly 4 bytes (== one int) which will tell you exactly how many bytes to receive for the actual payload.
Now I'm no python expert but after googling around a bit I think something like
def receive_image()
lengthBytes = sock.recv(4)
length = struct.unpack("!i", lengthBytes)[0]
data = sock.recv(length)
Note: After reading John Gordon's comment on the question I guess this still doesn't fully solve the waiting until according buffers are actually filled - as said no python expert - but I hope it gives you a idea where to go ;)
Related
I'm trying to read serial input from my device, and have gotten it to work in Python using pyserial, e.g.
import serial
port = serial.Serial(port='COM1', baudrate=19200, bytesize=8, parity='N', stopbits=1, timeout=None, xonxoff=False, rtscts=False, dsrdtr=False)
while 1:
N = port.in_waiting
if N>4:
msg = port.readline(N)
print(list(msg))
I'm trying to implement this same code in C#, but it does not quite seem to work, e.g.
port = new SerialPort(COM1);
port.BaudRate = baudRate;
port.DataBits = 8;
port.Parity = Parity.None;
port.StopBits = StopBits.One;
port.ReadTimeout = SerialPort.InfiniteTimeout;
port.Handshake = Handshake.None;
port.RtsEnable = false;
port.DtrEnable = false;
port.DataReceived += new SerialDataReceivedEventHandler(DataReceived);
and
void DataReceived(object sender, SerialDataReceivedEventArgs e)
{
int N = port.BytesToRead;
if (N > 4)
{
string line = port.ReadLine();
}
}
I'm able to read stuff properly using port.Read() in C#, however it seems like ReadLine doesn't quite work properly--it doesn't seem to be able to find the end-of-line character ("\n")? And the program just freezes. Yet I am not sure why it works with pyserial.ReadLine() which also seeks the same character (and works without timeout). As far as I can tell, the rest of the port settings are identical.
Thanks!
Because the DataReceived event indicates that data has been received through a port, the data is already read and you should call SerialPort.ReadExisting to get it.
void DataReceived(object sender, SerialDataReceivedEventArgs e)
{
var data = port.ReadExisting();
}
This event does not guarantee that its data is a single line, if you want a similar way to pySerial, you should not use the event and use ReadLine directly:
port.DataReceived += new SerialDataReceivedEventHandler(DataReceived);
while(true)
{
int N = port.BytesToRead;
if (N > 4)
{
string line = port.ReadLine();
}
}
I'm using the following code encode an image from python end,
context = zmq.Context()
socket = context.socket(zmq.PUB)
socket.bind("tcp://*:5558")
frame = cv2.imread("input.jpg")
encoded, buffer = cv2.imencode('.jpg', frame)
encoded_string = base64.b64encode(buffer)
socket.send_string("output", zmq.SNDMORE)
socket.send_pyobj(encoded_string)
Eventually, I'm using the following code to decrypt it in Unity end.
void Start()
{
AsyncIO.ForceDotNet.Force();
NetMQConfig.ManualTerminationTakeOver();
NetMQConfig.ContextCreate(false);
string topic = "output";
subSocket = new SubscriberSocket();
var timeout = new System.TimeSpan(0, 0, 1); //1sec
string subport;
subSocket.Options.ReceiveHighWatermark = 1000;
subSocket.Connect("tcp://localhost:5558");
subSocket.Subscribe(topic);
bool is_connected = subSocket.TryReceiveFrameString(timeout, out subport);
Debug.Log(is_connected);
myTexture = new Texture2D(640, 480, TextureFormat.ARGB32, false);
}
// Update is called once per frame
void Update()
{
string base64string = subSocket.ReceiveFrameString();
Debug.Log(base64string.Length);
if (base64string.Length > 100)
{
byte[] imgBytes = Convert. FromBase64String(base64string);
myTexture.LoadRawTextureData(imgBytes);
myTexture.Apply();
rawImg.texture = myTexture;
}
}
Unfortunately it throws the following error,
FormatException: The input is not a valid Base-64 string as it
contains a non-base 64 character, more than two padding characters, or
an illegal character among the padding characters.
what am I missing?
It would be more helpful if you could try sending a very small image through the channel (encode, transfer, receive) and log-print what was received on the other end.
My suspicions:
Somehow you're not converting to base-64
You end up sending too many bytes, or unnecessary bytes (null-termination character?)
Receiving the image content in a manner that adds CR or LF at the end (such as 'HTTP Response' body)
More information is needed to provide a better answer...
I need to stream video from a Raspberry Pi's camera (Python server), to a C# script client (Unity, in fact).
I have successfully done this, using a Python to Python workflow, but I can't manage to make It work on C# client side.
This is my python server, where I'm sending the Stream over sockets:
Server.py:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind(('0.0.0.0', 24000))
s.listen(1)
connection = s.accept()[0].makefile('wb')
print ('Client connected')
while True:
stream = io.BytesIO()
for foo in camera.capture_continuous(stream, 'jpeg'):
connection.write(struct.pack('<L', stream.tell()))
connection.flush()
stream.seek(0)
connection.write(stream.read())
stream.seek(0)
stream.truncate()
And this is my Python client side, which is actually working:
Client.py:
def threadLoop(self):
self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.socket.connect((self.ip, self.port))
connection = self.socket.makefile('rb')
print('Recibiendo fotos')
while not self.haltSignal():
image_len, image_width, image_height = struct.unpack('<LLL', connection.read(struct.calcsize('<LLL')))
print ("L: " + str(image_len) + " W: " + str(image_width) + " H: " + str(image_height))
if not image_len:
break
image_stream = io.BytesIO()
image_stream.write(connection.read(image_len))
image_stream.seek(0)
self.callback(image_stream.getvalue(), image_width, image_height)
self.socket.close()
self.socket = None
self.thread = None
So, the question is... having this Server.py, how can I make a C# client that receives the stream and show the pictures?
I've tried to capture the received data in C#, over the socket, store It in a Byte array and convert It to a readable image, which sometimes, when It's a (very)small sized picture, works fine. I guess there must be a better way to achieve this.
Thanks!
Edit: This is what I've been trying in C#. Thanks again!
public int Connect()
{
if (_clientSocket.Connected == false)
{
_clientSocket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
mutexToSend = new Mutex();
mutexReceived = new Mutex();
ThreadStart ts = new ThreadStart(SendReceiveData);
threadSend = new Thread(ts);
threadSend.Start();
return 1;
}
else
return 0; //There is a socket already connected.
}
//----------------------------------------------------------
private void SendReceiveData()
{
try
{
_clientSocket.SendTimeout = 1000;
_clientSocket.ReceiveTimeout = 1000;
System.Net.IPAddress ipaddress = this.serverIP;
_clientSocket.Connect(new IPEndPoint(ipaddress, this.serverPort));
}
catch (SocketException ex)
{
//turn ex.Message;
}
_clientSocket.BeginReceive(_recieveBuffer, 0, _recieveBuffer.Length, SocketFlags.None, new AsyncCallback(ReceiveCallback), null);
}
//----------------------------------------------------------
private void ReceiveCallback(IAsyncResult AR)
{
//Check how much bytes are recieved and call EndRecieve to finalize handshake
int recieved = _clientSocket.EndReceive(AR);
if (recieved <= 0)
{
GestorCamaraSala.imgReceived = _recieveBuffer; //I copy the received data to the image texture
return;
}
//Start receiving again
_clientSocket.BeginReceive(_recieveBuffer, 0, _recieveBuffer.Length, SocketFlags.None, new AsyncCallback(ReceiveCallback), null);
}// end ReceivedCallback
I am trying to send an int number from Python to an Arduino using PySerial, using .write([data]) to send with Python and Serial.read() or Serial.readString() to recieve on the Arduino, then .setPixelColor() and .show() to light a LED on a matrix which position corresponds to the int sent by the arduino (I am using the Duinofun Neopixel Shield).
But It does not seem to work properly, and I can't use the Serial Monitor as I am sending my data as the port would be busy.
I have tried to input a number myself using Serial.readString() then converting the string to an int and finally putting in into my function that displays the LED.
It does work properly when I do this, but when I send some data over, all the previously lit LEDs suddenly switch off which can only be caused by a reset of the Arduino board as far as I know.
This is the python code, it simply sends an int chosen by the user
import serial
a = int(input('Enter pixel position : '))
ser = serial.Serial("COM16", 9600)
ser.write([a])
And this is the part of the Arduino program that reads the incoming data.
Adafruit_NeoPixel pixels = Adafruit_NeoPixel(40, 6, NEO_GRB + NEO_KHZ800);
void setup() {
// put your setup code here, to run once:
pixels.begin();
Serial.begin(9600);
}
void loop() {
String a = Serial.readString();
int b = a.toInt();
pixels.setPixelColor(b, 30,30,30);
pixels.show();
Serial.println(a);
delay(1000);
}
All the LED switch off when I send some data, except the first LED which position corresponds to a 0 used in the .setPixelColor() function.
Problem is, the LED should light to the corresponding int sent by Python (e.g light the fifth LED for an int of 4).
You don't need to send an int from your Python script. Just send a string and then convert it back to int on your Arduino. Also, you can verify the number simply on your Arduino code if the received value is valid.
Another problem with your Arduino code is you are not checking the Serial port availability which would return an empty string by Serial.readString().
A simple approach is shown below but you can extend it for other pixels.
Python script:
import serial
ser = serial.Serial("COM16", 9600)
while True:
input_value = input('Enter pixel position: ')
ser.write(input_value.encode())
Arduino code:
#define MIN_PIXEL_RANGE 0
#define MAX_PIXEL_RANGE 100
Adafruit_NeoPixel pixels = Adafruit_NeoPixel(40, 6, NEO_GRB + NEO_KHZ800);
void setup()
{
// put your setup code here, to run once:
pixels.begin();
Serial.begin(9600);
}
void loop()
{
if (Serial.available())
{
String a = Serial.readString();
Serial.print("Received Value: ");
Serial.println(a);
int b = a.toInt();
if ((b >= MIN_PIXEL_RANGE) && (b <= MAX_PIXEL_RANGE))
{
pixels.setPixelColor(b, 30, 30, 30);
pixels.show();
delay(1000);
}
}
}
You can communicate between Ardinos and Python really easily and reliably if you use the pip-installable package pySerialTransfer. The package is non-blocking, easy to use, supports variable length packets, automatically parses packets, and uses CRC-8 for packet corruption detection.
Here's an example Python script:
from pySerialTransfer import pySerialTransfer as txfer
if __name__ == '__main__':
try:
link = txfer.SerialTransfer('COM13')
link.txBuff[0] = 'h'
link.txBuff[1] = 'i'
link.txBuff[2] = '\n'
link.send(3)
while not link.available():
if link.status < 0:
print('ERROR: {}'.format(link.status))
print('Response received:')
response = ''
for index in range(link.bytesRead):
response += chr(link.rxBuff[index])
print(response)
link.close()
except KeyboardInterrupt:
link.close()
Note that the Arduino will need to use the library SerialTransfer.h. You can install SerialTransfer.h using the Arduino IDE's Libraries Manager.
Here's an example Arduino sketch:
#include "SerialTransfer.h"
SerialTransfer myTransfer;
void setup()
{
Serial.begin(115200);
Serial1.begin(115200);
myTransfer.begin(Serial1);
}
void loop()
{
myTransfer.txBuff[0] = 'h';
myTransfer.txBuff[1] = 'i';
myTransfer.txBuff[2] = '\n';
myTransfer.sendData(3);
delay(100);
if(myTransfer.available())
{
Serial.println("New Data");
for(byte i = 0; i < myTransfer.bytesRead; i++)
Serial.write(myTransfer.rxBuff[i]);
Serial.println();
}
else if(myTransfer.status < 0)
{
Serial.print("ERROR: ");
Serial.println(myTransfer.status);
}
}
Lastly, note that you can transmit ints, floats, chars, etc. using the combination of these libraries!
I'm working on a wireless sensor network in which I have one coordinator router (API mode 2) connected to a Raspberry Pi 2, 5 or more routers in API mode 2 as well. Each router is connected to an Arduino Uno. The Unos also have different sensors attached to them (temperature, humidity etc). I have to send data from the sensors to the coordinator and process it. I have successfully transferred data using one router and coordinator (just two XBee S2 modules). On the Arduini I'm using Andrew's library https://github.com/andrewrapp/xbee-arduino and on the Pi I'm using a Python-xbee library https://github.com/nioinnovation/python-xbee. For a single router and coordinator my codes are:
Arduino Code (Router):
#include <XBee.h>
#include <math.h>
// create the XBee object
XBee xbee = XBee();
int sensor = A5;
uint8_t payload[8] = {0, 0, 0, 0, 0, 0, 0, 0};
// union to convert float to byte string
union u_tag {
uint8_t b[4];
float fval;
} u;
// SH + SL Address of receiving XBee
XBeeAddress64 addr64 = XBeeAddress64(0x0013a200, 0x40DC7C90);
ZBTxRequest zbTx = ZBTxRequest(addr64, payload, sizeof(payload));
ZBTxStatusResponse txStatus = ZBTxStatusResponse();
int statusLed = 13;
int errorLed = 12;
void flashLed(int pin, int times, int wait) {
for (int i = 0; i < times; i++) {
digitalWrite(pin, HIGH);
delay(wait);
digitalWrite(pin, LOW);
if (i + 1 < times) {
delay(wait);
}
}
}
double Thermistor(int RawADC)
{
double Temp;
Temp = log(10000.0 * ((1024.0 / RawADC - 1)));
Temp = 1 / (0.001129148 + (0.000234125 + (0.0000000876741 * Temp * Temp )) * Temp );
Temp = Temp - 273.15; // Convert Kelvin to Celcius
//Temp = (Temp * 9.0)/ 5.0 + 32.0; // Convert Celcius to Fahrenheit
return Temp;
}
void setup() {
pinMode(statusLed, OUTPUT);
pinMode(errorLed, OUTPUT);
Serial.begin(9600);
}
void loop() {
float rawADC = analogRead(sensor);
float t = Thermistor (rawADC);
// check if returns are valid, if they are NaN (not a number) then something went wrong!
if (!isnan(t)) {
// convert temperature into a byte array and copy it into the payload array
u.fval = t;
for (int i=0;i<4;i++){
payload[i]=u.b[i];
}
u.fval = 100.33;
for (int i=0;i<4;i++){
payload[i+4]=u.b[i];
}
xbee.send(zbTx);
flashLed(statusLed, 1, 100); // flash TX indicator
// after sending a tx request, we expect a status response, wait up to half second for the status response
if (xbee.readPacket(500)) {
// got a response!
// should be a znet tx status
if (xbee.getResponse().getApiId() == ZB_TX_STATUS_RESPONSE) {
xbee.getResponse().getZBTxStatusResponse(txStatus);
// get the delivery status, the fifth byte
if (txStatus.getDeliveryStatus() == SUCCESS) {
// success. time to celebrate
flashLed(statusLed, 5, 50);
} else {
// the remote XBee did not receive our packet. is it powered on?
flashLed(errorLed, 3, 500);
}
}
} else if (xbee.getResponse().isError()) {
//nss.print("Error reading packet. Error code: ");
//nss.println(xbee.getResponse().getErrorCode());
} else {
// local XBee did not provide a timely TX Status Response -- should not happen
flashLed(errorLed, 1, 50);
}
}
delay(2000);
}
Raspberry Pi Code (Coordinator):
from xbee import ZigBee
import serial
import struct
import datetime
PORT = '/dev/ttyUSB0'
BAUD_RATE = 9600
def hex(bindata):
return ''.join('%02x' % ord(byte) for byte in bindata)
# Open serial port
ser = serial.Serial(PORT, BAUD_RATE)
# Create API object
xbee = ZigBee(ser,escaped=True)
# Continuously read and print packets
while True:
try:
response = xbee.wait_read_frame()
sa = hex(response['source_addr_long'])
rf = hex(response['rf_data'])
obj = createObject(response)
obj.createPacket()
print ("Temperature: %.2f" % obj.packet['temperature'],
"Humidity: %.2f" % obj.packet['humidity'],
"Source Address: 0x%s" % obj.packet['sourceAddressShort'],
"Timestamp: %s" % obj.packet['timestamp'].isoformat())
except KeyboardInterrupt:
break
ser.close()
class createObject:
def __init__(self, response):
self.sourceAddrLong = hex(response['source_addr_long'])
self.rfData = hex(response['rf_data'])
self.sourceAddrShort = hex(response['source_addr_long'][4:])
self.options = response.pop('options')
self.frameType = response['id']
self.temperature = struct.unpack('f',response['rf_data'][0:4])[0]
self.humidity = struct.unpack('f',response['rf_data'][4:])[0]
self.dataLength = len(response['rf_data'])
self.packet={}
self.dateNow = datetime.datetime.utcnow()
self.packetJson=0
def createPacket(self):
self.packet.update({
'timestamp' : self.dateNow,
'temperature' : self.temperature,
'humidity' : self.humidity,
'dataLength' : self.dataLength,
'sourceAddressLong' : self.sourceAddrLong,
'sourceAddressShort' : self.sourceAddrShort,
'options' : self.options,
'frameType' : self.frameType
})
I have a few questions that I can't find answers to. I have looked almost everywhere but still have some confusions.
In the Arduino code, there is a portion of code at the end where it checks for the status response (I did not write the code, found it on the internet). When I set it up, my errorLED connected to pin 12 blinks once and looking into the code it means that the "local XBee did not provide a timely TX Status Response". My question is, do I have to send a response myself from the coordinator in python or is it generated automatically? If I have to do it myself, how would I do it? Because right now, there is not response. My setup works fine as I'm getting correct values on my Pi.
When I have more than one router, how would I handle it in the code? Would I keep sending sensor values after every 2 seconds from the arduino and loop through the address on Pi or is there another way that it is done usually? I'm very confused about it.
Right now, if I add more routers, they will keep sending out frames with sensor values and the coordinator reads them in a loop. How can I setup the system such that the coordinator sends a signal to each router and asks for the data and then the router replies with the data? Is it possible?
The Transmit Status frame happens automatically when the local XBee has confirmed delivery to the remote XBee. It's a low-level acknowledgement. My guess is that there's a problem in that code's logic. Maybe the response comes back after 500ms. The only way to tell would be to restructure the code to constantly poll the local XBee for frames, send the sensor status frame every two seconds, and keep track of how long it's been since the last successful Transmit Status frame comes in. And I'd recommend increasing the baud rate here as well, especially since the existing Arduino code isn't processing bytes as often as it should (e.g., 2 seconds of idling without reading the serial port).
It looks like the Raspberry Pi code is already set up to handle data from multiple devices inside the "Continuously read and print packets" loop. I would suggest configuring your XBee module to 115200bps and updating the Python code with the new value, so you're not limiting the data rate.
Your current design is easier to manage -- routers on the network will always report their sensor readings. I think you can even update the code to use 00000000-00000000 as the destination address, and the router will always send to the coordinator. But you could modify the Raspberry Pi code to keep a list of router MAC addresses (discovered via ATND node discovery), and send requests to them as needed. You'd need to modify the Arduino code to watch for inbound frames, and generate an outbound frame when a request comes in.
I'd recommend adding a second router to your setup without any code changes, and then just seeing how that works. From what I can tell, the Raspberry Pi will just print packets with different source addresses as they come in.