We Keep Coding

Sending Real Time images captured using unity camera

server
private void SendImageByte()
{
image_bytes = cm.Capture();
print(image_bytes.Length);
if (connectedTcpClient == null)
{
return;
}
try
{
// Get a stream object for writing.
NetworkStream stream = connectedTcpClient.GetStream();
if (stream.CanWrite)
{
// string serverMessage = "This is a message from your server.";
// Convert string message to byte array.
byte[] serverMessageAsByteArray = Encoding.ASCII.GetBytes(image_bytes.ToString());
// Write byte array to socketConnection stream.
stream.Write(serverMessageAsByteArray, 0, serverMessageAsByteArray.Length);
Debug.Log("Server sent his message - should be received by client");
}
}
catch (SocketException socketException)
{
Debug.Log("Socket exception: " + socketException);
}
}
client
import socket
host = "127.0.0.1"
port = 1755
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((host, port))
def receive_image():
data = sock.recv(999999).decode('utf-8')
print(len(data))
while True:
receive_image()
here script capture images from unity camera
public byte[] Capture()
{
if(renderTexture == null)
{
// creates off-screen render texture that can rendered into
rect = new Rect(0, 0, captureWidth, captureHeight);
renderTexture = new RenderTexture(captureWidth, captureHeight, 24);
screenShot = new Texture2D(captureWidth, captureHeight, TextureFormat.RGB24, false);
}
// _camera = GetComponent<Camera>();
_camera.targetTexture = renderTexture;
_camera.Render();
// reset active camera texture and render texture
_camera.targetTexture = null;
RenderTexture.active = null;
// read pixels will read from the currently active render texture so make our offscreen
// render texture active and then read the pixels
RenderTexture.active = renderTexture;
screenShot.ReadPixels(rect, 0, 0);
screenShot.Apply();
byte[] imageBytes = screenShot.EncodeToPNG();
//Object.Destroy(screenShot);
//File.WriteAllBytes(Application.dataPath + "/../"+ imagePath + "/img{counter}.png", bytes);
//counter = counter + 1;
return imageBytes;
}
Am trying to send real-time images on Unity3D from C# to python using socket communication to be processed and return back values to unity, but the problem even the bytes length received on the client is not the same as the server. I send about 400K bytes but I receive only 13
C# is the server and python is the client
or am doing it wrong but the main goal I want to create simulator work as udacity self-driving

Are you sure that image_bytes.ToString() returns what you expect and not maybe just something like "System.Byte[]" => 13 chars => 13 bytes.
In general why would you convert an already byte[] into a string just in order to convert it back into byte[] to send? I'm pretty sure you do not want to transmit binary image data using UTF-8 ... one option might be Base64 string, but still that would be quite inefficient.
Just send the raw bytes like e.g.
stream.Write(image_bytes, 0, image_bytes.Length);
And then receive until you receive that length.
A typical solution is to prepend the length of the message to send and on the receiver side actually wait until you received exactly that amount of bytes like e.g.
var lengthBytes = BitConverter.GetBytes(image_bytes.Length);
stream.Write(lengthBytes, 0, lengthBytes.Length);
stream.Write(image_bytes, 0, image_bytes.Length);
Now you know that on the receiver side you first have to receive exactly 4 bytes (== one int) which will tell you exactly how many bytes to receive for the actual payload.
Now I'm no python expert but after googling around a bit I think something like
def receive_image()
lengthBytes = sock.recv(4)
length = struct.unpack("!i", lengthBytes)[0]
data = sock.recv(length)
Note: After reading John Gordon's comment on the question I guess this still doesn't fully solve the waiting until according buffers are actually filled - as said no python expert - but I hope it gives you a idea where to go ;)

Converting Python struct.pack to C#

I'm converting some python code to C# and was successful so far, but there's this one part that I don't understand and is arguably the most important part as it involves file decompression and turning into JSON and human readable format.
file_data: bytes = file.read() 'I understand this
file_data: bytes = struct.pack('B' * len(file_data), *file_data[::-1]) 'This is the part that I do not understand, everything after the 'B'
file_data: bytes = zlib.decompress(file_data) 'should be fine, c# should have same library
file_data: tuple = pickle.loads(file_data, encoding='windows-1251')

That python code line simply reverses the code and removes the header.
Here's the c# code that does the same.
Hashtable UnpickledGP;
byte[] FileBytes = File.ReadAllBytes("GameParams.data").Reverse().ToArray();
byte[] ModdedFileBytes = new byte[FileBytes.Length - 2]; //remove file header
Array.Copy(FileBytes, 2, ModdedFileBytes, 0, ModdedFileBytes.Length);
using (Stream StreamTemp = new MemoryStream(ModdedFileBytes))
{
using (MemoryStream MemoryStreamTemp = new MemoryStream())
{
using (DeflateStream DeflateStreamTemp = new DeflateStream(StreamTemp, CompressionMode.Decompress))
{
DeflateStreamTemp.CopyTo(MemoryStreamTemp);
}
ModdedFileBytes = MemoryStreamTemp.ToArray();
}
}
using (Unpickler UnpicklerTemp = new Unpickler())
{
object[] UnpickledObjectTemp = (object[])UnpicklerTemp.loads(ModdedFileBytes);
UnpickledGP = (Hashtable)UnpickledObjectTemp[0];
UnpicklerTemp.close();
}

How to send image in C# to a Flask Server that is decoded by OpenCV?

Here is part of my Flask API in Python:
image_data = flask.request.get_data() # image_data's data type
string image_vector = numpy.frombuffer(image_data, dtype=numpy.uint8)
image = cv2.imdecode(image_vector, cv2.IMREAD_COLOR)
How would I send a image that I encoded like below, in C#:
ResultString = "Loading...";
var surface = SKSurface.Create(new SKImageInfo((int)canvasView.CanvasSize.Width,
(int)canvasView.CanvasSize.Height));
var canvas = surface.Canvas;
canvas.Clear();
foreach (SKPath path in completedPaths)
canvas.DrawPath(path, paint);
foreach (SKPath path in inProgressPaths.Values)
canvas.DrawPath(path, paint);
canvas.Flush();
var snap = surface.Snapshot();
var pngImage = snap.Encode(SKEncodedImageFormat.Png, 100);
AnalyerResults analyerResults = mathclient.AnalyzeWork(pngImage);
try { ResultString = analyerResults.message; } catch { ResultString = "Error..."; }
How would I send the image to in C# to be able to be received and decoded like shown in part of my API?
I already tried:
HttpClient client = await GetClient();
var result = await client.PostAsync(Url + "analyzer", new ByteArrayContent(pngImage.ToArray()));
return JsonConvert.DeserializeObject<AnalyerResults>(await result.Content.ReadAsStringAsync());
I also tried:
var client = new RestClient(Url + "analyzer");
client.Timeout = -1;
var request = new RestRequest(Method.POST);
request.AddHeader("Content-Type", "image/png");
request.AddParameter("image/png", pngImage, ParameterType.RequestBody);
IRestResponse response = client.Execute(request);
return JsonConvert.DeserializeObject<AnalyerResults>(response.Content);
However in both the content returned null. This question is related to How to Replicate this Postman Request which has a Binary Content Body and contains a .PNG File in C#?.

Python UDP SocketServer can't read whole packet

At sender side I have the following code using processing language (portion code):
udp = new UDP( this, 6002 ); // create a new datagram connection on port 6000
//udp.log( true ); // <-- printout the connection activity
udp.listen( true ); // and wait for incoming message
escribeUDPLog3(1,TRANSMIT,getTime()); //call function
int[] getTime(){
int year = year();
int month = month()
int day = day();
int hour = hour();
int minute = minute();
int second = second();
int[] time_constructed = {year, month,day,hour,minute,second};
return time_constructed;
}
void escribeUDPLog3(int pkg_type, int state, int[] time){
short year = (short)(time[0]); //>> 8;
byte year_msb = byte(year >> 8);
byte year_lsb = byte(year & 0x00FF);
byte month = byte(time[1]);
byte day = byte(time[2]);
byte hour = byte(time[3]);
byte minute = byte(time[4]);
byte second = byte(time[5]);
byte[] payload = {byte(pkg_type), byte(state), year_msb, year_lsb, month, day, hour, minute,second};
try {
if (UDP5_flag) {udp.send(payload, UDP5_IP, UDP5_PORT);}
}
catch (Exception e) {
e.printStackTrace();
}
}
At receiver side I'm using SocketServer python structure to set up a server listening for udp datagrams, as following.
from datetime import datetime
import csv
import SocketServer
def nodeStateCheckout(nodeid, state, nodeState):
if (state == ord(nodeState)):
return "OK"
else:
return "FAIL"
def timeConstructor(time):
year = str(ord(time[0]) << 8 | ord(time[1]))
month = str(ord(time[2]))
day = str(ord(time[3]))
hour = str(ord(time[4]))
minute = str(ord(time[5]))
second = str(ord(time[6]))
time_formatted = year + "-" + month + "-" + day \
+ " " + hour + ":" + minute + ":" + second
return time_formatted
class MyUDPHandler(SocketServer.BaseRequestHandler):
"""
This class works similar to the TCP handler class, except that
self.request consists of a pair of data and client socket, and since
there is no connection the client address must be given explicitly
when sending data back via sendto().
"""
def handle(self):
try:
data = self.request[0].strip()
socket = self.request[1]
#print "{} wrote:".format(self.client_address[0])
pkg_type = ord(data[0])
if pkg_type == 1: # log 3
state = ord(data[1])
csvfile = open("log3.csv", "a+")
csvwriter = csv.writer(csvfile, delimiter=',')
time_reconstructed = timeConstructor(data[2:9])
if state == 3:
csvwriter.writerow(["STOP",time_reconstructed])
elif state == 2:
csvwriter.writerow(["START",time_reconstructed])
else:
print "unknown state"
csvfile.close()
else:
print "packet not known"
except IndexError:
print "Bad parsed byte"
if __name__ == "__main__":
HOST, PORT = "localhost", 8892
server = SocketServer.UDPServer((HOST, PORT), MyUDPHandler)
server.serve_forever()
Edited:
I have problem specifically when using timeConstructor(data[2:9]), because I'm accessing out of index data, sometimes (with the help of print) I can't received second byte from data, and one time it get me out of index because I didn't received minute and second. Most of the time the code works well, but this type of error get me curious.
Old:
The problem is when reading the payload, sometimes its seems that some bytes doesn't arrive, even when I captured the whole payload using Wireshark (but Wireshark didn't tell me if this is the sent packet or received packet because I'm using loopback interfaces, maybe duplicated info?). If the datagram has 16 bytes payload long, sometimes I received 15 because when parsing from data I get out of index error.
I think that there are some buffer problems. Isn't it? How to configured it properly? I know that I can get packet loss because of connectionless protocol but I dont think that bytes get lost. It is supposed that "data" has all payload data from one udp datagram.

I believe your problem is that socket.sendto() does not always send all the bytes that you give it. It returns the number of bytes sent and you may have to call it again. You might be better off with opening the socket yourself and calling socket.sendall()

Problem in downloading the file using sharepoint copy.asmx

I am trying to download the file from the document using sharepoint webservices called copy.asmx. its onlt 100 kb file size.
But its not downloading the file.
The web services iteself return empty stream (out byte[] Stream) in the web service response. is that any memory issue.
Also it returning like "download_document()out of memory"
Note: I am using the MFP printer to view this application.

Please try below function. you need to pass FileURL(Full web url for document), Title(Pass name you want to give for downloaded file.)
public string DownLoadfiletolocal(string FileURL, string Title)
{
//Copy.Copy is a webservice object that I consumed.
Copy.Copy CopyObj = new Copy.Copy();
CopyObj.Url = SiteURL + "/_vti_bin/copy.asmx"; // Dynamically passing SiteURL
NetworkCredential nc2 = new NetworkCredential();
nc2.Domain = string.Empty;
nc2.UserName = _UserName;
nc2.Password = _Password;
string copySource = FileURL; //Pass full url for document.
Copy.FieldInformation myFieldInfo = new Copy.FieldInformation();
Copy.FieldInformation[] myFieldInfoArray = { myFieldInfo };
byte[] myByteArray;
// Call the web service
uint myGetUint = CopyObj.GetItem(copySource, out myFieldInfoArray, out myByteArray);
// Convert into Base64 String
string base64String;
base64String = Convert.ToBase64String(myByteArray, 0, myByteArray.Length);
// Convert to binary array
byte[] binaryData = Convert.FromBase64String(base64String);
// Create a temporary file to write the text of the form to
string tempFileName = Path.GetTempPath() + "\\" + Title;
// Write the file to temp folder
FileStream fs = new FileStream(tempFileName, FileMode.Create, FileAccess.ReadWrite);
fs.Write(binaryData, 0, binaryData.Length);
fs.Close();
return tempFileName;
}