At sender side I have the following code using processing language (portion code):
udp = new UDP( this, 6002 ); // create a new datagram connection on port 6000
//udp.log( true ); // <-- printout the connection activity
udp.listen( true ); // and wait for incoming message
escribeUDPLog3(1,TRANSMIT,getTime()); //call function
int[] getTime(){
int year = year();
int month = month()
int day = day();
int hour = hour();
int minute = minute();
int second = second();
int[] time_constructed = {year, month,day,hour,minute,second};
return time_constructed;
}
void escribeUDPLog3(int pkg_type, int state, int[] time){
short year = (short)(time[0]); //>> 8;
byte year_msb = byte(year >> 8);
byte year_lsb = byte(year & 0x00FF);
byte month = byte(time[1]);
byte day = byte(time[2]);
byte hour = byte(time[3]);
byte minute = byte(time[4]);
byte second = byte(time[5]);
byte[] payload = {byte(pkg_type), byte(state), year_msb, year_lsb, month, day, hour, minute,second};
try {
if (UDP5_flag) {udp.send(payload, UDP5_IP, UDP5_PORT);}
}
catch (Exception e) {
e.printStackTrace();
}
}
At receiver side I'm using SocketServer python structure to set up a server listening for udp datagrams, as following.
from datetime import datetime
import csv
import SocketServer
def nodeStateCheckout(nodeid, state, nodeState):
if (state == ord(nodeState)):
return "OK"
else:
return "FAIL"
def timeConstructor(time):
year = str(ord(time[0]) << 8 | ord(time[1]))
month = str(ord(time[2]))
day = str(ord(time[3]))
hour = str(ord(time[4]))
minute = str(ord(time[5]))
second = str(ord(time[6]))
time_formatted = year + "-" + month + "-" + day \
+ " " + hour + ":" + minute + ":" + second
return time_formatted
class MyUDPHandler(SocketServer.BaseRequestHandler):
"""
This class works similar to the TCP handler class, except that
self.request consists of a pair of data and client socket, and since
there is no connection the client address must be given explicitly
when sending data back via sendto().
"""
def handle(self):
try:
data = self.request[0].strip()
socket = self.request[1]
#print "{} wrote:".format(self.client_address[0])
pkg_type = ord(data[0])
if pkg_type == 1: # log 3
state = ord(data[1])
csvfile = open("log3.csv", "a+")
csvwriter = csv.writer(csvfile, delimiter=',')
time_reconstructed = timeConstructor(data[2:9])
if state == 3:
csvwriter.writerow(["STOP",time_reconstructed])
elif state == 2:
csvwriter.writerow(["START",time_reconstructed])
else:
print "unknown state"
csvfile.close()
else:
print "packet not known"
except IndexError:
print "Bad parsed byte"
if __name__ == "__main__":
HOST, PORT = "localhost", 8892
server = SocketServer.UDPServer((HOST, PORT), MyUDPHandler)
server.serve_forever()
Edited:
I have problem specifically when using timeConstructor(data[2:9]), because I'm accessing out of index data, sometimes (with the help of print) I can't received second byte from data, and one time it get me out of index because I didn't received minute and second. Most of the time the code works well, but this type of error get me curious.
Old:
The problem is when reading the payload, sometimes its seems that some bytes doesn't arrive, even when I captured the whole payload using Wireshark (but Wireshark didn't tell me if this is the sent packet or received packet because I'm using loopback interfaces, maybe duplicated info?). If the datagram has 16 bytes payload long, sometimes I received 15 because when parsing from data I get out of index error.
I think that there are some buffer problems. Isn't it? How to configured it properly? I know that I can get packet loss because of connectionless protocol but I dont think that bytes get lost. It is supposed that "data" has all payload data from one udp datagram.
I believe your problem is that socket.sendto() does not always send all the bytes that you give it. It returns the number of bytes sent and you may have to call it again. You might be better off with opening the socket yourself and calling socket.sendall()
Related
I'm trying to read serial input from my device, and have gotten it to work in Python using pyserial, e.g.
import serial
port = serial.Serial(port='COM1', baudrate=19200, bytesize=8, parity='N', stopbits=1, timeout=None, xonxoff=False, rtscts=False, dsrdtr=False)
while 1:
N = port.in_waiting
if N>4:
msg = port.readline(N)
print(list(msg))
I'm trying to implement this same code in C#, but it does not quite seem to work, e.g.
port = new SerialPort(COM1);
port.BaudRate = baudRate;
port.DataBits = 8;
port.Parity = Parity.None;
port.StopBits = StopBits.One;
port.ReadTimeout = SerialPort.InfiniteTimeout;
port.Handshake = Handshake.None;
port.RtsEnable = false;
port.DtrEnable = false;
port.DataReceived += new SerialDataReceivedEventHandler(DataReceived);
and
void DataReceived(object sender, SerialDataReceivedEventArgs e)
{
int N = port.BytesToRead;
if (N > 4)
{
string line = port.ReadLine();
}
}
I'm able to read stuff properly using port.Read() in C#, however it seems like ReadLine doesn't quite work properly--it doesn't seem to be able to find the end-of-line character ("\n")? And the program just freezes. Yet I am not sure why it works with pyserial.ReadLine() which also seeks the same character (and works without timeout). As far as I can tell, the rest of the port settings are identical.
Thanks!
Because the DataReceived event indicates that data has been received through a port, the data is already read and you should call SerialPort.ReadExisting to get it.
void DataReceived(object sender, SerialDataReceivedEventArgs e)
{
var data = port.ReadExisting();
}
This event does not guarantee that its data is a single line, if you want a similar way to pySerial, you should not use the event and use ReadLine directly:
port.DataReceived += new SerialDataReceivedEventHandler(DataReceived);
while(true)
{
int N = port.BytesToRead;
if (N > 4)
{
string line = port.ReadLine();
}
}
server
private void SendImageByte()
{
image_bytes = cm.Capture();
print(image_bytes.Length);
if (connectedTcpClient == null)
{
return;
}
try
{
// Get a stream object for writing.
NetworkStream stream = connectedTcpClient.GetStream();
if (stream.CanWrite)
{
// string serverMessage = "This is a message from your server.";
// Convert string message to byte array.
byte[] serverMessageAsByteArray = Encoding.ASCII.GetBytes(image_bytes.ToString());
// Write byte array to socketConnection stream.
stream.Write(serverMessageAsByteArray, 0, serverMessageAsByteArray.Length);
Debug.Log("Server sent his message - should be received by client");
}
}
catch (SocketException socketException)
{
Debug.Log("Socket exception: " + socketException);
}
}
client
import socket
host = "127.0.0.1"
port = 1755
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((host, port))
def receive_image():
data = sock.recv(999999).decode('utf-8')
print(len(data))
while True:
receive_image()
here script capture images from unity camera
public byte[] Capture()
{
if(renderTexture == null)
{
// creates off-screen render texture that can rendered into
rect = new Rect(0, 0, captureWidth, captureHeight);
renderTexture = new RenderTexture(captureWidth, captureHeight, 24);
screenShot = new Texture2D(captureWidth, captureHeight, TextureFormat.RGB24, false);
}
// _camera = GetComponent<Camera>();
_camera.targetTexture = renderTexture;
_camera.Render();
// reset active camera texture and render texture
_camera.targetTexture = null;
RenderTexture.active = null;
// read pixels will read from the currently active render texture so make our offscreen
// render texture active and then read the pixels
RenderTexture.active = renderTexture;
screenShot.ReadPixels(rect, 0, 0);
screenShot.Apply();
byte[] imageBytes = screenShot.EncodeToPNG();
//Object.Destroy(screenShot);
//File.WriteAllBytes(Application.dataPath + "/../"+ imagePath + "/img{counter}.png", bytes);
//counter = counter + 1;
return imageBytes;
}
Am trying to send real-time images on Unity3D from C# to python using socket communication to be processed and return back values to unity, but the problem even the bytes length received on the client is not the same as the server. I send about 400K bytes but I receive only 13
C# is the server and python is the client
or am doing it wrong but the main goal I want to create simulator work as udacity self-driving
Are you sure that image_bytes.ToString() returns what you expect and not maybe just something like "System.Byte[]" => 13 chars => 13 bytes.
In general why would you convert an already byte[] into a string just in order to convert it back into byte[] to send? I'm pretty sure you do not want to transmit binary image data using UTF-8 ... one option might be Base64 string, but still that would be quite inefficient.
Just send the raw bytes like e.g.
stream.Write(image_bytes, 0, image_bytes.Length);
And then receive until you receive that length.
A typical solution is to prepend the length of the message to send and on the receiver side actually wait until you received exactly that amount of bytes like e.g.
var lengthBytes = BitConverter.GetBytes(image_bytes.Length);
stream.Write(lengthBytes, 0, lengthBytes.Length);
stream.Write(image_bytes, 0, image_bytes.Length);
Now you know that on the receiver side you first have to receive exactly 4 bytes (== one int) which will tell you exactly how many bytes to receive for the actual payload.
Now I'm no python expert but after googling around a bit I think something like
def receive_image()
lengthBytes = sock.recv(4)
length = struct.unpack("!i", lengthBytes)[0]
data = sock.recv(length)
Note: After reading John Gordon's comment on the question I guess this still doesn't fully solve the waiting until according buffers are actually filled - as said no python expert - but I hope it gives you a idea where to go ;)
I write python socket client, but server return remaining time how to decode this structure in python.
header = Buffer.from( [03, 00, 00, 00]);
time = new Uint16Array(1);
time[0] = remaining_time_in_sec;
packet = Buffer.concat( [header, Buffer.from(time.buffer)]);
this.socket.write(packet);
this function return like
b'\x03\x00\x00\x00\t\x01'
i think solve this problem
this return binary code
b'\x03\x00\x00\x00\x0e\x01'
this here packet type
b'\x03\x00\x00\x00'
unpack('h',b'\x0e\x01')
return <======================
270
I am trying to measure the responses back from DNS servers. Making a sniffer for a typical DNS response that is less than 512 bytes is no big deal. My issue is receiving large 3000+ byte responses - in some cases 5000+ bytes. I haven't been able to get a socket working that can receive that data reliably. Is there a way with Python sockets to receive from a specific source address?
Here is what I have so far:
import socket
import struct
def craft_dns(Qdns):
iden = struct.pack('!H', randint(0, 65535))
QR_thru_RD = chr(int('00000001', 2)) # '\x01'
RA_thru_RCode = chr(int('00100000', 2)) # '\x00'
Qcount = '\x00\x01' # question count is 1
ANcount = '\x00\x00'
NScount = '\x00\x00'
ARcount = '\x00\x01' # additional resource count is 1
pad = '\x00' #
Rtype_ANY = '\x00\xff' # Request ANY record
PROtype = '\x00\x01' # Protocol IN || '\x00\xff' # Protocol ANY
DNSsec_do = chr(int('10000000', 2)) # flips DNSsec bit to enable
edns0 = '\x00\x00\x29\x10\x00\x00\x00\x00\x00\x00\x00' # DNSsec disabled
domain = Qdns.split('.')
quest = ''
for x in domain:
quest += struct.pack('!B', len(x)) + x
packet = (iden+QR_thru_RD+RA_thru_RCode+Qcount+ANcount+NScount+ARcount+
quest+pad+Rtype_ANY+PROtype+edns0) # remove pad if asking <root>
return packet
def craft_ip(target, resolv):
ip_ver_len = int('01000101', 2) # IPvers: 4, 0100 | IP_hdr len: 5, 0101 = 69
ipvers = 4
ip_tos = 0
ip_len = 0 # socket will put in the right length
iden = randint(0, 65535)
ip_frag = 0 # off
ttl = 255
ip_proto = socket.IPPROTO_UDP # dns, brah
chksm = 0 # socket will do the checksum
s_addr = socket.inet_aton(target)
d_addr = socket.inet_aton(resolv)
ip_hdr = struct.pack('!BBHHHBBH4s4s', ip_ver_len, ip_tos, ip_len, iden,
ip_frag, ttl, ip_proto, chksm, s_addr, d_addr)
return ip_hdr
def craft_udp(sport, dest_port, packet):
#sport = randint(0, 65535) # not recommended to do a random port generation
udp_len = 8 + len(packet) # calculate length of UDP frame in bytes.
chksm = 0 # socket fills in
udp_hdr = struct.pack('!HHHH', sport, dest_port, udp_len, chksm)
return udp_hdr
def get_len(resolv, domain):
target = "10.0.0.3"
d_port = 53
s_port = 5353
ip_hdr = craft_ip(target, resolv)
dns_payload = craft_dns(domain) # '\x00' for root
udp_hdr = craft_udp(s_port, d_port, dns_payload)
packet = ip_hdr + udp_hdr + dns_payload
buf = bytearray("-" * 60000)
recvSock = socket.socket(socket.PF_PACKET, socket.SOCK_RAW, socket.ntohs(0x0800))
recvSock.settimeout(1)
sendSock = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_RAW)
sendSock.settimeout(1)
sendSock.connect((resolv, d_port))
sendSock.send(packet)
msglen = 0
while True:
try:
pkt = recvSock.recvfrom(65535)
msglen += len(pkt[0])
print repr(pkt[0])
except socket.timeout as e:
break
sendSock.close()
recvSock.close()
return msglen
result = get_len('75.75.75.75', 'isc.org')
print result
For some reason doing
pkt = sendSock.recvfrom(65535)
Recieves nothing at all. Since I'm using SOCK_RAW the above code is less than ideal, but it works - sort of. If the socket is extremely noisy (like on a WLAN), I could end up receiving well beyond the DNS packets, because I have no way to know when to stop receiving packets when receiving a multipacket DNS answer. For a quiet network, like a lab VM, it works.
Is there a better way to use a receiving socket in this case?
Obviously from the code, I'm not that strong with Python sockets.
I have to send with SOCK_RAW because I am constructing the packet in a raw format. If I use SOCK_DGRAM the custom packet will be malformed when sending to a DNS resolver.
The only way I could see is to use the raw sockets receiver (recvSock.recv or recvfrom) and unpack each packet, look if the source and dest address match within what is supplied in get_len(), then look to see if the fragment bit is flipped. Then record the byte length of each packet with len(). I'd rather not do that. It just seems there is a better way.
Ok I was stupid and didn't look at the protocol for the receiving socket. Socket gets kind of flaky when you try to receive packets on a IPPROTO_RAW protocol, so we do need two sockets. By changing to IPPROTO_UDP and then binding it, the socket was able to follow the complete DNS response over multiple requests. I got rid of the try/catch and the while loop, as it was no longer necessary and I'm able to pull the response length with this block:
recvSock = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_UDP)
recvSock.settimeout(.3)
recvSock.bind((target, s_port))
sendSock = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_RAW)
#sendSock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sendSock.settimeout(.3)
sendSock.bind((target, s_port))
sendSock.connect((resolv, d_port))
sendSock.send(packet)
pkt = recvSock.recvfrom(65535)
msglen = len(pkt[0])
Now the method will return the exact bytes received from a DNS query. I'll leave this up in case anyone else needs to do something similar :)
I have created a simple RAW socket based packet sniffer. But when I run it, it rarely captures up a packet. First I created this to capture packets in 1 second time intervals, but seeing no packets are captured I commented that line. I was connected to internet and a lot of http traffic are going here and there, but I could not capture a one. Is there a problem in this in the code where I created the socket? Please someone give me a solution. I am fairly new to python programming and could not understand how to solve this.
import socket, binascii, struct
import time
sock = socket.socket(socket.PF_PACKET, socket.SOCK_RAW, socket.htons(0x800))
print "Waiting.."
pkt = sock.recv(2048)
print "received"
def processEth(data):
#some code to process source mac and dest. mac
return [smac, dmac]
def processIP(data):
sip = str(binascii.hexlify(data[1]))
dip = str(binascii.hexlify(data[2]))
return [sip, dip]
def processTCP(data):
sport = str(data[0])
dport = str(data[1])
return [sport, dport]
while len(pkt) > 0 :
if(len(pkt)) > 54:
pkt = sock.recv(2048)
ethHeader = pkt[0][0:14]
ipHeader = pkt[0][14:34]
tcpHeader = pkt[0][34:54]
ethH = struct.unpack("!6s6s2s",ethHeader)
ethdata = processEth(ethH)
ipH = struct.unpack("!12s4s4s",ipHeader)
ipdata = processIP(ipH)
tcpH = struct.unpack("!HH16", tcpHeader)
tcpdata = processTCP(tcpH)
print "S.mac "+ethdata[0]+" D.mac "+ethdata[1]+" from: "+ipdata[0]+":"+tcpdata[0]+" to: "+ipdata[1]+":"+tcpdata[1]
#time.sleep(1);
else:
continue
If you showed all the code, you are running into an endless loop.
Whenever a paket is coming in which has not a length greater then 54 bytes, you end up reading the same packet all the time.
Additionally, socket.recv() returns a string/byte sequence; your approach of accessing the data is wrong. pkt[0] returns a string with length 1; pkt[0][x:y] will not return something useful.
I am not familiar with using sockets, but with some changes I got output that might look similar to what you intended (there is something missing in processEth() I think...).
[...]
while len(pkt) > 0:
print "Waiting.."
pkt = sock.recv(2048)
print "received"
if(len(pkt)) > 54:
ethHeader = pkt[0:14]
ipHeader = pkt[14:34]
tcpHeader = pkt[34:38]
ethH = struct.unpack("!6s6s2s",ethHeader)
ethdata = processEth(ethH)
ipH = struct.unpack("!12s4s4s",ipHeader)
ipdata = processIP(ipH)
tcpH = struct.unpack("!HH16", tcpHeader)
tcpdata = processTCP(tcpH)
print "S.mac "+ethdata[0]+" D.mac "+ethdata[1]+" from: "+ipdata[0]+":"+tcpdata[0]+" to: "+ipdata[1]+":"+tcpdata[1]
#time.sleep(1);
else:
continue