I have stable stream source with constant bitrate = 10.69Mbps.
I'm trying to create bitrate analyser using two approaches. First one is using scapy sniff function, another one is simple packet length counter using sock.recv().
The biggest problem is that in the first one, program is 'choking' every 3-5 seconds:(10.516,10.527,10.527,9.926,10.526[Mbps])
and in the second, results are much below true values:(10.251,10.201,10.201,10.221,10.201[Mbps])
What I concluded is that approach with scapy, has to capture about 40-50 datagrams more than another approach.
import socket
import struct
import timeit
from collections import Counter
from scapy.all import sniff
#___________________________part not affecting code_______________________
MCAST_GRP = '239.0.1.104'
MCAST_PORT = 12345
IS_ALL_GROUPS = True
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
if IS_ALL_GROUPS:
sock.bind(('', MCAST_PORT))
else:
sock.bind((MCAST_GRP, MCAST_PORT))
mreq = struct.pack("4sl", socket.inet_aton(MCAST_GRP), socket.INADDR_ANY)
sock.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, mreq)
#__________________________Actual problems___________________________
packet_counts = Counter()
capturedPacketsSize = 0
## Defining custom function that sums sizes of captured packets
def custom_action(packet):
global capturedPacketsSize
capturedPacketsSize += len(packet)
#key = tuple(sorted([packet[0][1].src, packet[0][1].dst]))
#packet_counts.update([key])
#Scapy approach
print("_____.:| Starting analyse of bitrate! |:._____")
for x in range(10):
pkt = sniff(iface="eno4", filter="ip host 239.0.1.104", prn=custom_action, timeout=1)
MCbitrate = round((capturedPacketsSize*8)/(1024*1024),3)
print(MCbitrate)
capturedPacketsSize = 0
#General approach
totalSize = 0
print("_____.:| Starting analyse of bitrate! |:._____")
while 1:
stop = time.time() + 1
while (time.time()<stop):
#datagram = sock.recv(bufferUDP)
totalSize += len(sock.recv(bufferUDP))
theoreticalBitrate = (8*totalSize)/(1024*1024)
print(round(theoreticalBitrate,3))
totalSize = 0
So as you can see, idea of both is the same. But the results aren't.
Do you have any ideas how to increase accuracy without scapy approach or remove 'choking' of scapy approach?
__________________________________SOLUTION__________________________________
I believe that i concluded what may be wrong. After #Shir hint, i checked that in each solution (not considering scapy chokes) there are ~1016 +/- 2 packets per second.
The result of print(len(sock.recv())) gives 1316 which is (I assume) size of decapsulated datagram. 1016*1316*8 = 10.696.448 bits.
By this simple math, I realised, that my denominator as Mega, should be (1000*1000) for Bits, where it was (1024*1024) which is correct for Bytes.
Related
I'm trying to code a python script through which I want to send packets from one host (desktop ubuntu) to a server (Ubuntu server) using multiple source addresses. I decided that would be best to use raw sockets and define my own TCP and IP header. I succeed to send the SYN packet and also get a response with the SYN-ACK packet but then my host doesn't respond with the ACK packet but with the RST packet. This is the first problem with I met in my solution. After the beginning of a three-way handshake I would like to send an HTTP GET keep-alive request. Does anyone know how to do it? Or does anyone know a better solution or library to use?
def send_raw_socket():
s = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_TCP)
s.setsockopt(socket.IPPROTO_IP, socket.IP_HDRINCL, 1)
version_ihl_type_of_service = b'\x45\x00'
total_length = b'\x00\x3c'
Indentification = b'\xab\xcd'
flags_fragment_offset = b'\x00\x00'
TTL_protocol = b'\x40\x06'
Dest_add = b'\x0a\x0a'
Dest_add2 = b'\x1e\x03'
Src_add = b'\x0a\x0a'
src_add2 =b'\x0a\x0a'
Header_Checksum = my_ip_header_checksum(version_ihl_type_of_service, total_length, Indentification, flags_fragment_offset, TTL_protocol, Src_add,src_add2, Dest_add, Dest_add2)
IP_protocol = b'\x00\x06'
TCP_header_lenght = b'\x00\x14'
src_port = (int(src_port.hex(), 16)+49607).to_bytes(2, byteorder='big')
dest_port = b'\x1f\x95' # Source Port | Destination Port
seq_n1 = b'\x00\x00'
seq_n2 = b'\x00\x00' # Sequence Number
ack_n1 = b'\x00\x00'
ack_n2 = b'\x00\x00' # Acknowledgement Number
do_res = b'\x50\x02'
flags_win_s = b'\x71\x10' # Data Offset, Reserved, Flags | Window Size
checksum= tcp_header_checksum(IP_protocol, Src_add, src_add2, Dest_add, Dest_add2, TCP_header_lenght, src_port, dest_port, seq_n1, seq_n2, ack_n1, ack_n2, do_res, flags_win_s)
u_pinter= b'\x00\x00' # Checksum | Urgent Pointer
packet = version_ihl_type_of_service+total_length+Indentification+flags_fragment_offset+TTL_protocol+Header_Checksum+Src_add+src_add2+Dest_add+Dest_add2+src_port+dest_port+seq_n1+seq_n2+ack_n1+ack_n2+do_res+flags_win_s+checksum+u_pinter
s.sendto(packet, ('10.10.30.3',8085))
Wireshark three-way handshake
I created a program, that receives multicast stream and analyzes its bitrate.
The problem started, when I wanted to use multithreading approach here, in order to analyze many multicasts at the same time. Every multicast has it's unique addres, however they have the same port. First stream: 239.0.1.104 has constant bitrate of 10.69 Mbps, second: 239.0.1.105 has also CBR of 6.082 Mbps. The problem is, my program sums up results, and at the end i have:
16.328928
16.328928
16.802688
16.802688
16.750048
16.750048
16.813216
16.813216
16.771104
16.771104
Whereas I expect:
10.69
6.082
10.69
6.082
etc
Important:
I use socket option SO_REUSEADDR which in case of multicast is equal to using both SO_REUSEADDR and SO_REUSEPORT.(Mentioned few lines below)
I read article:
https://lwn.net/Articles/542629/
And also(very compact information providing):
How do SO_REUSEADDR and SO_REUSEPORT differ?
Where i read:
The meaning of SO_REUSEADDR changes for multicast addresses as it allows multiple sockets to be bound to exactly the same combination of source multicast address and port. In other words, for multicast addresses SO_REUSEADDR behaves exactly as SO_REUSEPORT for unicast addresses. Actually, the code treats SO_REUSEADDR and SO_REUSEPORT identically for multicast addresses, that means you could say that SO_REUSEADDR implies SO_REUSEPORT for all multicast addresses and the other way round.
I tried providing socket_name as parameter in order to distinguish both sockets, I tried also to add bitrateList_name and totalSize name as parameters, but always there is the same problem.
import socket
import struct
import time
import threading
from collections import Counter
MCAST_GRP = ['239.0.1.104','239.0.1.105']
MCAST_PORT1 = 12345
MCAST_PORT2 = 12345
def mcanalysis(multicast_group, MCAST_PORT):
IS_ALL_GROUPS = True
#scan time in seconds
SCAN_TIME = 5
#sampling time in seconds
SAMPLING_TIME = 1
bufferUDP = 2048
totalSize = 0
bitrateList = []
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)
sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
if IS_ALL_GROUPS:
# on this port, receives ALL multicast groups
sock.bind(('', MCAST_PORT))
else:
# on this port, listen ONLY to MCAST_GRP
sock.bind((multicast_group, MCAST_PORT))
#Creating socket that gets UDP multicast packets
for group in MCAST_GRP:
mreq = struct.pack("4sl", socket.inet_aton(group), socket.INADDR_ANY)
sock.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, mreq)
print("_____.:| Starting analysis of multicasts! |:._____\n")
print("͞◌͞◌͞◌͞◌͞.:| IP: {} PORT: {} |:.͞◌͞◌͞◌͞͞◌͞◌".format(multicast_group,MCAST_PORT))
SCAN_TIME = int(SCAN_TIME *(SAMPLING_TIME**(-1)))
for x in range(SCAN_TIME):
stop = time.time() + SAMPLING_TIME
while (time.time()<stop):
data, address = sock.recvfrom(bufferUDP)
totalSize += len(data)
bitrateList.append(totalSize)
print(bitrateList[x]*8/(1000000*SAMPLING_TIME))
totalSize = 0
bitrateList.pop(0)
txtfile = open("Bitrate_history_ip_{}.txt".format(multicast_group),"w+")
for x in range(SCAN_TIME-1):
bitrateList[x] = bitrateList[x]*8/(1000000*SAMPLING_TIME)
txtfile.write("{}.Bitrate was equal to: {} Mbps\n".format(x+1,bitrateList[x]))
txtfile.write("Maximum bitrate value was: {} Mbps\n".format(max(bitrateList)))
txtfile.write("Minimum bitrate value was: {} Mbps\n".format(min(bitrateList)))
t1 = threading.Thread(target=mcanalysis, args=(MCAST_GRP[0],MCAST_PORT1))
t2 = threading.Thread(target=mcanalysis, args=(MCAST_GRP[1],MCAST_PORT2))
t1.start()
t2.start()
t1.join()
t2.join()
print('End of test')
time.sleep(5)
I would to thank in advance for any information that can bring me closer to resolving this problem
I am trying to measure the responses back from DNS servers. Making a sniffer for a typical DNS response that is less than 512 bytes is no big deal. My issue is receiving large 3000+ byte responses - in some cases 5000+ bytes. I haven't been able to get a socket working that can receive that data reliably. Is there a way with Python sockets to receive from a specific source address?
Here is what I have so far:
import socket
import struct
def craft_dns(Qdns):
iden = struct.pack('!H', randint(0, 65535))
QR_thru_RD = chr(int('00000001', 2)) # '\x01'
RA_thru_RCode = chr(int('00100000', 2)) # '\x00'
Qcount = '\x00\x01' # question count is 1
ANcount = '\x00\x00'
NScount = '\x00\x00'
ARcount = '\x00\x01' # additional resource count is 1
pad = '\x00' #
Rtype_ANY = '\x00\xff' # Request ANY record
PROtype = '\x00\x01' # Protocol IN || '\x00\xff' # Protocol ANY
DNSsec_do = chr(int('10000000', 2)) # flips DNSsec bit to enable
edns0 = '\x00\x00\x29\x10\x00\x00\x00\x00\x00\x00\x00' # DNSsec disabled
domain = Qdns.split('.')
quest = ''
for x in domain:
quest += struct.pack('!B', len(x)) + x
packet = (iden+QR_thru_RD+RA_thru_RCode+Qcount+ANcount+NScount+ARcount+
quest+pad+Rtype_ANY+PROtype+edns0) # remove pad if asking <root>
return packet
def craft_ip(target, resolv):
ip_ver_len = int('01000101', 2) # IPvers: 4, 0100 | IP_hdr len: 5, 0101 = 69
ipvers = 4
ip_tos = 0
ip_len = 0 # socket will put in the right length
iden = randint(0, 65535)
ip_frag = 0 # off
ttl = 255
ip_proto = socket.IPPROTO_UDP # dns, brah
chksm = 0 # socket will do the checksum
s_addr = socket.inet_aton(target)
d_addr = socket.inet_aton(resolv)
ip_hdr = struct.pack('!BBHHHBBH4s4s', ip_ver_len, ip_tos, ip_len, iden,
ip_frag, ttl, ip_proto, chksm, s_addr, d_addr)
return ip_hdr
def craft_udp(sport, dest_port, packet):
#sport = randint(0, 65535) # not recommended to do a random port generation
udp_len = 8 + len(packet) # calculate length of UDP frame in bytes.
chksm = 0 # socket fills in
udp_hdr = struct.pack('!HHHH', sport, dest_port, udp_len, chksm)
return udp_hdr
def get_len(resolv, domain):
target = "10.0.0.3"
d_port = 53
s_port = 5353
ip_hdr = craft_ip(target, resolv)
dns_payload = craft_dns(domain) # '\x00' for root
udp_hdr = craft_udp(s_port, d_port, dns_payload)
packet = ip_hdr + udp_hdr + dns_payload
buf = bytearray("-" * 60000)
recvSock = socket.socket(socket.PF_PACKET, socket.SOCK_RAW, socket.ntohs(0x0800))
recvSock.settimeout(1)
sendSock = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_RAW)
sendSock.settimeout(1)
sendSock.connect((resolv, d_port))
sendSock.send(packet)
msglen = 0
while True:
try:
pkt = recvSock.recvfrom(65535)
msglen += len(pkt[0])
print repr(pkt[0])
except socket.timeout as e:
break
sendSock.close()
recvSock.close()
return msglen
result = get_len('75.75.75.75', 'isc.org')
print result
For some reason doing
pkt = sendSock.recvfrom(65535)
Recieves nothing at all. Since I'm using SOCK_RAW the above code is less than ideal, but it works - sort of. If the socket is extremely noisy (like on a WLAN), I could end up receiving well beyond the DNS packets, because I have no way to know when to stop receiving packets when receiving a multipacket DNS answer. For a quiet network, like a lab VM, it works.
Is there a better way to use a receiving socket in this case?
Obviously from the code, I'm not that strong with Python sockets.
I have to send with SOCK_RAW because I am constructing the packet in a raw format. If I use SOCK_DGRAM the custom packet will be malformed when sending to a DNS resolver.
The only way I could see is to use the raw sockets receiver (recvSock.recv or recvfrom) and unpack each packet, look if the source and dest address match within what is supplied in get_len(), then look to see if the fragment bit is flipped. Then record the byte length of each packet with len(). I'd rather not do that. It just seems there is a better way.
Ok I was stupid and didn't look at the protocol for the receiving socket. Socket gets kind of flaky when you try to receive packets on a IPPROTO_RAW protocol, so we do need two sockets. By changing to IPPROTO_UDP and then binding it, the socket was able to follow the complete DNS response over multiple requests. I got rid of the try/catch and the while loop, as it was no longer necessary and I'm able to pull the response length with this block:
recvSock = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_UDP)
recvSock.settimeout(.3)
recvSock.bind((target, s_port))
sendSock = socket.socket(socket.AF_INET, socket.SOCK_RAW, socket.IPPROTO_RAW)
#sendSock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sendSock.settimeout(.3)
sendSock.bind((target, s_port))
sendSock.connect((resolv, d_port))
sendSock.send(packet)
pkt = recvSock.recvfrom(65535)
msglen = len(pkt[0])
Now the method will return the exact bytes received from a DNS query. I'll leave this up in case anyone else needs to do something similar :)
I have created a simple RAW socket based packet sniffer. But when I run it, it rarely captures up a packet. First I created this to capture packets in 1 second time intervals, but seeing no packets are captured I commented that line. I was connected to internet and a lot of http traffic are going here and there, but I could not capture a one. Is there a problem in this in the code where I created the socket? Please someone give me a solution. I am fairly new to python programming and could not understand how to solve this.
import socket, binascii, struct
import time
sock = socket.socket(socket.PF_PACKET, socket.SOCK_RAW, socket.htons(0x800))
print "Waiting.."
pkt = sock.recv(2048)
print "received"
def processEth(data):
#some code to process source mac and dest. mac
return [smac, dmac]
def processIP(data):
sip = str(binascii.hexlify(data[1]))
dip = str(binascii.hexlify(data[2]))
return [sip, dip]
def processTCP(data):
sport = str(data[0])
dport = str(data[1])
return [sport, dport]
while len(pkt) > 0 :
if(len(pkt)) > 54:
pkt = sock.recv(2048)
ethHeader = pkt[0][0:14]
ipHeader = pkt[0][14:34]
tcpHeader = pkt[0][34:54]
ethH = struct.unpack("!6s6s2s",ethHeader)
ethdata = processEth(ethH)
ipH = struct.unpack("!12s4s4s",ipHeader)
ipdata = processIP(ipH)
tcpH = struct.unpack("!HH16", tcpHeader)
tcpdata = processTCP(tcpH)
print "S.mac "+ethdata[0]+" D.mac "+ethdata[1]+" from: "+ipdata[0]+":"+tcpdata[0]+" to: "+ipdata[1]+":"+tcpdata[1]
#time.sleep(1);
else:
continue
If you showed all the code, you are running into an endless loop.
Whenever a paket is coming in which has not a length greater then 54 bytes, you end up reading the same packet all the time.
Additionally, socket.recv() returns a string/byte sequence; your approach of accessing the data is wrong. pkt[0] returns a string with length 1; pkt[0][x:y] will not return something useful.
I am not familiar with using sockets, but with some changes I got output that might look similar to what you intended (there is something missing in processEth() I think...).
[...]
while len(pkt) > 0:
print "Waiting.."
pkt = sock.recv(2048)
print "received"
if(len(pkt)) > 54:
ethHeader = pkt[0:14]
ipHeader = pkt[14:34]
tcpHeader = pkt[34:38]
ethH = struct.unpack("!6s6s2s",ethHeader)
ethdata = processEth(ethH)
ipH = struct.unpack("!12s4s4s",ipHeader)
ipdata = processIP(ipH)
tcpH = struct.unpack("!HH16", tcpHeader)
tcpdata = processTCP(tcpH)
print "S.mac "+ethdata[0]+" D.mac "+ethdata[1]+" from: "+ipdata[0]+":"+tcpdata[0]+" to: "+ipdata[1]+":"+tcpdata[1]
#time.sleep(1);
else:
continue
I'd need build up an UDP packet with Python setting a particular value of its TTL. Could anyone show me the minimal-length code to do this?
Using PyIP.
Not tested, but shows the idea:
import ip
import udp
import socket
# build UDP
udp_packet = udp.Packet()
udp_packet.sport = 1024;
udp_packet.dport = 3024;
udp_packet.data = "\xFF\xFF\xFF\xFFrcon \"test\" test\0"
udp_data = udp.assemble(udp_packet, 0)
# build IP packet
ip_packet = ip.Packet()
ip_packet.src = "1.1.1.1"
ip_packet.dst = "2.2.2.2"
ip_packet.ttl = 10
ip_packet.data = udp_data
packet = ip.assemble(ip_packet, 0)
# send the packet here