I am trying to extract a specific byte from each packet in a pcap file. All packets are ICMP.
In the data section, there is a byte that changes each packet. It is in the same position for each one. I would like to extract that byte.
Using scapy:
pkts = rdpcap('test.pcap')
pl = PacketList([p for p in pkts])
bytes(pl[12].payload)
returns the following:
b'E\x00\x00T]\xa7\x00\x00***J***\x01!A\xc0\xa88\x01\xc0\xa88o\x08\x004\xe9\xbf2\x00\x00^"\x87\xbe\x00\x0c2\xf4\x08\t\n\x0b\x0c\r\x0e\x0f\x10\x11\x12\x13\x14\x15\x16\x17\x18\x19\x1a\x1b\x1c\x1d\x1e\x1f !"#$%&\'()*+,-./01234567'
I have enclosed the byte that I want to extract within three stars. However, if I print out the bytes for each packet, the byte that I want to extract will be in a different offset.
If I run a hexdump for each packet as follows:
hexdump(bytes(pl[12].payload))
The specific byte I want to extract is always in the same position, but I don't know how to extract it.
How can I extract a specific byte from a pcap using scapy?
Following this answer here: Get specific bytes in payload from a pcap file
If I execute the same command, it doesn't do anything useful:
>>> hexdump(pkts[14][2].load[8])
0000 00 00 00 00 00 00 00 00 ........
>>>
You want the TTL?
Lets start at the high level, and move down.
Scapy is giving you the constructed packet. If you want the TTL of the packet, call the attribute:
>>> plist[182].ttl
64
If you want to get the specific byte of the packet, lets look at the hexdump:
>>> hexdump(plist[182])
0000 AA BB CC 66 42 DE AA BB CC 3F 52 A3 08 00 45 00 .a.lM..M.AK...E.
0010 00 5B 58 B9 40 00 40 06 64 96 C0 A8 01 28 AC D9 .[X.#.#.d....(..
...
These are in hex, the first field 0000 is the offset, then 16 bytes in hex, then ascii.
Offset Bytes ASCII
====== =============================================== ================
0000 AA BB CC 66 42 DE AA BB CC 3F 52 A3 08 00 45 00 .a.lM..M.AK...E.
Things start at 0, so the the byte addresses are 0..15 for the first line.
The second line the offset is 16 (16 * 1). So the byte addresses are 16..31.
The third line, the offset is 32 (16 * 2). So the byte addresses are 32..47
You highlighted the 7th byte on row 2:
Offset Bytes ASCII
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
====== =============================================== ================
0010 00 5B 58 B9 40 00 40 06 64 96 C0 A8 01 28 AC D9 .[X.#.#.d....(..
That address is:
offset + byte_address.
offset = 16 * 1
byte_address = 6
Which gives us:
16 + 6 = 22
With that, we can now get byte address 22 out of the raw packet:
>>> b = raw(plist[182])
>>> b[22]
64
Note that wireshark packet numbers start at 1. The packets in python are going to start at 0. So in my example, packet 182 corresponded to packet 183 in Wireshark.
plist[182].payload gives you the IP portion of the packet, so the offsets are going to be different since we aren't looking at the whole packet anymore. We can get the same value using the '.ttl' attribute. Or, knowing that the address is byte 8 in the IP header:
>>> plist[182].payload.ttl
64
>>> raw(plist[182].payload)[8]
64
Related
Here is my python code:
import socket
data = bytes.fromhex("47 A2 62 19 20 00 00 00 00")
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, 0)
ADDRESS = ("192.168.0.1",9000)
s.connect(ADDRESS)
s.send(data)
I want to put 4 random hexadecimal bytes after "47 A2 62 19 20 00 00 00 00". For example "47 A2 62 19 20 00 00 00 00 20 1E 4A 72".
I wonder what to do. Or is there a better way?
random.randbytes can generate random bytes for you. Note that a hexdecimal pair is two bytes, not one byte, so you'll need eight random bytes, not four:
import random
data = bytes.fromhex("47 A2 62 19 20 00 00 00 00") + random.randbytes(8)
So I'm trying to create a python script to generate a level for a game made in MMF2+Lua, and I've run into something I can't figure out how to fix.
Generating a 16x16 empty level with borders with the game gives this (deflated):
78 5E 63 20 0A FC 27 00 40 86 8C AA C1 1D 02 23 3D 7C 08 27 32 00 9F 62 FE 10
which should be a flattened 18x18 array with the edge having 0x00, and the rest having 0xFF.
My python script generates this with the exact same input to zlib.deflate:
78 9C 63 60 20 06 FC 27 00 46 D5 8C AA C1 A7 86 30 00 00 9F 62 FE 10
They're different, but inflating them gives the same exact data. However, when I put the data into the game, it crashes when trying to load the level.
What's really different between the two values, and am I able to fix it?
Those are two different encodings of the same data, both valid. They differ in the sequence of copies. Here are readable forms of both, first from the game:
! infgen 2.6 output
!
zlib
!
last
fixed
literal 0
match 37 1
literal 255
match 31 1
match 4 69
match 258 36
match 26 258
match 256 288
match 34 613
end
!
adler
then from zlib:
! infgen 2.6 output
!
zlib
!
last
fixed
literal 0 0
match 36 1
literal 255
match 31 1
match 258 36
match 258 36
match 28 36
match 34 1
end
!
adler
literal gives a byte or bytes inserted in the stream. match is a copy of previous bytes in the stream (possibly overlapped with bytes being copied), where the first parameter is the number of bytes to copy, and the second parameter is the distance back in bytes to copy from.
Good day,
I am not sure where I should place this question , I am learning about DNS and how it works and as I understand I send a request out on UDP port 53 to a server and the host should respond to me on that port correct?
Here is a script that I am working with and it works and accurately describes the DNS message query and usage and even gets a DNS answer back for me.
How is this possible if it cannot listen on port 53 with out having root on a system?
DNS PACKET DETAILS
;DNS HEADER;
; AA AA - ID
; 01 00 - Query parameters
; 00 01 - Number of questions
; 00 00 - Number of answers
; 00 00 - Number of authority records
; 00 00 - Number of additional records
; DNS QUESTION --
; 07 - 'example' has length 7, ;so change this to be the length of domain ; keep in ming there are not '.' in the question.
; 65 - e
; 78 - x
; 61 - a
; 6D - m
; 70 - p
; 6C - l
; 65 - e
; 03 - subdomain '.com' length 03 ; change this to be the length of type.
; 63 - c
; 6F - o
; 6D - m
CODE :
import binascii
import socket
def send_udp_message(message, address, port):
"""send_udp_message sends a message to UDP server
message should be a hexadecimal encoded string
"""
message = message.replace(" ", "").replace("\n", "")
server_address = (address, port)
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
try:
sock.sendto(binascii.unhexlify(message), server_address)
data, _ = sock.recvfrom(4096)
finally:
sock.close()
return binascii.hexlify(data).decode("utf-8")
def format_hex(hex):
"""format_hex returns a pretty version of a hex string"""
octets = [hex[i:i+2] for i in range(0, len(hex), 2)]
pairs = [" ".join(octets[i:i+2]) for i in range(0, len(octets), 2)]
return "\n".join(pairs)
message = "AA AA 01 00 00 01 00 00 00 00 00 00 " \
"07 65 78 61 6d 70 6c 65 03 63 6f 6d 00 00 01 00 01"
response = send_udp_message(message, "8.8.8.8", 53)
print(format_hex(response))
RESPONSE:
aa aa
81 80
00 01
00 01
00 00
00 00
07 65
78 61
6d 70
6c 65
03 63
6f 6d
00 00
01 00
01 c0
0c 00
01 00
01 00
00 32
98 00
04 5d
b8 d8
22
If you look at the last four bytes you'll see that this is the IP for example.com in hex 5db8d822
You can go here to check it out.
HEX to IP converter Online
No, your source port is not port 53. User processes are allocated outbound port numbers above 1023, which are unprivileged.
A simple synchronous Python DNS client will basically block and hold the same port open until the server responds. The IP packet you send contains the information that the server needs in order to know where to reply (this is in the headers of the IP packet itself, before the DNS query payload).
The following Fortran code:
INTEGER*2 :: i, Array_A(32)
Array_A(:) = (/ (i, i=0, 31) /)
OPEN (unit=11, file = 'binary2.dat', form='unformatted', access='stream')
Do i=1,32
WRITE(11) Array_A(i)
End Do
CLOSE (11)
Produces streaming binary output with numbers from 0 to 31 in integer 16bit. Each record is taking up 2 bytes, so they are written at byte 1, 3, 5, 7 and so on. The access='stream' suppresses the standard header of Fortran for each record (I need to do that to keep the files as tiny as possible).
Looking at it with a Hex-Editor, I get:
00 00 01 00 02 00 03 00 04 00 05 00 06 00 07 00
08 00 09 00 0A 00 0B 00 0C 00 0D 00 0E 00 0F 00
10 00 11 00 12 00 13 00 14 00 15 00 16 00 17 00
18 00 19 00 1A 00 1B 00 1C 00 1D 00 1E 00 1F 00
which is completely fine (despite the fact that the second byte is never used, because decimals are too low in my example).
Now I need to import these binary files into Python 2.7, but I can't. I tried many different routines, but I always fail in doing so.
1. attempt: "np.fromfile"
with open("binary2.dat", 'r') as f:
content = np.fromfile(f, dtype=np.int16)
returns
[ 0 1 2 3 4 5 6 7 8 9 10 11
12 13 14 15 16 17 18 19 20 21 22 23
24 25 0 0 26104 1242 0 0]
2. attempt: "struct"
import struct
with open("binary2.dat", 'r') as f:
content = f.readlines()
struct.unpack('h' * 32, content)
delivers
struct.error: unpack requires a string argument of length 64
because
print content
['\x00\x00\x01\x00\x02\x00\x03\x00\x04\x00\x05\x00\x06\x00\x07\x00\x08\x00\t\x00\n', '\x00\x0b\x00\x0c\x00\r\x00\x0e\x00\x0f\x00\x10\x00\x11\x00\x12\x00\x13\x00\x14\x00\x15\x00\x16\x00\x17\x00\x18\x00\x19\x00']
(note the delimiter, the t and the n which shouldn't be there according to what Fortran's "streaming" access does)
3. attempt: "FortranFile"
f = FortranFile("D:/Fortran/Sandbox/binary2.dat", 'r')
print(f.read_ints(dtype=np.int16))
With the error:
TypeError: only length-1 arrays can be converted to Python scalars
(remember how it detected a delimiter in the middle of the file, but it would also crash for shorter files without line break (e.g. decimals from 0 to 8))
Some additional thoughts:
Python seems to have troubles with reading parts of the binary file. For np.fromfile it reads Hex 19 (dec: 25), but crashes for Hex 1A (dec: 26). It seems to be confused with the letters, although 0A, 0B ... work just fine.
For attempt 2 the content-result is weird. Decimals 0 to 8 work fine, but then there is this strange \t\x00\n thing. What is it with hex 09 then?
I've been spending hours trying to find the logic, but I'm stuck and really need some help. Any ideas?
The problem is in open file mode. Default it is 'text'. Change this mode to binary:
with open("binary2.dat", 'rb') as f:
content = np.fromfile(f, dtype=np.int16)
and all the numbers will be readed successfull. See Dive in to Python chapter Binary Files for more details.
I am trying to write a simple program in python to use telegram api, (not bot api, main messaging api) Now i have written this code
#!/usr/bin/env python
import socket
import random
import time
import struct
import requests
def swap64(i):
return struct.unpack("<L", struct.pack(">L", i))[0]
MESSAGE = '0000000000000000'+format(swap32(int(time.time()*1000%1000)<<21|random.randint(0,1048575)<<3|4),'x')+format(swap32(int(time.time())),'x')+'140000007897466068edeaecd1372139bbb0394b6fd775d3'
res = requests.post(url='http://149.154.167.40',
data=bytes.fromhex(MESSAGE),
headers={'connection': 'keep-alive'})
print("received data:", res)
For payload of post data i used the source code of telegram web, The 0 auth id, message id is generated using the algo in telegram web, next is length (14000000) just like in the source and main doc and then the method and so on,
When i run this code i get received data: <Response [404]> i have used both tcp and http transport with this and tcp one gives me nothing as answer from server, i don't know where i'm wrong in my code
i would be glad if someone can show the error in my code
btw here is hex dump of my generated req:
0000 34 08 04 17 7a ec 48 5d 60 84 ba ed 08 00 45 00
0010 00 50 c6 07 40 00 40 06 76 28 c0 a8 01 0d 95 9a
0020 a7 28 c9 62 00 50 0d 1a 3b df 41 5a 40 7f 50 18
0030 72 10 ca 39 00 00 00 00 00 00 00 00 00 00 6c 28
0040 22 4a 94 a9 c9 56 14 00 00 00 78 97 46 60 68 ed
0050 ea ec d1 37 21 39 bb b0 39 4b 6f d7 75 d3
i have already read this and this and many other docs but cant find out my problem
thanks in advance
update
i used this code as suggested
TCP_IP = '149.154.167.40'
TCP_PORT = 80
MESSAGE = 'ef0000000000000000'+"{0:0{1}x}".format(int(time.time()*4294.967296*1000),16)+'140000007897466068edeaecd1372139bbb0394b6fd775d3'
BUFFER_SIZE = 1024
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((TCP_IP, TCP_PORT))
s.send(bytes.fromhex(MESSAGE))
data = s.recv(BUFFER_SIZE)
s.close()
and i still get no response
hex dump of my request:
0000 34 08 04 17 7a ec 48 5d 60 84 ba ed 08 00 45 00
0010 00 51 e1 44 40 00 40 06 5a ea c0 a8 01 0d 95 9a
0020 a7 28 df 8c 00 50 e4 0d 12 46 e2 98 bf a3 50 18
0030 72 10 af 66 00 00 ef 00 00 00 00 00 00 00 00 00
0040 16 37 dc e1 28 39 23 14 00 00 00 78 97 46 60 68
0050 ed ea ec d1 37 21 39 bb b0 39 4b 6f d7 75 d3
Fixed code
Finally got it working with this code
import socket
import random
import time
import struct
import requests
def swap32(i):
return struct.unpack("<L", struct.pack(">L", i))[0]
TCP_IP = '149.154.167.40'
TCP_PORT = 80
z = int(time.time()*4294.967296*1000000)
z = format(z,'x')
q = bytearray.fromhex(z)
e = q[::-1].hex()
MESSAGE = 'ef0a0000000000000000'+e+'140000007897466068edeaecd1372139bbb0394b6fd775d3'
BUFFER_SIZE = 1024
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((TCP_IP, TCP_PORT))
s.send(bytes.fromhex(MESSAGE))
data = s.recv(BUFFER_SIZE)
s.close()
print(data)
here is sample data from a simple TCP handshake with Telegram Servers:
Connect:Success:0
Connected to 149.154.167.40:443
raw_data: 000000000000000000F011DB3B2AA9561400000078974660A9729A4F5B51F18F7943F9C0D61B1315
auth_key_id: 0000000000000000 0
message_id: 56A92A3BDB11F000 6244568794892726272
data_length: 00000014 20
message_data: 78974660A9729A4F5B51F18F7943F9C0D61B1315
message_type: 60469778
>> EF0A000000000000000000F011DB3B2AA9561400000078974660A9729A4F5B51F18F7943F9C0D61B1315
Send:Success:42
Receive:Success:85
<< 15000000000000000001CC0CC93D2AA9564000000063241605A9729A4F5B51F18F7943F9C0D61B1315B4445B94718B3C6DD4136466FAC62DCD082311272BE9FF8F9700000015C4B51C01000000216BE86C022BB4C3
raw_data: 000000000000000001CC0CC93D2AA9564000000063241605A9729A4F5B51F18F7943F9C0D61B1315B4445B94718B3C6DD4136466FAC62DCD082311272BE9FF8F9700000015C4B51C01000000216BE86C022BB4C3
auth_key_id: 0000000000000000 0
message_id: 56A92A3DC90CCC01 6244568803180334081
data_length: 00000040 64
message_data: 63241605A9729A4F5B51F18F7943F9C0D61B1315B4445B94718B3C6DD4136466FAC62DCD082311272BE9FF8F9700000015C4B51C01000000216BE86C022BB4C3
message_type: 05162463
classid: resPQ#05162463
nonce: A9729A4F5B51F18F7943F9C0D61B1315
server_nonce: B4445B94718B3C6DD4136466FAC62DCD
pq: 2311272BE9FF8F97 2526843935494475671
count: 00000001 1
fingerprints: C3B42B026CE86B21 14101943622620965665
Lets break it down:
We are using the TCP abridged version, so we start off with 0xEF
The format for plain-text Telegram messages is auth_ke_id + msg_id + msg_len + msg
auth_key_id is always 0 for plain-text messages hence we always start with 0000000000000000
msg_id must approximately equal unixtime*2^32(see here) I have also seen that some variant of this works quite well for msg_id in any language on any platform: whole_part_of(current_micro_second_time_stamp * 4294.967296)
The first message you start with for Auth_key generation is reqPQ which is defined as: reqPQ#0x60469778 {:nonce, :int128} so it is simply a TL-header + a 128-bit random integer the total length will always be 4 + 16 = 20 encoded as little-endian that would be msg_len = 14000000
say we have a 128-bit random integer= 55555555555555555555555555555555, then our reqPQ message would be 7897466055555555555555555555555555555555, which is simply TL-type 60469778 or 78974660 in little-endian followed by your randomly chooses 128-bit nonce.
Before you send out the packet, again recall that TCP-abridged mode required you to include the total packet length in front of the other bytes just after the initial 0xEA . This packet length is computed as follows
let len = total_length / 4
a) If len < 127 then len_header = len as byte
b) If len >=127 then len_header = 0x7f + to_3_byte_little_endian(len)
finally we have:
EF0A000000000000000000F011DB3B2AA956140000007897466055555555555555555555555555555555
or
EF0A
0000000000000000
00F011DB3B2AA956
14000000
78974660
55555555555555555555555555555555
compared to yours:
0000000000000000
6C28224A94A9C956
14000000
78974660
68EDEAECD1372139BBB0394B6FD775D3
I would say, try using TCP-abriged mode by include the 0xEF starting bit and re-check your msg_id computation
cheers.