I have a script that connects to a mine craft server, receives packets, and sends packets.
So, I send a 'login' packet, and I receive a 'login' packet. Unfortunately, the received login packet is encoded (Information about encoding here: http://wiki.vg/Protocol#0x01).
The received login packet is stored in a variable named received_login_packet. I need to decode it so that I can get the separate bits of data, such as the packet type, the dimension, etc...
I've looked around a bit, but I have absolutely no idea as to how to go about doing this.
Here's some code if it helps:
#encoding the packet to send
encuserlen = str(len(enc_user)) # enc_user is just my username
packfmt = '>bih%sshiibBB' % encuserlen
packetbytes = struct.pack(packfmt, 1, 28, len(data['user']), enc_user, 0, 0, 0, 0, 0, 0)
s.send(packetbytes)
time.sleep(2)
#login packet sent, waited for response
response = s.recv(1024) #this is the raw login response.
#it's encoded as detailed above. i want to decode it
Any help would be appreciated and please don't hesitate to say if it's not clear enough.
So, if I understand this right, you want to decode the packet response, which is a 1024 byte, to get the correct information out. It seems you were able to use the struct.pack statement, there is a similar statement to unpack, as seen in the documentation. Basically, it looks like this.
packfmt = '>issiibBB'
output=struct.unpack(packfmt,response)
Also, I'm not quite convinced that your request was being sent right, but I'll leave that as an exercise for you to figure out how to set it. See format strings.
Related
I'm writing a bittorrent client in python, and have been using a loop to continually read messages from the peer sockets using recv().
When I run my program I look in wireshark to see what bittorrent messages I'm getting. It's pretty easy to tell what kind of message you got from the first 5 bytes of the message, since the length and message ID are specified there.
I'm running into some problems when dealing with receiving data containing multiple messages.
I've tried tackling it by writing a method like this:
def handleMultiple(self, message, peer):
total_length = len(message)
parsed = 0
while parsed < total_length:
m_len, m_id = struct.unpack(">IB", message[parsed:parsed + 5])
m_total = m_len + 4
print(m_len, total_length, parsed, m_id, peer.made_handshake, peer.ip)
self.handleMessage(message[parsed:m_total + parsed], peer)
parsed += m_total
The function just breaks down the received bytes into its constituent messages and hands it off to the message handler that knows how to deal with individual messages.
The problem is that when I printed out the length prefix and message ID from a message I received using recv(), sometimes it looks like just garbage numbers.
This is really my first time experimenting with sockets, so I lack the intuition to know what I'm really getting when calling recv(). Should I just call receive on the first 5 bytes of data I get, then do some checking to make sure that the length and ID are valid, then call recv() on the rest of the message?
How should I go about handling multiple messages incoming at a time?
Edit:
I wanted to provide some images of the results I'm seeing to see if anyone can help identify the issue I'm having.
Here's a picture of the bittorrent messages I'm receiving:
Here's a corresponding logging output:
The columns are supposed to be message length + 4, total message length, message id, and the IP from the sender:
As I can see, the length prefix for the first messages, (the ones that are multiple messages sent to me at a time) are completely too large. The fifth message I got from 95.211.212.26 is a well formed bitfield message.
Another thing I noticed is that the supposed message ID from each of the multi-message messages is 255. Also given that the total length of a bitfield message for this given torrent is 126, the total lengths (303, 328, 325) are not inconceivable for messages of a bitfield followed by several have messages.
Alright so I've managed to figure out where I was going wrong. I was reading from the socket assuming that my message would be there in full. In reality, I was reading the initial snippet of the message, and at a later time I was reading the middle of the message. The 255 values I was seeing weren't message IDs but actually the middle of the peer's bitfield (0xff).
I changed my approach to store the read in bytes from the socket to the peer's message buffer. Once the message buffer was at least as long as the expected payload, I read the message and trimmed the buffer to exclude what I just read. Now all of my messages' IDs are looking as I expect.
I'm forwarding a machine's packets through mine. I tested with tcpdump host <machine_ip> to see if everything is alright, and it is. Now I need to capture these packets. I chose to do this with Python. Right now I'm trying to filter the packets, but the ip that is printed is different from the right one. It should've been 192.168.0.8:
import socket
from struct import *
print("Started")
with socket.socket(socket.AF_PACKET,socket.SOCK_RAW, socket.ntohs(0x0003)) as s:
while True:
packet=s.recvfrom(65565)
content=packet[0]
ip_header=unpack('!BBHHHBBH4s4s', content[:20])
source_ip=socket.inet_ntoa(ip_header[8])
print(source_ip)
The printed ones are 8.0.69.0 and 8.0.69.16, which none of these matches the expected form.
This is because in front of the raw data is the MAC header.
If You change the line:
ip_header=unpack('!BBHHHBBH4s4s', content[:20])
to
ip_header=unpack('!BBHHHBBH4s4s', content[14:34])
You will probably get your ip address. I said probably becase it really depends on the link layer, as there might be a vlan tag present, thus shifting the ip header even further.
I need to:
1) Create a socket to a specific IP and Port
2) Send a string of data via the connection
3) Await the response and check it is valid
I have seen a lot of conflicting advice at the moment and so decided to ask a new question to try and clear this up.
I'm trying to use the socket library in Python to achieve this however am facing a couple of issues. I've tried a few different methods, however I am facing an issue sending the string of data. My data is in XML format so getting this as a string is proving difficult for me. I have attempted converting it to binary however would prefer it in plain text format, just that it should be as a string. Any pointers as to whether the code would fulfil the 3 steps I wish to complete would be perfect!
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('109.73.xxx.xxx', 29006))
s.sendall('*XML SHOULD GO HERE*')
data = s.recv(1024)
s.close()
print 'Received', repr(data)
I haven't got a response back from the server at the moment because I suspect of the erroneous format of the string of data sent. This means I haven't been able to check whether the code prints the response data from the server.
The XML data I need to pass as a string is in the following format:
<Message xmlns:xsi="http://www.w3.org/CANNOTPROVIDETHISURL" xmlns:xsd="http://www.w3.org/CANNOTPROVIDETHISURL">
<ClientHeader>
<element1>RC1234</element1>
<element2>1234</element2>
<element3>12345678</element3>
<element4>123456789</element4>
<element5>-1</element5>
<element6>-1</element6>
<element7>A123</element7>
<element8>TMS</element8>
</ClientHeader>
<MsgType>TYPE</MsgType>
<MsgData></MsgData>
</Message>
Hello kind folks of StackOverflow.
I am trying to make a sort of 'bot' which can connect to a Minecraft Classic server, post messages and possibly build.
Anyway, I'm having some trouble understanding how to send packets in python, and how to correctly encode them.
Here are the packets I need to send, I want to send the 'Player Identification' one: http://www.minecraftwiki.net/wiki/Classic_server_protocol#Client_.E2.86.92_Server_packets
I know I need to be using sockets, and I need to be using struct.pack, but how exactly can I send it?
An example piece code that sends a login packet would be marvellous.
Thanks.
I'll get the ball rolling:
import socket
import struct
username = "username_value"
verification_key = "verification_key"
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # boilerplate
s.connect(("example.com", 1234)) # adjust accordingly
# now for the packet
# note that the String type is specified as having a length of 64, we'll pad that
packet = ""
packet += struct.pack("B", 1) # packet type
packet += struct.pack("B", 7) # protocol version
packet += "%-64s" % username # magic!
packet += "%-64s" % verification_key
packet += struct.pack("B", 0) # that unused byte, assuming a NULL byte here
# send what we've crafted
s.send(packet)
The "%-20s" may be weird for you if you've never used format strings. Essentially..
print "%s" % 5
.. would print out 5 ..
print "%10s" % 5
.. would pad the output to be exactly 10 characters in width. But it pads them on the right side, we want that padding on the left -- hence the - ..
print "%-10s" % s, "<this will be 9 spaces away from the '5'>"
.. play around with it.
If anything is unclear, let me know. I like what you're doing, reminds me of an old project of mine. Except that I didn't have a neat protocol specification like you did, lucky bastard. ;)
I've got a simple TCP server and client. The client receives data:
received = sock.recv(1024)
It seems trivial, but I can't figure out how to recieve data larger than the buffer. I tried chunking my data and sending it multiple times from the server (worked for UDP), but it just told me that my pipe was broken.
Suggestions?
If you have no idea how much data is going to pour over the socket, and you simply want to read everything until the socket closes, then you need to put socket.recv() in a loop:
# Assumes a blocking socket.
while True:
data = sock.recv(4096)
if not data:
break
# Do something with `data` here.
Mike's answer is the one you're looking for, but that's not a situation you want to find yourself in. You should develop an over-the-wire protocol that uses a fixed-length field that describes how much data is going to be sent. It's a Type-Length-Value protocol, which you'll find again and again and again in network protocols. It future-proofs your protocol against unforeseen requirements and helps isolate network transmission problems from programmatic ones.
The sending side becomes something like:
socket.write(struct.pack("B", type) #send a one-byte msg type
socket.write(struct.pack("H", len(data)) #send a two-byte size field
socket.write(data)
And the receiving side something like:
type = socket.read(1) # get the type of msg
dataToRead = struct.unpack("H", socket.read(2))[0] # get the len of the msg
data = socket.read(dataToRead) # read the msg
if TYPE_FOO == type:
handleFoo(data)
elif TYPE_BAR == type:
handleBar(data)
else:
raise UnknownTypeException(type)
You end up with an over-the-wire message format that looks like:
struct {
unsigned char type;
unsigned short length;
void *data;
}
Keep in mind that:
Your operating system has it's own idea of what it's TCP/IP socket buffer size is.
TCP/IP packet maximum size (generally is 1500 bytes)
pydoc for socket suggests that 4096 is a good buffer size
With that said, it'd really be helpful to see the code around that one line. There are a few things that could play into this, if you're using select or just polling, is the socket non-blocking, etc.
It also matters how you're sending the data, if your remote end disconnects. More details.