We Keep Coding

How to decode a string representation of a bytes object?

I have a string which includes encoded bytes inside it:
str1 = "b'Output file \xeb\xac\xb8\xed\x95\xad\xeb\xb6\x84\xec\x84\x9d.xlsx Created'"
I want to decode it, but I can't since it has become a string. Therefore I want to ask whether there is any way I can convert it into
str2 = b'Output file \xeb\xac\xb8\xed\x95\xad\xeb\xb6\x84\xec\x84\x9d.xlsx Created'
Here str2 is a bytes object which I can decode easily using
str2.decode('utf-8')
to get the final result:
'Output file 문항분석.xlsx Created'

You could use ast.literal_eval:
>>> print(str1)
b'Output file \xeb\xac\xb8\xed\x95\xad\xeb\xb6\x84\xec\x84\x9d.xlsx Created'
>>> type(str1)
<class 'str'>
>>> from ast import literal_eval
>>> literal_eval(str1).decode('utf-8')
'Output file 문항분석.xlsx Created'

Based on the SyntaxError mentioned in your comments, you may be having a testing issue when attempting to print due to the fact that stdout is set to ascii in your console (and you may also find that your console does not support some of the characters you may be trying to print). You can try something like the following to set sys.stdout to utf-8 and see what your console will print (just using string slice and encode below to get bytes rather than the ast.literal_eval approach that has already been suggested):
import codecs
import sys
sys.stdout = codecs.getwriter('utf-8')(sys.stdout.buffer)
s = "b'Output file \xeb\xac\xb8\xed\x95\xad\xeb\xb6\x84\xec\x84\x9d.xlsx Created'"
b = s[2:-1].encode().decode('utf-8')

A simple way is to assume that all the characters of the initial strings are in the [0,256) range and map to the same Unicode value, which means that it is a Latin1 encoded string.
The conversion is then trivial:
str1[2:-1].encode('Latin1').decode('utf8')

Finally I have found an answer where i use a function to cast a string to bytes without encoding.Given string
str1 = "b'Output file \xeb\xac\xb8\xed\x95\xad\xeb\xb6\x84\xec\x84\x9d.xlsx Created'"
now i take only actual encoded text inside of it
str1[2:-1]
and pass this to the function which convert the string to bytes without encoding its values
import struct
def rawbytes(s):
"""Convert a string to raw bytes without encoding"""
outlist = []
for cp in s:
num = ord(cp)
if num < 255:
outlist.append(struct.pack('B', num))
elif num < 65535:
outlist.append(struct.pack('>H', num))
else:
b = (num & 0xFF0000) >> 16
H = num & 0xFFFF
outlist.append(struct.pack('>bH', b, H))
return b''.join(outlist)
So, calling the function would convert it to bytes which then is decoded
rawbytes(str1[2:-1]).decode('utf-8')
will give the correct output
'Output file 문항분석.xlsx Created'

Is the output of scapy.sprintf a raw string?why the length is wrong?

I want to analyze TCP packets by scapy. And I use pkt.sprintf('%Raw.load%') to extract tcp data. But the output string has something wrong with length. but the '\' is deemed to be a str instead of a Escaped character.so '\x11' is considered as 4 different strings instead of a ASCII character.
Here are my codes:
from scapy.all import *
def findTCPdata(pkt):
raw = pkt.sprintf("%Raw.load%")
print raw
print 'length of TCP data: '+ str(len(raw))
def main():
pkts = rdpcap('XXX.pcap')
for pkt in pkts:
findTCPdata(pkt)
if __name__ == '__main__':
main()
enter image description here
The length of each tcp data should be 17 instead of the value in screen(53,52,46,52).
4 tcp data are:
'U\x11\x04\x92\x02\x03\x1e\x03#\x03\xf8q=e\xcb\x15\r'
'U\x11\x04\x92\x02\x03.\x03#\x03\xf8q=e\xcb\xb8\x05'
'U\x11\x04\x92\x02\x03X\x03#\x03\xf8q=e\xcbiO'
'U\x11\x04\x92\x02\x03n\x03#\x03\xf8q=e\xcb\xdb\xe3'
Please help me solve the problem.Thank you!

i see. i need a function to transform rawstring to string.
so i add codes after line 3(raw = pkt.sprintf("%Raw.load%")) like:
raw = raw.replace('\'','')
string = raw.decode('string_escape')
then the output is right

Backporting bytearray to Python 2.5

I'm trying to convert this websocket example for use in Python 2.5, but am running into errors with the use of the bytearray type.
The code stops working for Python 2.5 here (in the send_text method of websocket_server/websocket_server.py):
FIN = 0x80
OPCODE = 0x0f
def send_text(self, message):
header = bytearray();
payload = encode_to_UTF8(message)
payload_length = len(payload)
header.append(FIN | OPCODE_TEXT)
header.append(payload_length)
self.request.send(header + payload)
The message variable stores the string input that is sent to clients.
It attempts to create an array of bytes and send that using the self.request.send method. How would I change this to make it work in Python 2.5 which doesn't have the bytes type or bytearray?

Using struct MIGHT work, I haven't tested this.
What I would do, as a workaround, would be to use struct.pack to pack byte by byte.
mensaje = "saludo"
FIN = 0x80
OPCODE = 0x0f
payload = ''
for c in mensaje:
payload += struct.pack("H", ord(c))
msj = struct.pack("H",FIN | OPCODE )
msj+= struct.pack("H",len(payload))
print msj + payload
I'm using "H" as the 'fmt' parameter in the struct.pack function, but you better check how is your package sent and how many bytes per 'character' (since I'm guessing you're using unicode, I'm using 'H', unsigned short = 2 bytes).
More info: https://docs.python.org/2/library/struct.html, section 7.3.2.1 and 7.3.2.2.
EDIT:
I'll answer here, what do I mean by using 'chr()' instead of 'struct.pack()':
mensaje = "saludo"
FIN = 0x80
OPCODE = 0x0f
payload = mensaje
msj = chr( FIN | OPCODE )
msj+= chr(len(payload))
print msj + payload
if you print the message, then you should see the same output when using struct.pack("B", ord(something)) than when using ord(something), I just used struct.pack() because I thought your message was two bytes per char (as unicode).

Python get packet data TCP

Hello I have python sniffer
def receiveData(s):
data = ''
try:
data = s.recvfrom(65565)
#k = data.rstrip()
except timeout:
data = ''
except:
print "An Error Occurred."
sys.exc_info()
return data[0]
data = receiveData(s)
s is socket. Im getting data but it contains symbols please help me somebody how i can convert it into plain text
Im newbie in Python and if its very silly question sorry : )
this is data example E\x00\x00)\x1a*#\x00\x80\x06L\xfd\xc0\xa8\x01\x84\xad\xc2#\xb9M\xb8\x00P\xed\xb3\x19\xd9\xedY\xc1\xfbP\x10\x01\x04\x16=\x00\x00\x00'

You can't really convert it to "plain text". Characters such as the NUL (ASCII 0, shown as \x00) can't be displayed so python shows them in their hex representation.
What most sniffing/hexdump tools do is to replace unprintable characters with e.g. a dot. You could do it like this:
import string
printable = set(string.printable)
print ''.join(x if x in printable else '.' for x in data)
Example:
>>> data = 'E\x00\x00)\x1a*#\x00\x80\x06L\xfd\xc0\xa8\x01\x84\xad\xc2#\xb9M\xb8\x00P\xed\xb3\x19\xd9\xedY\xc1\xfbP\x10\x01\x04\x16=\x00\x00\x00'
>>> print ''.join(x if x in printable else '.' for x in data)
E..).*#...L.......#.M..P.....Y..P....=...

The conversion to "plain text" depends on what your data mean.
Do you have compressed text? Then uncompress it.
Do you have encoded numbers? Then decode it and display the numbers.
Without knowing the semantic of the data, no one can tell you.
Of course, you can just display the raw data with print data.encode("hex"), but I am not sure if that is what you want.

Str to hex in Python

I receive on my socket a 4 bytes value that I want to print as hex value. I am trying:
print "%08x" % (nonce)
However, I get an error message that string can't be converted to hex. Anyone an idea
how this could be quickly resolved?

Use the struct module to unpack the octets received from the network into an actual number. The %08x format will work on the number:
import struct
n, = struct.unpack('>I', nonce)
print "%08x" % n

You most likely have a string containing the bytes. However, to print them as a number you need well.. a number.
You can easily create the hex string you are looking for like this:
''.join('%02x' % ord(x) for x in nonce)
Demo:
>>> nonce = os.urandom(4)
>>> nonce
'X\x19e\x07'
>>> ''.join('%02x' % ord(x) for x in nonce)
'58196507'

Another option is:
from binascii import hexlify
>>> hexlify('a1n4')
'61316e34'

num = "9999"
print hex(int(num))
#0x270f

If your data can be converted to a string, you could use the str.encode() method:
>>> s = "XYZ"
>>> s.encode('hex')
'58595a'