I have done the following.
from struct import pack, unpack
t = 1234
tt = str(pack("<I", t))
printing tt gives \xf3\xe0\x01\x00. How do I get original value of t back from tt?
I tried using unpacking the repr(tt) but that does not work out. How do I go about doing this?
>>> t=1234
>>> tt=pack('<I', t)
>>> tt
'\xd2\x04\x00\x00'
>>> unpack('<I', tt)
(1234,)
>>> ttt, = unpack('<I', tt)
>>> ttt
1234
you are using the wrong package for serialization. the struct package is only useful for python code which interacts with C code.
for serialization into a string, you should use the pickle module.
import pickle
t = 1234
tt = pickle.dumps(t)
t = pickle.loads(tt)
unpack('<I', tt) will give you (1234,).
repr doesn't work since it adds quotes to the string:
>>> repr('foo')
'"foo"'
Related
In my code, I get a path from the database that may contain special escaping characters that I need to convert them to a real path name. I'm using python 3.7 on Windows.
Suppose this path: C:\Files\2c2b2541\00025\test.x
IMPORTANT: the path is not a fixed value in the code and it is an output of executing a Stored Procedure from pyodbc.
When I try to convert it to an absolute path I get this error:
ValueError: _getfullpathname: embedded null character in path
I also tried to replace "\" with "/" but with no luck.
import os
# path = cursor.execute(query, "some_input").fetchone()[0]
path = 'C:\Files\2c2b2541\00025\test.x'
print(os.path.abspath(path))
Judging by your comments on the other answers, it sounds like the data is already corrupted in the database you're using. That is, you have a literal null byte stored there, and perhaps other bogus bytes (like \2 perhaps turning into \x02). So you probably need two fixes.
First, you should fix whatever code is putting values into the database, so it won't put bogus data in any more. You haven't described how the data gets into the database, so I we can't give you much guidance on how to do this. But most programming languages (and DB libraries) have tools to prevent escape sequences from being evaluated in strings where they're not wanted.
Once you've stopped new bad data from getting added, you can work on fixing the values that are already in the database. It probably shouldn't be too hard to write a query that will replace \0 null bytes with \\0 (or whatever the appropriate escape sequence is for your DB). You may want to look for special characters like newlines (\n) and unprintable characters (like \x02) as well.
I'd only try to fix this issue on the output end if you don't have any control of the database at all.
I think the below is the right way to solve your problem.
>>> def get_fixed_path(path):
... path = repr(path)
... path = path.replace("\\", "\\\\")
... path = path.replace("\\x", "\\\\0")
... path = os.path.abspath(path3).split("'")[1]
... return path
...
>>>
>>> path = 'C:\Files\2c2b2541\00025\test.x'
>>> path
'C:\\Files\x02c2b2541\x0025\test.x'
>>>
>>> print(path)
C:\Filesc2b2541 25 est.x
>>>
>>> final_path = get_fixed_path(path)
>>> final_path
'C:\\Files\\002c2b2541\\00025\\test.x'
>>>
>>> print(final_path)
C:\Files\002c2b2541\00025\test.x
>>>
And here is the detailed description of each and every steps/statements in the above solution.
First step (problem)
>>> import os
>>>
>>> path = 'C:\Files\2c2b2541\00025\test.x'
>>> path
'C:\\Files\x02c2b2541\x0025\test.x'
>>>
>>> print(path)
C:\Filesc2b2541 25 est.x
>>>
Second step (problem)
>>> path2 = repr(path)
>>> path2
"'C:\\\\Files\\x02c2b2541\\x0025\\test.x'"
>>>
>>> print(path2)
'C:\\Files\x02c2b2541\x0025\test.x'
>>>
Third step (problem)
>>> path3 = path2.replace("\\", "\\\\")
>>> path3
"'C:\\\\\\\\Files\\\\x02c2b2541\\\\x0025\\\\test.x'"
>>>
>>> print(path3)
'C:\\\\Files\\x02c2b2541\\x0025\\test.x'
>>>
>>> path3 = path3.replace("\\x", "\\\\0")
>>> path3
"'C:\\\\\\\\Files\\\\\\002c2b2541\\\\\\00025\\\\test.x'"
>>>
>>> print(path3)
'C:\\\\Files\\\002c2b2541\\\00025\\test.x'
>>>
Fourth step (problem)
>>> os.path.abspath(path3)
"C:\\Users\\RISHIKESH\\'C:\\Files\\002c2b2541\\00025\\test.x'"
>>>
>>> os.path.abspath(path2)
"C:\\Users\\RISHIKESH\\'C:\\Files\\x02c2b2541\\x0025\\test.x'"
>>>
>>> os.path.abspath('k')
'C:\\Users\\RISHIKESH\\k'
>>>
>>> os.path.abspath(path3).split("'")
['C:\\Users\\RISHIKESH\\', 'C:\\Files\\002c2b2541\\00025\\test.x', '']
>>> os.path.abspath(path3).split("'")[1]
'C:\\Files\\002c2b2541\\00025\\test.x'
>>>
Final step (solution)
>>> final_path = os.path.abspath(path3).split("'")[1]
>>>
>>> final_path
'C:\\Files\\002c2b2541\\00025\\test.x'
>>>
>>> print(final_path)
C:\Files\002c2b2541\00025\test.x
>>>
Replace "\" by "\\".
That's it.
You need to either use a raw string literal or double backslashes \\.
import os
path = r'C:\Files\2c2b2541\00025\test.x' #r before the string
print(os.path.abspath(path))
Is there a build-in function in python 3 to let me get b from a?
a = '\\xe9\\x82\\xa3'
b = b'\xe9\x82\xa3'
You can use unicode-escape encoding:
>>> a = '\\xe9\\x82\\xa3'
>>> a.encode().decode('unicode-escape').encode('latin1')
b'\xe9\x82\xa3'
>>> import codecs
>>> codecs.decode(a, 'unicode-escape').encode('latin1')
b'\xe9\x82\xa3'
how can i print the output of os.urandom(n) in terminal?
I try to generate a SECRET_KEY with fabfile and will output the 24 bytes.
Example how i implement both variants in the python shell:
>>> import os
>>> out = os.urandom(24)
>>> out
'oS\xf8\xf4\xe2\xc8\xda\xe3\x7f\xc75*\x83\xb1\x06\x8c\x85\xa4\xa7piE\xd6I'
>>> print out
oS�������5*������piE�I
If what you want is hex-encoded string, use binascii.a2b_hex (or hexlify):
>>> out = 'oS\xf8\xf4\xe2\xc8\xda\xe3\x7f\xc75*\x83\xb1\x06\x8c\x85\xa4\xa7piE\xd6I'
>>> import binascii
>>> print binascii.hexlify(out)
6f53f8f4e2c8dae37fc7352a83b1068c85a4a7706945d649
To use just built-ins, you can get the integer value with ord and then convert that back to a hex number:
list_of_hex = [str(hex(ord(z)))[2:] for z in out]
print " ".join(list_of_hex)
If you just want the hex list, then the str() and [2:] are unnecessary
The output of this and the hexify() version are both type str and should work fine for the web app.
I am observing a different output in the unpack function of python when I accept the string input from the console and when I read the string input from a variable.
I read the string input from the variable, input:
>>> import struct
>>> input="\x0d\x00\x00\x00"
>>> print struct.unpack("I",input)[0]
13
I read the string input from the console:
>>> import sys
>>> import struct
>>> print struct.unpack("I",sys.stdin.read(4))[0]
\x0d\x00\x00\x00
1680898140
The input string is the same but the output is different. Does it interpret the input read from the console in a different way? How can I get the same input by reading the data from console?
"\x0d\x00\x00\x00" (from the first code) is different from r"\x0d\x00\x00\x00" (== "\\x0x\\x00\x00\x00") from the second code.
>>> struct.unpack("I", '\x0d\x00\x00\x00')[0]
13
>>> struct.unpack("I", r'\x0d\x00\x00\x00'[:4])[0]
1680898140
Try following:
>>> struct.unpack("I", sys.stdin.readline().decode('string-escape')[:4])[0]
\x0d\x00\x00\x00
13
seems like you are unpacking the wrong data...
>>> struct.unpack('I','\\x0d')[0]
1680898140
your call to sys.stdin.read(4) reads only 4 characters: \, x, 0 and d.
>>> import sys
>>> import struct
>>> value = raw_input().decode('string-escape')
\x0d\x00\x00\x00
>>> print struct.unpack("I", value)[0]
13
I have a string with escaped data like
escaped_data = '\\x50\\x51'
print escaped_data # gives '\x50\x51'
What Python function would unescape it so I would get
raw_data = unescape( escaped_data)
print raw_data # would print "PQ"
You can decode with string-escape.
>>> escaped_data = '\\x50\\x51'
>>> escaped_data.decode('string-escape')
'PQ'
In Python 3.0 there's no string-escape, but you can use unicode_escape.
From a bytes object:
>>> escaped_data = b'\\x50\\x51'
>>> escaped_data.decode("unicode_escape")
'PQ'
From a Unicode str object:
>>> import codecs
>>> escaped_data = '\\x50\\x51'
>>> codecs.decode(escaped_data, "unicode_escape")
'PQ'
You could use the 'unicode_escape' codec:
>>> '\\x50\\x51'.decode('unicode_escape')
u'PQ'
Alternatively, 'string-escape' will give you a classic Python 2 string (bytes in Python 3):
>>> '\\x50\\x51'.decode('string_escape')
'PQ'
escaped_data.decode('unicode-escape') helps?
Try:
eval('"' + raw_data + '"')
It should work.