Is there any way I could save hexdump() to byte list so the list can be accessed by index. what I need is like this
byte = hexdump(packet)
for i in range(0, len(byte)):
print %x byte[i]
The byte content of the packet may be accessed by invoking str(packet), as follows:
content = str(packet) # decoded hex string, such as '\xde\xad\xbe\xef'
print content
for byte in content:
pass # do something with byte
EDIT - This answer specifies how this can be converted to a byte array, for example:
byte_array = map(ord, str(packet)) # list of numbers, such as [0xDE, 0xAD, 0xBE, 0xEF]
print byte_array
for byte in byte_array:
pass # do something with byte
Related
ciphertext = base64.b64decode(xxxxxx) //output is b'148,240,50,66,81,26,240,2,101,31'
bytearray(ciphertext) // output is bytearray(b'148,240,50,66,81,26,240,2,101,31')
What am looking for is output of bytearray([148,240,50,66,81,26,240,2,101,31])
Full code:
ciphertext = base64.b64decode("MTQ4LDI0MCw1MCw2Niw4MSwyNiwyNDAsMiwxMDEsMzEsMjM3LDEwMSw4OCwxODQsMTQsMTM1LDEzMCw0Miw0NywxODksMTkyLDE1MSw0OCwyMjQsMTU1LDQxLDM5LDE0MywyMDksMTA0LDE5NywyMywxMDUsMjMsMTYzLDUzLDQsMTQ0LDE2MSwxNDgsMjMwLDI1NCwxMzQsMjEzLDE3NCwyNDcsMTkxLDUyLDY0LDE2LDYzLDk0LDE1NCwxMzMsMzksMTMzLDIyNCwxODcsMTE0LDE1OCwyMzksMzUsMTUxLDM4LDE3NSwxNTIsOTksMTAyLDIxNCwyNTEsMTk0LDIxMywxNzMsMTc0LDcyLDIyNSwyMDIsMTcyLDE1NCw4OCwxMzksMTE1LDIzNywyMzYsMTIxLDAsMjE0LDIxNiwxOTYsNDAsMzgsMjA0LDgzLDEzNiwxNjAsMTczLDY5LDcsMzgsMjI1LDExOCw0OSw0OCw3MCwxNjYsMTIxLDI0NSwxOTEsMTgzLDEyMiwxOTksMTg3LDgsNDMsNDUsOTMsMTI0LDIxNSwxNjEsNzAsMjU0LDI2LDE4OCwxMywyMjYsMTMxLDMsNCw0MywxOTgsMjEyLDEwMywxMTcsMjE1LDEyNywyNDMsMzksNzIsNzYsMTE0LDUwLDE5Niw1NSwxMjEsODYsMjUxLDUzLDI0MiwzMCwxMDksNDcsMjEwLDI1MywxNjMsOTAsOTgsMTQsNjAsMTE1LDc1LDE0OSwyMTAsMTc1LDI2LDEyNCwyMjgsMjQ3LDIwLDIwMyw5NiwyMTAsMjYsODEsNjUsMTg4LDEyMSwxMjgsOTEsMTA3LDE2OCwxMywyMDcsMTc1LDE3MCwyNTUsMjM2LDE0OSwxMDksNTksMjQsMTcyLDExLDU4LDEzLDAsMTUyLDExNiwxMTAsMTExLDIyLDIzMSwzLDIzNyw0Miw4MSw3Nyw2MywyMjMsMTAzLDEwOSw1NiwxNTgsNDMsMjA2LDIwMiwzOCwxNDgsMTM3LDE4OSwyMTQsMjE2LDkwLDE4LDIyNCwyNTQsMzcsMTA5LDE4LDg0LDIyMiwyMDksMjUsNTMsMjE5LDE2OSwyMTEsNTAsMTgyLDQwLDExMiwyMDksMzEsNTIsMjEsNTMsOTgsMTIyLDI1NCwxMDgsMzksMzgsMTM0LDE1MCwxMzksMTk0LDMw=")
Replace:
bytearray(ciphertext)
with:
bytearray(map(int, ciphertext.split(b',')))
# Or if you prefer genexprs:
bytearray(int(x) for x in ciphertext.split(b','))
The former is just converting the raw bytes to an equivalent bytearray, the latter splits it up by commas and parses the components as ints.
I'm creating my RSA Signature like this.
transactionStr = json.dumps(GenesisTransaction())
signature = rsa.sign(transactionStr.encode(), client.privateKey, 'SHA-1')
But I'm unable to get it to a string so I can save it.
I have tried decoding it using utf8
signature.decode("utf8")
but I get the error "'utf-8' codec can't decode byte 0xe3 in position 2"
any way I can do this?
A RSA signature looks like this
b'aL\xe3\xf4\xbeEM\xc4\x9e\n\x9e\xf4M`\xba\x85*\x13\xd52x\xd9\\\xe8F\x1c\x07\x90[/\x9dy\xce\xa9IV\x89\xe0\xcd9\\_3\x1e\xaa\x80\xdea\xd1\xbem/\x8e\x91\xbd\x13\x12o\x8c\xed\xf6\x89\xb5\x0b'
.decode('utf8') is used to decode text encoded in UTF8, not arbitrary bytes. Convert the byte string to a hexadecimal string instead:
>>> sig = b'aL\xe3\xf4\xbeEM\xc4\x9e\n\x9e\xf4M`\xba\x85*\x13\xd52x\xd9\\\xe8F\x1c\x07\x90[/\x9dy\xce\xa9IV\x89\xe0\xcd9\\_3\x1e\xaa\x80\xdea\xd1\xbem/\x8e\x91\xbd\x13\x12o\x8c\xed\xf6\x89\xb5\x0b'
>>> s = sig.hex()
>>> s
'614ce3f4be454dc49e0a9ef44d60ba852a13d53278d95ce8461c07905b2f9d79cea9495689e0cd395c5f331eaa80de61d1be6d2f8e91bd13126f8cedf689b50b'
To convert back, if needed:
>>> b = bytes.fromhex(s)
>>> b
b'aL\xe3\xf4\xbeEM\xc4\x9e\n\x9e\xf4M`\xba\x85*\x13\xd52x\xd9\\\xe8F\x1c\x07\x90[/\x9dy\xce\xa9IV\x89\xe0\xcd9\\_3\x1e\xaa\x80\xdea\xd1\xbem/\x8e\x91\xbd\x13\x12o\x8c\xed\xf6\x89\xb5\x0b'
>>> b==sig
True
sql-server and pyodbc return all SQL_GUID datatypes in uppercase, while queries are case-insensitive:
psql.py --sql="select controllerid from controllers where controllerid='F573A57D-9247-44CB-936A-D16DD4E8327F'"
[('F573A57D-9247-44CB-936A-D16DD4E8327F', )]
psql.py --sql="select controllerid from controllers where controllerid='f573a57d-9247-44cb-936a-d16dd4e8327f'"
[('F573A57D-9247-44CB-936A-D16DD4E8327F', )]
I want it to output in lowercase.
Added a pyodbc.add_output_converter() method which just does a lower on SQL_GUID but this is a packed structure:
def guid_to_lowercase(value):
return value.lower()
pyodbc.add_output_converter(pyodbc.SQL_GUID, guid_to_lowercase)
[(b'}\xa5s\xf5g\x92\xcbd\x93j\xd1m\xd4\xe82\x7f', )]
It looks like a byte, but changing it:
def guid_to_lowercase(value):
#log.error("type:{}".format(type(value)))
return value.decode("ascii").lower()
UnicodeDecodeError: 'ascii' codec can't decode byte 0xa5 in position 1: ordinal not in range(128)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xa5 in position 1: invalid start byte
I assume I need to unpack it, but how is the format.
Outside of pyodbc I can do a uuid.lower()
When we print out the raw bytes returned by the server for the GUID
'F573A57D-9247-44CB-936A-D16DD4E8327F'
we get
b'}\xa5s\xf5g\x92\xcbd\x93j\xd1m\xd4\xe82\x7f'
Converting the ASCII characters to their hex values, removing the \x prefixes, and adding some spaces we get
7da573f5 4792 cb44 936a d16dd4e8327f
revealing that SQL Server ODBC is returning the bytes for the GUID, but the first three segments are in little-endian order, while the last two segments are in big-endian order.
So, if we change our output converter function to
def guid_to_lowercase(value):
first_three_values = struct.unpack('<I2H', value[:8])
fourth_value = struct.unpack('>H', value[8:10])[0]
fifth_value = struct.unpack('>Q', b'\x00\x00' + value[10:16])[0]
guid_string_parts = (
'{:08x}'.format(first_three_values[0]),
'{:04x}'.format(first_three_values[1]),
'{:04x}'.format(first_three_values[2]),
'{:04x}'.format(fourth_value),
'{:012x}'.format(fifth_value),
)
return '-'.join(guid_string_parts)
it returns
'f573a57d-9247-44cb-936a-d16dd4e8327f'
This question already has answers here:
Best way to convert string to bytes in Python 3?
(5 answers)
Closed 11 days ago.
I have the following function to parse a utf-8 string from a sequence of bytes
Note -- 'length_size' is the number of bytes it take to represent the length of the utf-8 string
def parse_utf8(self, bytes, length_size):
length = bytes2int(bytes[0:length_size])
value = ''.join(['%c' % b for b in bytes[length_size:length_size+length]])
return value
def bytes2int(raw_bytes, signed=False):
"""
Convert a string of bytes to an integer (assumes little-endian byte order)
"""
if len(raw_bytes) == 0:
return None
fmt = {1:'B', 2:'H', 4:'I', 8:'Q'}[len(raw_bytes)]
if signed:
fmt = fmt.lower()
return struct.unpack('<'+fmt, raw_bytes)[0]
I'd like to write the function in reverse -- i.e. a function that will take a utf-8 encoded string and return it's representation as a byte string.
So far, I have the following:
def create_utf8(self, utf8_string):
return utf8_string.encode('utf-8')
I run into the following error when attempting to test it:
File "writer.py", line 229, in create_utf8
return utf8_string.encode('utf-8')
UnicodeDecodeError: 'ascii' codec can't decode byte 0x98 in position 0: ordinal not in range(128)
If possible, I'd like to adopt a structure for the code similar to the parse_utf8 example. What am I doing wrong?
Thank you for your help!
UPDATE: test driver, now correct
def random_utf8_seq(self, length):
# from http://www.w3.org/2001/06/utf-8-test/postscript-utf-8.html
test_charset = u" !\"#$%&'()*+,-./0123456789:;<=>?#ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~ ¡¢£¤¥¦§¨©ª«¬ ®¯°±²³´µ¶·¸¹º»¼½¾¿ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿĂ㥹ĆćČčĎďĐđĘęĚěĹ弾ŁłŃńŇňŐőŒœŔŕŘřŚśŞşŠšŢţŤťŮůŰűŸŹźŻżŽžƒˆˇ˘˙˛˜˝–—‘’‚“”„†‡•…‰‹›€™"
utf8_seq = u""
for i in range(length):
utf8_seq += random.choice(test_charset)
return utf8_seq
I get the following error:
input_str = self.random_utf8_seq(200)
File "writer.py", line 226, in random_utf8_seq
print unicode(utf8_seq, "utf-8")
UnicodeDecodeError: 'utf8' codec can't decode byte 0xbb in position 0: invalid start byte
If utf-8 => bytestring conversion is what do you want then you may use str.encode, but first you need to properly mark the type of source string in your example - prefix with u for unicode:
# coding: utf-8
import random
def random_utf8_seq(length):
# from http://www.w3.org/2001/06/utf-8-test/postscript-utf-8.html
test_charset = u" !\"#$%&'()*+,-./0123456789:;<=>?#ABCDEFGHIJKLMNOPQRSTUVWXYZ[\]^_`abcdefghijklmnopqrstuvwxyz{|}~ ¡¢£¤¥¦§¨©ª«¬ ®¯°±²³´µ¶·¸¹º»¼½¾¿ÀÁÂÃÄÅÆÇÈÉÊËÌÍÎÏÐÑÒÓÔÕÖ×ØÙÚÛÜÝÞßàáâãäåæçèéêëìíîïðñòóôõö÷øùúûüýþÿĂ㥹ĆćČčĎďĐđĘęĚěĹ弾ŁłŃńŇňŐőŒœŔŕŘřŚśŞşŠšŢţŤťŮůŰűŸŹźŻżŽžƒˆˇ˘˙˛˜˝–—‘’‚“”„†‡•…‰‹›€™"
utf8_seq = u''
for i in range(length):
utf8_seq += random.choice(test_charset)
print utf8_seq.encode('utf-8')
return utf8_seq.encode('utf-8')
print( type(random_utf8_seq(200)) )
-- output --
õ3×sÔP{Ć.s(Ë°˙ě÷xÓ#bűV—û´ő¢uZÓČn˜0|_"Ðyø`êš·ÏÝhunÍÅ=ä?
óP{tlÇűpb¸7s´ňƒG—čøň\zčłŢXÂYqLĆúěă(ÿî ¥PyÐÔŇnל¦Ì˝+•ì›
ŻÛ°Ñ^ÝC÷ŢŐIñJĹţÒył"MťÆ‹ČČ4þ!»šåŮ#Öhň-
ÈLGĄ¢ß˛Đ¯.ªÆź˘Ř^ĽÛŹËaĂŕ¹#¢éüÜńlÊqš=VřU…‚–MŽÎÉèoÙŹŠ¨Ð
<type 'str'>
I got a list = [0x97, 0x52], not unicode object. this is unicode of a charactor '青'(u'\u9752'). How could I change this list to unicode object first, then encode to 'UTF-8'?
bytes = [0x97, 0x52]
code = bytes[0] * 256 + bytes[1] # build the 16-bit code
char = unichr(code) # convert code to unicode
utf8 = char.encode('utf-8') # encode unicode as utf-8
print utf8 # prints '青'
Not sure if this is the most elegant way, but it works for this particular example.
>>> ''.join([chr(x) for x in [0x97, 0x52]]).decode('utf-16be')
u'\u9752'