I am trying to convert a base64 string back to a GUID style hex number in python and having issues.
Base64 encoded string is: bNVDIrkNbEySjZ90ypCLew==
And I need to get it back to: 2243d56c-0db9-4c6c-928d-9f74ca908b7b
I can do it with the following PHP code but can't work out how to to it in Python
function Base64ToGUID($guid_b64) {
$guid_bin = base64_decode($guid_b64);
return join('-', array(
bin2hex(strrev(substr($guid_bin, 0, 4))),
bin2hex(strrev(substr($guid_bin, 4, 2))),
bin2hex(strrev(substr($guid_bin, 6, 2))),
bin2hex(substr($guid_bin, 8, 2)),
bin2hex(substr($guid_bin, 10, 6))
));
}
Here is the GUIDtoBase64 version:
function GUIDToBase64($guid) {
$guid_b64 = '';
$guid_parts = explode('-', $guid);
foreach ($guid_parts as $k => $part) {
if ($k < 3)
$part = join('', array_reverse(str_split($part, 2)));
$guid_b64 .= pack('H*', $part);
}
return base64_encode($guid_b64);
}
Here are some of the results using some of the obvious and not so obvious options:
import base64
import binascii
>>> base64.b64decode("bNVDIrkNbEySjZ90ypCLew==")
'l\xd5C"\xb9\rlL\x92\x8d\x9ft\xca\x90\x8b{'
>>> binascii.hexlify(base64.b64decode("bNVDIrkNbEySjZ90ypCLew=="))
'6cd54322b90d6c4c928d9f74ca908b7b'
Python port of the existing function (bitstring required)
import bitstring, base64
def base64ToGUID(b64str):
s = bitstring.BitArray(bytes=base64.b64decode(b64str)).hex
def rev2(s_):
def chunks(n):
for i in xrange(0, len(s_), n):
yield s_[i:i+n]
return "".join(list(chunks(2))[::-1])
return "-".join([rev2(s[:8]),rev2(s[8:][:4]),rev2(s[12:][:4]),s[16:][:4],s[20:]])
assert base64ToGUID("bNVDIrkNbEySjZ90ypCLew==") == "2243d56c-0db9-4c6c-928d-9f74ca908b7b"
First off, the b64 string and the resultant GUID doesn't match if we decode properly.
>>> import uuid
>>> import base64
>>> u = uuid.UUID("2243d56c-0db9-4c6c-928d-9f74ca908b7b")
>>> u
UUID('2243d56c-0db9-4c6c-928d-9f74ca908b7b')
>>> u.bytes
'"C\xd5l\r\xb9Ll\x92\x8d\x9ft\xca\x90\x8b{'
>>> base64.b64encode(u.bytes)
'IkPVbA25TGySjZ90ypCLew=='
>>> b = base64.b64decode('bNVDIrkNbEySjZ90ypCLew==')
>>> u2 = uuid.UUID(bytes=b)
>>> print u2
6cd54322-b90d-6c4c-928d-9f74ca908b7b
The base64 encoded version of the resultant GUID you posted is wrong. I'm not sure I understand the way you're encoding the GUID in the first place.
Python has in its arsenal all the tools required for you to be able to answer this problem. However, here's the rough scratching I did in a python terminal:
import uuid
import base64
base64_guid = "bNVDIrkNbEySjZ90ypCLew=="
bin_guid = base64.b64decode(base64_guid)
guid = uuid.UUID(bytes=bin_guid)
print guid
This code should give you enough of a hint to build your own functions. Don't forget, the python shell gives you a powerful tool to test and play with code and ideas. I would investigate using something like IPython notebooks.
I needed to do this to decode a BASE64 UUID that had been dumped from Mongodb. Originally the field had been created by Mongoose. The code I used, based on the code by #tpatja is here:
def base64ToGUID(b64str):
try:
bytes=base64.urlsafe_b64decode(b64str)
except Exception as e:
print("Can't decode base64 ", e)
s = bitstring.BitArray(bytes).hex
return "-".join([s[:8],s[8:][:4],s[12:][:4],s[16:][:4],s[20:]])
Based on good answers above, I wrote a version that does not require the bitstring package and includes validations and support for more input options.
import base64
import regex
import uuid
from typing import Optional
def to_uuid(obj) -> Optional[uuid.UUID]:
if obj is None:
return None
elif isinstance(obj, uuid.UUID):
return obj
elif isinstance(obj, str):
if regex.match(r'[0-9a-fA-F]{8}[-]{0,1}[0-9a-fA-F]{4}[-]{0,1}[0-9a-fA-F]{4}[-]{0,1}[0-9a-fA-F]{4}[-]{0,1}[0-9a-fA-F]{12}', obj):
return uuid.UUID(hex=obj)
elif regex.match(r'[0-9a-zA-Z\+\/]{22}[\=]{2}', obj):
b64_str = base64.b64decode(obj).hex()
uid_str = '-'.join([b64_str[:8], b64_str[8:][:4], b64_str[12:][:4], b64_str[16:][:4], b64_str[20:]])
return uuid.UUID(hex=uid_str)
raise ValueError(f'{obj} is not a valid uuid/guid')
else:
raise ValueError(f'{obj} is not a valid uuid/guid')
Related
I have this python code:
import os
from base64 import b64encode
from typing import Union
from hashlib import sha1
from hmac import new
PREFIX = bytes.fromhex("42")
SIG_KEY = bytes.fromhex("F8E7A61AC3F725941E3AC7CAE2D688BE97F30B93")
DEVICE_KEY = bytes.fromhex("02B258C63559D8804321C5D5065AF320358D366F")
def deviceId(data: bytes = None) -> str:
if isinstance(data, str): data = bytes(data, 'utf-8')
identifier = PREFIX + (data or os.urandom(20))
mac = new(DEVICE_KEY, identifier, sha1)
return f"{identifier.hex()}{mac.hexdigest()}".upper()
def signature(data: Union[str, bytes]) -> str:
data = data if isinstance(data, bytes) else data.encode("utf-8")
return b64encode(PREFIX + new(SIG_KEY, data, sha1).digest()).decode("utf-8")
I need to convert it to kotlin code. After few hours i wrote this:
import java.util.*
import javax.crypto.Mac
import android.util.Base64
import javax.crypto.spec.SecretKeySpec
fun String.decodeTheHex(): ByteArray = chunked(2).map { it.toInt(16).toByte() }.toByteArray()
fun ByteArray.toHexString() = joinToString("") { "%02x".format(it) }
const val HMAC_SHA_1 = "HmacSHA1"
const val NDC_MSG_SIG_KEY = "F8E7A61AC3F725941E3AC7CAE2D688BE97F30B93"
const val DEVICE_ID_KEY = "02B258C63559D8804321C5D5065AF320358D366F"
fun getSignature(data: String): String {
val mac = Mac.getInstance("HmacSHA1")
mac.init(SecretKeySpec(NDC_MSG_SIG_KEY.decodeTheHex(), HMAC_SHA_1))
return Base64.encodeToString("42".decodeTheHex() + mac.doFinal(data.toByteArray()), Base64.NO_WRAP)
}
fun getDeviceId(): String {
val mac = Mac.getInstance("HmacSHA1")
mac.init(SecretKeySpec(DEVICE_ID_KEY.decodeTheHex(), HMAC_SHA_1))
val bytes = ByteArray(20)
Random().nextBytes(bytes)
val id = "42" + bytes.toHexString()
return (id + mac.doFinal("2".toByteArray(Charsets.UTF_8) + bytes).toHexString()).uppercase()
}
But the result after compiling is not correct (the app doesn't allow the result, i mean the app, for that i'm sending the request. And i know, that the result from function getDeviceId() is completly wrong). How to convert it?
I have this requirement where I need to encrypt a dictionary in python using password based AES256 encryption. I was only given a sample vb.net project by my boss to take reference from regarding the steps that I need to follow for encryption but I am not really aware of how vb.net works. I have converted as much code from vb.net to python as much I could but still the output that my code produces is different from the vb.net project. Can someone please explain what exactly am I missing out on due to which the encryption output of my python program is different than the encryption output of the vb.net program? Thanks in advance!
Here is the vb.net code
Imports System.IO
Imports System.Security.Cryptography
Imports System.Text
Imports System.Web.Script.Serialization
Imports System.Net
Imports System.Security.Cryptography.X509Certificates
Imports System.Net.Security
Public Class Form1
Private Sub Form1_Load(sender As Object, e As EventArgs) Handles MyBase.Load
loginrequest()
End Sub
Private Sub loginrequest()
Dim lobjLogin As New loginrequest
lobjLogin.jsonProperty1 = "jsonProperty1"
lobjLogin.jsonProperty2 = "jsonProperty2"
lobjLogin.jsonProperty3 = "jsonProperty3"
lobjLogin.jsonProperty4 = "jsonProperty4"
Dim lRequestJson As String = ""
lRequestJson = (New JavaScriptSerializer()).Serialize(lobjLogin)
Dim lchecksum As String = EncryptText(lRequestJson, "key")
End Sub
Public Function EncryptText(pInput As String, password As String) As String
Dim bytesToBeEncrypted As Byte() = Encoding.UTF8.GetBytes(GenerateSHA256String(pInput))
Dim passwordBytes As Byte() = Encoding.UTF8.GetBytes(password)
passwordBytes = SHA256.Create().ComputeHash(passwordBytes)
Dim bytesEncrypted As Byte() = AES_Encrypt(bytesToBeEncrypted, passwordBytes)
Dim result As String = Convert.ToBase64String(bytesEncrypted)
Return result
End Function
Private Function AES_Encrypt(bytesToBeEncrypted As Byte(), passwordBytes As Byte()) As Byte()
Dim encryptedBytes As Byte() = Nothing
Dim saltBytes As Byte() = New Byte() {1, 2, 3, 4, 5, 6, _
7, 8}
Using ms As New MemoryStream()
Using AES As New RijndaelManaged()
AES.KeySize = 256
AES.BlockSize = 128
Dim key = New Rfc2898DeriveBytes(passwordBytes, saltBytes, 1000)
AES.Key = key.GetBytes(AES.KeySize / 8)
AES.IV = key.GetBytes(AES.BlockSize / 8)
AES.Mode = CipherMode.CBC
Using cs = New CryptoStream(ms, AES.CreateEncryptor(), CryptoStreamMode.Write)
cs.Write(bytesToBeEncrypted, 0, bytesToBeEncrypted.Length)
cs.Close()
End Using
encryptedBytes = ms.ToArray()
End Using
End Using
Return encryptedBytes
End Function
Private Function GenerateSHA256String(ByVal inputString) As String
Dim sha256 As SHA256 = SHA256Managed.Create()
Dim bytes As Byte() = Encoding.UTF8.GetBytes(inputString)
Dim hash As Byte() = sha256.ComputeHash(bytes)
Dim stringBuilder As New StringBuilder()
For i As Integer = 0 To hash.Length - 1
stringBuilder.Append(hash(i).ToString("X2"))
Next
Return stringBuilder.ToString()
End Function
End Class
Public Class loginrequest
Public Property jsonProperty1 As String
Public Property jsonProperty2 As String
Public Property jsonProperty3 As String
Public Property jsonProperty4 As String
End Class
Here is the corresponding python code that I wrote
import base64
from Crypto.Cipher import AES
from Crypto.Protocol.KDF import PBKDF2
from Crypto.Util.Padding import pad
import hashlib, json
payload = {
"jsonProperty1": "jsonProperty1",
"jsonProperty2": "jsonProperty2",
"jsonProperty3": "jsonProperty3",
"jsonProperty4": "jsonProperty4",
}
payload = json.dumps(payload)
bytesToBeEncrypted = hashlib.sha256(payload.encode("utf-8")).hexdigest()
class AESCipher(object):
def __init__(self, key, interactions=1000):
self.bs = AES.block_size
self.key = hashlib.sha256(key.encode("utf-8")).hexdigest()
self.interactions = interactions
def pkcs7padding(self, data, block_size=16):
if type(data) != bytearray and type(data) != bytes:
raise TypeError("Only support bytearray/bytes !")
pl = block_size - (len(data) % block_size)
return data + bytearray([pl for i in range(pl)])
def encrypt(self, raw):
import os
raw = "".join([x.upper() for x in bytesToBeEncrypted])
keyiv = PBKDF2(self.key, os.urandom(8), 48, self.interactions)
key = keyiv[:32]
iv = keyiv[32:48]
cipher = AES.new(key, AES.MODE_CBC, iv)
encoded = raw.encode("utf-8")
encodedpad = self.pkcs7padding(encoded)
ct = cipher.encrypt((encodedpad))
cip = base64.b64encode(ct)
print(cip, len(cip))
enc = AESCipher("key")
dec = enc.encrypt(bytesToBeEncrypted)
Please note that I took some reference from some other threads as well regarding my python code because encryption is a new concept for me.
P.S. I also found out that the vb.net code is using .toString("X2") to generate a hexadecimal string in uppercase but unfortunately, I was not able to find the corresponding equivalent in python for the same. Could that be a problem?
<?php
$password = "SECRETPASSWORD";
$nonce = random_bytes(32); # requires PHP 7
date_default_timezone_set("UTC");
$timestamp = date(DATE_ATOM);
$encodedNonce = base64_encode($nonce);
$passSHA = base64_encode(sha1($nonce . $timestamp . sha1($password, true), true));
?>
it generates the below result with a 28 characters password digest, which I am using in soap requests, and it works fine
password_digest = '/pBYmwwc2cM87CUr8oB4Wkmyc0Q='
nonce = '���>�!��g��q�[�`�R��=J�o�'
nonce_base_64_encode = 'uqbkProhsR3JZxjC93HWW8BghQFSgqo9Sv9vGgUa4hs='
timestamp = '2022-01-13T18:28:52+00:00'
I need to do this same in python, but python is somehow generating longer password_digest and the soap request fails. I don't know if I am not generating the random_bytes correctly in python or some other issue. Below is python code:
import secrets
import hashlib
import datetime
clearPassword = 'MYSECRETPASSWORD'
created_at_timestamp_utc = datetime.datetime.utcnow().strftime("%Y-%m-%dT%H:%M:%SZ")
def getSha1String(string):
stringInBytes = string.encode('ascii')
hash_object = hashlib.sha1(stringInBytes)
hex_dig = hash_object.hexdigest()
return hex_dig
def getBase64NonceString(nonce):
nonce_bytes = nonce.encode('ascii')
base64_bytes = base64.b64encode(nonce_bytes)
base64_nonce = base64_bytes.decode('ascii')
return base64_nonce
def getBase64String(string):
string_bytes = string.encode('ascii')
base64_bytes = base64.b64encode(string_bytes)
base64_string = base64_bytes.decode('ascii')
return base64_string
nonce = secrets.token_bytes(32)
base64_nonce = getBase64Nonce(nonce)
sha1_password = getSha1String(clearPassword)
password_digest = getBase64String(getSha1String(str(nonce) + created_at_timestamp_utc + sha1_password))
Your python code has 3 problems:
You're using binary output from sha1() in PHP, but hex output in python. Use digest(), not hexdigest(). This is why your output is longer in python.
Your timestamp format is incorrect.
The PHP format DATE_ATOM is "Y-m-d\TH:i:sP", where P outputs the UTC offset in the format +00:00. Unfortunately Python's strftime() doesn't seem to have an equivalent, but it's all in UTC anyway and your python code simply specifies the static string Z. So change that to +00:00, otherwise your tokens won't match.
Eg: timestamp = datetime.datetime.utcnow().strftime('%Y-%m-%dT%H:%M:%S+00:00')
You're using SECRETPASSWORD in PHP and MYSECRETPASSWORD in python, and I am emabarrased at how long I bashed my head against that one without noticing.
Succint working code:
import hashlib, datetime, base64
password = 'SECRETPASSWORD'
timestamp = '2022-01-13T18:28:52+00:00'
nonce = base64.b64decode('uqbkProhsR3JZxjC93HWW8BghQFSgqo9Sv9vGgUa4hs=')
def quickSHA(input):
return hashlib.sha1(input).digest()
def makeToken(password, timestamp, nonce):
return base64.b64encode(
quickSHA( nonce + timestamp + quickSHA(password) )
)
print makeToken(password, timestamp, nonce)
Output: /pBYmwwc2cM87CUr8oB4Wkmyc0Q=
I am trying to upgrade some code written in python2, so that it works in python3 and still works in python2. This is the code snippet in question that works under python2:
import ctypes
import struct
class Ethernet(ctypes.Structure):
_fields_ = [('dst', ctypes.c_char_p)]
def __init__(self, packet):
(dst, ) = struct.unpack('!6s', packet[:6])
self.dst = ':'.join(['%02x' % (ord(octet), ) for octet in dst])
def main():
p = b'\xab\xad\xba\xbe\x0f\xe9'
e = Ethernet(p)
if b"ab:ad:ba:be:0f:e9" == e.dst:
print(True)
else:
print(False)
if __name__ == '__main__':
main()
My problems start when I move to python3 i get this warning for the same code:
File "test2.py", line 11, in <listcomp>
self.dst = ':'.join(['%02x' % (ord(octet), ) for octet in dst])
TypeError: ord() expected string of length 1, but int found
So obviously this code doesn't work as it is in python3.
The only way I can seem to get it to work is like this :
try: # Python 3
self.dst = b':'.join([struct.pack('bb', ord('{:x}'.format((octet >> 4) & 0xf)), ord('{:x}'.format((octet >> 0) & 0xf))) for octet in dst])
except TypeError: # Python 2
self.dst = ':'.join(['%02x' % (ord(octet), ) for octet in dst])
Which is very messy, but every which way I try it it the fields definition of ctypes.c_char_p that blocks anything simple.
Is there a better way to do this that is a single line that works in both python2 and python3 (no exceptions needed)
I don't know why I can't mark the correct answer as such but eryksun response is what worked for me:
2.6+ has the bytearray type, which iterates as integers. It also has b'' literals. For the formatting part in Python 3 you can use a string and then encode it, which is harmless in Python 2. For example:
dst = bytearray(packet[:6]);
self.dst = b':'.join([('%02x' % o).encode('ascii') for o in dst])`
– eryksun Jul 16 at 15:39
with using python 2.7:
>myCity = 'Isparta'
>myCity.lower()
>'isparta'
#-should be-
>'ısparta'
tried some decoding, (like, myCity.decode("utf-8").lower()) but could not find how to do it.
how can lower this kinds of letters? ('I' > 'ı', 'İ' > 'i' etc)
EDIT: In Turkish, lower case of 'I' is 'ı'. Upper case of 'i' is 'İ'
Some have suggested using the tr_TR.utf8 locale. At least on Ubuntu, perhaps related to this bug, setting this locale does not produce the desired result:
import locale
locale.setlocale(locale.LC_ALL, 'tr_TR.utf8')
myCity = u'Isparta İsparta'
print(myCity.lower())
# isparta isparta
So if this bug affects you, as a workaround you could perform this translation yourself:
lower_map = {
ord(u'I'): u'ı',
ord(u'İ'): u'i',
}
myCity = u'Isparta İsparta'
lowerCity = myCity.translate(lower_map)
print(lowerCity)
# ısparta isparta
prints
ısparta isparta
You should use new derived class from unicode from emre's solution
class unicode_tr(unicode):
CHARMAP = {
"to_upper": {
u"ı": u"I",
u"i": u"İ",
},
"to_lower": {
u"I": u"ı",
u"İ": u"i",
}
}
def lower(self):
for key, value in self.CHARMAP.get("to_lower").items():
self = self.replace(key, value)
return self.lower()
def upper(self):
for key, value in self.CHARMAP.get("to_upper").items():
self = self.replace(key, value)
return self.upper()
if __name__ == '__main__':
print unicode_tr("kitap").upper()
print unicode_tr("KİTAP").lower()
Gives
KİTAP
kitap
This must solve your problem.
You can just use .replace() function before changing to upper/lower. In your case:
myCity.replace('I', 'ı').lower()
I forked and redesigned Emre's solution by monkey-patching method to built-in unicode module. The advantage of this new approach is no need to use a subclass of unicode and
redefining unicode strings by my_unicode_string = unicode_tr(u'bla bla bla')
Just importing this module, integrates seamlessly with builtin native unicode strings
https://github.com/technic-programming/unicode_tr
# -*- coding: utf8 -*-
# Redesigned by #guneysus
import __builtin__
from forbiddenfruit import curse
lcase_table = tuple(u'abcçdefgğhıijklmnoöprsştuüvyz')
ucase_table = tuple(u'ABCÇDEFGĞHIİJKLMNOÖPRSŞTUÜVYZ')
def upper(data):
data = data.replace('i',u'İ')
data = data.replace(u'ı',u'I')
result = ''
for char in data:
try:
char_index = lcase_table.index(char)
ucase_char = ucase_table[char_index]
except:
ucase_char = char
result += ucase_char
return result
def lower(data):
data = data.replace(u'İ',u'i')
data = data.replace(u'I',u'ı')
result = ''
for char in data:
try:
char_index = ucase_table.index(char)
lcase_char = lcase_table[char_index]
except:
lcase_char = char
result += lcase_char
return result
def capitalize(data):
return data[0].upper() + data[1:].lower()
def title(data):
return " ".join(map(lambda x: x.capitalize(), data.split()))
curse(__builtin__.unicode, 'upper', upper)
curse(__builtin__.unicode, 'lower', lower)
curse(__builtin__.unicode, 'capitalize', capitalize)
curse(__builtin__.unicode, 'title', title)
if __name__ == '__main__':
print u'istanbul'.upper()
print u'İSTANBUL'.lower()
You need to set the proper locale (I'm guessing tr-TR) with locale.setLocale(). Otherwise the default upper-lower mappings will be used, and if that default is en-US, the lowercase version of I is i.