We Keep Coding

Using pycryptodome to decrypt encrypted data

I'm new to encryption/decryption, but I have sensitive data that I need to store as encrypted data. Our ETL has a built in encryption process which outputs the following
{
"data":{
"transformation":"AES/GCM/noPadding",
"iv":"jlemHiOD8uiyMsqY",
"type":"JSON",
"ciphertext":"TOtsmTYG1jKCZXewFNPBk5xWf+q4AO8OSZPoYw==",
"key_params":{
"symmetric":{
"key_algorithm":"AES"
}
}
}
}
From here, I'd like to use pycryptodome to decrypt the data when I need to consume the data. I am trying to run the following python script to decrypt but am running into some errors. I think it has to do with how the data is formatted?
import pandas as pd
from Crypto.Cipher import AES
test_encrypted_value = TOtsmTYG1jKCZXewFNPBk5xWf+q4AO8OSZPoYw==
aes_iv = 'jIemHiOD8uiyMsqY'
aes_key = '4E645267556B586E3272357538782F41'
cipher = AES.new(aes_key, AES.MODE_GCM, aes_iv)
error - TypeError: Nonce must be bytes, bytearray or memoryview
If I remove the IV, I also get an error on the key passed. Which leads me to think I am passing the wrong data type/format.
error - TypeError: Object type <class 'str'> cannot be passed to C code
UPDATE
Per the responses, I updated my code to transform data format. Additionally, I changed my example and saved the expected value. I am expecting the decrypted value to be 158100.
import pandas as pd
from Crypto.Cipher import AES
from Crypto.Util.Padding import pad, unpad
import codecs
import base64
test_encrypted_value = 'SUXiDF6Dgtc8y3eY8Euqi/IYbSlQquLJAUKmZw=='
aes_iv = 'lMF2Jrruo9rR57Uy'
aes_key = '4E645267556B586E3272357538782F41'
byte_key = codecs.decode(aes_key, 'hex_codec')
base64_iv = base64.b64decode(aes_iv)
base64_encrypted_value = base64.b64decode(test_encrypted_value)
cipher = AES.new(byte_key, AES.MODE_GCM, base64_iv)
plaintext = cipher.decrypt(ciphertext)
print(plaintext.decode())
I am now getting the below error
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb9 in position 1: invalid start byte
For what it's worth, encryption is through snaplogic 'Encrypt Field' found here - https://docs-snaplogic.atlassian.net/wiki/spaces/SD/pages/1438346/Encrypt+Field
I am also generating the AES key at the following link with the 128-bit and HEX option set - https://www.allkeysgenerator.com/Random/Security-Encryption-Key-Generator.aspx

You need to pass the data in bytes format. Your aes_iv and test_encrypted_value is in the base64 format, while your aes_key is in the hex format. In order to use it, you must first convert those to bytes.
byte_key = codecs.decode(aes_key, 'hex_codec')
base64_iv = base64.b64decode(aes_iv)
base64_encrypted_value = base64.b64decode(test_encrypted_value)

PyCryptodome AES, 'EcbMode' object has no attribute 'encrypt_and_digest'

I am using Pycharm Text Editor and PyCryptodome Library for encryption of message using Advanced Encryption Standard (AES). It is also one of the majorly used Symmetric Encryption Algorithm. My code of AES Encryption was stated below:
from Crypto.Cipher import AES
key = os.urandom(16)
cipher = AES.new(key, AES.MODE_ECB)
ciphertext, tag = cipher.encrypt_and_digest(message)
And I am encountering the error:-
AttributeError: 'EcbMode' object has no attribute 'encrypt_and_digest
at line:-
ciphertext, tag = cipher.encrypt_and_digest(message)
I have tried uninstalling the module a couple of times but the error persists.
Pycharm does not mark the line with red underscores and I can use ctrl-click to go into the source and see that the encrypt_and_digest() function exists.
My question is:-
Why can't the code go through the compiler?
Is there another module in python that I can use to perform AES encryption?

If you read the documentation you will see that encrypt_and_digest() is only supported by the modern modes:
MODE_CCM
MODE_EAX
MODE_GCM
MODE_SIV
MODE_OCB
ECB mode really shouldn't be used for anything as it is not semantically secure.

From Error we can see that encrypt_and_digest() attribute is not available in ECB (Electronic Code Book) Mode of AES Encryption. So, there are two Solution for your query let's have a look at one by one:-
1. By Changing Mode:-
By Changing mode to modern modes we can use encrypt_and_digest() module. Basically encrypt_and_digest() module is combination of encrypt() and digest() Module.
encrypt():- This module used to Encrypt your Message
digest():- This module used to Generate MAC Tag of Message
# List of 'Modern Modes' was given below:-
1. MODE_EAX
2. MODE_CCM
3. MODE_SIV
4. MODE_GCM
5. MODE_OCB
Code for the given Scenario Using EAX Mode was stated below:-
# Import all the Important Libraries
from Crypto.Cipher import AES
import os
# 'pad_message()' function declaration for padding purpose.
# Because 'message' length should be always multiple of '16'
def pad_message(message):
while len(message) % 16 != 0:
message = message + " "
return message
# Initialization of 'Key' and 'Message'
key = os.urandom(16)
message = input("Enter your Message for AES Encryption:- ")
# If the length of the message is not multiple of '16' then pad it
message = pad_message(message)
# Print Message, Key and Length of Message before Encryption Process
print("\nMessage:-", message)
print("Key:-", key)
print("Length of the Message:-", len(message))
# Declare New module for AES Encryption in 'EAX' Mode
cipher = AES.new(key, AES.MODE_EAX)
# Encrypt 'Message' and Generate 'MAC Tag' Using 'encrypt_and_digest()' method
cipher_text, mac_tag = cipher.encrypt_and_digest(message.encode('utf-8'))
# Print Encrypted Message
print("\nEncryption of Message Using AES:-", cipher_text)
print("MAC Tag of our Encrypted Message is:-", mac_tag)
# Output of Above Code:-
Enter your Message for AES Encryption:- Stack Overflow
Message:- Stack Overflow
Key:- b'\xf1\x9a\xc1\x12\xdcI7\xc8\xe4\xcf\x1e5\xe4\x93i\xc4'
Length of the Message:- 16
Encryption of Message Using AES:- b'\x97\x0e+\xcb^\x82\xeelhs2_\x90m\x1c+'
MAC Tag of our Encrypted Message is:- b'c!\xb2\xf4\x82\xceT3\x0cM1\x04\x87(y?'
2. Without Changing Mode:-
If you want to Encrypt Using ECB (Electronic Code Book) Mode. then we can Use encrypt() and hex() module.
Code for the given Scenario Using ECB Mode was stated below:-
# Import all the Important Libraries
from Crypto.Cipher import AES
import os
# 'pad_message()' function declaration for padding purpose.
# Because 'message' length should be always multiple of '16'
def pad_message(message):
while len(message) % 16 != 0:
message = message + " "
return message
# Initialization of 'Key' and 'Message'
key = os.urandom(16)
message = input("Enter your Message for AES Encryption:- ")
# If the length of the message is not multiple of '16' then pad it
message = pad_message(message)
# Print Message, Key and Length of Message before Encryption Process
print("\nMessage:-", message)
print("Key:-", key)
print("Length of the Message:-", len(message))
# Declare New module for AES Encryption in 'ECB (Electronic Codebook)' Mode
cipher = AES.new(key, AES.MODE_ECB)
# Encrypt 'Message'
cipher_text = cipher.encrypt(message.encode('utf-8'))
# Print Encrypted Message
print("\nEncryption of Message Using AES:-", cipher_text)
print("Hex of Cipher Text:-", cipher_text.hex())
# Verify by decrypting Cipher Text whether you are recieving same message or not
decrypted_message = cipher.decrypt(cipher_text)
print("\nDecryption of Cipher Text Using AES:-", decrypted_message)
# Output of Above Code:-
Enter your Message for AES Encryption:- Stack Overflow
Message:- Stack Overflow
Key:- b'\x94\x88o\xf0\x8f\xbe\xec\x0e\x1e\xdf\x06A\xdf<\xbe\xe3'
Length of the Message:- 16
Encryption of Message Using AES:- b'\xf6c)\xee\xea\x13\xdcX\x9c\x06E\x82~{c\xc6'
Hex of Cipher Text:- f66329eeea13dc589c0645827e7b63c6
The decryption of Cipher Text Using AES:- b'Stack Overflow
Hope this Solution helps you.

Python, why does decrypt fails the second time

Below is a test. When I try to decode the same string for the second time I get a error message and I really have no clue why or where to search for. Please advise.
Error:
Traceback (most recent call last):
File "test.py", line 28, in <module>
telnr_to_string = str(telnr_decrypt, 'utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x9e in position 0: invalid start byte
Code:
from Crypto.Cipher import AES
from Crypto.Util.Padding import *
from urllib.request import urlopen
import os
import urllib
import subprocess
import re
import time
import base64
import random
file_key = open("key.txt")
key = file_key.read()
key = key.encode()
file_iv = open("iv.txt")
iv = file_iv.read()
iv = iv.encode()
obj2 = AES.new(key, AES.MODE_CBC, iv)
z=1
while z < 3:
x=''
telnr_decrypt=''
telnr=''
telnr_to_string=''
telnr_cleaned=''
x=(b'\x9e\xde\x98p:\xa3\x04\x92\xb5!2K[\x8e\\\xee')
telnr_decrypt = obj2.decrypt(x)
telnr_to_string = str(telnr_decrypt, 'utf-8')
telnr_cleaned = re.sub("[^0-9]", "", telnr_to_string)
print (telnr_cleaned)
z=z+1

Just move obj2 = AES.new(key, AES.MODE_CBC, iv) into the while loop. Don't worry, cipher objects are not very stateful objects; they don't do much other than to store the key (or, for AES, probably the internal subkeys) and maintain a small buffer. Generally it is therefore a bad idea to reuse or cache them.
If you call encrypt / decrypt multiple times in succession then the methods act as if you would be encrypting one big message. In other words, they allow you to encryption / decrypt in a piecemeal fashion so that message-sized buffers can be avoided. This is not very explicitly documented, but you can see this in figure 2 in the documentation.
For CBC mode it means that the operation is identical to setting the IV to the last block of ciphertext. If the IV is wrong then the first block of plaintext will be randomized. Random bytes do generally not contain valid UTF-8 encoding, which means that decoding to string will (likely) fail.

Encode in Laravel, decode in Python

I'm using Laravel's encryptString method to encrypt some data on my website. This uses OpenSSL's 256-bit AES-CBC encryption without any serialization. I'm now trying to decrypt that data in Python but I keep getting errors about key length and can't seem to figure out why.
Example data to decrypt: eyJpdiI6ImdxY0VcLzFodmpISFV4allSWmJDdEpRPT0iLCJ2YWx1ZSI6IkxXd0ZJaUd2bTUweW5pNm0wUjQwOFM2N1wvWEs5SlYrNB4xNlR7Qkh1U3FvPSIsIm1hYyI6Ijc5ZWM0YTYxYjljZGFiNzgwNjY2NDU1ZmQ5Yjc1ZmJlOGU4NzBkMjQzMzA3MmVhYzE3NzY4ZmU1MWIyMjZlOTQifQ==
Example Key to use for decryption (from laravel .env):
base64:/AZejP0lh3McL/+Vy5yZcADdTcR65qnx5Jqinuw7raK=
I changed those values around so actually decrypting with those values won't give any real data, just figured it'd be good for example. I then try to decrypt this data in Python 3.7 with:
import base64
from Crypto.Cipher import AES
def decrypt(enc, key):
IV = 16 * '\x00'
decobj = AES.new(key, AES.MODE_CBC, IV)
data = decobj.decrypt(base64.b64decode(enc))
print(str(data.decode()))
if __name__ == "__main__":
key = b"/AZejP0lh3McL/+Vy5yZcADdTcR65qnx5Jqinuw7raK="
decrypt("eyJpdiI6ImdxY0VcLzFodmpISFV4allSWmJDdEpRPT0iLCJ2YWx1ZSI6IkxXd0ZJaUd2bTUweW5pNm0wUjQwOFM2N1wvWEs5SlYrNB4xNlR7Qkh1U3FvPSIsIm1hYyI6Ijc5ZWM0YTYxYjljZGFiNzgwNjY2NDU1ZmQ5Yjc1ZmJlOGU4NzBkMjQzMzA3MmVhYzE3NzY4ZmU1MWIyMjZlOTQifQ==", key)
And it seems like this should work, but when I run it I get the error: ValueError: Incorrect AES key length (60 bytes) so I'm not sure what I'm doing wrong. I've tried padding/unpadding the data/key but that doesn't seem to change anything. I'm wondering if I'm getting the wrong key to use for decryption from Laravel, but from what I could tell in the linked documentation, it should just be the APP_KEY in my .env file.
If anyone could help me or point me in the right direction, that would be amazing!
This question is unique to other similar questions because I'm trying to figure out primarily if I'm getting the correct AES key from Laravel, I don't actually overly need help decrypting, I just think I'm grabbing the wrong key from Laravel.
EDIT: New code that seems like it works:
import base64
from Crypto.Cipher import AES
from Crypto.Util.Padding import pad, unpad
def decrypt(enc, key):
IV = 16 * '\x00'.encode()
decobj = AES.new(key, AES.MODE_CBC, IV)
data = decobj.decrypt(pad(base64.b64decode(enc), 16))
print(base64.b64decode(data))
if __name__ == "__main__":
key = base64.b64decode(b"/AZejP0lh3McL/+Vy5yZcADdTcR65qnx5Jqinuw7raK=")
decrypt("eyJpdiI6ImdxY0VcLzFodmpISFV4allSWmJDdEpRPT0iLCJ2YWx1ZSI6IkxXd0ZJaUd2bTUweW5pNm0wUjQwOFM2N1wvWEs5SlYrNB4xNlR7Qkh1U3FvPSIsIm1hYyI6Ijc5ZWM0YTYxYjljZGFiNzgwNjY2NDU1ZmQ5Yjc1ZmJlOGU4NzBkMjQzMzA3MmVhYzE3NzY4ZmU1MWIyMjZlOTQifQ==", key)
The print statement now prints some bytes, but when I run .decode() on it I get the error: UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfa in position 0: invalid start byte and can't seem to figure out what I need to do to make it be able to be printed as a string.

Question: ...trying to decrypt that data in Python but I keep getting errors about key length
I can use your key, in the code of the linked answer, after doing .b64decode(....
The example code .encode(... and decode(... works as expecte.
Conclusion: There is nothing wrong, with your Key!
key = b"/AZejP0lh3McL/+Vy5yZcADdTcR65qnx5Jqinuw7raK="
key = base64.b64decode(key)
But with your code, I got TypeError, related to the IV parameter:
expect_byte_string(iv)
File "/usr/local/lib/python3.4/dist-packages/Crypto/Util/_raw_api.py", line 172, in expect_byte_string
TypeError: Only byte strings can be passed to C code
Fixed with IV = 16 * '\x00'.encode(), results in ValueError, related to enc:
data = decobj.decrypt(base64.b64decode(enc))
File "/usr/local/lib/python3.4/dist-packages/Crypto/Cipher/_mode_cbc.py", line 209, in decrypt
ValueError: Error 3 while decrypting in CBC mode
This leads to github issues:10
Error 3 means "ERR_NOT_ENOUGH_DATA"
According to the linked GitHub page, you have to reread the documentation, about padding data while you are encoding.
Working example from GitHub:
import base64
from Crypto.Cipher import AES
from Crypto.Util.Padding import pad, unpad
key = b"/AZejP0lh3McL/+Vy5yZcADdTcR65qnx5Jqinuw7raK="
key = base64.b64decode(key)
BLOCK_SIZE = 32
encryption_suite = AES.new(key, AES.MODE_CBC, b'This is an IV...')
cipher_text = encryption_suite.encrypt(pad(b'A really secret message...', BLOCK_SIZE))
decryption_suite = AES.new(key, AES.MODE_CBC, b'This is an IV...')
print(unpad(decryption_suite.decrypt(cipher_text), BLOCK_SIZE).decode())
>>> A really secret message...
Tested with Python: 3.4.2

Full Working Code with MAC Validation
Adding to Gonzalo's submission, Laravel's encrypted message is a base64 json-encoded array consisting of the following key pairs:
[
'iv' => 'generated initialization vector (iv)',
'value' => 'encrypted, base64ed, signed value',
'mac' => 'message authentication code (mac)'
]
The 'value' is signed using a message authentication code (MAC) to verify that the value has not changed during transit.
The MAC extracted from the payload (encrypted message) should be checked against the
mac extracted from the 'value' (this can be done using the key, iv, and value). Laravel's encryption scheme can be found on GitHub: src/Illuminate/Encryption/Encrypter.php
Referencing a discussion thread on Laravel, I traced a partial solution on Github: orian/crypt.py (which is a fork of fideloper/crypt.py).
I have forked off of Orian's code and fixed an input parameter issue. The code works as expected as long as the encryption key (passed in as an input to decrypt()) is base64 decoded and does not include the 'base64:' that is typically prepended to the APP_KEY environmental variable string assignment found in the .env file.
Solution: pilatuspc12ng/crypt.py
The code snippet of the crypt.py is shown below:
import base64
import json
from Crypto.Cipher import AES
from phpserialize import loads
import hashlib
import hmac
def decrypt(payload, key):
"""
Decrypt strings that have been encrypted using Laravel's encrypter (AES-256 encryption).
Plain text is encrypted in Laravel using the following code:
>>> ciphertext = Crypt::encrypt('hello world');
The ciphertext is a base64's json-encoded array consisting of the following keys:
[
'iv' => 'generated initialization vector (iv)',
'value' => 'encrypted, base64ed, signed value',
'mac' => 'message authentication code (mac)'
]
The 'value' is signed using a message authentication code (MAC) so verify that the value has not changed during
transit.
Parameters:
payload (str): Laravel encrypted text.
key (str): Encryption key (base64 decoded). Make sure 'base64:' has been removed from string.
Returns:
str: plaintext
"""
data = json.loads(base64.b64decode(payload))
if not valid_mac(key, data):
return None
value = base64.b64decode(data['value'])
iv = base64.b64decode(data['iv'])
return unserialize(mcrypt_decrypt(value, iv, key)).decode("utf-8")
def mcrypt_decrypt(value, iv, key):
AES.key_size=128
crypt_object=AES.new(key=key,mode=AES.MODE_CBC,IV=iv)
return crypt_object.decrypt(value)
def unserialize(serialized):
return loads(serialized)
def valid_mac(key, data):
dig = hmac.new(key, digestmod=hashlib.sha256)
dig.update(data['iv'].encode('utf8'))
dig.update(data['value'].encode('utf8'))
dig = dig.hexdigest()
return dig==data['mac']

I notice this haven't been active for a while but I'm trying to do the same and can't seem to make it work.
What I noticed is that the encoded password string stored by Laravel is a base 64 encoded json object, which wasn't taken into consideration in the original question:
pass_obj = base64.b64decode('eyJpdiI6ImdxY0VcLzFodmpISFV4allSWmJDdEpRPT0iLCJ2YWx1ZSI6IkxXd0ZJaUd2bTUweW5pNm0wUjQwOFM2N1wvWEs5SlYrNB4xNlR7Qkh1U3FvPSIsIm1hYyI6Ijc5ZWM0YTYxYjljZGFiNzgwNjY2NDU1ZmQ5Yjc1ZmJlOGU4NzBkMjQzMzA3MmVhYzE3NzY4ZmU1MWIyMjZlOTQifQ==')
print(pass_obj)
>>> b'{"iv":"gqcE\\/1hvjHHUxjYRZbCtJQ==","value":"LWwFIiGvm50yni6m0R408S67\\/XK9JV+4\x1e16T{BHuSqo=","mac":"79ec4a61b9cdab780666455fd9b75fbe8e870d2433072eac17768fe51b226e94"}'
From that you can get the IV and the encrypted value, both seem to be base64 encoded. But I still get a decoding error at the end like;
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xfa in position 0: invalid start byte
Here is my complete code;
password = 'eyJpdiI6ImJGNDNNZjN3YWtpcDQ5VEJVXC9IazF3PT0iLCJ2YWx1ZSI6IkNVRW1VQUY1dXArYlFkU3NlY1pnZUE9PSIsIm1hYyI6ImM3ODk0NWQ0NjgxMzM4YjE0M2JhN2MzZWRmOWEwMWJiMjI2Y2FhYmUxYjFhYzAyYjY4YWZkZGE3N2EyMDYwNWYifQ=='
key = 'some secret key that i can't share'.encode()
p_obj = json.loads(base64.b64decode(password).decode())
decobj = AES.new(key, AES.MODE_CBC, base64.b64decode(p_obj['iv']))
data = decobj.decrypt(base64.b64decode(p_obj['value']))
print(data)
>>> b'l\xee:f\x9eZ\x90rP\x99\xca&#\x1d1\x9f'
data.decode()
>>> Traceback (most recent call last):
File "<stdin>", line 1, in <module>
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xee in position 1: invalid continuation byte
#Pecans did you ever figure this out?
Thank you.
Full working code EDIT
I fugured it out, I had a problem with my key initially. So here's the full snippet for future reference;
import base64
import json
from phpserialize import unserialize
from Crypto.Cipher import AES
from Crypto.Util.Padding import pad, unpad
key = b'my secret key'
enc_pass = 'eyJpdiI6ImJGNDNNZjN3YWtpcDQ5VEJVXC9IazF3PT0iLCJ2YWx1ZSI6IkNVRW1VQUY1dXArYlFkU3NlY1pnZUE9PSIsIm1hYyI6ImM3ODk0NWQ0NjgxMzM4YjE0M2JhN2MzZWRmOWEwMWJiMjI2Y2FhYmUxYjFhYzAyYjY4YWZkZGE3N2EyMDYwNWYifQ=='
p_obj = json.loads(base64.b64decode(password).decode())
decobj = AES.new(key, AES.MODE_CBC, base64.b64decode(p_obj['iv']))
data = decobj.decrypt(base64.b64decode(p_obj['value']))
dec_pass = unserialize(unpad(data, 16)).decode()
You will have the decrypted password in dec_pass.
Note that sometimes Laravel generates the key in base64. In that case the string will be something like base64:sdfsdjfhjsdf32, then you have to decode first.
Cheers!

Encryption of a JPG file using pycrypro's AES failing

Given below is the code(not complete yet) I have written to encrypt and decrypt files using python with the pycrypto module.
from Crypto.Hash import SHA256
from Crypto.Cipher import AES
import getpass
class ED(object):
def getfromuser(self,choice):
if choice=='key':
key=getpass.getpass('Enter AES Key (minimum 16 characters): ')
if len(key)<16:
print 'Key entered too short. Please try again.'
self.getfromuser(choice)
key=key+str(8-len(key)%8)*(8-len(key)%8)
return key
if choice=='IV':
IV_seed=raw_input('Enter a seed for the IV: ')
IV=SHA256.new()
IV.update(IV_seed)
IV.digest()
return str(IV)[0:16]
def AESEncrypt(self,key,IV,source,dest):
f=open(source,"r")
fstream=f.read()
f.close()
AES_stream=AES.new(key,AES.MODE_CBC,IV)
AES_encrypted=AES_stream.encrypt(fstream)
with open(dest,"w") as write_file:
write_file.write(AES_encrypted)
def AESDecrypt(self,key,IV,source,dest):
f=open(source,"r")
fstream=f.read()
f.close()
AES_stream=AES.new(key,AES.MODE_CBC,IV)
AES_decrypted=AES_stream.decrypt(fstream)
with open(dest,"w") as write_file:
write_file.write(AES_decrypted)
When I tried to encrypt a JPG file using this I got the following error:
AES_encrypted=AES_stream.encrypt(fstream)
ValueError: Input strings must be a multiple of 16 in length
I tried it on an mp4 file and it worked fine: the encryption and the decryption too.
What is the cause of this error and how do I fix it?

I found the solution. If one does not want the hassle of padding, one can use the CFB (Cipher feedback) mode as follows:
AES_stream=AES.new(key,AES.MODE_CFB,IV)