Okay, so basically I am having issues decrypting with Python.
I've managed to encrypt/decrypt data with Node.js - using "aes-128-ctr", the same goes for PyCrypto, but when I try to encrypt with Node.js and decrypt with Python I get invalid deciphered text.
Node.js code:
var key = "1234567890123456";
var cipher = crypto.createCipher("aes-128-ctr",key)
var ctext = cipher.update('asasasa','utf8','hex') + cipher.final('hex')
console.log(ctext) // outputs: "f2cf6ecd8f"
Python code:
counter = Counter.new(128)
cipher = AES.new("1234567890123456", AES.MODE_CTR, counter=counter)
cipher.decrypt("f2cf6ecd8f") // outputs: weird encoding characters
By the way, I don't care about the level of security of this encryption, I care about performance more.
crypto.createCipher takes a password and EVP_BytesToKey to derive a key and IV from that, but pycrypto directly expects a key and IV. You need to use exactly the same procedure.
crypto.createCipher must never be used with CTR-mode, because the key and IV generation are not randomized. Since the CTR-mode is a streaming mode, it will always produce the same key stream which might enable an attacker who only observes multiple ciphertexts that are encrypted with the same password to deduce the plaintext. This is possible because of the resulting many-time pad issue.
If you must use CTR-mode, then you have to use crypto.createCipheriv. If you use the same key, you have to use a different IV every time. This is why this is actually called a nonce for CTR-mode. For AES-CTR, a nonce of 96 bit is a good compromise between security and size of possibly encryptable plaintexts.
var key = "1234567890123456"
var iv = Buffer.concat([crypto.randomBytes(12), Buffer.alloc(4, 0)])
var cipher = crypto.createCipheriv("aes-128-ctr", key, iv)
var ctext = iv.toString('hex') + cipher.update('asasasa','utf8','hex') + cipher.final('hex')
console.log(ctext)
Example output:
5b88aeb265712b6c8bfa8dbd0000000063012d1e52eb42
The IV is not secret and you have to use the exact same IV during decryption. Usually, it is sent along with the ciphertext by being prefixed to it. It is then sliced off before decryption:
ct = codecs.decode('5b88aeb265712b6c8bfa8dbd0000000063012d1e52eb42', 'hex') # I'm using Python 3
counter = Counter.new(32, prefix=ct[:12], initial_value=0)
cipher = AES.new("1234567890123456", AES.MODE_CTR, counter=counter)
cipher.decrypt(ct[16:])
Output:
b'asasasa'
Keep in mind that a key needs to be randomly chosen. You can generate a random key and keep it in an encoded form in the source code (i.e. as Hex). If you do that, you must not give the source code or the bytecode to anyone that you wouldn't trust the key with.
Related
I am working on converting the encryption function of AES/GCM-256 in C# to Python.
i found the following code in Python and i am using it, but the problem is that my data in the Python function is the same as the C# data.
The output string (encoded) produced by my Python function is not the same as the output of the C# function, although my inputs (key, iv and data) are the same in both functions.
I would be grateful if someone could help me
My keyis: b'4fda3c622e966e0839441401bbd3b8f191d4267bf5f19b40812a34b212fd3ed9'
My iv is: b'4fda3c622e966e0839441401bbd3b8f191d4267bf5f19b40812a34b212fd3ed9'
C# function
public static string AesEncrypt(byte[] payload, byte[] key, byte[] iv)
{
var cipher = new GcmBlockCipher(new AesEngine());
byte[] baPayload = new byte[0];
cipher.Init(true, new AeadParameters(new KeyParameter(key), 128, iv, baPayload));
var cipherBytes = new byte[cipher.GetOutputSize(payload.Length)];
int len = cipher.ProcessBytes(payload, 0, payload.Length, cipherBytes, 0);
cipher.DoFinal(cipherBytes, len);
return Convert.ToBase64String(cipherBytes);
}
It can be converted to bytes using the following function PASSPHRASE in C#:
public static byte[] StringToByteArray(string hex)
{
return Enumerable.Range(0, hex.Length)
.Where(x => x % 2 == 0)
.Select(x => Convert.ToByte(hex.Substring(x, 2), 16))
.ToArray();
}
python:
PASSPHRASE= b'a4b42ed2702cb1b00a14f39a88c719cb04e5e8b29e2479634c990258e327483'
def AES_encrypt(data,iv):
global PASSPHRASE
data_json_64 = data
key = binascii.unhexlify(PASSPHRASE)
cipher = AES.new(key, AES.MODE_GCM, iv)
x = cipher.encrypt(data)
return x
Because all my data is the same, I expect my output to be the same, but it is not
My input test string to both C# and Python is::
"In publishing and graphic design, Lorem ipsum is a"
My iv to both C# and Python is:
'4fda3c622e966e0839441401bbd3b8f191d4267bf5f19b40812a34b212fd3ed9'
The encoded output in C# is:
02Em9Vve6fWtAcVNesIXzagoB327EmskwMZdRippAAaxqAzkp0VeGSjctbaguqA/01CnPHB2PkRDDOxjgZ9pAfu2
The encoded output in Python is:
HudpKzIov7lNt4UNng+a9P/FLXrzdenwDBT4uFYhIUc3XOS7TpaCzxja8I+zHCdXnvk=
After implementing suggested changes from your comments and some minor tweaks to your code, it is possible to reproduce your C# result. The main problem was the handle of a ciphertext and a tag. As #Topaco has stated in the comments, C# implicitly concatenates ciphertext and tag as ciphertext|tag, which will mean you need to do the same in your code. If you follow the GCM documentation you will see that you need to use encrypt_and_digest() function in order to obtain the ciphertext and the tag. Here is what you need to change:
cipher = AES.new(key, AES.MODE_GCM, iv) → cipher = AES.new(key, AES.MODE_GCM, binascii.unhexlify(iv))
x = cipher.encrypt(data) → x, key = cipher.encrypt_and_digest(data)
return x → return x + key
These changes should help you fix your code. I also have some other suggestions. I wouldn't use global variables in your code. For starters, this will mean that your function should be AES_encrypt(data, key, iv) and not AES_encrypt(data, iv). Why are global variables a bad idea? In this particular case it makes no sense you use a global variable for password as you shouldn't use the same password for encryption of everything. Additionally, when talking about encryption, you will see that every algorithm is using a key which is not the same as a password. Here you are implementing the password based encryption. However, you are not doing it correctly, as you are using a password (passphrase) as a cryptographic key. The idea behind a password based encryption is to use a good digest/key stretching technique to create a key from your password. Finally, you should take a look at the NIST Special Publication 800-38D to learn more about the GCM and NIST's suggested parameters when it comes to implementing a safe encryption.
I am trying to use python 3.7.3 to encrypt a given plaintext with a given key and a given iv. I am unfamiliar with python encryption libraries but have looked around for a couple hours trying to figure this out, and I am unable to.
I have tired using the pycryptodome library as I have been unable to use crypto and pycrypto (keep getting errors when I install them, maybe because I am on windows?).
from Crypto.Cipher import AES
from Crypto.Util import Counter
data = b"plaintext"
key = b"5A557AC90890B2ACD59C536FE4279BBC"
iv = b"5A557AC90890B2ACD59C536FE4279BBC"
counter = Counter.new(128, initial_value=iv)
cipher = AES.new(key, AES.MODE_CTR, counter=counter)
enc = cipher.encrypt(data)
print(enc)
My end result is to print out the encrypted string, but currently I am getting various errors such as "not supported between instances of 'bytes' and 'int'"
Your code does not use Counter correctly. You are trying to use hexidecimal where an integer should be, and even if you converted your iv value to an integer it would be too large.
initial_value (integer) - The initial value of the counter. Default
value is 1.
So changing the code to the following would work for example:
counter = Counter.new(128, initial_value=1)
If you want to randomize the iv you could do something such as:
iv = int.from_bytes(os.urandom(16), sys.byteorder)
counter = Counter.new(128, initial_value=iv)
I wrote an application in Nodejs that encrypts user passwords using AES-256-CTR :
const crypto = require('crypto')
const masterkey = 'azertyuiopazertyuiopazertyuiopaz'
const cipher = crypto.createCipher('aes-256-ctr', masterkey)
console.log(cipher.update('antoine', 'utf8', 'hex') + cipher.final('hex')) //=> 6415bc70ad76c6
It then gets persisted into a database and now I'm trying to decipher it from a Python script using PyCrypto like this :
masterkey = 'azertyuiopazertyuiopazertyuiopaz'
password = '6415bc70ad76c6'
from Crypto.Cipher import AES
import os
import binascii
counter = os.urandom(16)
# counter = bytes(16) # does not work
# counter = masterkey[0:16].encode() # does not work
cipher = AES.new(masterkey, AES.MODE_CTR, counter=lambda: counter)
print(cipher.decrypt(binascii.a2b_hex(password)))
But it gives me completely wrong results here.
Do you know what I am missing ?
EDIT
Thanks to zaph, it appears that the way my Javascript code encrypts data is insecure. I still have to figure out what IV is being used internally by Node. I've tried many without success
masterkey[0:16].encode()
bytes(16)
Update based on new information in the question: The best bet is that Nodejs is using a default counter value.
The same counter value must be used for both encryption and decryption. But no counter value is provided on encryption and a random value is used on decryption so it can never work.
Use: crypto.createCipheriv(algorithm, key, iv) where iv is the random counter initial value.
It is necessary to create a random counter value on encryption and save it so that the same initial counter value can be used on decryption. One option is to prefix the encrypted data with the counter value, it does not need to be secret. Then on decryption it can be split from the encrypted data and used.
Also when using CTR mode the same initial counter value must never be use again with the same key.
See CTR mode
PyCrypto documentation CTR mode:
MODE_CBC
Cipher-Block Chaining (CBC). Each of the ciphertext blocks depends on the current and all previous plaintext blocks. An Initialization Vector (IV) is required.
The IV is a data block to be transmitted to the receiver. The IV can be made public, but it must be authenticated by the receiver and it should be picked randomly.)
The IV is the initial counter value.
[Nodejs dociumewnrtation: Class: Cipher:
crypto.createCipheriv(algorithm, key, iv)
algorithm <string>
key <string> | <Buffer> | <TypedArray> | <DataView>
iv <string> | <Buffer> | <TypedArray> | <DataView>
Creates and returns a Cipher object, with the given algorithm, key and initialization vector (iv).
I'm working with PyCrypto in Django and I need to encrypt a string using the user's secret key they made themselves. I successfully wrote an encryption method as follows:
from Crypto.Cipher import AES
from Crypto.Random import get_random_string
def encrypt(value, key):
"""
Return an encryption of value under key, as well as IV.
Pads value with extra bytes to make it multiple of 16.
"""
extra = 16 - (len(value) % 16)
data = value + chr(extra) * extra
iv = get_random_bytes(16)
encryption_suite = AES.new(key, AES.MODE_CBC, iv)
cipher_text = encryption_suite.encrypt(data)
return cipher_text, iv
Why am I not using Django's encryptions? Because there is a client application that is NOT written in Django (and won't ever be) that accepts the encrypted value the user stored previously and decrypts it once the user enters their secret key.
Problem is that I can't seem to save the encrypted value to the database for the User model. For example:
user = User.objects.get(id=user_id)
cipher, iv = encrypt(user_value, user_key)
user.secret_value = cipher
user.iv = iv
user.save()
This results in this error:
Warning: Incorrect string value: '\xE7\xAA\x13\x036\xC8...' for column 'iv' at row 1
(same error for secret_value)
I know this must be something to do with improper encoding. What's the right way to go about fixing this? Should I convert each byte into a string character?
Thanks.
I guess you're trying to save binary data into CharFields. Either change field types of user.iv and user.secret_value to BinaryField, or encode these values using for example base64 encoder.
I'm working on a project using Pyramid 1.3 (Python 2.7) and storing data in MySQL. I have a table of email addresses, and I would like to encrypt them for storage. I am trying to encrypt them in the application, and then will decrypt them for viewing. I'm not going for complete security but am mainly aiming to obfuscate the data enough were the database itself compromised.
I'm using PyCrypto with AES, and have been trying to follow some posts on here and some web tutorials I found. The closest I found so far is this post, and it seems to work, at least encrypting it. I follow that and get something like "7hBAQrWhJRnL9YdBGJfRErGFwGi3aC6noGzYTrGwAoQ=" stored in the database. But the decrypt function keeps erroring with this:
UnicodeDecodeError: 'ascii' codec can't decode byte 0xa1 in position 1: ordinal not in range(128)
I came across some unicode presentation about Python which sort of helped me make more sense of it but I still keep getting the same error.
Is there a straightforward tutorial on how to encode, store in a database, pull out of database, and decode a source data string?
Do I need a specific collation on the database column? Does the field need to be a certain type? So far I've been using a default collation and setting it to VARCHAR, assuming that I was storing a string. It sounds like I've got some encoding problem somewhere with incompatible types or something but my head is spinning on where I need to change something.
Any better pointers or anything else I can provide? I can show my code but its basically a copy of the link above... I was just trying to get a proof of concept working before modifying it too much.
edit:
some sample source...
In MySQL, the table is
id (int)
client_id (int)
emailaddress varchar(100) utf8mb4_general_ci (I've been playing around with the collations, I have no idea what it should be!)
Python:
from base64 import b64encode, b64decode, urlsafe_b64decode, urlsafe_b64encode
BLOCK_SIZE = 32
INTERRUPT = u'\u0001'
PAD = u'\u0000'
def AddPadding(data, interrupt, pad, block_size):
new_data = ''.join([data, interrupt])
new_data_len = len(new_data)
remaining_len = block_size - new_data_len
to_pad_len = remaining_len % block_size
pad_string = pad * to_pad_len
return ''.join([new_data, pad_string])
def StripPadding(data, interrupt, pad):
return data.rstrip(pad).rstrip(interrupt)#data.rsplit(interrupt,1)[0]#rstrip(pad).rstrip(interrupt)
SECRET_KEY = u'a1b2c3d4e5f6g7h8a1b2c3d4e5f6g7h8'
IV = u'12345678abcdefgh'
cipher_for_encryption = AES.new(SECRET_KEY, AES.MODE_CBC, IV)
cipher_for_decryption = AES.new(SECRET_KEY, AES.MODE_CBC, IV)
def EncryptWithAES(encrypt_cipher, plaintext_data):
plaintext_padded = AddPadding(plaintext_data, INTERRUPT, PAD, BLOCK_SIZE)
encrypted = encrypt_cipher.encrypt(plaintext_padded)
return urlsafe_b64encode(encrypted)
def DecryptWithAES(decrypt_cipher, encrypted_data):
decoded_encrypted_data = urlsafe_b64decode(encrypted_data)
decrypted_data = decrypt_cipher.decrypt(decoded_encrypted_data)
return StripPadding(decrypted_data, INTERRUPT, PAD)
#encrypts it
posted_singleaddress = EncryptWithAES(cipher_for_encryption, posted_singleaddress)
#"me#mail.com" inserts "Ktpr49Uzn99HZXbmqEzGKlWo9wk-XBMXGZl_iyna-8c=" into the database
clientemails is the list of emails from the table above. I get the error when uncommenting out:
#if clientemails:
# decrypted = DecryptWithAES(cipher_for_decryption, clientemails[0].emailaddress)
I was just trying to decode the first item just to try and get it to work but that's the part that seems to be giving it fits now....
The general rule with PyCrypto is that cryptographic keys, IVs, plaintexts, paddings, and ciphertexts should always be defined as binary strings, not text. The fact you use Unicode for them is by itself a source of problems.
Another problems is that you pass to AES.new key and IV in hexadecimal encoded form, so that the former is 256 bits and the latter 128 bits. That seems still to work, but I guess your intention was to use AES128 - which has a 128 bit key. You therefore need to convert it to binary, for instance via unhexlify: the two character string b'34' will map to the single byte '\x34'. The IV needs to be twice as long.
In your code it's therefore better to have:
from binascii import unhexlify
INTERRUPT = b'\x01'
PAD = b'\x00'
SECRET_KEY = unhexlify('a1b2c3d4e5f6g7h8a1b2c3d4e5f6g7h8')
IV = unhexlify('12345678abcdefgh'*2)
If you need to encrypt text, you would first encode it (e.g. to UTF-8) and then pass it to your function EncryptWithAES(). See also this example taken from the PyCrypto API:
from Crypto.Cipher import AES
from Crypto import Random
key = b'Sixteen byte key'
iv = Random.new().read(AES.block_size)
cipher = AES.new(key, AES.MODE_CFB, iv)
msg = iv + cipher.encrypt(b'Attack at dawn')
The result of the encryption step (that is, the ciphertext) is again a binary string. In order to store it directly in the MySQL DB you must use either a BINARY or a VARBINARY type column.