We Keep Coding

Is the output of the RSA algorithm for signing the same string the same at each time of encryption (signature) or does it change?

I am rewriting the codes written in C# with Python. In a part of the C# program, a string is signed with the RSA algorithm and sent to an API.
I have rewritten all parts of the C# program in Python, including the string signature section
The output in the signature of the C# program is the same as the signature of the Python program
But the APi in question does not confirm the signature generated by Python, even though the signature string in C# and Python is exactly the same.
I would be grateful if someone could help me
And whether the output of the RSA algorithm for signature is the same or changes for the same string at each time of encryption (signature)?
In Python, I use the following
from Cryptodome. Signature import pkcs1_15
from Cryptodome. Hash import SHA256
from Cryptodome.PublicKey import RSA
import base64
...
I rewrote the signature of a string that is in C# programming language in Python language (with RSA algorithm and key length of 2048) and it showed me the output of the same string from both programs (C# and Python).
When I send this signature string to an API, it does not receive an error and accepts the signature generated with C#, but it receives an error with the signature generated in Python and does not accept the signature.
What I expected was that since the signature output from both programming languages is the same (the same strings), the desired API should not have any problem with it.
Is my view wrong?
C#
public static string SignData(String stringToBeSigned, string privateKey)
{
var pem = "-----BEGIN PRIVATE KEY-----\n" + privateKey + "\n-----END PRIVATE KEY-----";
PemReader pr = new PemReader(new StringReader(pem));
AsymmetricKeyParameter privateKeyParams = (AsymmetricKeyParameter)pr.ReadObject();
RSAParameters rsaParams = DotNetUtilities.ToRSAParameters((RsaPrivateCrtKeyParameters)privateKeyParams);
RSACryptoServiceProvider csp = new RSACryptoServiceProvider();
csp.ImportParameters((RSAParameters)rsaParams);
var dataBytes = Encoding.UTF8.GetBytes(stringToBeSigned);
return Convert.ToBase64String(csp.SignData(dataBytes, HashAlgorithmName.SHA256, RSASignaturePadding.Pkcs1));
}
python :
key = RSA.import_key(open('rsa.private').read())
h = SHA256.new(normalize.encode('utf-8'))
signature = pkcs1_15.new(key).sign(h)
base64_bytes = base64.b64encode(signature)
base64_signature = base64_bytes.decode('ascii')
print(base64_signature)

Generated sha1 key in PBKDF2 Python do not match in .NET Rfc2898

I have a python code that generates a PBKDF2 sha1 hash of a password using the hashlib.pbkdf2_hmac method. Then I use that password digest in a dotnet framework 4.5 program to verify it against the same password. The C# program returns false which suggests that the hash produced from the python program is incorrect.
The key is in this format: #iterations|salt|key.
Then, I take that key and I try to verify it using a dotnet framework app using via method:
public static bool IsValid(string testPassword, string originalDelimitedHash)
{
//extract original values from delimited hash text
var originalHashParts = originalDelimitedHash.Split('|');
var origIterations = Int32.Parse(originalHashParts[0]);
var origSalt = Convert.FromBase64String(originalHashParts[1]);
var originalHash = originalHashParts[2];
//generate hash from test password and original salt and iterations
var pbkdf2 = new Rfc2898DeriveBytes(testPassword, origSalt, origIterations, HashAlgorithmName.SHA1);
byte[] testHash = pbkdf2.GetBytes(20);
var hashStr = Convert.ToBase64String(testHash);
if (hashStr == originalHash)
return true;
return false;
}
my python program:
from hashlib import pbkdf2_hmac
from base64 import b64encode
from os import urandom
def generate_password_hash(password:string):
encodedPass = password.encode('utf8')
random_bytes = urandom(20)
salt = b64encode(random_bytes)
iterations = 5000
key = pbkdf2_hmac('sha1', encodedPass, salt, iterations, dklen=20)
result = f'{iterations}|{salt.decode("utf-8")}|{binascii.hexlify(key).decode("utf-8")}'
return result
So if my password is hDHzJnMg0O the resulting digest from the above python method would be something like 5000|J5avBy0q5p9R/6cgxUpu6+6sW7o=|2445594504c9ffb54d1f11bbd0b385e3e37a5aca
So if I take that and supply it to my C# IsValid method (see below) it returns false which means the passwords do not match
static void Main(string[] args)
{
var pass = "hDHzJnMg0O";
var hash = "5000|J5avBy0q5p9R/6cgxUpu6+6sW7o=|2445594504c9ffb54d1f11bbd0b385e3e37a5aca";
var isValid = IsValid(pass, hash); // returns False
}

The Python code:
uses b64encode(random_bytes) as salt for the PBKDF2 call. This is rather unusual (but not a bug). Typically the raw data, i.e. random_bytes, is applied as salt and passed to the PBKDF2 call. With the Base64 encoding only the string would be created.
hex encodes the key (i.e. the return value of the PBKDF2 call).
The C# code is different in these points and:
uses the raw data (i.e. random_bytes from the Python side) for the PBKDF2 call, i.e. the salt from the Python side is Base64 decoded.
Base64 encodes the key (i.e. the return value of the PBKDF2 call)
Changes in the C# code for compatibility with the Python code (of course the changes could also be made in the Python code, but the Python code seems to be the reference):
...
var origSalt = Encoding.UTF8.GetBytes(originalHashParts[1]); // Convert.FromBase64String(originalHashParts[1]);
...
var hashStr = Convert.ToHexString(testHash); // Convert.ToBase64String(testHash);
...
For the latter, Convert.ToHexString() was used, which is available since .NET 5. For other .NET versions see e.g. here.
Furthermore, since the hex encoded values are compared and the different implementations are not standardized regarding lower (e.g. binascii.hexlify(key)) and upper case letters (e.g. Convert.ToHexString(testHash)), it is more robust to convert both strings uniformly, e.g.:
if (hashStr.ToUpper() == originalHash.ToUpper())
return true;
With these changes, validation with the C# code works.
Edit (with regard to the change in the Python code addressed in the comment):
If in the Python code random_bytes is used as salt and the salt is Base64 encoded for concatenation, then in the C# code the Base64 encoded salt must be Base64 decoded again (as in the original C# code).

How to limit key length with Passlib 1.7+

A server protocol requires me to derive a password hash with a limited key size. This is the given JavaScript + CryptoJS implementation:
var params = {keySize: size/32, hasher: CryptoJS.algo.SHA512, iterations: 5000}
var output = CryptoJS.PBKDF2(password, salt, params).toString();
I want to re-implement this in Python using Passlib, i.e. something like
from passlib.hash import pkbdf2_sha512
output = pbkdf2_sha512.hash(password, salt=salt, rounds=5000)
The Passlib API does not allow me to specify the key size. How to do it though?

If the derived key it to long just truncate it to the required length. Each byte is just as valid as every other byte, it makes no difference which bytes you use, there is no ordering.

Django PyCrypto - Save Encrypted String to Database Bad Unicode Data

I'm working with PyCrypto in Django and I need to encrypt a string using the user's secret key they made themselves. I successfully wrote an encryption method as follows:
from Crypto.Cipher import AES
from Crypto.Random import get_random_string
def encrypt(value, key):
"""
Return an encryption of value under key, as well as IV.
Pads value with extra bytes to make it multiple of 16.
"""
extra = 16 - (len(value) % 16)
data = value + chr(extra) * extra
iv = get_random_bytes(16)
encryption_suite = AES.new(key, AES.MODE_CBC, iv)
cipher_text = encryption_suite.encrypt(data)
return cipher_text, iv
Why am I not using Django's encryptions? Because there is a client application that is NOT written in Django (and won't ever be) that accepts the encrypted value the user stored previously and decrypts it once the user enters their secret key.
Problem is that I can't seem to save the encrypted value to the database for the User model. For example:
user = User.objects.get(id=user_id)
cipher, iv = encrypt(user_value, user_key)
user.secret_value = cipher
user.iv = iv
user.save()
This results in this error:
Warning: Incorrect string value: '\xE7\xAA\x13\x036\xC8...' for column 'iv' at row 1
(same error for secret_value)
I know this must be something to do with improper encoding. What's the right way to go about fixing this? Should I convert each byte into a string character?
Thanks.

I guess you're trying to save binary data into CharFields. Either change field types of user.iv and user.secret_value to BinaryField, or encode these values using for example base64 encoder.

Python AES encoding/decoding string and storing in MySQL

I'm working on a project using Pyramid 1.3 (Python 2.7) and storing data in MySQL. I have a table of email addresses, and I would like to encrypt them for storage. I am trying to encrypt them in the application, and then will decrypt them for viewing. I'm not going for complete security but am mainly aiming to obfuscate the data enough were the database itself compromised.
I'm using PyCrypto with AES, and have been trying to follow some posts on here and some web tutorials I found. The closest I found so far is this post, and it seems to work, at least encrypting it. I follow that and get something like "7hBAQrWhJRnL9YdBGJfRErGFwGi3aC6noGzYTrGwAoQ=" stored in the database. But the decrypt function keeps erroring with this:
UnicodeDecodeError: 'ascii' codec can't decode byte 0xa1 in position 1: ordinal not in range(128)
I came across some unicode presentation about Python which sort of helped me make more sense of it but I still keep getting the same error.
Is there a straightforward tutorial on how to encode, store in a database, pull out of database, and decode a source data string?
Do I need a specific collation on the database column? Does the field need to be a certain type? So far I've been using a default collation and setting it to VARCHAR, assuming that I was storing a string. It sounds like I've got some encoding problem somewhere with incompatible types or something but my head is spinning on where I need to change something.
Any better pointers or anything else I can provide? I can show my code but its basically a copy of the link above... I was just trying to get a proof of concept working before modifying it too much.
edit:
some sample source...
In MySQL, the table is
id (int)
client_id (int)
emailaddress varchar(100) utf8mb4_general_ci (I've been playing around with the collations, I have no idea what it should be!)
Python:
from base64 import b64encode, b64decode, urlsafe_b64decode, urlsafe_b64encode
BLOCK_SIZE = 32
INTERRUPT = u'\u0001'
PAD = u'\u0000'
def AddPadding(data, interrupt, pad, block_size):
new_data = ''.join([data, interrupt])
new_data_len = len(new_data)
remaining_len = block_size - new_data_len
to_pad_len = remaining_len % block_size
pad_string = pad * to_pad_len
return ''.join([new_data, pad_string])
def StripPadding(data, interrupt, pad):
return data.rstrip(pad).rstrip(interrupt)#data.rsplit(interrupt,1)[0]#rstrip(pad).rstrip(interrupt)
SECRET_KEY = u'a1b2c3d4e5f6g7h8a1b2c3d4e5f6g7h8'
IV = u'12345678abcdefgh'
cipher_for_encryption = AES.new(SECRET_KEY, AES.MODE_CBC, IV)
cipher_for_decryption = AES.new(SECRET_KEY, AES.MODE_CBC, IV)
def EncryptWithAES(encrypt_cipher, plaintext_data):
plaintext_padded = AddPadding(plaintext_data, INTERRUPT, PAD, BLOCK_SIZE)
encrypted = encrypt_cipher.encrypt(plaintext_padded)
return urlsafe_b64encode(encrypted)
def DecryptWithAES(decrypt_cipher, encrypted_data):
decoded_encrypted_data = urlsafe_b64decode(encrypted_data)
decrypted_data = decrypt_cipher.decrypt(decoded_encrypted_data)
return StripPadding(decrypted_data, INTERRUPT, PAD)
#encrypts it
posted_singleaddress = EncryptWithAES(cipher_for_encryption, posted_singleaddress)
#"me#mail.com" inserts "Ktpr49Uzn99HZXbmqEzGKlWo9wk-XBMXGZl_iyna-8c=" into the database
clientemails is the list of emails from the table above. I get the error when uncommenting out:
#if clientemails:
# decrypted = DecryptWithAES(cipher_for_decryption, clientemails[0].emailaddress)
I was just trying to decode the first item just to try and get it to work but that's the part that seems to be giving it fits now....

The general rule with PyCrypto is that cryptographic keys, IVs, plaintexts, paddings, and ciphertexts should always be defined as binary strings, not text. The fact you use Unicode for them is by itself a source of problems.
Another problems is that you pass to AES.new key and IV in hexadecimal encoded form, so that the former is 256 bits and the latter 128 bits. That seems still to work, but I guess your intention was to use AES128 - which has a 128 bit key. You therefore need to convert it to binary, for instance via unhexlify: the two character string b'34' will map to the single byte '\x34'. The IV needs to be twice as long.
In your code it's therefore better to have:
from binascii import unhexlify
INTERRUPT = b'\x01'
PAD = b'\x00'
SECRET_KEY = unhexlify('a1b2c3d4e5f6g7h8a1b2c3d4e5f6g7h8')
IV = unhexlify('12345678abcdefgh'*2)
If you need to encrypt text, you would first encode it (e.g. to UTF-8) and then pass it to your function EncryptWithAES(). See also this example taken from the PyCrypto API:
from Crypto.Cipher import AES
from Crypto import Random
key = b'Sixteen byte key'
iv = Random.new().read(AES.block_size)
cipher = AES.new(key, AES.MODE_CFB, iv)
msg = iv + cipher.encrypt(b'Attack at dawn')
The result of the encryption step (that is, the ciphertext) is again a binary string. In order to store it directly in the MySQL DB you must use either a BINARY or a VARBINARY type column.