I'm creating a script that communicates with a server that is using PGP encryption to encrypt it's responses.
I can request the public key of the server, and use that key to encrypt my request via the PGPy library, and the server is able to decrypt it and respond.
The scripts encrypted request has it's public key in it, which the server is expecting as a part of it's API, so it takes that key and encrypts the response and sends it back to my script. My script receives this encrypted response fine, but PGPy appears to lack the facilities to decrypt it.
From what I can tell, PGPy can only decrypt a message that it encrypted because the the PGPKey.decrypt() takes a PGPMessage object, not an armored string, and I cannot find any method that takes an armored string and either decrypts it or turns it into a PGPMessage so that I can decrypt it.
This is as far as I've gotten:
def get_token_from_payload(encrypted_payload):
print("Payload: ", encrypted_payload)
privkey, _ = pgpy.PGPKey.from_file('private.key')
message_blob = pgpy.PGPMessage.new(encrypted_payload)
token_string = privkey.decrypt(message_blob).message
print("Token: ", token_string)
return json.load(token_string)
The message_blob = pgpy.PGPMessage.new(encrypted_payload) that I thought would do the right thing, doesn't, and instead makes it's own encrypted text blob that the line token_string = privkey.decrypt(message_blob).message simply turns back into the armored ascii string I started with. Of course the last line fails because there is no json to do anything with so we'll ignore that for right now as that should work just fine once PGPy is decrypting things.
I must be missing something. This library is essentially useless if it can't do this. How do I either decrypt the armored string or convert the armored string into a PGPMessage that can then be decrypted?
Update:
def get_token_from_payload(encrypted_payload):
ep_file = open("ep.pgp", "w")
ep_file.write(encrypted_payload)
ep_file.close()
privkey, _ = pgpy.PGPKey.from_file('private.key')
message = pgpy.PGPMessage.from_file("ep.pgp")
token_string = privkey.decrypt(message).message
return json.loads(token_string)
This solution does work, but I don't find it acceptable as it requires an extraneous set of IO steps, creating a file and then reading it right back in. I already have the armored ascii message in memory, I just want to decrypt it directly.
You should be able to create a PGPMessage object from the ascii armored string in memory by using pgpy.PGPMessage.from_blob()
def get_token_from_payload(encrypted_payload):
print("Payload: ", encrypted_payload)
privkey, _ = pgpy.PGPKey.from_file('private.key')
message_blob = pgpy.PGPMessage.from_blob(encrypted_payload)
token = privkey.decrypt(message_blob).message
print("Token: ", token)
return json.load(token)
You might also need to convert the token value from a bytearray to a string. Something like this should work:
token_string = str(token, 'UTF-8')
print("Token: ", token_string)
return json.load(token_string)
Related
I'm accessing an API that returns the following public key:
"publicKey": "MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEArZj/8FWa9e2PmHIBzdMwA/Wo5HYyOHBOxORU5bVBOsb8ZJekhgNWplZxskpuMx1GC9m0WTvCHK+lLmlxKyOomu85q7MxocM8n7iF8Cc0Qrgjushut35FM1bT36em46eCCuO4WqG9/GhCsUeLTsQFBTUxF2Zk6++EJcmBgwU1yNvFZNUScfTmNSMpOcnWlGgt0GpOCdsx8GECOgZhwkJFDnUa01k4BeHYDJEufgNkq4lXh8wxep03S6RyZIAye9zDTaGhGvA5+loQq8bBWCbBzNTJWNhn1kpsnPQJHFcugLMYUyglzxk6phy1Et/s1ANH8H8jdRojhoJEVjg7+Y0JwwIDAQAB"
I need to use this public key to encrypt a password and send it to another endpoint in Base64 format.
What I need to do is exactly what this website does: https://www.devglan.com/online-tools/rsa-encryption-decryption
I've followed several tutorials, asked gpt chat for help and tested it in several ways but I can't.
When I encrypt the password using the website above and call the API directly through Postman it works, but when I encrypt it via Python it doesn't work. It says the encrypted password was not recognized.
My code:
#staticmethod
def get_pem_format(publickey: str):
lenkey = int(len(publickey) / 64)
key = ""
for x in range(0, lenkey+1):
startpos = x*64
endpos = startpos + 64
keylen = str(publickey[startpos:endpos])
key += f"{keylen}\n" if len(keylen) == 64 else keylen
key = f"-----BEGIN PUBLIC KEY-----\n{key}\n-----END PUBLIC KEY-----"
return key
def get_rsa_credentials(self, sourceid, newpass):
userkeyinfo = self._query_password_info(sourceid)
# Save public key
with open('/tmp/public_key.pem', 'w') as f:
f.write(self.get_pem_format(userkeyinfo["publicKey"]))
# Get public key
with open('/tmp/public_key.pem', 'rb') as f:
publickey = RSA.importKey(f.read())
cipher = PKCS1_OAEP.new(publickey)
encryptedpass = cipher.encrypt(newpass.encode())
base64encryptedpass = base64.b64encode(encryptedpass).decode()
return base64encryptedpass, userkeyinfo["publicKeyId"]
newpass = "Oliveir4souz#"
sourceid = "2c9180878168627f018192ff06f66ccb"
ecryptedpass, publickeyid = self.get_rsa_credentials(sourceid, newpass)
The _query_password_info method is where I call the api and get the public key.
And I created this method get_pem_format that generates the file in pem format, because all the libraries I found only carry the key of a file.
The code above does not generate any errors. But the encrypted value is not valid. But when I use the website as in the image below, it works perfectly in the API call.
I cannot understand what I am doing wrong.
The problem is caused by different paddings: With RSA (just like with RSA/ECB/PKCS1Padding) in the Cipher Type field, the website applies PKCS#1 v1.5 padding. Since the ciphertext generated with the website can be successfully decrypted by the endpoint, the endpoint obviously uses PKCS#1 v1.5 padding as well.
However, in the Python code, cipher = PKCS1_OAEP.new(publickey) specifies OAEP as padding, which is why the Python code on the one hand and the website (or the endpoint) on the other hand are incompatible. To fix the problem, cipher = PKCS1_v1_5.new(publickey) must be used in the Python code so that PKCS#1 v1.5 padding is applied there as well. With this change, the ciphertext generated with the Python code can be successfully decrypted using the website with RSA in the Cipher Type field (and should also be successfully decrypted by the endpoint).
For completeness: PKCS1_OAEP() applies OAEP as padding using the default SHA-1 for content digest and MGF1 digest, which is equivalent to the RSA/ECB/OAEPWithSHA-1AndMGF1Padding option of the website.
As side note: PyCryptdome supports import and export of keys in different formats and encodings. This way you can convert your Base64 encoded ASN.1/DER encoded key (in X.509/SPKI format) into a PEM encoded key as follows:
key = RSA.import_key(base64.b64decode(publickey)).exportKey(format='PEM').decode('utf8')
where publickey is: MIIBIj....
Regarding the posted screenshot on encryption with the website: Keep in mind that RSA encryptions (both with OAEP and PKCS#1 v1.5) are non-deterministic, i.e. encryptions with the same key and plaintext generate different ciphertexts. Therefore, it is not an indication of an error if tests with identical key and plaintext result in different ciphertexts.
file_path = 'upload/abc.tar'
in source=source_plaintext I want to give a file but how to do this
if I assign file_path then it will take as a string
kms_kwargs = dict(key_ids=[key_arn])
if botocore_session is not None:
kms_kwargs['botocore_session'] = botocore_session
master_key_provider = aws_encryption_sdk.KMSMasterKeyProvider(**kms_kwargs)
# Encrypt the plaintext source data
ciphertext, encryptor_header = aws_encryption_sdk.encrypt(
source=source_plaintext,
key_provider=master_key_provider
)
I assume you are using the provided aws documentation
If you want to encrypt the file itself, you may check the part Encrypting and Decrypting Byte Streams, there you can encrypt whole file (any stream).
The part you are trying is to encrypt a single string. This is used for encypting data encryption key. Basic idea is that you can generate a random data encryption key, encrypt data using the data key and then encrypt the data key with the master (kms) key.
My server encrypts files using pycrypto with AES in CTR mode. My counter is a simple counter like this:
\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01
\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x02
\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x03
I wanna decrypt the cipher text with c++'s cryptopp library in my clients. How should I do so?
Python code:
encryptor = AES.new(
CRYPTOGRAPHY_KEY,
AES.MODE_CTR,
counter=Counter.new(128),
)
cipher = encryptor.encrypt(plain_text)
C++ code so far:
byte ctr[] = "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x01"
mDecryptor = new CryptoPP::CTR_Mode<CryptoPP::AES>::Decryption(key, 32, ctr);
std::string plain;
CryptoPP::StringSource(std::string(data, len), true, new CryptoPP::StreamTransformationFilter(*mDecryptor, new CryptoPP::StringSink(plain)));
but after running this plain is garbage.
Update:
Sample encrypted data you can try to decrypt with crypto++ so that you can help me even if you don't know python and you're just experienced with crypto++:
Try to decrypt this base64 encoded text:
2t0lLuSBY7NkfK5I4kML0qjcZl3xHcEQBPbDo4TbvQaXuUT8W7lNbRCl8hfSGJA00wgUXhAjQApcuTCZckb9e6EVOwsa+eLY78jo2CqYWzhGez9zn0D2LMKNmZQi88WuTFVw9r1GSKIHstoDWvn54zISmr/1JgjC++mv2yRvatcvs8GhcsZVZT8dueaNK6tXLd1fQumhXCjpMyFjOlPWVTBPjlnsC5Uh98V/YiIa898SF4dwfjtDrG/fQZYmWUzJ8k2AslYLKGs=
with this key:
12341234123412341234123412341234
with counter function described in the beginning of this post using crypto++. If you succeed post the decrypted text (which contains only numbers) and your solution please.
Update2:
I'm not providing an IV in python code, the python module ignores IV. I the IV thing is what causing the problem.
As I read their source codes I can say PyCrypto and Crypto++ Both are perfect libraries for cryptography for Python and C++. The problem was that I was prefixing the encrypted data with some meta information about file and I totally forgot about that, after handling these meta data in client Crypto++ decrypted my files.
As I didn't find this documented explicitly anywhere (not even in Wikipedia) I write it here:
Any combination of Nonce, IV and Counter like concatenation, xor, or likes will work for CTR mode, but the standard that most libraries implement is to concatenate these values in order. So the value that is used in block cipher algorithm is usually: Nonce + IV + Counter. And counter usually starts from 1 (not 0).
I'm trying to encrypt a file using OpenPGP in python via the pycrypto application. I've been following the sample provided in their code here: https://github.com/dlitz/pycrypto/blob/master/lib/Crypto/Cipher/CAST.py
So I'm using mode.openPGP, but I can't seem to encrypt anything using a public key. My public key is well over the 16byte limit they specify (and any generation I've seen is over this limit as well). Is there a different value I'm supposed to use here, like the fingerprint ID?
I'm trying to read the contents of a file, encrypt it with a key, then print it into a new file to be sent (both will be deleted later on).
My code is as follows:
iv = CryptoRandom.new().read(CAST.block_size)
cipher = CAST.new(public_key, CAST.MODE_OPENPGP, iv)
file = open(filename)
contents = ''.join(file.readlines())
encrypted_contents = cipher.encrypt(contents)
encrypted_filename = filename.replace('/tmp/', '/tmp/encrypted')
encrypted_filename = encrypted_filename.replace('.csv', '.asc')
encrypted_file = open(encrypted_filename, 'w')
encrypted_file.write(encrypted_contents)
return encrypted_filename
I think you may be misunderstanding the algorithm you're using here. CAST is a symmetric-key algorithm, but whilst this implementation has an "OpenPGP mode", that doesn't mean that you simply pass your public key to it.
You should be generating a unique 16 byte key and passing that to CAST.new(). You would then generally encrypt that randomly-generated key using the public-key, and store/transmit the cipher text, and encrypted random-key together. The decryption process would decrypt the random-key using the private-key, then use the decrypted random-key to decrypt the cipher text.
I'm developing a web app (using gevent, but that is not significant) that has to write some confidential information in log. The obvious idea is to encrypt the confidential information using a public key that is hard-coded into my application. To read it, one would need a private key, and 2048-bit RSA seems to be safe enough. I have chosen pycrypto (tried M2Crypto as well, but found nearly no differences for my purpose) and implemented log encryption as a logging.Formatter subclass. However, I'm new to pycrypto and cryptoraphy, and I am not sure my choice of the way my data is encrypted is reasonable. Is PKCS1_OAEP module what I need? Or there are more friendly ways of encryption without dividing the data in small chunks?
So, what I did is:
import logging
import sys
from Crypto.Cipher import PKCS1_OAEP as pkcs1
from Crypto.PublicKey import RSA
PUBLIC_KEY = """ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDe2mtK03UhymB+SrIbJJUwCPhWNMl8/gA9d7jex0ciSuFfShDaqJ4wYWG4OOl\
VqKMxPrPcZ/PMSwtc021yI8TXfgewb65H/YQw4JzzGANq2+mFT8jWRDn+xUc6vcWnXIG3OPg5DvIipGQvIPNIUUP3qE7yDHnS5xdVdFrVe2bUUXmZJ9\
0xJpyqlTuRtIgfIfEQC9cggrdr1G50tXdXZjS0M1WXl5P6599oH/ykjpDFrCnh5fz9WDwUc0mNJ+11Qh+yfDp3k7AhzhRaROKLVWnfkklFaFm7LsdVX\
KPjp7dPRcTb84c2OnlIjU0ykL74Fy0K3eaPvM6TLe/K1XuD3933 pupkin#pupkin"""
PUBLIC_KEY = RSA.importKey(PUBLIC_KEY)
LOG_FORMAT = '[%(asctime)-15s - %(levelname)s: %(message)s]'
# May be more, but there is a limit.
# I suppose, the algorithm requires enough padding,
# and size of padding depends on key length.
MAX_MSG_LEN = 128
# Size of a block encoded with padding. For a 2048-bit key seems to be OK.
ENCODED_CHUNK_LEN = 256
def encode_msg(msg):
res = []
k = pkcs1.new(PUBLIC_KEY)
for i in xrange(0, len(msg), MAX_MSG_LEN):
v = k.encrypt(msg[i : i+MAX_MSG_LEN])
# There are nicer ways to make a readable line from data than using hex. However, using
# hex representation requires no extra code, so let it be hex.
res.append(v.encode('hex'))
assert len(v) == ENCODED_CHUNK_LEN
return ''.join(res)
def decode_msg(msg, private_key):
msg = msg.decode('hex')
res = []
k = pkcs1.new(private_key)
for i in xrange(0, len(msg), ENCODED_CHUNK_LEN):
res.append(k.decrypt(msg[i : i+ENCODED_CHUNK_LEN]))
return ''.join(res)
class CryptoFormatter(logging.Formatter):
NOT_SECRET = ('CRITICAL',)
def format(self, record):
"""
If needed, I may encode only certain types of messages.
"""
try:
msg = logging.Formatter.format(self, record)
if not record.levelname in self.NOT_SECRET:
msg = encode_msg(logging.Formatter.format(self, record))
return msg
except:
import traceback
return traceback.format_exc()
def decrypt_file(key_fname, data_fname):
"""
The function decrypts logs and never runs on server. In fact,
server does not have a private key at all. The only key owner
is server admin.
"""
res = ''
with open(key_fname, 'r') as kf:
pkey = RSA.importKey(kf.read())
with open(data_fname, 'r') as f:
for l in f:
l = l.strip()
if l:
try:
res += decode_msg(l, pkey) + '\n'
except Exception: # A line may be unencrypted
res += l + '\n'
return res
# Unfortunately dictConfig() does not support altering formatter class.
# Anyway, in demo code I am not going to use dictConfig().
logger = logging.getLogger()
handler = logging.StreamHandler(sys.stderr)
handler.setFormatter(CryptoFormatter(LOG_FORMAT))
logger.handlers = []
logger.addHandler(handler)
logging.warning("This is secret")
logging.critical("This is not secret")
UPDATE: Thanks to the accepted answer below, now I see:
My solution seems to be pretty valid for now (very few log entries, no performance considerations, more or less trusted storage). Concerning security, the best thing I can do right now is not forgetting to prohibit the user who runs my daemon from writing to the .py and .pyc files of the program. :-) However, if the user is compromised, he still may try to attach a debugger to my daemon process, so I should also disable login for him. Pretty obvious moments, but very important ones.
Surely there are solutions being much more scalable. A very common technique is to encrypt AES keys with slow but reliable RSA, and to encrypt data with the AES that is pretty fast. Data encryption in the case is symmetric, but retrieving the AES key requires either breaking RSA, or getting it from memory when my program is running. Stream encryption with higher-level libraries and binary log file format also are a way to go, though binary log format encrypted as a stream should be very vulnerable to log corruption, even a sudden reboot due to electricity blackout may be a problem unless I do some things at a lower level (at least log rotation on each daemon start).
I changed .encode('hex') to .encode('base64').replace('\n').replace('\r'). Fortunately, the base64 codec works fine with no line ends. It saves some space.
Using an untrusted storage may require signing records, but that seems to be another story.
Checking if the string is encrypted based on catching exceptions is ok, since, unless the log is tampered with by a malicious user, it's base64 codec who raises an exception, not RSA decryption.
You seem to encrypt data directly with RSA. This is relatively slow, and has the problem that you can only encrypt small parts of data. Distinguishing encrypted from plaintext data based on "decryption doesn't work" is also not a very clean solution, although it will probably work. You do use OAEP, which is good. You may want to use base64 instead of hex to save space.
However, crypto is easy to get wrong. For this reason, you should always use high-level crypto libraries wherever possible. Anything where you have to specify padding schemes yourself isn't "high-level". I am not sure if you will be able to create an efficient, line-based log encryption system without resorting to rather low-level libraries, though.
If you have no reason to encrypt only individual parts of the log, consider just encrypting the entire thing.
If you are really desperate for a line-based encryption, what you could do is the following: Create a random symmetric AES key from a secure randomness source, and give it a short but unique ID. Encrypt this key with RSA, and write the result to the log file in a line prefixed with a tag, e.g. "KEY", together with the ID. For each log line, generate a random IV, encrypt the message with AES256 in CBC mode using said IV (you don't have any length limits per line now!) and write the key ID, IV and the encrypted message to the log, prefixed with a tag, e.g. "ENC". After a certain time, destroy the symmetric key and repeat (generate new one, write to log). The disadvantage of this approach is that an attacker who can recover the symmetric key from memory can read the messages encrypted with said key. The advantage is that you can use higher-level building blocks and it is much, much faster (on my CPU, you can encrypt 70,000 log lines of 1 KB per second with AES-128, but only around 3,500 chunks of max. 256 bytes with RSA2048). RSA decryption is REALLY slow, by the way (around 100 chunks per second).
Note that you have no authentication, i.e. you won't notice modifications to your logs. For this reason, I assume you trust the log storage. Otherwise, see RFC 5848.