I am having a hard time creating a signature.
I am needing to make a signature using HMAC with SHA256 using a Checkout Request JSON and a secret key. I need to do it by concatenating signature, pipe character (|) and Checkout Request JSON and then encoding it with BASE64.
This is a formula I found in the documentations:
$signed_checkout_request = base64( hmac_sha256( $checkout_request, $private_key ) + "|" + $checkout_request )
I have made this based on some online code:
import hashlib
import hmac
import base64
checkout_request = '{"charge":{"amount":499,"currency":"EUR"}}'.encode('utf-8');
private_key = b'44444444444';
digest = hmac.new(private_key, msg=checkout_request, digestmod=hashlib.sha256).digest()
signature = base64.b64encode(digest).decode()
However I am not sure how to get the "|" into it. I am also not sure if I am even on the right track if I am honest... I don't have much experience in this section and I have failed at googling.
private_key = 'blahblahblah'
checkout_request = json.dumps({"charge":{"amount":4999,"currency":"EUR"}}, sort_keys=True, separators=(",", ":"))
digest = hmac.new(private_key.encode(), msg=checkout_request.encode(), digestmod=hashlib.sha256,).hexdigest()
signature = base64.b64encode((digest + "|" + checkout_request).encode()).decode()
I was able to get it to work with that :)
Related
I have a project written in python. I use cryptography library to encrypt and decrypt data.
I do it how is shown in their tutorial.
Here is my python code:
import base64
import os
from cryptography.fernet import Fernet
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.kdf.pbkdf2 import PBKDF2HMAC
password = b"my password"
salt = os.urandom(16)
kdf = PBKDF2HMAC(algorithm=hashes.SHA256(),
length=32,
salt=salt,
iterations=100000,
backend=default_backend())
key = base64.urlsafe_b64encode(kdf.derive(password))
f = Fernet(key)
data = b"my data..."
token = f.encrypt(data)
Then for decryption I can just use:
f.decrypt(token)
Everything works perfectly in python but now I need to do the same thing in kotlin. I found out about fernet java-8 library but I don't know how to use it in the same way.
The problem is that I have two tools: one is written in python and another I want to write in kotlin. Both tools are meant to do the same thing - the python one is for desktop and the kotlin one is gonna be an android app. So it is really important for their encryption to be the same, so that files encrypted in python (desktop tool) can be decrypted in kotlin (android app) and vice versa.
But I don't know how to write analogous kotlin code.
You see there is a function (or class) called PBKDF2HMAC and there is also base64.urlsafe_b64encode and others. And I don't know what are analogous functions in kotlin or fernet java-8.
So how should I do it? Assuming that in kotlin I have to use password and salt I used in python.
Thanks!
In Java/Kotlin, using fernet-java8, the token generated with the Python code could be decrypted as follows:
import java.security.SecureRandom
import java.util.Base64
import javax.crypto.spec.PBEKeySpec
import javax.crypto.SecretKeyFactory
import com.macasaet.fernet.Key
import com.macasaet.fernet.Token
import com.macasaet.fernet.StringValidator
import com.macasaet.fernet.Validator
import java.time.Duration
import java.time.temporal.TemporalAmount
...
// Data from encryption
val salt = Base64.getUrlDecoder().decode("2Yb8EwpYkMlycHxoKcmHuA==")
val token = Token.fromString("gAAAAABfoAmp7C7IWVgA5urICEIspm_MPAGZ-SyGnPEVUBBNerWQ-K6mpSoYTwRkUt3FobyAFHbYfhNtiGMe_96yyLvUoeLIIg==");
// Derive Fernet key
val key = deriveKey("my password", salt)
val fernetKey = Key(key)
// Decrypt
val validator: Validator<String> = object : StringValidator {
override fun getTimeToLive(): TemporalAmount {
return Duration.ofHours(24)
}
}
val data = token.validateAndDecrypt(fernetKey, validator)
println(data) // my data...
with:
fun deriveKey(password: String, salt: ByteArray): String {
val iterations = 100000
val derivedKeyLength = 256
val spec = PBEKeySpec(password.toCharArray(), salt, iterations, derivedKeyLength)
val secretKeyFactory = SecretKeyFactory.getInstance("PBKDF2WithHmacSHA256")
val key = secretKeyFactory.generateSecret(spec).encoded
return Base64.getUrlEncoder().encodeToString(key)
}
Here the Fernet key is derived using the key derivation function PBKDF2. PBKDF2 expects various input parameters, such as a password, a digest, a salt, an iteration count and the desired key length. In the posted example the key is returned Base64url encoded.For decryption the same parameters must be used as for encryption. Since the salt is usually (as in the posted code) randomly generated during encryption, it must be passed to the decryption side along with the ciphertext (note: the salt is not a secret).
The validator sets the time-to-live (by default 60s) to 24h, see here for more details.
In the posted Python code the export of the salt has to be added, e.g. by Base64url encoding it analogous to key and token (and printing it for simplicity). In practice, salt and token could also be concatenated during encryption and separated during decryption.
Update:
The encryption part is analogous:
// Generate salt
val salt = generateSalt()
println(Base64.getUrlEncoder().encodeToString(salt))
// Derive Fernet key
val key = deriveKey("my password", salt)
val fernetKey = Key(key)
// Encrypt
val data = "my data..."
val token = Token.generate(fernetKey, data)
println(token.serialise()) // the Base64url encoded token
with
fun generateSalt(): ByteArray {
val random = SecureRandom()
val salt = ByteArray(16)
random.nextBytes(salt)
return salt
}
I am communicating with our clients server. For an api I need to sign a string with my private key. They have the following condition to follow
User SHA 256 algorithm to calculate the hash of the string
Use the private key and RSA (PKCS1_PADDING) algorithm to sign the Hash Value.
Base64 encode the encrypted Hash Value
and I am doing following
from Crypto.Signature import PKCS1_v1_5
from Crypto.Hash import SHA256
from Crypto.PublicKey import RSA
import base64
pkey = RSA.importKey(keystring)
message = "Hello world"
h = SHA256.new(message.encode())
signature = PKCS1_v1_5.new(pkey).sign(h)
result = base64.b64encode(signature).decode()
Here I am getting a string as result. But on the server side my signature is not matching.
Is there anything I am going wrong with ?? Can anyone help me on this ?
I came back to this question recently and noticed it was never resolved. I don't know what was going wrong with the OPs setup but the following code worked for me.
First, the python code that generates the signature of "Hello world":
from Cryptodome.Signature import PKCS1_v1_5
from Cryptodome.Hash import SHA256
from Cryptodome.PublicKey import RSA
import base64
def sign(message: str, private_key_str: str) -> str:
priv_key = RSA.importKey(private_key_str)
h = SHA256.new(message.encode('utf-8'))
signature = PKCS1_v1_5.new(priv_key).sign(h)
result = base64.b64encode(signature).decode()
return result
And now the Java code that verifies it:
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.security.GeneralSecurityException;
import java.security.KeyFactory;
import java.security.PublicKey;
import java.security.Signature;
import java.security.spec.X509EncodedKeySpec;
import java.util.Base64;
...
...
public static boolean verify(String message, String b64Sig, byte[] pubkey_spki) throws GeneralSecurityException {
var pubKey = (PublicKey) KeyFactory.getInstance("RSA").generatePublic(new X509EncodedKeySpec(pubkey_spki));
var verifier = Signature.getInstance("SHA256withRSA");
verifier.initVerify(pubKey);
verifier.update(message.getBytes(StandardCharsets.UTF_8));
return verifier.verify(Base64.getDecoder().decode(b64Sig));
}
Perhaps the trickiest part of this is specifying the correct padding scheme in each language/library. These signatures use the scheme identified as RSASSA-PKCS1-v1_5 in the PKCS#1 RFC 8017. On the python side this is accomplished by providing the SHA256 hash object to the PKCS1_v1_5 signature object. In Java it is perhaps a little more straightforward in that you ask for Signature object that implements the RSA algorithm with SHA256 as the hash function, but still have to know that this is RSASSA-PKCS1-v1_5 and not some other possibility in RFC 8017.
I think if one is not already something of an expert then understanding that these magic choices in python and Java produce compatible code is going to be difficult.
I am using RSA with private/public key. I am trying to encrypt a string, save it in a database (sqlite) and then retrieve it again and encrypt.
I can't decrypt the data again when it is coming out of the sqlite. The string is identical and I am a bit lost.
#!/usr/bin/env python3
from Crypto.PublicKey import RSA
from Crypto import Random
import sqlite3
import base64
# database layout
#
# CREATE TABLE secrets ( id INT, secret TEXT );
# INSERT INTO secrets (id,secret) VALUES (1,"");
# database
conn = sqlite3.connect('database.db')
c = conn.cursor()
# generate keys
private_key = RSA.generate(1024, Random.new().read)
public_key = private_key.publickey()
# save keys
f = open('public.pem', 'wb+')
f.write(public_key.exportKey('PEM'))
f.close()
f = open('private.pem', 'wb+')
f.write(private_key.exportKey('PEM'))
f.close()
# crypt
f = open('public.pem','rb')
encrypt_public_key = RSA.importKey(f.read())
secret = "123456"
enc_secret = encrypt_public_key.encrypt(secret.encode("utf-8"), 32)[0]
enc_secret_encoded = base64.b64encode(enc_secret)
print("Base64: " + str(enc_secret_encoded))
# save in db
c.execute('UPDATE secrets SET secret="%s" WHERE id=1' % (enc_secret_encoded))
conn.commit()
print("--------------- DECRYPTION ------------------------")
# decrypt
p = open('private.pem','rb')
decrypt_private_key = RSA.importKey(p.read())
c.execute('SELECT secret FROM secrets WHERE id=1')
result = c.fetchone()
encoded_secret = result[0]
print("Base64: " + encoded_secret)
decoded_secret = base64.b64decode(encoded_secret)
enc_secret = decrypt_private_key.decrypt(decoded_secret)
print("Decrypted: " + str(enc_secret))
Output:
$ ./stuck.py
Base64: b'bfAERXPFvrDRdr5Pcexu8JgHlKfDaUhkqJrSWZJbLwlKLWY8XHtIlBwrRfP7eMX9PTKo4t2CtpdXS6Fam4B+jR3/bYPxji0rHt1Aed64sLH4xAnxgh5B/qWidcYT5cPmvwMekGbCaMSgGjvNB4Js/yDRrW4+N8dqx3IoUAl8zgA='
--------------- DECRYPTION ------------------------
Base64: b'bfAERXPFvrDRdr5Pcexu8JgHlKfDaUhkqJrSWZJbLwlKLWY8XHtIlBwrRfP7eMX9PTKo4t2CtpdXS6Fam4B+jR3/bYPxji0rHt1Aed64sLH4xAnxgh5B/qWidcYT5cPmvwMekGbCaMSgGjvNB4Js/yDRrW4+N8dqx3IoUAl8zgA='
Decrypted: b'\x90\x07\xa2}\x96w\xda\xd3h\xf1\xd4\xc6z\xa5\xf3\x85\x97\xeb\xcfL\x0e\x1f;\x18\xd5\x98\xb3\xb2\xd0\x93.\xc9z\x1c\xc8\xac\xe4x\xbfT\xe4{\x1b\x19\xda\xfb/?A\xda_\xceHc\xd14X\x94\x8a\x94\xfc\x12\xc4\x86\xc9\x16\xc9b\xbf\xdaJ\xcf\xff\xe1J\x95\x03&\xda\x98\x9f\x10\xb1\tzW\xea\x9b\xd2\x13\xc1\x8d\x19\xe97\xd6\xeay\xf3\x83\xb7\xcf\xd3v\\`~\x07\xcea(\x81\xe1c\x08\x0b\x8c\xee\xc2\x87\xed\xc8\x08D\x8e\xe5\x83\xf4'
When you run my example, you will see that the same encrypted string gets into the sqlite and out again, but why can't I decrypt it again and get the same result as secret?
UPDATE: When I remove the sqlite database then it works as expected. So the problem must be somewhere in storing or retrieving the data.
Any hint appreciated.
The Python base 64 library returns a bytes object rather than a string when encoding (which is a bit odd since the whole point of base 64 encoding is to create a printable string).
This means that when you convert the result to a string to save it in sqlite it is in the form b'XXX...XX', i.e. it is saved as a string starting with a b with quotes around the actual base64 encoded data.
When decoding, the default is to discard any non base64 characters. So this removes the quotes but not the initial b. This means the data you are decoding has an extra b at the front so you end up trying to decrypt the wrong cipher text.
You can see this by adding validate=True to the call to decode the base 64 data to force it to validate the input. This will cause a binascii.Error because of the ' character.
enc_secret = base64.b64decode(enc_secret_b64, validate=True)
The fix is to decode the bytes object from base 64 encoding into an ASCII string before saving to sqlite. Then only the “real” base 64 characters will saved to the database:
enc_secret_encoded = base64.b64encode(enc_secret).decode("ASCII")
you got the wrong order of base64/RSA while decoding. this works:
#!/usr/bin/env python3
from Crypto.PublicKey import RSA
from Crypto import Random
import base64
key_pair = RSA.generate(1024, Random.new().read(1024 // 8))
public_key = key_pair.publickey()
secret = "123456"
enc_secret = public_key.encrypt(secret.encode("utf-8"), 32)[0]
enc_secret_b64 = base64.b64encode(enc_secret)
print(enc_secret_b64)
enc_secret = base64.b64decode(enc_secret_b64)
secret = key_pair.decrypt(enc_secret)
print(secret.decode("utf-8"))
# 123456
also note that you need to call the .read method of Random.
apart from that: that is not what RSA is meant for. if you want to encrypt data using RSA you should use RSA for key encapsulation only and encrypt the data using a symmetric crypto system (e.g. AES).
The E*Trade API allows you to use RESTful to log on to the site and manipulate an account or retrieve quote information. Though I am having trouble generating an oauth_signature that matches their "practice problem" located toward the bottom of
https://us.etrade.com/ctnt/dev-portal/getContent?contentId=306a9d46-58c2-4cac-85f6-7717aea056bd
The simple HMAC-SMA1 algorithm has been coded below and reproduces the oauth core 1.0a signature value from here https://oauth.net/core/1.0a/#sig_base_example. Though I cannot get E*Trade signature value to reproduce.
def generate_oauth_signature():
from urllib.parse import quote_plus
from hashlib import sha1
import binascii
import hmac
key = quote_plus('7d30246211192cda43ede3abd9b393b9') + \
'&' + \
quote_plus('XCF9RzyQr4UEPloA+WlC06BnTfYC1P0Fwr3GUw/B0Es=')
key = key.encode()
raw = quote_plus('GET') + '&' + \
quote_plus('https://etws.etrade.com/accounts/rest/accountlist') + '&' + \
quote_plus('oauth_consumer_key=c5bb4dcb7bd6826c7c4340df3f791188&oauth_nonce=0bba225a40d1bbac2430aa0c6163ce44&oauth_signature_method=HMAC-SHA1&oauth_timestamp=1344885636&oauth_token=VbiNYl63EejjlKdQM6FeENzcnrLACrZ2JYD6NQROfVI=')
raw = raw.encode()
hashed = hmac.new(key, raw, sha1)
sig = hashed.digest()
oauth_signature = quote_plus(binascii.b2a_base64(hashed.digest())[:-1])
The function is supposed to yield "%2FXiv96DzZabnUG2bzPZIH2RARHM%3D", but I'm not there yet. Has anyone worked out the hashing for the E*Trade API?
I am aware of the etradepy.py, which is a nice package, but is a little outdated and does not match the current E*Trade website.
One problem is that the oauth_token needs to be encoded in the parameter string (it will end up being double encoded). Mine is the following:
oauth_consumer_key=c5bb4dcb7bd6826c7c4340df3f791188&oauth_nonce=0bba225a40d1bbac2430aa0c6163ce44&oauth_signature_method=HMAC-SHA1&oauth_timestamp=1344885636&oauth_token=VbiNYl63EejjlKdQM6FeENzcnrLACrZ2JYD6NQROfVI%3D
I have a valid certificate issued by the spanish authority (FNMT) and I want to play with it to learn more about it.
The file has extension .p12
I would like to read the information in it (first and last name) and check if the certificate is valid. Is it possible to do that with pyOpenSSL? I guess I have to use the crypto module in OpenSSL.
Any help or useful link? Trying reading here: http://packages.python.org/pyOpenSSL/openssl-crypto.html but not much information :-(
It's fairly straight-forward to use. This isn't tested, but should work:
# load OpenSSL.crypto
from OpenSSL import crypto
# open it, using password. Supply/read your own from stdin.
p12 = crypto.load_pkcs12(open("/path/to/cert.p12", 'rb').read(), passwd)
# get various properties of said file.
# note these are PyOpenSSL objects, not strings although you
# can convert them to PEM-encoded strings.
p12.get_certificate() # (signed) certificate object
p12.get_privatekey() # private key.
p12.get_ca_certificates() # ca chain.
For more examples, have a look through the unit test code of pyopenssl. Pretty much every way you might want to use the library is there
See also here or without adverts here.
As pyOpenSSL.crypto.load_pkcs12 is now deprecated, here is the equivalent solution using cryptography, with loading inside a requests Session as a bonus.
from cryptography.hazmat.primitives import serialization
from requests import Session
with open("./cert.p12", "rb") as f:
(
private_key,
certificate,
additional_certificates,
) = serialization.pkcs12.load_key_and_certificates(
f.read(), CLIENT_CERT_KEY.encode()
)
# key will be available in user readable temporary file for the time of the
# program run (until key and cert get gc'ed)
key = tempfile.NamedTemporaryFile()
cert = tempfile.NamedTemporaryFile()
key.write(
private_key.private_bytes(
encoding=serialization.Encoding.PEM,
format=serialization.PrivateFormat.PKCS8,
encryption_algorithm=serialization.NoEncryption(),
)
)
key.flush()
cert.write(
certificate.public_bytes(serialization.Encoding.PEM),
)
cert.flush()
session = Session()
session.cert = (cert.name, key.name)
Maybe is wrong answering to an old Q, but I thought that it may help someone that find this Q after me. This solution work for python 3, and I think is a little bit better. I found it in the repo of zeep and is a class to encapsule the usage.
Class
import os
from OpenSSL import crypto
class PKCS12Manager():
def __init__(self, p12file, passphrase):
self.p12file = p12file
self.unlock = passphrase
self.webservices_dir = ''
self.keyfile = ''
self.certfile = ''
# Get filename without extension
ext = os.path.splitext(p12file)
self.filebasename = os.path.basename(ext[0])
self.createPrivateCertStore()
self.p12topem()
def getKey(self):
return self.keyfile
def getCert(self):
return self.certfile
def createPrivateCertStore(self):
home = os.path.expanduser('~')
webservices_dir = os.path.join(home, '.webservices')
if not os.path.exists(webservices_dir):
os.mkdir(webservices_dir)
os.chmod(webservices_dir, 0o700)
self.webservices_dir = webservices_dir
def p12topem(self):
p12 = crypto.load_pkcs12(open(self.p12file, 'rb').read(), bytes(self.unlock, 'utf-8'))
# PEM formatted private key
key = crypto.dump_privatekey(crypto.FILETYPE_PEM, p12.get_privatekey())
self.keyfile = os.path.join(self.webservices_dir, self.filebasename + ".key.pem")
open(self.keyfile, 'a').close()
os.chmod(self.keyfile, 0o600)
with open(self.keyfile, 'wb') as f:
f.write(key)
# PEM formatted certificate
cert = crypto.dump_certificate(crypto.FILETYPE_PEM, p12.get_certificate())
self.certfile = os.path.join(self.webservices_dir, self.filebasename + ".crt.pem")
open(self.certfile, 'a').close()
os.chmod(self.certfile, 0o644)
with open(self.certfile, 'wb') as f:
f.write(cert)
Usage
from requests import Session
from zeep import Client
from zeep.transports import Transport
# https://github.com/mvantellingen/python-zeep/issues/824
pkcs12 = PKCS12Manager('cert.p12', 'password_for_cert')
session = Session()
session.cert = (pkcs12.getCert(), pkcs12.getKey())
transport = Transport(session=session)
client = Client('url_service', transport=transport)