We Keep Coding

How to Do EC Compression on a Public Key in Python?

I am trying to find the Python-equivalent of running openssl ec -pubin -in example.pem -inform PEM -outform DER conv_form compressed
Example using the following public key:
-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE4q+ot7o3PuJqsBonZni2spVPvqLk
6FCiEF9GXzTyYZ1snzreGB+pyoiUUkz2/H60XWmQsgC7zZ60TBT0rVimtg==
-----END PUBLIC KEY-----
and running the following command gives the following output:
echo '-----BEGIN PUBLIC KEY-----\r\nMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE4q+ot7o3PuJqsBonZni2spVPvqLk\r\n6FCiEF9GXzTyYZ1snzreGB+pyoiUUkz2/H60XWmQsgC7zZ60TBT0rVimtg==\r\n-----END PUBLIC KEY-----\r\n' | openssl ec -pubin -inform PEM -outform DER -conv_form compressed | base64
read EC key
writing EC key
MDkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDIgAC4q+ot7o3PuJqsBonZni2spVPvqLk6FCiEF9GXzTyYZ0=
now providing a bit more verbosity with the same command plus the -text flag and minus the base64 encoding:
echo '-----BEGIN PUBLIC KEY-----\r\nMFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE4q+ot7o3PuJqsBonZni2spVPvqLk\r\n6FCiEF9GXzTyYZ1snzreGB+pyoiUUkz2/H60XWmQsgC7zZ60TBT0rVimtg==\r\n-----END PUBLIC KEY-----\r\n' | openssl ec -pubin -inform PEM -outform DER -conv_form compressed -text
read EC key
Private-Key: (256 bit)
pub:
02:e2:af:a8:b7:ba:37:3e:e2:6a:b0:1a:27:66:78:
b6:b2:95:4f:be:a2:e4:e8:50:a2:10:5f:46:5f:34:
f2:61:9d
ASN1 OID: prime256v1
NIST CURVE: P-256
writing EC key
090*�H�*�H�="⯨��7>�j�'fx���O����P�_F_4�a�%
So far I am able to do something like:
import base64
import cryptography
csr_crypto = cryptography.x509.load_pem_x509_csr(csr_encoded) # csr_encoded being the CSR in PEM format that the public key is derived from
pub_key = csr_crypto.public_key()
compressed_bytes = pub_key.public_bytes(cryptography.hazmat.primitives.serialization.Encoding.X962, cryptography.hazmat.primitives.serialization.PublicFormat.CompressedPoint).hex()
this leads to compressed_bytes being equal to
02e2afa8b7ba373ee26ab01a276678b6b2954fbea2e4e850a2105f465f34f2619d
which if you look closely is equal to what is in the pub above.
How are these hex bytes ultimately converted to the string MDkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDIgAC4q+ot7o3PuJqsBonZni2spVPvqLk6FCiEF9GXzTyYZ0= that openssl outputs as a compressed public key?

The first key posted is a public key in X.509/SPKI format (PEM encoded), which contains the actual key in uncompressed format 0x04 + <x> + <y>.
The key derived from this using the OpenSSL statement is also a public key in X.509/SPKI format, but contains the actual key in compressed format 0x02 + <x> or 0x03 + <x> for even or odd y, respectively. This format also contains the complete information, since for a given curve an uncompressed key can be derived from a compressed key.
X.509/SPKI keys can be parsed with an ASN.1 parser (in addition to OpenSSL), e.g. online: https://lapo.it/asn1js.
In Python, an X.509/SPKI key with uncompressed key can be converted to an X.509/SPKI key with compressed key using the PyCryptodome library, see export_key():
from base64 import b64encode
from Crypto.PublicKey import ECC
x509PemWithUncompressed = '''-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE4q+ot7o3PuJqsBonZni2spVPvqLk
6FCiEF9GXzTyYZ1snzreGB+pyoiUUkz2/H60XWmQsgC7zZ60TBT0rVimtg==
-----END PUBLIC KEY-----'''
publicKey = ECC.import_key(x509PemWithUncompressed);
x509withCompressed = publicKey.export_key(format='DER', compress=True)
print(b64encode(x509withCompressed).decode('utf-8')) # MDkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDIgAC4q+ot7o3PuJqsBonZni2spVPvqLk6FCiEF9GXzTyYZ0=
The Cryptography library on the other hand only allows the export of an X.509/SPKI key with uncompressed key. For this, the format has to be specified with SubjectPublicKeyInfo. An export as X.509/SPKI key with compressed key is not possible. The options UncompressedPoint and CompressedPoint only allow export in the format 0x04 + x + y or 0x02/0x03 + x, but not in the X.509/SPKI format.
But there is a workaround for the desired conversion. In the X.509/SPKI format, the actual key (compressed or uncompressed) is located at the end. The preceding part contains information about the curve, data lengths, etc. In the case of an X.509/SPKI key with compressed key and curve P-256, the preceding part is: 0x3039301306072a8648ce3d020106082a8648ce3d030107032200 (note that the prefix for an X.509 /SPKI key with uncompressed key differs because of the different data lengths) and can be used as a prefix for converting a compressed key in 0x02/0x03 + x format to X.509/SPKI format:
from base64 import b64encode
from cryptography.hazmat.primitives import serialization
x509PemWithUncompressed = b'''-----BEGIN PUBLIC KEY-----
MFkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDQgAE4q+ot7o3PuJqsBonZni2spVPvqLk
6FCiEF9GXzTyYZ1snzreGB+pyoiUUkz2/H60XWmQsgC7zZ60TBT0rVimtg==
-----END PUBLIC KEY-----'''
publicKey = serialization.load_pem_public_key(x509PemWithUncompressed)
compressedKey = publicKey.public_bytes(serialization.Encoding.X962, serialization.PublicFormat.CompressedPoint)
x509withCompressed = bytes.fromhex('3039301306072a8648ce3d020106082a8648ce3d030107032200') + compressedKey
print(b64encode(x509withCompressed).decode('utf-8')) # MDkwEwYHKoZIzj0CAQYIKoZIzj0DAQcDIgAC4q+ot7o3PuJqsBonZni2spVPvqLk6FCiEF9GXzTyYZ0=

From M2Crypto to pyCrypto

I'm trying to port an M2Crypto function to pyCrypto or any other solution, 'cause the last version of M2Crypto doesn't work with pypy.
So, i'm trying to port an existing code from pysimplesoap:
https://github.com/pysimplesoap/pysimplesoap/blob/master/pysimplesoap/xmlsec.py
This is the only function that i don't know how to port:
def x509_extract_rsa_public_key(cert, binary=False):
"Return the public key (PEM format) from a X509 certificate"
x509 = x509_parse_cert(cert, binary)
return x509.get_pubkey().get_rsa().as_pem()
The original public certificate is:
-----BEGIN CERTIFICATE-----
MIIFhTCCA20CBFOF3SIwDQYJKoZIhvcNAQEFBQAwgYYxIDAeBgkqhkiG9w0BCQEW
EWpjZXJkYUBleHBlcnRpLmNsMQswCQYDVQQGEwJDTDERMA8GA1UECAwIU2FudGlh
Z28xETAPBgNVBAcMCFNhbnRpYWdvMRAwDgYDVQQKDAdFeHBlclRJMRAwDgYDVQQL
DAdFeHBlclRJMQswCQYDVQQDDAIxMDAeFw0xNDA1MjgxMjU3MDZaFw0xNjA1Mjcx
MjU3MDZaMIGGMSAwHgYJKoZIhvcNAQkBFhFqY2VyZGFAZXhwZXJ0aS5jbDELMAkG
A1UEBhMCQ0wxETAPBgNVBAgMCFNhbnRpYWdvMREwDwYDVQQHDAhTYW50aWFnbzEQ
MA4GA1UECgwHRXhwZXJUSTEQMA4GA1UECwwHRXhwZXJUSTELMAkGA1UEAwwCMTAw
ggIiMA0GCSqGSIb3DQEBAQUAA4ICDwAwggIKAoICAQCX1t11QZSgfodg+NPwKdof
lakbpxsFmCvjbY3Vpp6/bTv56XIZe/Z3gBlU4zeGslEaqzs1k4cGAcdZPHxSMydC
oLxmyXpdS2hVFUZTYAeanXHhQzUKmNlgYj3uObprPEEQzD/vEuqpwz2/ZGyaHsWs
exu9aeuLAUUSNne0yqobrzRfEp2FqCp4sJB80cXgGfPA4Cr5jROHqUi8sVWpWRy5
ai5ZaiXRPo3YKF1336twuw9lRS3cRtJh9/AoJElGT7G7BMDpxyTTa08y1iRliDGu
mwWivQMHoKqEs0lCtF9Uz8cFdmiIoRPTt6McpmLoapL9v19xjNnim4lke6DPvtcg
uato7T+frDqA5Cj5GRP/8jbe90Y+YjHuJTkw+fkV6gDTRmJ3wCWDIw/07aY6nZ+H
24Imu6N2YBsMEsa8j9OW04mNgtppRC4dFBh0FIKXC35kJgN38y+6T7MsQThX1XZS
SlK0FygJJADVGelmxtsrRRfnp4yLYRjwdkRGExRjVs/+fkOyKI+fX0o68z6MEDyA
5epVHpgwJ/Yz3Lo7cgXy0hO5a/MfZc0Y0ofb29g8sJMJ8j/SSR85i4pFxudn+HH0
SmkkzE/P10adF/X+pqjscOE+aXwnX09lUUQ9TIlpYaR3hUhONsuefYJ2sHz2z+vt
K5btQwN7u9+QeXLgb20PMwIDAQABMA0GCSqGSIb3DQEBBQUAA4ICAQAqCepOSFi7
8d6lKTfWpMuj5ygpplBa3Bj81AKkNfSGDN7zkNX2sCbqn9aEjniIPtldG6I2fgCs
FYTZE9oEPNiuEuH7PNtAjTExIUi9Jzm7bqjCdSs7Ioek1cPePgst017zJ6NSTkaM
r7pUDaS3855xO92uaJskppAeegwz9Dv3d5wY+wVViqUki4pZyxa95IvBJz4NR1Xr
cO6XtUUT9M0wbd0jAkRp7DPQfkihZj8vLSvlUYTRdlF2swIBE/ME2T3NCa0/kt1c
IA1Aq/zn7t0yKvyaJ/O//LrHA1Lfa/uC61O/9P3t+eXDsYl73CeGQdSYZp2DAZmA
Ek3tzwhFa6HR+POIo8MptWMT3DQ0ISHH+EW1Xp8GHIGsk2ELsXuA6XTNwpfz9yvl
9d7IGsq4cdX88cNUCbXm4tj7F3s6i8pNWeCImaYcXKGCBdsLM+lbmqbuV7o3d1Ei
efbR1TQkCxRBNCMUI0pF7NW8PvY3QER9/jEnN52SX+tuQRVdpgl+PyTdSASr4FhV
+HHmgeOgeOewXDnZ7aA1F6f8+CY8Niv4FGZIAptdxTqdynY4nUy/wFowBouO3LEF
6nIcQ3Jx1pDXoEmcLa03JaL7qQNSHyqSe/YEl8E5fdDr7vApzw9pvpAjj1aslidL
bNd4l1YGlL2vbGsIXZlbdBLiblXRi78AyQ==
And when i execute that function, the result is:
-----BEGIN PUBLIC KEY-----
MIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEAl9bddUGUoH6HYPjT8Cna
H5WpG6cbBZgr422N1aaev207+elyGXv2d4AZVOM3hrJRGqs7NZOHBgHHWTx8UjMn
QqC8Zsl6XUtoVRVGU2AHmp1x4UM1CpjZYGI97jm6azxBEMw/7xLqqcM9v2Rsmh7F
rHsbvWnriwFFEjZ3tMqqG680XxKdhagqeLCQfNHF4BnzwOAq+Y0Th6lIvLFVqVkc
uWouWWol0T6N2Chdd9+rcLsPZUUt3EbSYffwKCRJRk+xuwTA6cck02tPMtYkZYgx
rpsFor0DB6CqhLNJQrRfVM/HBXZoiKET07ejHKZi6GqS/b9fcYzZ4puJZHugz77X
ILmraO0/n6w6gOQo+RkT//I23vdGPmIx7iU5MPn5FeoA00Zid8AlgyMP9O2mOp2f
h9uCJrujdmAbDBLGvI/TltOJjYLaaUQuHRQYdBSClwt+ZCYDd/Mvuk+zLEE4V9V2
UkpStBcoCSQA1RnpZsbbK0UX56eMi2EY8HZERhMUY1bP/n5DsiiPn19KOvM+jBA8
gOXqVR6YMCf2M9y6O3IF8tITuWvzH2XNGNKH29vYPLCTCfI/0kkfOYuKRcbnZ/hx
9EppJMxPz9dGnRf1/qao7HDhPml8J19PZVFEPUyJaWGkd4VITjbLnn2CdrB89s/r
7SuW7UMDe7vfkHly4G9tDzMCAwEAAQ==
-----END PUBLIC KEY-----
I was trying to do something like this, but it doesn't work:
pub = open("certificate_server.crt", "r").read()
a2 = crypto.load_certificate(FILETYPE_PEM, pub)
b2 = a2.get_pubkey()
But then i don't know how to proceed, can someone help me?

Using pyOpenSSL, after you call get_pubkey() you can call OpenSSL.crypto.dump_publickey(OpenSSL.crypto.FILETYPE_PEM, b2) to get the PEM output.

Convert PEM file to DER

I am currently trying to write a script that allows me to compute the Tor HS address from the hiddens service's private key file.
In order to do this the file needs to be brought into the DER format.
Using OpenSSL this can be done with:
openssl rsa -in private_key -pubout -outform DER
Piping this into python with:
base64.b32encode(hashlib.sha1(sys.stdin.read()[22:]).digest()[:10]).lower()'
will return the address correctly.
However I would like to perform the same using only python. My problem is that using the pycrypto module the DER output is different and the address therefore incorrect.
key = RSA.importKey(keyfile.read()).publickey()
print(key.exportKey(format='DER'))
Will result in a different output than the openssl call.
Is this just a matter of implementation that allows different results? Or am I making a mistake somewhere?
Any help would be appreciated

convert certificate to der using python
first we load the file
cert_file = keyfile.read()
Then we convert it to pem format
from OpenSSL import crypto
cert_pem = crypto.load_certificate(crypto.FILETYPE_PEM, cert_file)
now we are generating the der-output
i.e.: output equals to openssl x509 -outform der -in certificate.pem -out certificate.der.
cert_der = crypto.dump_certificate(crypto.FILETYPE_ASN1, cert_pem)

I was looking for something similar and, as of March 2019, OpenSSL recommends using pyca/cryptography instead of the crypto module. (source)
Here after is then what you intend to do: convert PEM to DER
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives import serialization
with open("id_rsa", "rb") as keyfile:
# Load the PEM format key
pemkey = serialization.load_pem_private_key(
keyfile.read(),
None,
default_backend()
)
# Serialize it to DER format
derkey = pemkey.private_bytes(
serialization.Encoding.DER,
serialization.PrivateFormat.TraditionalOpenSSL,
serialization.NoEncryption()
)
# And write the DER format to a file
with open("key.der", "wb") as outfile:
outfile.write(derkey)

I want Convert Certificate file not the key file from DER to PEM, but Google took me here. thanks #alleen1's answer, I can convert certificate or key from DER to PEM and vice versa.
Step one, load the file.
Step two,save it to the format you want.
I ommit the process to get the "pem_data" and "der_data",you can get it from file or anywhere else. they should be bytes not string, use method .encode() when needed.
from cryptography import x509
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.backends import default_backend
# Step one, load the file.
# Load key file
# PEM
key = serialization.load_pem_private_key(pem_data, None, default_backend())
# DER
key = serialization.load_pem_private_key(der_data, None, default_backend())
# Load cert file
# PEM
cert = x509.load_pem_x509_certificate(pem_data, default_backend())
# DER
cert = x509.load_der_x509_certificate(der_data, default_backend())
# Step two,save it to the format you want.
# PEM key
key_val = key.private_bytes(
serialization.Encoding.PEM,
serialization.PrivateFormat.TraditionalOpenSSL,
serialization.NoEncryption()
)
# DER key
key_val = key.private_bytes(
serialization.Encoding.DER,
serialization.PrivateFormat.TraditionalOpenSSL,
serialization.NoEncryption()
)
# PEM cert
cert_val = cert.public_bytes(serialization.Encoding.PEM)
# DER cert
cert_val = cert.public_bytes(serialization.Encoding.DER)

The inital question is: "Exact the public key from private key", this because the openSSL command states "pubout" in initial question.
Using OpenSSL this can be done with: (note that "pubout" defines OUTPUT as public key only)
openssl ALGORITHM_USED -in private_key -pubout -outform DER
But with Python cryptography module you can exact the public key from private key (note this seems applicable for RSA and EC based cryptography).
With Python:
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives.asymmetric import ec
from cryptography.hazmat.backends import default_backend
# Create private key (example uses elliptic curve encryption)
priv_key = ec.generate_private_key(ec.SECP256K1, default_backend())
pub_key = priv_key.public_key()
pub_key_pem = pub_key.public_bytes(
encoding=serialization.Encoding.PEM,
format=serialization.PublicFormat.SubjectPublicKeyInfo
)
with open('public_key.pem', 'wb') as outfile:
outfile.write(public_key_pem)
More info on cryptography documentation: https://cryptography.io/en/latest/hazmat/primitives/asymmetric/ec/#cryptography.hazmat.primitives.asymmetric.ec.EllipticCurvePublicKey

Python Openssl generate rsa key pair and write to a file

I want to generate a private , public key pair and put them into private.key and public.key files respectively.
I have the following code.
from OpenSSL import crypto, SSL
def gen_rsa_key_pair():
k = crypto.PKey()
k.generate_key(crypto.TYPE_RSA, 1024)
open("Priv.key", "wt").write(crypto.dump_privatekey(crypto.FILETYPE_PEM, k))
crypto.dump_publickey() is not available.
How do I dump public key to a file?

The OpenSSL functions to print the public RSA key do not seem to be exported by the Python OpenSSL wrapper. By accessing the internals of the crypto module, you could still do it yourself (assuming that you have this package installed locally), as this code snippet shows:
>>> bio = crypto._new_mem_buf()
>>> rsa = crypto._lib.EVP_PKEY_get1_RSA(k._pkey)
>>> crypto._lib.PEM_write_bio_RSAPublicKey(bio, rsa)
1
>>> s = crypto._bio_to_string(bio)
>>> print(s)
-----BEGIN RSA PUBLIC KEY-----
MIGJAoGBANF1gYh10F8HTQdM6+bkwAwJ0Md6bMciKbP3qS6KTki3v3m+cM17Szqq
Mp4xxWbvnS2oeotYfn8eaZg0QUTOVDd1F7tuOxVEdvQ9ZEp1aeOCRU3b9QZSmVfg
wJrqDG3f149mNdexI12plwaxyt6odonv6+fEQJrbhrV/nIA8N/EFAgMBAAE=
-----END RSA PUBLIC KEY-----
This is just for illustration purposes. A proper solution should be added to the crypto module itself, via a new method dump_publickey() or the like.

How to read a RSA public key in PEM + PKCS#1 format

I have a RSA public key in PEM format + PKCS#1(I guess):
-----BEGIN RSA PUBLIC KEY-----
MIGJAoGBAJNrHWRFgWLqgzSmLBq2G89exgi/Jk1NWhbFB9gHc9MLORmP3BOCJS9k
onzT/+Dk1hdZf00JGgZeuJGoXK9PX3CIKQKRQRHpi5e1vmOCrmHN5VMOxGO4d+zn
JDEbNHODZR4HzsSdpQ9SGMSx7raJJedEIbr0IP6DgnWgiA7R1mUdAgMBAAE=
-----END RSA PUBLIC KEY-----
I want to get the SHA1 digest of its ASN1 encoded version in Python. The first step should be to read this key, but I failed to do it in PyCrypto:
>> from Crypto.PublicKey import RSA
>> RSA.importKey(my_key)
ValueError: RSA key format is not supported
The documentation of PyCrypto says PEM + PKCS#1 is supported, so I'm confused.
I've also tried M2Crypto, but it turns out that M2Crypto does not support PKCS#1 but only X.509.

PyCrypto supports PKCS#1 in the sense that it can read in X.509 SubjectPublicKeyInfo objects that contain an RSA public key encoded in PKCS#1.
Instead, the data encoded in your key is a pure RSAPublicKey object (that is, an ASN.1 SEQUENCE with two INTEGERs, modulus and public exponent).
You can still read it in though. Try something like:
from Crypto.PublicKey import RSA
from Crypto.Util import asn1
from base64 import b64decode
key64 = 'MIGJAoGBAJNrHWRFgWLqgzSmLBq2G89exgi/Jk1NWhbFB9gHc9MLORmP3BOCJS9k\
onzT/+Dk1hdZf00JGgZeuJGoXK9PX3CIKQKRQRHpi5e1vmOCrmHN5VMOxGO4d+znJDEbNHOD\
ZR4HzsSdpQ9SGMSx7raJJedEIbr0IP6DgnWgiA7R1mUdAgMBAAE='
keyDER = b64decode(key64)
seq = asn1.DerSequence()
seq.decode(keyDER)
keyPub = RSA.construct( (seq[0], seq[1]) )
Starting from version 2.6, PyCrypto can import also RsaPublicKey ASN.1 objects.
The code is then much simpler:
from Crypto.PublicKey import RSA
from base64 import b64decode
key64 = b'MIGJAoGBAJNrHWRFgWLqgzSmLBq2G89exgi/Jk1NWhbFB9gHc9MLORmP3BOCJS9k\
onzT/+Dk1hdZf00JGgZeuJGoXK9PX3CIKQKRQRHpi5e1vmOCrmHN5VMOxGO4d+znJDEbNHOD\
ZR4HzsSdpQ9SGMSx7raJJedEIbr0IP6DgnWgiA7R1mUdAgMBAAE='
keyDER = b64decode(key64)
keyPub = RSA.importKey(keyDER)