Decrypt Kamstrup WMBUS in Python - python

I have a Kamstrup WMbus water meter sending out frames like this:
21442D2C529027581B168D2814900939201F0775C6452FBBAC155B46A546035219D51AB8
I am trying to decrypt this in Python.
Breaking the frame up I have the following values:
Key 16ce383ebc0790e928215525cd4f7abf
Input IV 2D2C529027581B162890093920000000
Input Data 1F0775C6452FBBAC155B46A546035219D5
Pasting this into http://www.cryptogrium.com/aes-ctr.html results in a valid decrypted frame:
bbe57934ddc46a71004400000044000000
I have tried both PyCrypto and PyCryptodome but neither gives the same correct answer than cryptogrium.com.
from Cryptodome.Cipher import AES
# ctr = Crypto.Util.Counter.new(128, initial_value=int("0000002039099028161B582790522C2D", 16))
# cipher = Crypto.Cipher.AES.new(ecryptionKey, Crypto.Cipher.AES.MODE_CTR, counter=ctr)
# secret = Secret()
spec = AES.new(ecryptionKey, AES.MODE_CTR, iv=inputIV)
dataDec = cipher.decrypt(data)
The first commented out approach runs but gives the wrong result.
The second one stops with the error:
TypeError: Invalid parameters for CTR mode: {'iv': bytearray(b"-,R\x90\'X\x1b\x16(\x90\t9 \x00\x00\x00")}
In C# we are using the following which works:
IBufferedCipher cipher = CipherUtilities.GetCipher("AES/CTR/NoPadding");
ParametersWithIV ivAndKey = new ParametersWithIV(new KeyParameter(keyBytes), inputIVBytes);
cipher.Init(false, ivAndKey);
...
int length1 = cipher.ProcessBytes(data, 0, data.Length, outBuf, 0);
int length2 = cipher.DoFinal(outBuf, length1);
...
I am confused because C# uses the parameters I have: key, data, IV
But Python expects: key, data, counter
Does anyone have an example how I can decrypt this in Python3? Or maybe explain how I should use the IV to set up the counter for AES-CNT?

In the end one of the examples from https://www.programcreek.com/python/example/87998/Crypto.Cipher.AES.MODE_CTR put me on the right track. After fixing a couple of typos in my code it worked correctly.
def kamstrupDecrypt(meterSerial, frame, payload):
inputIV = getInputIV(frame)
print("IV:", binascii.hexlify(inputIV))
ecryptionKey = getEncryptionKey(meterSerial)
print("Key:", binascii.hexlify(ecryptionKey))
counter = Counter.new(128, initial_value = bytes_to_long(inputIV))
cipher = AES.new(ecryptionKey, AES.MODE_CTR, counter=counter)
payloadDec = cipher.decrypt(payload)
print("Decrypted: ", binascii.hexlify(payloadDec))
return payloadDec

Related

Object type <class 'tuple'> cannot be passed to C code when encrypting file in django

so i want to encrypt file .txt with keyword i was input. but i got error message
Object type <class 'tuple'> cannot be passed to C code
my problem is with this code:
aes = AES.new(key.encode('utf8'), AES.MODE_CTR, counter=ctr)
if i add encode to ctr, then i got error message 'dict' object has no attribute 'encode'
if i remove don't add any encode to key and ctr, then i got error message Object type <class 'str'> cannot be passed to C code
can someone please help me to fix it? i was using django to encrypt with AES 128 with method CTR. or maybe someone can give me example aes encryption with another method but can be run in django. here's my full function code:
# AES supports multiple key sizes: 16 (AES128), 24 (AES192), or 32 (AES256).
key_bytes = 16
# Takes as input a 32-byte key and an arbitrary-length plaintext and returns a
# pair (iv, ciphtertext). "iv" stands for initialization vector.
def encrypt(key, testpass):
assert len(key) == key_bytes
print(testpass)
print(key)
# Choose a random, 16-byte IV.
iv = Random.new().read(AES.block_size)
# Convert the IV to a Python integer.
iv_int = int(binascii.hexlify(iv), 16)
# Create a new Counter object with IV = iv_int.
ctr = Counter.new(AES.block_size * 8, initial_value=iv_int)
print(ctr)
# Create AES-CTR cipher.
aes = AES.new(key.encode('utf8'), AES.MODE_CTR, counter=ctr)
# Encrypt and return IV and ciphertext.
ciphertext = aes.encrypt(testpass)
print(iv)
print(ciphertext)
return (iv, ciphertext)
here's how i called that function:
testpass = Audio_store.objects.all().values_list('password').last()
enkripsi = encrypt("testingtesting11", testpass)
when i print testpass, it contains ('testpass_3kEMV2T.txt',)
but when i print testpass.encode("utf-8"), it shows nothing
Your test data is ('testpass_3kEMV2T.txt',), you need to pull out the element from the tuple and pass it in as bytes to your encrypt method, because the AES.encrypt method requires bytes | bytearray | memoryview as its argument. Here's a version of your encrypt that determines if the argument you pass it is a str and encodes it to bytes if so (but it doesn't check if it's otherwise bytes, I leave that as an exercise for you if you want more stringent checking).
import binascii
from Crypto.Cipher import AES
from Crypto.Util import Counter
from Crypto import Random
key_bytes = 16
# here's a rudimentary change that accepts bytes or
# str
def encrypt(key, plaintext):
if isinstance(plaintext, str):
plaintext = plaintext.encode("utf-8")
assert len(key) == key_bytes
# Choose a random, 16-byte IV.
iv = Random.new().read(AES.block_size)
# Convert the IV to a Python integer.
iv_int = int(binascii.hexlify(iv), 16)
# Create a new Counter object with IV = iv_int.
ctr = Counter.new(AES.block_size * 8, initial_value=iv_int)
print(ctr)
# Create AES-CTR cipher.
aes = AES.new(key.encode('utf8'), AES.MODE_CTR, counter=ctr)
# Encrypt and return IV and ciphertext.
ciphertext = aes.encrypt(plaintext)
return (iv, ciphertext)
# for now, since you're getting your data from the db which is a 1-tuple, you have to pull out the first elem
testpass = ('testpass_3kEMV2T.txt',)
print(encrypt("a"*16, testpass[0]))
Output:
{'counter_len': 16, 'prefix': b'', 'suffix': b'', 'initial_value': 302861312392273214314458272471454024973, 'little_endian': False}
(b'\xe3\xd8\xf7\x90\xcd\x96m\xcb\xa5g)\xd1\xda\xc3\x85\r', b'-F)\x83\x9a\xf9\xe1\xc3\xfb\xfa_<^\x1c:q\x07\xa1#\xbb')

Exchanging encrypted messages between python and swift

I need to have a python code and a swift code exchange encrypted message.
Here's what I tried:
Fernet
After a review of the options, I thought that a symetric key algorithm could work well.
In python (as usual), it is straightforward to encrypt and decrypt:
Fernet(key).encrypt(b"mdg") # encrypt
Fernet(key).decrypt(encryptedMsg) # decrypt
In swift, it seemed initially straightforward with something along the lines of:
func encrypt(key: String, msg: String) throws -> String {
let data = Data(base64URL: key)!
let symetricKey = try! SymmetricKey(data: d)
let msgUtf8 = msg.data(using: .utf8)!
let sealBox = try! AES.GCM.seal(msgUtf8, using: symetricKey, nonce: nil)
return sealBox.combined.base64EncodedString();
}
However, I have been unable to find the algorithm in swift matching python's Fernet.
ChaCha
While searching for the problem, I landed on this amazing answer from Bram. Very unfortunately it only solves one side of my problem : encrypting messages in python and decoding them in swift. I also need the reverse process.
How to solve this?
To start, we first need a way to create secure random values to generate the IV and keys. You can also generate the keys using CryptoKit's SymmetricKey and extract the data from them, but for now, I'll use this function.
extension Data {
static func secureRandom(ofSize size: Int) -> Data {
var output = [UInt8](repeating: 0, count: size)
_ = SecRandomCopyBytes(kSecRandomDefault, size, &output)
return Data(output)
}
}
We then require the possibility to compute the AES CBC ciphertext, which can be done using CommonCrypto.
func encrypt(plaintext: Data, key: Data, iv: Data) -> Data {
var encryptor: CCCryptorRef?
defer {
CCCryptorRelease(encryptor)
}
var key = Array(key)
var iv = Array(iv)
var plaintext = Array(plaintext)
CCCryptorCreate(CCOperation(kCCEncrypt), CCAlgorithm(kCCAlgorithmAES), CCOperation(kCCOptionPKCS7Padding), &key, key.count, &iv, &encryptor)
var outputBytes = [UInt8](repeating: 0, count: CCCryptorGetOutputLength(encryptor, plaintext.count, false))
CCCryptorUpdate(encryptor, &plaintext, plaintext.count, &outputBytes, outputBytes.count, nil)
var movedBytes = 0
var finalBytes = [UInt8](repeating: 0, count: CCCryptorGetOutputLength(encryptor, 0, true))
CCCryptorFinal(encryptor, &finalBytes, finalBytes.count, &movedBytes)
return Data(outputBytes + finalBytes[0 ..< movedBytes])
}
and the HMAC with the SHA-256 hash function. I recommend using CryptoKit's HMAC implementation here, but to keep things simple, I went with the CommonCrypto implementation.
func computeHMAC(_ data: Data, using key: Data) -> Data {
var data = Array(data)
var key = Array(key)
var macOut = [UInt8](repeating: 0, count: Int(CC_SHA256_DIGEST_LENGTH))
CCHmac(CCHmacAlgorithm(kCCHmacAlgSHA256), &key, key.count, &data, data.count, &macOut)
return Data(macOut)
}
This brings all of this together into the following
let plaintext = Data("Hello world!".utf8)
let signingKey = Data.secureRandom(ofSize: kCCKeySizeAES128)
let cryptoKey = Data.secureRandom(ofSize: kCCKeySizeAES128)
let fernetKey = (signingKey + cryptoKey).base64EncodedString()
let version: [UInt8] = [0x80]
let timestamp: [UInt8] = {
let timestamp = Int(Date().timeIntervalSince1970).bigEndian
return withUnsafeBytes(of: timestamp, Array.init)
}()
let iv = Data.secureRandom(ofSize: kCCBlockSizeAES128)
let ciphertext = encrypt(plaintext: plaintext, key: cryptoKey, iv: iv)
let hmac = computeHMAC(version + timestamp + iv + ciphertext, using: signingKey)
let fernetToken = (version + timestamp + iv + ciphertext + hmac).base64EncodedString()
print("Fernet key: \(fernetKey)")
print("Fernet token: \(fernetToken)")
An example output can be
Fernet key: 7EwFlYNKTGfj+2fSgL3AUqtrRqRs4D1TWNK7t2XbGJQ=
Fernet token: gAAAAABivCLM0y0poDtGOohT1yK4XTDJppYPJdu4fuDTZ5tb9P9KP5ACgX8aJq4imsSdbzOCcvY3Tueo4FYbwyG+ZugozILL+Q==
We can use this in python using cryptography.io's implementation
from cryptography.fernet import Fernet
key = b'7EwFlYNKTGfj+2fSgL3AUqtrRqRs4D1TWNK7t2XbGJQ='
token = b'gAAAAABivCLM0y0poDtGOohT1yK4XTDJppYPJdu4fuDTZ5tb9P9KP5ACgX8aJq4imsSdbzOCcvY3Tueo4FYbwyG+ZugozILL+Q=='
Fernet(key).decrypt(token)
# b'Hello world!'

Implementing PKCS7 padding in python for a C# script

I have this encrypt with AES function in python:
def encrypt_text(self):
raw = self.PKCS7_padding()
iv = Random.new().read(self.block_size)
cipher = AES.new(self.key, AES.MODE_CBC, iv)
encrypted = base64.b64encode(iv + cipher.encrypt(raw))
return encrypted
And then padding it with:
def PKCS7_padding(self):
return self.text+chr(16-len(self.text)%16)*(16-len(self.text)%16)
But when I send it to this c# function
public static string DecryptString(string key, string cipherText)
{
byte[] iv = new byte[16];
byte[] buffer = Convert.FromBase64String(cipherText);
using (Aes aes = Aes.Create())
{
aes.Key = Convert.FromBase64String(key);
aes.IV = iv;
ICryptoTransform decryptor = aes.CreateDecryptor(aes.Key, aes.IV);
using (MemoryStream memoryStream = new MemoryStream(buffer))
{
using (CryptoStream cryptoStream = new CryptoStream((Stream)memoryStream, decryptor, CryptoStreamMode.Read))
{
using (StreamReader streamReader = new StreamReader((Stream)cryptoStream))
{
return streamReader.ReadToEnd();
}
}
}
}
}
Say i send "hello world"
I get a weird mix of the string i sent plus random non pritable characters. EX: "�Ȯ�Ŗf�"��Xhello world"
The key is generated as follows:
key = base64.b64encode(os.urandom(32))
return key.decode()
I do not get any errors but weird text any ideas Thanks! If you have any questions please feel free to ask
In the Python code, a random IV is generated and concatenated with the ciphertext. However, in the C# code, the separation of IV and ciphertext is missing. Instead, a zero IV is applied and the concatenation of IV and ciphertext is decrypted, which causes the problem.
The separation of IV and ciphertext can be added to the C# code e.g. as follows:
byte[] buffer = ...
byte[] ct = new byte[buffer.Length - iv.Length];
Buffer.BlockCopy(buffer, 0, iv, 0, iv.Length);
Buffer.BlockCopy(buffer, iv.Length, ct, 0, ct.Length);
buffer = ct;
using ...
Note that PyCryptodome supports padding in a dedicated module: Crypto.Util.Padding.
Also, in the Python code the Base64 encoded key is nowhere Base64 decoded (but maybe the corresponding code was just not posted).

Correct way to use Poly1305 with ChaCha20?

I'm trying to use ChaCha20-Poly1305 cipher from the cryptography module,
but there is only ChaCha20 cipher and Poly1305 MAC available.
This is the way I tried to combine them at first:
from cryptography.hazmat.primitives.poly1305 import Poly1305
from cryptography.hazmat.primitives.ciphers import (
Cipher,
algorithms as algo,
)
from cryptography.hazmat.backends import default_backend as defb
class ChaCha20Poly1305:
def __init__(self, locking, key, nonce):
self._locking = locking
# only accepts 16 bytes nonce
cipher = Cipher(algo.ChaCha20(key, nonce), None, defb())
if locking:
self._cipher = cipher.encryptor()
else:
self._cipher = cipher.decryptor()
self._auth = Poly1305(key)
self._auth.update(nonce)
def update(self, data):
ctxt = self._cipher.update(data)
self._auth.update(ctxt)
return ctxt
def finalize(self, tag=None):
if not self._locking
if tag is None:
raise ValueError('tag required')
self._auth.verify(tag)
def calculate_tag(self):
return self._auth.calculate_tag()
Is this the correct way to use this cipher with Poly1305?
Edit: Although cryptography provides ChaCha20Poly1305,
it does NOT support encrypting data continuously.
It simply takes a piece of data, encrypts it and returns the ciphertext
with appended MAC. And that is not what I want.
It's possible to implement a streaming authenticated encryption / decryption with the Cryptography implementations ChaCha20 and Poly1305 analogous to the PyCryptodome implementation ChaCha20_Poly1305. The posted code already does this essentially, whereby the following points are missing or buggy:
The cryptography implementation ChaCha20 expects the complete 16 bytes IV, i.e. nonce (12 bytes) and counter (4 bytes), in little endian format, s. RFC 7539 sec 2.3.
The counter value 0 is used to generate the Poly1305 key, the counter values from and including 1 for encryption, s. RFC 7539 sec 2.4 and sec 2.6.
The Poly1305 key does not simply correspond to the encryption key, but must be derived from this and the nonce with the counter 0, s. RFC 7539 sec 2.6.
Apart from the derivation of the Poly1305 key, the nonce is not further involved in the calculation of the tag.
The additional authenticated data (AAD) must be taken into account, s. RFC 7539 sec 2.8.
Before the tag is calculated, the data (AAD, if present, and ciphertext) are formatted in a defined way, using Zero padding for AAD and ciphertext and appending the lengths of AAD and ciphertext each as an 8 byte integer in little endian byte order, s. RFC 7539 sec 2.8.
The following code takes these points into account, should illustrate the fundamentals and must / can be adapted to individual needs:
from cryptography.hazmat.backends import default_backend as defb
from cryptography.hazmat.primitives.poly1305 import Poly1305
from cryptography.hazmat.primitives.ciphers import Cipher
from cryptography.hazmat.primitives.ciphers import algorithms as algo
class ChaCha20Poly1305:
def __init__(self, encrypt, key, nonce):
self._encrypt = encrypt
self._dataLength = 0;
self._aadLength = 0;
self._nonceCounter = (0).to_bytes(4, byteorder='little') + nonce # Create 16 bytes IV for Poly1305 key derivation
self._nonceEncrypt = (1).to_bytes(4, byteorder='little') + nonce # Create 16 bytes IV for encryption / decryption
cipher = Cipher(algo.ChaCha20(key, self._nonceEncrypt), None, defb())
if encrypt:
self._cipher = cipher.encryptor()
else:
self._cipher = cipher.decryptor()
polyKey = self.__getPolyKey(key) # Get Poly1305 key
self._auth = Poly1305(polyKey)
# Add AAD and zero pad if nnecessary (optional, may only be called once and before first 'update' call)
def updateAAD(self, aad):
self._auth.update(aad)
self._aadLength = len(aad)
self._auth.update(self.__getZeroBytes(self._aadLength))
# Add ciphertext / plaintext for encryption / decryption and actualize tag
def update(self, data):
ctxt = self._cipher.update(data)
self._dataLength += len(ctxt)
if self._encrypt:
self._auth.update(ctxt)
else:
self._auth.update(data)
return ctxt
# Complete padding and verify tag (only decryption)
def verify_tag(self, tag=None):
if not self._encrypt:
self.__pad()
if tag is None:
raise ValueError('tag required')
self._auth.verify(tag)
else:
raise ValueError('Tag verification only during decryption')
# Complete padding and calculate tag (only encryption)
def calculate_tag(self):
if self._encrypt:
self.__pad()
return self._auth.finalize()
else:
raise ValueError('Tag calculation only during encryption')
# Complete formatting: zero pad ciphertext, append AAD and ciphertext lengths
def __pad(self):
self._auth.update(self.__getZeroBytes(self._dataLength))
self._auth.update(self._aadLength.to_bytes(8, byteorder='little'))
self._auth.update(self._dataLength.to_bytes(8, byteorder='little'))
# Zero pad data (AAD or ciphertext)
def __getZeroBytes(self, len):
spareBytes = len % 16
if (spareBytes != 0):
length = 16 - spareBytes
return bytes([0]) * length
return b''
# Derive Poly1305 key
def __getPolyKey(self, key):
cipher = Cipher(algo.ChaCha20(key, self._nonceCounter), None, defb())
cipher = cipher.encryptor()
key = cipher.update(b"\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0");
return key
The implementation satisfies the test vector from RFC 7539, sec 2.8.2:
# Test vector from RFC 7539, sec 2.8.2
plaintext1 = b"Ladies and Gentlemen of the class "
plaintext2 = b"of '99: If I could offer you only one"
plaintext3 = b" tip for the future, sunscreen would be it."
nonce = bytes.fromhex("070000004041424344454647")
key = bytes.fromhex("808182838485868788898a8b8c8d8e8f909192939495969798999a9b9c9d9e9f")
# Encryption
ccEnc = ChaCha20Poly1305(True, key, nonce)
ccEnc.updateAAD(bytes.fromhex('50515253c0c1c2c3c4c5c6c7'))
ct1 = ccEnc.update(plaintext1)
ct2 = ccEnc.update(plaintext2)
ct3 = ccEnc.update(plaintext3)
tag = ccEnc.calculate_tag()
print("Ciphertext:\n%s\n" % (ct1 + ct2 + ct3).hex())
print("Tag:\n%s\n" % tag.hex())
# Decryption
ccDec = ChaCha20Poly1305(False, key, nonce)
ccDec.updateAAD(bytes.fromhex('50515253c0c1c2c3c4c5c6c7'))
dt1 = ccDec.update(ct1)
dt2 = ccDec.update(ct2)
dt3 = ccDec.update(ct3)
ccDec.verify_tag(tag)
print("Decrypted:\n%s\n" % (dt1 + dt2 + dt3))
Note: It's of course important not to trust the decrypted data until it has been successfully authenticated! Just like the PyCryptodome implementation, the construct tempts to work with unauthenticated (and possibly corrupted) data. This problem has already been pointed out in detail in the comments and more reliable alternatives have also been suggested (see also the linked post in the other answer).
It simply takes a piece of data, encrypts it and returns the ciphertext with appended MAC. And that is not what I want.
I think this is what you really want. This is how the popular AE (and AEAD) works.
BTW, have you seen this?

Python AES.MODE_CTR does not decrypt as expected

I am trying to rewrite a node.js decoding function in python
this is the function I am trying to rewrite
module.exports.decrypt = function (data, key, iv, hmacKey) {
iv = iv.slice(0, 16)
var decipherer = crypto.createDecipheriv('aes-128-ctr', key, iv)
var hmac
if (hmacKey) {
hmac = crypto.createHmac('sha1', hmacKey).update(data).digest()
}
return {
data: decipherer.update(data),
hmac: hmac
}
}
the key parameter is generated by crypto.pbkdf2Sync function of node's standard library.
What I implemented in python is
from Crypto.Cipher import AES
from Crypto.Util import Counter
def decrypt(data, key, iv, hmac=None):
iv = iv[:16]
ctr = Counter.new(128)
cipher = AES.new(key, mode=AES.MODE_CTR, IV=iv, counter=ctr)
return cipher.decrypt(data)
I provided the python decrypt function the same same data that the js function receives.
since the data and key, iv params that the js function expects are binary buffers, I copied them to the python code as hex strings and then used binascii.unhexlify in the python code before passing them to the ecrypt python function.
so the arguments that I am providing to both functions are (hex)
data 972acf88c5d7
key 129b6e542600889a75ec7659d9dc23df
iv 31323331323331323331323331323331323331323331313233313233313233313233
The python function returns gibberish.
can anyone help?
thanks!

Categories