I have 2 functions: encrypt/decrypt implemented by AES
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
DEFAULT_MODE = modes.ECB()
def encrypt(data, key):
encryptor = Cipher(algorithms.AES(key), DEFAULT_MODE, backend=default_backend()).encryptor()
return encryptor.update(data) + encryptor.finalize()
def decrypt(data, key):
if len(data) == 0 or len(data)%16 != 0:
raise ValueError
decryptor = Cipher(algorithms.AES(key), DEFAULT_MODE, backend=default_backend()).decryptor()
return decryptor.update(data)
The code can be executed in below:
>>> encrypt(b'a'*16, b'I_got_one_key_in_32bytes_length.')
'\xab\x07\x9d\xa0\xf0\xa0g\x9ae\xd9\x10\x9e\xea2\xb4\x17'
>>> decrypt(b'\xab\x07\x9d\xa0\xf0\xa0g\x9ae\xd9\x10\x9e\xea2\xb4\x17', b'I_got_one_key_in_32bytes_length.')
'aaaaaaaaaaaaaaaa'
The mission here is to change the mode from ECB to others, say GCM.
DEFAULT_MODE = modes.GCM('iv')
The code can still be executed in below:
>>> encrypt(b'a'*16, b'I_got_one_key_in_32bytes_length.')
'Y5y\xbe\xeeK\xb9\x10\xcdf\x99\xa6\x1d\xf2\xa0\x1e'
>>> decrypt(b'Y5y\xbe\xeeK\xb9\x10\xcdf\x99\xa6\x1d\xf2\xa0\x1e', b'I_got_one_key_in_32bytes_length.')
'aaaaaaaaaaaaaaaa'
However, since those encrypt/decrypt function have been used broadly for a while, an issue is raised: How to migrate to new mode without impacting original encrypted data?
PS: my draft thought is logic like below: decrypting data in GCM mode if possible, otherwise decrypting in ECB mode; and encrypting all of them in GCM mode later. But AFAIK, this idea won't work since I have no way to raise the Error. With this approach, is that possible to know the AES mode of encrypted data?
def decrypt(data, key):
try:
# decrypt data by key in GCM
except GCMDecryptFailedError:
# decrypt data by key in ECB
return decrypted_data
The error here is to use a cryptographic algorithm directly, without specifying a specific protocol, and a protocol version number that comes with that. Fortunately, there are ways of getting out of this, and specify a specific protocol.
One of the tricks employed by cryptography is that it is impossible to generate specific content randomly simply because the chance of generating the a specific value reduces by each bit. So what you can do is to prefix your ciphertext with e.g. a 128 / 16 byte bit value. The chance that you generate this value at the start of your ciphertext is 2 to the power of 128 for each message (less if the messages are not random themselves for a specific key). In other words, the chance is as low as guessing an AES-128 key; we call such a chance "negligible". This trick of course depends on the output of ECB encryption using a random key to be random as well.
However, for the future you might want to include one or more bytes as protocol version indicator. So you could send e.g. the byte value 01 as new version, and then the 16 byte magic value, followed by the random nonce for GCM, the ciphertext and the GCM authentication tag (if that's not already included in the GCM ciphertext). Once you got rid of the ECB version of your protocol (version 00 not indicated in your messages) then you can get rid of the magic and re-purpose the 16 bytes in the protocol header of your messages, for protocol 2 or higher.
If you want to generate a nice magic then you could use any kind of 16 byte string, say "Protocol 1, GCM:" (without the quotes) in ASCII. You could also use the leftmost 128 bits of a hash if you want to use a larger string.
So initially your logic would be, in pseudo-code:
versionByte = message[0]
if message.length >= 17 && versionByte == 01h then
magic = message[1- 16]
if magic == "Protocol 1, GCM:" then
gcmDecrypt(message, 17)
else
ecbDecrypt(message, 0)
// --- include other versions here ---
else
ecbDecrypt(message, 0)
Of course this is still a very basic protocol. But at least you can change it later on. You might want to look at more complete protocol specs, such as Fernet, CMS or - of course - TLS instead of just AES-GCM.
Whatever you do: write down your protocol in a separate document and refer to it from your code. You may quote your simple protocol in the source code as easy lookup.
Related
When I used rsa library to encrypt content in Python,
I found that even if the same public key and the same plaintext were used,
the output content was different each time, and the output ciphertext could be decrypted perfectly.
So I want to know how the RSA encryption algorithm implements this algorithm with different encryption results each time.
The following is the source code and the ciphertext output for many times.
import rsa
data = b'hello, world'
pk = rsa.PublicKey(21968272887747488664299300886573437453854580842272801065486318320328573181104433915148345103361664593733184722692105149694142557011266255075972021704711966860643495011049367729520386363274015109405027569939049707059547205662044677513224725454246882263137472476944688288600202939249708651097639414591301098996178101611307541565108035735952182518865647460401330824147744542993709272159435504287548711774248609991298003738752699597664282754244110245104529559246443251024491287411685325071990133422302961361831613169335261576570530061643400976849033234171349450189113706076777344091951159628029458250885131329209309850429, 65537)
sk = rsa.PrivateKey(21968272887747488664299300886573437453854580842272801065486318320328573181104433915148345103361664593733184722692105149694142557011266255075972021704711966860643495011049367729520386363274015109405027569939049707059547205662044677513224725454246882263137472476944688288600202939249708651097639414591301098996178101611307541565108035735952182518865647460401330824147744542993709272159435504287548711774248609991298003738752699597664282754244110245104529559246443251024491287411685325071990133422302961361831613169335261576570530061643400976849033234171349450189113706076777344091951159628029458250885131329209309850429, 65537, 7180742814003184493745817226790609535628314246962295259545720906634095162818242875479619891118201610188935763454388765380592975819694916096822751254380575157372246976924478622789961650274744826184819271605876418277150620865958482714928972468695190683750109638846897363602141498155351308783613387153774908482554823734710213533339079775940427840254792667407339506634483414544868884993644469123554250547973774825288728499603644573043340903253662627022861078040710813466717381393318974263956822836617559198769733538785368579523554468493535497334351910973554355558084517450711717078208243534059900951053098416621979162953, 2892399658197458942905975614589062229163400545478597547382814345027395128547900843767403239802516658965367060847402270250006453487328128143951683257674546551047677883067394312961875875837583648708792776670850392284514504120294996660277476938434444686489314576152155327763997732075822518345380214599954128122325100250621109610911, 7595171996887213720796562116779069406951367089854155042546817829399701614804640519699383335239152053864712615020908685785110173445687693446414448808069297671341400340127530462352491976340390927112062123224788804186559233620266300549932283394695195359373967318632526999572685782623554155939)
print(rsa.encrypt(data, pk)
# 1
b'\x17T\xc0\x03\xa4\xa6\xc06\x83\xdcM\xe5\xf9\xd8t\xc9>\xad}\xc9\x15[\xcc!\x19\x97/\xbf\xc7\xe4\xcbhu\x8d\xfb&\x18\x84\xc8e\xec\xe1\n\xfd$\x92\xda\x12S\x0f\r\xba\x81y\x88E\x9ceu\xd9\xd2Z\xf8\xc3\xd3&\xf2\xf7j\t\t\xf2\xc6w\xf6\x9a7\xbd\x01\x96\xad\xf5\x9e\xf4\xa8,\xd2\x19b\x0f\x05\x0c\xd8G\xe66\x91\x85.\xbdX\x0b\xd9H\xb14\xc6\x88\xb5\xd7\x1f\xed\xf7\xb4\x10\xb7\xad\x9f\xab\x01\r(\r*\xd90\x84\xba\xfb\xd9\x94HK\xdf\xaf\xa0\xf2\x98\x96\xb6*b\xb5\xc0\xa6\xe5A[\x9fwf\x18\x08v\x85\t\xb7\xf7\x97\xc74\xe5{;9qw\xb1u>\t`\xfd\x10\xfbu\xfb\xf5\x11\xe9\xc1\xa0I\x96\x03\xa5\x84\x0b\xcd\x060\xa1\xb1\xbcs|\xfe\xf3N\xad\xddA\xe2l\xf83N\xae\x9c\xbe\x1568\xe9\xf5\xfdn\xe9\xbc\x98\xb5\xb9Bn\xf1]!\x86\xd39\xd2<&\xd6}\x9a\xe2\xa4|\xf0\x9a\xaf\xac\x08^\x93\x174\n~L<+=\x8d\x95'
# 2
b'5\xbc\xb2\xaa\x16\'\xa2\x93\x16D\'S\xfc\x9fm\xc9\xbbF\xa6:dN\x91f\xc1\xaa\x05\xeb\xe4\x16|\xd3\x07#\xd5\xda\xe9\x9b\xd0V\xd4\xb0#Y\xf2G\x0c\xae\xb7A\x9a\xaa\xb8^\xf8\xea\xddj%\xd0\xe8w\xb2\xf1\x9c\xf8D\xcc\x9b\xfe\xea\x16hT\x81\'u`\x10"\xaf\xe3\xd3#\xa0\xc2\x18\x8f^lE\xb0H\xe8\xd5\xf2\x8e\xd8\x8fq;\xd7B]\xc8j\x94\'0\xb0\x80\x0f\xd3\xd1\x90I\x1eL\x91y\x8dA\x01\xda>x`\x0b}6:\xb6o\xcf\xd1=\x15p\xdb\x16\xd3bF\xd5\xc9\\\x86\x1b\xeb\xc4H\x11\x04\xa9o\xe1\xffSF\xe3\xc1\x99\x05\xc44\x03\x86\x81\xbb#>\xfb\xc2\x0bscbW\x0f\xb8\x92\x81\xbb\x19c\xd1n\t\xa4sI\x91+\x97\x9e\x0b\xf1\x8b\xd2;\xa9NV\xc1\xb0#\xd1\xa24P\xce\x93US\xf5\x97=m\xb3\xb6\xd3\x9b\'\xade\x1e\xbc\x80\x13C\x99\x93\x89&\xbd\xde\x83f\\H6\xad2\nFM\xf07q\xe9`\xb1H\x98#X'
# 3
b"'E\xdb\xfd\xe4\xf9\x0c\xe1\xa4l\xaaq\x0e#\xde2\xe9\xe4\x12\xb3\xc2d\xd1W\xde*\x8d<\xcb\x1a\xea\xb4\xb86\x9bV0\r\xef\xfb\xafg\xe8\x1eHzg\x03I\x99ta\xad\x84[r.E\xbb\xc2\xae\xf1\xc2\xafd\xcb\xa6`\xf0)U\x85\xb1\n0\xb2\x05\x17s\xa3\xe3f\xb7\xda\x08\xd1\xae#\xd8\xa7\x90Tce\xc2\xac\xf3Q\x81\xbe1\x92\x8d\xcb\xbf\xfa\x88\xf3'\xe8\xa1\x9e\x9e\xae~\xb90Uq\x98\xe6\x17b\x9d]1\xf6\xabirw\xbc\x89\xae\xd8\xdf\x8a\xf5\xf1\xd4*~\x94\xe38\x1f$\x0e\x94t\xb64\x83q\xf8\x8f\xd6pR\xd4%\xf8\x1cv\xc5\xfe\x8d]\xcfy\xff\xb9\xc7\x10\xaao%\xa8\x13\xce6#Y\xfa\x06\xb8\xab(H^\xd8\x1a\xb63\xb0\xb0c\xe0\x11#\xa9\t\xdd\xa8\\\xeag\xc6H\xa5L\x0b\x10\xdb\xa9\xc44\xdcZ\xf1`\xa2\xc1^;\x1d\xdf\xbf\x92\x894\x847\xe9\x16\x15\xad\xd1c\xf9.\xc21\x02\x85\xb1\x0b\x96=\xf3D\xdf\xf7\xbep\x9c"
# 4
b'$\x82\xc8\x95\xcb\xdaq\xc0\x16\x0e\xef\xb6\xc8\x89\xabKQafM\x10^\x11\xea2\xfc\x8b\x0b~H\xfd\xe5\xe0\x80\x81<\xae\xb7\xfeT)K\xb3\x96\xc0y\x83e\x93\xae\xdb\x93\x82\xea\xb7\xb7\xdbQJX\xb2\xfdM\xf2(A6+e\xb7\x89\x8a\xba6\xb7\xa3\xde*\xea\xe0\x1cR\xa9i\x8a\x9aEK\xa2T\xebM\xa9\x1d\x96\x87\xaf\xb2I\xcej!"\xe2\xc8\xc08\x94\x8a\x18\x1d\t\x11`\xdf*\xbc\xb9\xf6J\xbci\xb3\xcc\xde\xb0\xa5\x98b}o\x94\xbe\xe0\x7f\xe2J\x8a\xa2)R{U\xdfu\xf6UO\xc2C\xf3\'\x87c\x1e\xc6\xe0\xbe\x879\xa5N\xb3J\xc8Cz\x9b\xa7\xec\x90[\xa8\x8a\xac\xeep\\ar\xbd\x94O\xce]\x1fw\x1bm|K\xce\x15\xf6\xcc\xc5\xc84\x9a\x00Z\x0b\xfd\xe9\xfb^6\x9b\xfd\xeb\x8c\xf1h\xda\x17\xc4\xb0\x08\\-\n7\x9e\x1f\x1d\xa7\xb4\xb9\xf0wq\x9a\x15G\xc5\x90\xf5\x00\x89\tI\x16\x90\xbcI\x80z\x90\xdb\nO\xdc\xe5\x8fh\xca'
Any asymmetric encryption method has to be randomized, so that if you encrypt the same plaintext twice, you don't get the same ciphertext. Otherwise it would be very insecure. Anyone who has the public key can encrypt something. Suppose an adversary has a ciphertext, they want to find out the plaintext, and they have partial information about the plaintext (e.g. they know it's a message in a certain format, but they don't know the exact content). They can try encrypting possible values of the plaintext until the result is the ciphertext they want to break. But since the encryption is randomized, they need to use the same data input and the same random value, otherwise they won't get the same ciphertext. And the adversary can't know what random value went into the ciphertext they want to break.
For RSA, in practice, there are two methods for doing encryption. Both are defined by the document known as PKCS#1. Both take the plaintext to encrypt and apply a transformation to it that involves either appending random data (PKCS#1 v1.5) or masking with random data (PSS). Then the result undergoes the well-known exponentiation part of RSA.
You can use the exponentiation to inspect a ciphertext.
n = 21968272887747488664299300886573437453854580842272801065486318320328573181104433915148345103361664593733184722692105149694142557011266255075972021704711966860643495011049367729520386363274015109405027569939049707059547205662044677513224725454246882263137472476944688288600202939249708651097639414591301098996178101611307541565108035735952182518865647460401330824147744542993709272159435504287548711774248609991298003738752699597664282754244110245104529559246443251024491287411685325071990133422302961361831613169335261576570530061643400976849033234171349450189113706076777344091951159628029458250885131329209309850429
e = 65537
d = 7180742814003184493745817226790609535628314246962295259545720906634095162818242875479619891118201610188935763454388765380592975819694916096822751254380575157372246976924478622789961650274744826184819271605876418277150620865958482714928972468695190683750109638846897363602141498155351308783613387153774908482554823734710213533339079775940427840254792667407339506634483414544868884993644469123554250547973774825288728499603644573043340903253662627022861078040710813466717381393318974263956822836617559198769733538785368579523554468493535497334351910973554355558084517450711717078208243534059900951053098416621979162953
c1 = b'\x17T\xc0\x03\xa4\xa6\xc06\x83\xdcM\xe5\xf9\xd8t\xc9>\xad}\xc9\x15[\xcc!\x19\x97/\xbf\xc7\xe4\xcbhu\x8d\xfb&\x18\x84\xc8e\xec\xe1\n\xfd$\x92\xda\x12S\x0f\r\xba\x81y\x88E\x9ceu\xd9\xd2Z\xf8\xc3\xd3&\xf2\xf7j\t\t\xf2\xc6w\xf6\x9a7\xbd\x01\x96\xad\xf5\x9e\xf4\xa8,\xd2\x19b\x0f\x05\x0c\xd8G\xe66\x91\x85.\xbdX\x0b\xd9H\xb14\xc6\x88\xb5\xd7\x1f\xed\xf7\xb4\x10\xb7\xad\x9f\xab\x01\r(\r*\xd90\x84\xba\xfb\xd9\x94HK\xdf\xaf\xa0\xf2\x98\x96\xb6*b\xb5\xc0\xa6\xe5A[\x9fwf\x18\x08v\x85\t\xb7\xf7\x97\xc74\xe5{;9qw\xb1u>\t`\xfd\x10\xfbu\xfb\xf5\x11\xe9\xc1\xa0I\x96\x03\xa5\x84\x0b\xcd\x060\xa1\xb1\xbcs|\xfe\xf3N\xad\xddA\xe2l\xf83N\xae\x9c\xbe\x1568\xe9\xf5\xfdn\xe9\xbc\x98\xb5\xb9Bn\xf1]!\x86\xd39\xd2<&\xd6}\x9a\xe2\xa4|\xf0\x9a\xaf\xac\x08^\x93\x174\n~L<+=\x8d\x95'
print(binascii.unhexlify('0' + hex(pow(int(binascii.hexlify(c1), 16), d, n))[2:]))
That last value is the padded plaintext. You can see the data in there, with padding before it. This is the PKCS#1 v1.5 padding method (which is insecure unless used very carefully, and should not be used except for backward compatibility with systems that require it).
Cryptography noob here. I'm trying to write a script in NodeJS that encrypts a string and produces output that matches the output of my Python script that uses the cryptography.fernet library. My overall goal is to use the original key to encrypt messages in Node that will later be decrypted using Python.
Sample of my Python code:
from cryptography.fernet import Fernet
key = Fernet.generate_key() # For example: 6saGtiTFEXej729GUWSeAyQdIpRFdGhfY2XFUDpvsu8=
f = Fernet(key)
message = 'Hello World'
encoded = message.encode()
encrypted = f.encrypt(encoded)
Which produces the output: gAAAAABhJs_E-dDVp_UrLK6PWLpukDAM0OT5M6bfcqvVoCvg7r63NSi4OWOamLpABuYQG-5wsts_9h7cLbCsWmctArXcGqelXz_BXl_o2C7KM9o7_eq7VTc=
My Node script uses the built-in Crypto module and must also use the same 32-byte key that is being used in my Python program. I know that fernet uses is AES-128-CBC as its algorithm, so that's what I'm using for my Node script.
My NodeJS code:
const crypto = require("crypto");
const key = '6saGtiTFEXej729GUWSeAyQdIpRFdGhfY2XFUDpvsu8=';
const algorithm = 'aes-128-cbc';
const message = 'Hello World';
const iv = crypto.randomBytes(16);
const cipher = crypto.createCipheriv(algorithm, key, iv);
const encrypted = cipher.update(message, 'utf8', 'hex') + cipher.final('hex');
Which is giving me: Error: Invalid key length
My first problem is that I'm unsure how to convert the key so that it's the proper length. I also know from looking at fernet's source code that the key is split into two parts: the first 16 bytes are the signing_key and the last 16 bytes are the encryption_key - I haven't found much information on whether/how I need to deal with those two pieces of the original key in my Node implementation.
Since I'm new to this I'm a little confused on how to accomplish what I'm after. Any tips or advice is very much appreciated.
The specs for the Fernet format can be found on https://github.com/fernet/spec/blob/master/Spec.md
There they specify both a generating and a veryfying steps, here is the generating which should give enough information for your implementation:
Record the current time for the timestamp field.
Choose a unique IV.
Construct the ciphertext:
Pad the message to a multiple of 16 bytes (128 bits) per RFC 5652, section 6.3. This is the same padding technique used in PKCS #7 v1.5 and all versions of SSL/TLS (cf. RFC 5246, section 6.2.3.2 for TLS 1.2).
Encrypt the padded message using AES 128 in CBC mode with the chosen IV and user-supplied encryption-key.
Compute the HMAC field as described above using the user-supplied signing-key.
Concatenate all fields together in the format above.
base64url encode the entire token.
From this we can see that the signing key (first half of full key) is used in HMAC, while the second half is used in the AES128-CBC, so just dividing the key into two separate elements (with proper conversion from hex string to bytes) should be enough for using Node.js crypto module (https://nodejs.org/en/knowledge/cryptography/how-to-use-crypto-module/) to construct your own implementation.
So, in pyDes, a cryptographic library of DES, there is an API, which goes like this pyDes.des(key, [mode], [IV], [pad], [padmode]). An usage of it goes like this k = des("DESCRYPT", CBC, "\0\0\0\0\0\0\0\0", pad=None, padmode=PAD_PKCS5) - where I can either use CBC or ECB mode of encryption.However, as an assignment from my professor, I am told to encrypt using pyDes library, but using CBC and Counter Mode manually.
I managed to do CBC mode fine, unfortunately I am stuck with the counter mode.
Using the given api of des(key, CBC, IV ...) I can only use IV when I use CBC or ECB mode of operation. I can not use it something like des("hello", mode = None, "foo",....) where "foo" is my IV.( I am supposed to implement Counter mode of operation and the iv is random in every single iteration)
So, my question is did anyone faced this issue, and tried to overcome it.
The main operation that you need to isolate in order to implement some mode is the actual block cipher without a mode of operation or padding. pyDes doesn't seem to provide direct access to the block cipher directly, but you can emulate it easily with the ECB mode. ECB is a simple execution of the block cipher on all input blocks in the same way.
The idea would be to create a counter input stream, execute ECB on the input stream to get the key stream and then XOR every byte of the plaintext with the corresponding byte in the key stream.
Steps for CTR mode:
Generate a random nonce (IV) in the range of 0 to 1<<64 (DES block size) which is the starting counter:
import random
r = random.SystemRandom()
nonce = r.randrange(0, 1<<64)
Convert the counter for each block of the plaintext to bytes with struct.pack('>Q', counter) and increase the counter by one
Repeat until you have at least as much bytes in the counter stream as in the plaintext
Encrypt the counter input stream with ECB mode and any available padding
XOR the key stream and the plaintext and throw away the rest of the key stream (if any)
Since CTR mode is a stream cipher, you can use the exact same operation for decrypting with the only difference that the nonce must be supplied from outside. You can prepend the nonce to the ciphertext so that it can be used for decryption. It doesn't have to be secret, but it needs to be unique if the same key is used.
Note that the block size of DES and 3DES doesn't permit to encrypt many ciphertexts or long ciphertexts with CTR under the same key. If you do then you need to change to a block cipher with a bigger block size like AES.
Short Question: Is there a proved strong reversible encryption method (in Python)?
Requirement: Do not require 3rd part of Python libraries.
Apply environment: transport data through networks.
I saw a method using str.translate() with a key-generated table. Here is the table generating function:
def get_table(key):
m = hashlib.md5()
m.update(key)
s = m.digest()
(a, b) = struct.unpack('<QQ', s)
table = [c for c in string.maketrans('', '')]
for i in xrange(1, 1024):
table.sort(lambda x, y: int(a % (ord(x) + i) - a % (ord(y) + i)))
return ''.join(table)
Questions about this function:
Is this a good/strong reversible encryption?
In the function 1024 is a big number, need we loop so many times to get a table that strong enough?
Thanks in advance.
If you want strong encryption without a third-party library, you're out of luck--the Python Standard Library only has hash functions. If you want secure encryption you'll have to either implement something like AES yourself (this is not a good idea, as it's really easy for the inexperienced to mess up when implementing an encryption algorithm), or change your requirements and use PyCrypto.
An xor cipher would work nicely (if you bitwise-XOR each character of the message with its counterpart in the key, you can get back to the message again by XORing the ciphertext with the key again).
XOR Cipher
EDIT: Exactly how you acquire the key will determine the security of this cipher, but it's a fast, easily reversible cipher.
EDIT2: Specifically, see these lines from the Wiki on how you might make this a secure cipher system...
"If the key is random and is at least as long as the message, the XOR cipher is much more secure than when there is key repetition within a message.[3] When the keystream is generated by a pseudo-random number generator, the result is a stream cipher. With a key that is truly random, the result is a one-time pad, which is unbreakable even in theory."
you could make your own encrpytion program by using an offset factor.
ie; convert each letter into a number using ord().
add to the number using a randomly generated offset key.
convert back into letter using chr()
and to decrypt:
convert each character into a number using ord()
subtract by the offset key
convert back into letter using chr()
you know have the original message.
hope it helps you
I'm writing a piece of code to encrypt a text using symmetric encryption. But it's not coming back with the right result...
from Crypto.Cipher import AES
import os
crypto = AES.new(os.urandom(32), AES.MODE_CTR, counter = lambda : os.urandom(16))
encrypted = crypto.encrypt("aaaaaaaaaaaaaaaa")
print crypto.decrypt(encrypted)
Here, the decrypted text is different from the original.
I don't really understand much about cryptography so please bear with me. I understand the CTR mode requires a "counter" function to supply a random counter each time, but why does it need it to be 16 bytes when my key is 32 bytes and it insists that my message is in multiples of 16 bytes too? Is this normal?
I'm guessing that it doesn't get back to the original message because the counter changed between encrypt and decrypt. But then, how is it supposed to work theoretically anyway? What am I doing wrong? Anyway, I'm forced to resort back to ECB until I figure this out :(
The counter must return the same on decryption as it did on encryption, as you intuit, so, one (NOT SECURE AT ALL) way to do it is:
>>> secret = os.urandom(16)
>>> crypto = AES.new(os.urandom(32), AES.MODE_CTR, counter=lambda: secret)
>>> encrypted = crypto.encrypt("aaaaaaaaaaaaaaaa")
>>> print crypto.decrypt(encrypted)
aaaaaaaaaaaaaaaa
CTR is a block cipher, so the "16-at-a-time" constraint that seems to surprise you is a pretty natural one.
Of course, a so-called "counter" returning the same value at each call is grossly insecure. Doesn't take much to do better, e.g....:
import array
class Secret(object):
def __init__(self, secret=None):
if secret is None: secret = os.urandom(16)
self.secret = secret
self.reset()
def counter(self):
for i, c in enumerate(self.current):
self.current[i] = c + 1
if self.current: break
return self.current.tostring()
def reset(self):
self.current = array.array('B', self.secret)
secret = Secret()
crypto = AES.new(os.urandom(32), AES.MODE_CTR, counter=secret.counter)
encrypted = crypto.encrypt(16*'a' + 16*'b' + 16*'c')
secret.reset()
print crypto.decrypt(encrypted)
AES is a block cipher: it's an algorithm (more precisely, a pair of algorithms) that takes a key and a message block and either encrypts or decrypts the block. The size of a block is always 16 bytes, regardless of the key size.
CTR is a mode of operation. It's a pair of algorithms that builds on a block cipher to produce a stream cipher, which can encrypt and decrypt messages of arbitrary lengths.
CTR works by combining successive message blocks with the encryption of successive values of a counter. The size of the counter needs to be one block so that it's valid input for the block cipher.
Functionally, it doesn't matter what the successive values of the counter are, as long as the encryption and decryption side use the same sequence. Usually the counter is treated as a 256-bit number and incremented for each successive block, with an initial value chosen at random. Thus, usually, the incrementation method is baked into the code, but the decryption side needs to know what the initial value is, so encryption side sends or stores the initial counter value at the beginning of the encrypted message.
For security, it is vital to never repeat the same counter value with a given key. So for a single-use key, it's ok to start with '\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'. But if the key is used multiple times then the second message is not allowed to reuse any of the counter values used by the first message, and the easiest way to ensure that is to generate the initial counter value at random (with a 2^128 space, the chances of a collision are acceptably negligible).
By letting the caller pick a counter function, the PyCrypto library gives you plenty of rope to hang yourself. You should use Crypto.Util.Counter, not just “for better performance” as the documentation puts it, but because it's easier to build something secure than what you're likely to come up with on your own. And even so, take care to use a random initial value, which is not the default.
import binascii
import os
from Crypto.Cipher import AES
from Crypto.Util import Counter
def int_of_string(s):
return int(binascii.hexlify(s), 16)
def encrypt_message(key, plaintext):
iv = os.urandom(16)
ctr = Counter.new(128, initial_value=int_of_string(iv))
aes = AES.new(key, AES.MODE_CTR, counter=ctr)
return iv + aes.encrypt(plaintext)
def decrypt_message(key, ciphertext):
iv = ciphertext[:16]
ctr = Counter.new(128, initial_value=int_of_string(iv))
aes = AES.new(key, AES.MODE_CTR, counter=ctr)
return aes.decrypt(ciphertext[16:])
I may be definitely late and I may have overlooked the previous answers, but I didn't find a clear statement of how this should (at least IMHO) be done according with the PyCrypto packages.
The Crypto.Util.Counter package provides callable stateful counters, which are very useful, but it was easy at least for me to use them improperly.
You have to create a counter, with e.g. ctr = Counter.new('parameters here'). Every time your counter is called by your counter mode cipher object to encrypt the message, it is incremented. This is needed for good cryptography practices, otherwise information about equal blocks may leak from the ciphertext.
Now you cannot call the decryption function on the same cipher object, because it would call again the same counter which in the meanwhile has been incremented, possibly several times. What you need to do is to create a new cipher object with a different counter initialized with the same parameters. In this way the decryption works properly, starting the counter from the same point as the encryption was done.
Working example below:
# Import modules
from Crypto.Cipher import AES
from Crypto import Random
from Crypto.Util import Counter
# Pad for short keys
pad = '# constant pad for short keys ##'
# Generate a random initialization vector, to be used by both encryptor and decryptor
# This may be sent in clear in a real communication
random_generator = Random.new()
IV = random_generator.read(8)
# Encryption steps
# Ask user for input and pad or truncate to a 32 bytes (256 bits) key
prompt = 'Input your key. It will padded or truncated at 32 bytes (256 bits).\n-: '
user_keye = raw_input(prompt)
keye = (user_keye + pad)[:32]
# Create counter for encryptor
ctr_e = Counter.new(64, prefix=IV)
# Create encryptor, ask for plaintext to encrypt, then encrypt and print ciphertext
encryptor = AES.new(keye, AES.MODE_CTR, counter=ctr_e)
plaintext = raw_input('Enter message to cipher: ')
ciphertext = encryptor.encrypt(plaintext)
print ciphertext
print
# Decryption steps
# Ask user for key: it must be equal to that used for encryption
prompt = 'Input your key. It will padded or truncated at 32 bytes (256 bits).\n-: '
user_keyd = raw_input(prompt)
keyd = (user_keyd + pad)[:32]
# Create counter for decryptor: it is equal to the encryptor, but restarts from the beginning
ctr_d = Counter.new(64, prefix=IV)
# Create decryptor, then decrypt and print decoded text
decryptor = AES.new(keyd, AES.MODE_CTR, counter=ctr_d)
decoded_text = decryptor.decrypt(ciphertext)
print decoded_text
why does it need it to be 16 bytes when my key is 32 bytes
It has to be the same length as the cipher's block size. CTR mode just encrypts the counter and XORs the plaintext with the encrypted counter block.
Notes:
the counter value MUST be unique -- if you EVER use the same counter value to encrypt two different plaintexts under the same key, you just gave away your key.
like an IV, the counter is NOT secret -- just send it along with the ciphertext. If you make the code more complicated by trying to keep it secret, you will probably shoot yourself in the foot.
the counter value need not be unpredictable -- starting with zero and adding one for each block is perfectly fine. But note that if you encrypt multiple messages, you need to keep track of the counter values that have already been consumed, i.e. you need to keep track of how many blocks have already been encrypted with that key (and you can't use the same key in different instances of your program or on different machines).
the plain text can be any length -- CTR mode turns a block cipher into a stream cipher.
Standard disclaimer: Crypto is hard. If you don't understand what you are doing, you will get it wrong.
I just want to store some passwords across sessions.
Use scrypt. scrypt includes encrypt and decrypt which use AES-CTR with a password-derived key.
$ pip install scrypt
$ python
>>> import scrypt
>>> import getpass
>>> pw = getpass.getpass("enter password:")
enter password:
>>> encrypted = scrypt.encrypt("Guido is a space alien.",pw)
>>> out = scrypt.decrypt(encrypted,pw)
>>> out
'Guido is a space alien.'
The initialization vector ("counter") needs to stay the same, just as the key does, between encryption and decryption. It is used so that you can encode the same text a million times, and get different ciphertext each time (preventing some known plaintext attacks and pattern matching / attacks). You still need to use the same IV when decrypting as when encrypting. Usually when you start decrypting a stream, you initialize the IV to the same value that you started with when you started encrypting that stream.
See http://en.wikipedia.org/wiki/Initialization_vector for info on initialization vectors.
Note that os.urandom(16) is not 'deterministic', which is a requirement for counter functions. I suggest you use the increment function, as that is how CTR mode is designed. The initial counter value should be random, but the successive values should be fully predictable from the initial value (deterministic). The initial value may even be taken care of for you (I don't know the details)
About the key, IV, and input sizes, it sounds like the cipher you chose has a block size of 16 bytes. Everything you describe fits that and seems normal to me.