We Keep Coding

How is the random ciphertext in RSA encryption algorithm implemented?

When I used rsa library to encrypt content in Python,
I found that even if the same public key and the same plaintext were used,
the output content was different each time, and the output ciphertext could be decrypted perfectly.
So I want to know how the RSA encryption algorithm implements this algorithm with different encryption results each time.
The following is the source code and the ciphertext output for many times.
import rsa
data = b'hello, world'
pk = rsa.PublicKey(21968272887747488664299300886573437453854580842272801065486318320328573181104433915148345103361664593733184722692105149694142557011266255075972021704711966860643495011049367729520386363274015109405027569939049707059547205662044677513224725454246882263137472476944688288600202939249708651097639414591301098996178101611307541565108035735952182518865647460401330824147744542993709272159435504287548711774248609991298003738752699597664282754244110245104529559246443251024491287411685325071990133422302961361831613169335261576570530061643400976849033234171349450189113706076777344091951159628029458250885131329209309850429, 65537)
sk = rsa.PrivateKey(21968272887747488664299300886573437453854580842272801065486318320328573181104433915148345103361664593733184722692105149694142557011266255075972021704711966860643495011049367729520386363274015109405027569939049707059547205662044677513224725454246882263137472476944688288600202939249708651097639414591301098996178101611307541565108035735952182518865647460401330824147744542993709272159435504287548711774248609991298003738752699597664282754244110245104529559246443251024491287411685325071990133422302961361831613169335261576570530061643400976849033234171349450189113706076777344091951159628029458250885131329209309850429, 65537, 7180742814003184493745817226790609535628314246962295259545720906634095162818242875479619891118201610188935763454388765380592975819694916096822751254380575157372246976924478622789961650274744826184819271605876418277150620865958482714928972468695190683750109638846897363602141498155351308783613387153774908482554823734710213533339079775940427840254792667407339506634483414544868884993644469123554250547973774825288728499603644573043340903253662627022861078040710813466717381393318974263956822836617559198769733538785368579523554468493535497334351910973554355558084517450711717078208243534059900951053098416621979162953, 2892399658197458942905975614589062229163400545478597547382814345027395128547900843767403239802516658965367060847402270250006453487328128143951683257674546551047677883067394312961875875837583648708792776670850392284514504120294996660277476938434444686489314576152155327763997732075822518345380214599954128122325100250621109610911, 7595171996887213720796562116779069406951367089854155042546817829399701614804640519699383335239152053864712615020908685785110173445687693446414448808069297671341400340127530462352491976340390927112062123224788804186559233620266300549932283394695195359373967318632526999572685782623554155939)
print(rsa.encrypt(data, pk)
# 1
b'\x17T\xc0\x03\xa4\xa6\xc06\x83\xdcM\xe5\xf9\xd8t\xc9>\xad}\xc9\x15[\xcc!\x19\x97/\xbf\xc7\xe4\xcbhu\x8d\xfb&\x18\x84\xc8e\xec\xe1\n\xfd$\x92\xda\x12S\x0f\r\xba\x81y\x88E\x9ceu\xd9\xd2Z\xf8\xc3\xd3&\xf2\xf7j\t\t\xf2\xc6w\xf6\x9a7\xbd\x01\x96\xad\xf5\x9e\xf4\xa8,\xd2\x19b\x0f\x05\x0c\xd8G\xe66\x91\x85.\xbdX\x0b\xd9H\xb14\xc6\x88\xb5\xd7\x1f\xed\xf7\xb4\x10\xb7\xad\x9f\xab\x01\r(\r*\xd90\x84\xba\xfb\xd9\x94HK\xdf\xaf\xa0\xf2\x98\x96\xb6*b\xb5\xc0\xa6\xe5A[\x9fwf\x18\x08v\x85\t\xb7\xf7\x97\xc74\xe5{;9qw\xb1u>\t`\xfd\x10\xfbu\xfb\xf5\x11\xe9\xc1\xa0I\x96\x03\xa5\x84\x0b\xcd\x060\xa1\xb1\xbcs|\xfe\xf3N\xad\xddA\xe2l\xf83N\xae\x9c\xbe\x1568\xe9\xf5\xfdn\xe9\xbc\x98\xb5\xb9Bn\xf1]!\x86\xd39\xd2<&\xd6}\x9a\xe2\xa4|\xf0\x9a\xaf\xac\x08^\x93\x174\n~L<+=\x8d\x95'
# 2
b'5\xbc\xb2\xaa\x16\'\xa2\x93\x16D\'S\xfc\x9fm\xc9\xbbF\xa6:dN\x91f\xc1\xaa\x05\xeb\xe4\x16|\xd3\x07#\xd5\xda\xe9\x9b\xd0V\xd4\xb0#Y\xf2G\x0c\xae\xb7A\x9a\xaa\xb8^\xf8\xea\xddj%\xd0\xe8w\xb2\xf1\x9c\xf8D\xcc\x9b\xfe\xea\x16hT\x81\'u`\x10"\xaf\xe3\xd3#\xa0\xc2\x18\x8f^lE\xb0H\xe8\xd5\xf2\x8e\xd8\x8fq;\xd7B]\xc8j\x94\'0\xb0\x80\x0f\xd3\xd1\x90I\x1eL\x91y\x8dA\x01\xda>x`\x0b}6:\xb6o\xcf\xd1=\x15p\xdb\x16\xd3bF\xd5\xc9\\\x86\x1b\xeb\xc4H\x11\x04\xa9o\xe1\xffSF\xe3\xc1\x99\x05\xc44\x03\x86\x81\xbb#>\xfb\xc2\x0bscbW\x0f\xb8\x92\x81\xbb\x19c\xd1n\t\xa4sI\x91+\x97\x9e\x0b\xf1\x8b\xd2;\xa9NV\xc1\xb0#\xd1\xa24P\xce\x93US\xf5\x97=m\xb3\xb6\xd3\x9b\'\xade\x1e\xbc\x80\x13C\x99\x93\x89&\xbd\xde\x83f\\H6\xad2\nFM\xf07q\xe9`\xb1H\x98#X'
# 3
b"'E\xdb\xfd\xe4\xf9\x0c\xe1\xa4l\xaaq\x0e#\xde2\xe9\xe4\x12\xb3\xc2d\xd1W\xde*\x8d<\xcb\x1a\xea\xb4\xb86\x9bV0\r\xef\xfb\xafg\xe8\x1eHzg\x03I\x99ta\xad\x84[r.E\xbb\xc2\xae\xf1\xc2\xafd\xcb\xa6`\xf0)U\x85\xb1\n0\xb2\x05\x17s\xa3\xe3f\xb7\xda\x08\xd1\xae#\xd8\xa7\x90Tce\xc2\xac\xf3Q\x81\xbe1\x92\x8d\xcb\xbf\xfa\x88\xf3'\xe8\xa1\x9e\x9e\xae~\xb90Uq\x98\xe6\x17b\x9d]1\xf6\xabirw\xbc\x89\xae\xd8\xdf\x8a\xf5\xf1\xd4*~\x94\xe38\x1f$\x0e\x94t\xb64\x83q\xf8\x8f\xd6pR\xd4%\xf8\x1cv\xc5\xfe\x8d]\xcfy\xff\xb9\xc7\x10\xaao%\xa8\x13\xce6#Y\xfa\x06\xb8\xab(H^\xd8\x1a\xb63\xb0\xb0c\xe0\x11#\xa9\t\xdd\xa8\\\xeag\xc6H\xa5L\x0b\x10\xdb\xa9\xc44\xdcZ\xf1`\xa2\xc1^;\x1d\xdf\xbf\x92\x894\x847\xe9\x16\x15\xad\xd1c\xf9.\xc21\x02\x85\xb1\x0b\x96=\xf3D\xdf\xf7\xbep\x9c"
# 4
b'$\x82\xc8\x95\xcb\xdaq\xc0\x16\x0e\xef\xb6\xc8\x89\xabKQafM\x10^\x11\xea2\xfc\x8b\x0b~H\xfd\xe5\xe0\x80\x81<\xae\xb7\xfeT)K\xb3\x96\xc0y\x83e\x93\xae\xdb\x93\x82\xea\xb7\xb7\xdbQJX\xb2\xfdM\xf2(A6+e\xb7\x89\x8a\xba6\xb7\xa3\xde*\xea\xe0\x1cR\xa9i\x8a\x9aEK\xa2T\xebM\xa9\x1d\x96\x87\xaf\xb2I\xcej!"\xe2\xc8\xc08\x94\x8a\x18\x1d\t\x11`\xdf*\xbc\xb9\xf6J\xbci\xb3\xcc\xde\xb0\xa5\x98b}o\x94\xbe\xe0\x7f\xe2J\x8a\xa2)R{U\xdfu\xf6UO\xc2C\xf3\'\x87c\x1e\xc6\xe0\xbe\x879\xa5N\xb3J\xc8Cz\x9b\xa7\xec\x90[\xa8\x8a\xac\xeep\\ar\xbd\x94O\xce]\x1fw\x1bm|K\xce\x15\xf6\xcc\xc5\xc84\x9a\x00Z\x0b\xfd\xe9\xfb^6\x9b\xfd\xeb\x8c\xf1h\xda\x17\xc4\xb0\x08\\-\n7\x9e\x1f\x1d\xa7\xb4\xb9\xf0wq\x9a\x15G\xc5\x90\xf5\x00\x89\tI\x16\x90\xbcI\x80z\x90\xdb\nO\xdc\xe5\x8fh\xca'

Any asymmetric encryption method has to be randomized, so that if you encrypt the same plaintext twice, you don't get the same ciphertext. Otherwise it would be very insecure. Anyone who has the public key can encrypt something. Suppose an adversary has a ciphertext, they want to find out the plaintext, and they have partial information about the plaintext (e.g. they know it's a message in a certain format, but they don't know the exact content). They can try encrypting possible values of the plaintext until the result is the ciphertext they want to break. But since the encryption is randomized, they need to use the same data input and the same random value, otherwise they won't get the same ciphertext. And the adversary can't know what random value went into the ciphertext they want to break.
For RSA, in practice, there are two methods for doing encryption. Both are defined by the document known as PKCS#1. Both take the plaintext to encrypt and apply a transformation to it that involves either appending random data (PKCS#1 v1.5) or masking with random data (PSS). Then the result undergoes the well-known exponentiation part of RSA.
You can use the exponentiation to inspect a ciphertext.
n = 21968272887747488664299300886573437453854580842272801065486318320328573181104433915148345103361664593733184722692105149694142557011266255075972021704711966860643495011049367729520386363274015109405027569939049707059547205662044677513224725454246882263137472476944688288600202939249708651097639414591301098996178101611307541565108035735952182518865647460401330824147744542993709272159435504287548711774248609991298003738752699597664282754244110245104529559246443251024491287411685325071990133422302961361831613169335261576570530061643400976849033234171349450189113706076777344091951159628029458250885131329209309850429
e = 65537
d = 7180742814003184493745817226790609535628314246962295259545720906634095162818242875479619891118201610188935763454388765380592975819694916096822751254380575157372246976924478622789961650274744826184819271605876418277150620865958482714928972468695190683750109638846897363602141498155351308783613387153774908482554823734710213533339079775940427840254792667407339506634483414544868884993644469123554250547973774825288728499603644573043340903253662627022861078040710813466717381393318974263956822836617559198769733538785368579523554468493535497334351910973554355558084517450711717078208243534059900951053098416621979162953
c1 = b'\x17T\xc0\x03\xa4\xa6\xc06\x83\xdcM\xe5\xf9\xd8t\xc9>\xad}\xc9\x15[\xcc!\x19\x97/\xbf\xc7\xe4\xcbhu\x8d\xfb&\x18\x84\xc8e\xec\xe1\n\xfd$\x92\xda\x12S\x0f\r\xba\x81y\x88E\x9ceu\xd9\xd2Z\xf8\xc3\xd3&\xf2\xf7j\t\t\xf2\xc6w\xf6\x9a7\xbd\x01\x96\xad\xf5\x9e\xf4\xa8,\xd2\x19b\x0f\x05\x0c\xd8G\xe66\x91\x85.\xbdX\x0b\xd9H\xb14\xc6\x88\xb5\xd7\x1f\xed\xf7\xb4\x10\xb7\xad\x9f\xab\x01\r(\r*\xd90\x84\xba\xfb\xd9\x94HK\xdf\xaf\xa0\xf2\x98\x96\xb6*b\xb5\xc0\xa6\xe5A[\x9fwf\x18\x08v\x85\t\xb7\xf7\x97\xc74\xe5{;9qw\xb1u>\t`\xfd\x10\xfbu\xfb\xf5\x11\xe9\xc1\xa0I\x96\x03\xa5\x84\x0b\xcd\x060\xa1\xb1\xbcs|\xfe\xf3N\xad\xddA\xe2l\xf83N\xae\x9c\xbe\x1568\xe9\xf5\xfdn\xe9\xbc\x98\xb5\xb9Bn\xf1]!\x86\xd39\xd2<&\xd6}\x9a\xe2\xa4|\xf0\x9a\xaf\xac\x08^\x93\x174\n~L<+=\x8d\x95'
print(binascii.unhexlify('0' + hex(pow(int(binascii.hexlify(c1), 16), d, n))[2:]))
That last value is the padded plaintext. You can see the data in there, with padding before it. This is the PKCS#1 v1.5 padding method (which is insecure unless used very carefully, and should not be used except for backward compatibility with systems that require it).

Encryption with NodeJS doesn't match encryption using Python (cryptography.fernet)

Cryptography noob here. I'm trying to write a script in NodeJS that encrypts a string and produces output that matches the output of my Python script that uses the cryptography.fernet library. My overall goal is to use the original key to encrypt messages in Node that will later be decrypted using Python.
Sample of my Python code:
from cryptography.fernet import Fernet
key = Fernet.generate_key() # For example: 6saGtiTFEXej729GUWSeAyQdIpRFdGhfY2XFUDpvsu8=
f = Fernet(key)
message = 'Hello World'
encoded = message.encode()
encrypted = f.encrypt(encoded)
Which produces the output: gAAAAABhJs_E-dDVp_UrLK6PWLpukDAM0OT5M6bfcqvVoCvg7r63NSi4OWOamLpABuYQG-5wsts_9h7cLbCsWmctArXcGqelXz_BXl_o2C7KM9o7_eq7VTc=
My Node script uses the built-in Crypto module and must also use the same 32-byte key that is being used in my Python program. I know that fernet uses is AES-128-CBC as its algorithm, so that's what I'm using for my Node script.
My NodeJS code:
const crypto = require("crypto");
const key = '6saGtiTFEXej729GUWSeAyQdIpRFdGhfY2XFUDpvsu8=';
const algorithm = 'aes-128-cbc';
const message = 'Hello World';
const iv = crypto.randomBytes(16);
const cipher = crypto.createCipheriv(algorithm, key, iv);
const encrypted = cipher.update(message, 'utf8', 'hex') + cipher.final('hex');
Which is giving me: Error: Invalid key length
My first problem is that I'm unsure how to convert the key so that it's the proper length. I also know from looking at fernet's source code that the key is split into two parts: the first 16 bytes are the signing_key and the last 16 bytes are the encryption_key - I haven't found much information on whether/how I need to deal with those two pieces of the original key in my Node implementation.
Since I'm new to this I'm a little confused on how to accomplish what I'm after. Any tips or advice is very much appreciated.

The specs for the Fernet format can be found on https://github.com/fernet/spec/blob/master/Spec.md
There they specify both a generating and a veryfying steps, here is the generating which should give enough information for your implementation:
Record the current time for the timestamp field.
Choose a unique IV.
Construct the ciphertext:
Pad the message to a multiple of 16 bytes (128 bits) per RFC 5652, section 6.3. This is the same padding technique used in PKCS #7 v1.5 and all versions of SSL/TLS (cf. RFC 5246, section 6.2.3.2 for TLS 1.2).
Encrypt the padded message using AES 128 in CBC mode with the chosen IV and user-supplied encryption-key.
Compute the HMAC field as described above using the user-supplied signing-key.
Concatenate all fields together in the format above.
base64url encode the entire token.
From this we can see that the signing key (first half of full key) is used in HMAC, while the second half is used in the AES128-CBC, so just dividing the key into two separate elements (with proper conversion from hex string to bytes) should be enough for using Node.js crypto module (https://nodejs.org/en/knowledge/cryptography/how-to-use-crypto-module/) to construct your own implementation.

How to migrate AES mode from one to another?

I have 2 functions: encrypt/decrypt implemented by AES
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives.ciphers import Cipher, algorithms, modes
DEFAULT_MODE = modes.ECB()
def encrypt(data, key):
encryptor = Cipher(algorithms.AES(key), DEFAULT_MODE, backend=default_backend()).encryptor()
return encryptor.update(data) + encryptor.finalize()
def decrypt(data, key):
if len(data) == 0 or len(data)%16 != 0:
raise ValueError
decryptor = Cipher(algorithms.AES(key), DEFAULT_MODE, backend=default_backend()).decryptor()
return decryptor.update(data)
The code can be executed in below:
>>> encrypt(b'a'*16, b'I_got_one_key_in_32bytes_length.')
'\xab\x07\x9d\xa0\xf0\xa0g\x9ae\xd9\x10\x9e\xea2\xb4\x17'
>>> decrypt(b'\xab\x07\x9d\xa0\xf0\xa0g\x9ae\xd9\x10\x9e\xea2\xb4\x17', b'I_got_one_key_in_32bytes_length.')
'aaaaaaaaaaaaaaaa'
The mission here is to change the mode from ECB to others, say GCM.
DEFAULT_MODE = modes.GCM('iv')
The code can still be executed in below:
>>> encrypt(b'a'*16, b'I_got_one_key_in_32bytes_length.')
'Y5y\xbe\xeeK\xb9\x10\xcdf\x99\xa6\x1d\xf2\xa0\x1e'
>>> decrypt(b'Y5y\xbe\xeeK\xb9\x10\xcdf\x99\xa6\x1d\xf2\xa0\x1e', b'I_got_one_key_in_32bytes_length.')
'aaaaaaaaaaaaaaaa'
However, since those encrypt/decrypt function have been used broadly for a while, an issue is raised: How to migrate to new mode without impacting original encrypted data?
PS: my draft thought is logic like below: decrypting data in GCM mode if possible, otherwise decrypting in ECB mode; and encrypting all of them in GCM mode later. But AFAIK, this idea won't work since I have no way to raise the Error. With this approach, is that possible to know the AES mode of encrypted data?
def decrypt(data, key):
try:
# decrypt data by key in GCM
except GCMDecryptFailedError:
# decrypt data by key in ECB
return decrypted_data

The error here is to use a cryptographic algorithm directly, without specifying a specific protocol, and a protocol version number that comes with that. Fortunately, there are ways of getting out of this, and specify a specific protocol.
One of the tricks employed by cryptography is that it is impossible to generate specific content randomly simply because the chance of generating the a specific value reduces by each bit. So what you can do is to prefix your ciphertext with e.g. a 128 / 16 byte bit value. The chance that you generate this value at the start of your ciphertext is 2 to the power of 128 for each message (less if the messages are not random themselves for a specific key). In other words, the chance is as low as guessing an AES-128 key; we call such a chance "negligible". This trick of course depends on the output of ECB encryption using a random key to be random as well.
However, for the future you might want to include one or more bytes as protocol version indicator. So you could send e.g. the byte value 01 as new version, and then the 16 byte magic value, followed by the random nonce for GCM, the ciphertext and the GCM authentication tag (if that's not already included in the GCM ciphertext). Once you got rid of the ECB version of your protocol (version 00 not indicated in your messages) then you can get rid of the magic and re-purpose the 16 bytes in the protocol header of your messages, for protocol 2 or higher.
If you want to generate a nice magic then you could use any kind of 16 byte string, say "Protocol 1, GCM:" (without the quotes) in ASCII. You could also use the leftmost 128 bits of a hash if you want to use a larger string.
So initially your logic would be, in pseudo-code:
versionByte = message[0]
if message.length >= 17 && versionByte == 01h then
magic = message[1- 16]
if magic == "Protocol 1, GCM:" then
gcmDecrypt(message, 17)
else
ecbDecrypt(message, 0)
// --- include other versions here ---
else
ecbDecrypt(message, 0)
Of course this is still a very basic protocol. But at least you can change it later on. You might want to look at more complete protocol specs, such as Fernet, CMS or - of course - TLS instead of just AES-GCM.
Whatever you do: write down your protocol in a separate document and refer to it from your code. You may quote your simple protocol in the source code as easy lookup.

DCPcrypt encryption decrypt in PyCrypto

I am working to decrypt data that was encrypted with DCPcrypt using Rijndael. I wanted to use Python to decrypt it but I'm running into issues. I'll mention that I'm not particularly crypto savvy (I took a college course, but that's about it) and I'm also not a Delphi programmer, so that is also probably hindering my efforts to decipher what precisely DCPcrypt is doing.
This is the meat of the Delphi code:
Cipher: TDCP_rijndael;
begin
Cipher:= TDCP_rijndael.Create(nil);
Cipher.InitStr(PasswordField.Text);
Cipher.EncryptCBC(encryptString[1],encryptString[1],Length(encryptString));
So the implementation uses a key (obtained from the password field) but no IV. PyCrypto on the other hand requires an IV. Searching through the internals of the DCPcrypt code, it appears that if the IV is nil, then an ECB encryption is used to populate the IV from a string of 0xff?
procedure TDCP_rijndael.Init(var Key; Size: longint; IVector: pointer);
....
if IVector= nil then
begin
FillChar(IV,Sizeof(IV),$FF);
{$IFDEF CFORM}Encrypt(IV,IV){$ELSE}RijndaelEncryptECB(Data,IV,IV){$ENDIF};
Move(IV,LB,Sizeof(LB));
end
It appears that I'm using a static IV. However, I am not able to make this work. Here's my implementation in PyCrypto. Any ideas what I'm doing wrong?
key = "password"
s = hashlib.sha1()
s.update(key)
key = s.digest()
key = key[:16]
# Set up the IV, note that in ECB the third parameter to the AES.new function is ignored since ECB doesn't use an IV
ecb = AES.new(key, AES.MODE_ECB, '\xff' * 16)
iv = ecb.encrypt('\xff' * 16)
cipher = AES.new(key, AES.MODE_CFB, iv)
msg = cipher.decrypt(ct[:16])
I have some plain text that was encrypted using the Delphi code and then base64 encoded. The key used was the string password, as hardcoded in above. Using my implementation, I decrypt a bunch of garbled bytes.
k8b+uce5Fkp7Hbk/CaGYcuEWTfxlI05as88lJL0mHmJxLsKWqki2YwiFPU9Rx8qiUC2cvWZrQIOnkw==
Any help is greatly appreciated.

A random assortment of suggestions and thoughts:
Static IVs are generally a security risk as they open the door to known plaintext attacks.
Looking at the dcpcrypt source, it looks like there is a method to specify an IV. Any reason not to, if for no other reason than to eliminate incorrect IV as an error source?
Likewise, you could experiment with ECB instead of CBC to eliminate the IV altogether and isolate whether the problem is in the IV or elsewhere (key, data, or configuration).

DISCLAIMER: I'm extremely new to Python, but try this:
Try changing
key = key[:16]
to
key = key + bytes([0,0,0,0])
This will give you a 24 byte key which, I think, should work.
DCP allows any length of key whereas Crypto insists on keys of 16, 24 or 32 bytes. By default DCP will use SHA1 to generate the key, which is then 20 bytes long. Based on this DCP uses logic for keylength <= 24 and just zero pads the key rather than logic for keylength <= 16 which is the effect that key[:16] is having.
Also, don't know if this is just a typo but try changing the AES mode
cipher = AES.new(key, AES.MODE_CFB, iv)
to
cipher = AES.new(key, AES.MODE_CBC, iv)
That will give a decryption of your input. But then you'll need to consider padding of the original source text because DCP does not pad (I think) but Crypto requires multiples of 16 for decryption.

PyCrypto problem using AES+CTR

I'm writing a piece of code to encrypt a text using symmetric encryption. But it's not coming back with the right result...
from Crypto.Cipher import AES
import os
crypto = AES.new(os.urandom(32), AES.MODE_CTR, counter = lambda : os.urandom(16))
encrypted = crypto.encrypt("aaaaaaaaaaaaaaaa")
print crypto.decrypt(encrypted)
Here, the decrypted text is different from the original.
I don't really understand much about cryptography so please bear with me. I understand the CTR mode requires a "counter" function to supply a random counter each time, but why does it need it to be 16 bytes when my key is 32 bytes and it insists that my message is in multiples of 16 bytes too? Is this normal?
I'm guessing that it doesn't get back to the original message because the counter changed between encrypt and decrypt. But then, how is it supposed to work theoretically anyway? What am I doing wrong? Anyway, I'm forced to resort back to ECB until I figure this out :(

The counter must return the same on decryption as it did on encryption, as you intuit, so, one (NOT SECURE AT ALL) way to do it is:
>>> secret = os.urandom(16)
>>> crypto = AES.new(os.urandom(32), AES.MODE_CTR, counter=lambda: secret)
>>> encrypted = crypto.encrypt("aaaaaaaaaaaaaaaa")
>>> print crypto.decrypt(encrypted)
aaaaaaaaaaaaaaaa
CTR is a block cipher, so the "16-at-a-time" constraint that seems to surprise you is a pretty natural one.
Of course, a so-called "counter" returning the same value at each call is grossly insecure. Doesn't take much to do better, e.g....:
import array
class Secret(object):
def __init__(self, secret=None):
if secret is None: secret = os.urandom(16)
self.secret = secret
self.reset()
def counter(self):
for i, c in enumerate(self.current):
self.current[i] = c + 1
if self.current: break
return self.current.tostring()
def reset(self):
self.current = array.array('B', self.secret)
secret = Secret()
crypto = AES.new(os.urandom(32), AES.MODE_CTR, counter=secret.counter)
encrypted = crypto.encrypt(16*'a' + 16*'b' + 16*'c')
secret.reset()
print crypto.decrypt(encrypted)

AES is a block cipher: it's an algorithm (more precisely, a pair of algorithms) that takes a key and a message block and either encrypts or decrypts the block. The size of a block is always 16 bytes, regardless of the key size.
CTR is a mode of operation. It's a pair of algorithms that builds on a block cipher to produce a stream cipher, which can encrypt and decrypt messages of arbitrary lengths.
CTR works by combining successive message blocks with the encryption of successive values of a counter. The size of the counter needs to be one block so that it's valid input for the block cipher.
Functionally, it doesn't matter what the successive values of the counter are, as long as the encryption and decryption side use the same sequence. Usually the counter is treated as a 256-bit number and incremented for each successive block, with an initial value chosen at random. Thus, usually, the incrementation method is baked into the code, but the decryption side needs to know what the initial value is, so encryption side sends or stores the initial counter value at the beginning of the encrypted message.
For security, it is vital to never repeat the same counter value with a given key. So for a single-use key, it's ok to start with '\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'. But if the key is used multiple times then the second message is not allowed to reuse any of the counter values used by the first message, and the easiest way to ensure that is to generate the initial counter value at random (with a 2^128 space, the chances of a collision are acceptably negligible).
By letting the caller pick a counter function, the PyCrypto library gives you plenty of rope to hang yourself. You should use Crypto.Util.Counter, not just “for better performance” as the documentation puts it, but because it's easier to build something secure than what you're likely to come up with on your own. And even so, take care to use a random initial value, which is not the default.
import binascii
import os
from Crypto.Cipher import AES
from Crypto.Util import Counter
def int_of_string(s):
return int(binascii.hexlify(s), 16)
def encrypt_message(key, plaintext):
iv = os.urandom(16)
ctr = Counter.new(128, initial_value=int_of_string(iv))
aes = AES.new(key, AES.MODE_CTR, counter=ctr)
return iv + aes.encrypt(plaintext)
def decrypt_message(key, ciphertext):
iv = ciphertext[:16]
ctr = Counter.new(128, initial_value=int_of_string(iv))
aes = AES.new(key, AES.MODE_CTR, counter=ctr)
return aes.decrypt(ciphertext[16:])

I may be definitely late and I may have overlooked the previous answers, but I didn't find a clear statement of how this should (at least IMHO) be done according with the PyCrypto packages.
The Crypto.Util.Counter package provides callable stateful counters, which are very useful, but it was easy at least for me to use them improperly.
You have to create a counter, with e.g. ctr = Counter.new('parameters here'). Every time your counter is called by your counter mode cipher object to encrypt the message, it is incremented. This is needed for good cryptography practices, otherwise information about equal blocks may leak from the ciphertext.
Now you cannot call the decryption function on the same cipher object, because it would call again the same counter which in the meanwhile has been incremented, possibly several times. What you need to do is to create a new cipher object with a different counter initialized with the same parameters. In this way the decryption works properly, starting the counter from the same point as the encryption was done.
Working example below:
# Import modules
from Crypto.Cipher import AES
from Crypto import Random
from Crypto.Util import Counter
# Pad for short keys
pad = '# constant pad for short keys ##'
# Generate a random initialization vector, to be used by both encryptor and decryptor
# This may be sent in clear in a real communication
random_generator = Random.new()
IV = random_generator.read(8)
# Encryption steps
# Ask user for input and pad or truncate to a 32 bytes (256 bits) key
prompt = 'Input your key. It will padded or truncated at 32 bytes (256 bits).\n-: '
user_keye = raw_input(prompt)
keye = (user_keye + pad)[:32]
# Create counter for encryptor
ctr_e = Counter.new(64, prefix=IV)
# Create encryptor, ask for plaintext to encrypt, then encrypt and print ciphertext
encryptor = AES.new(keye, AES.MODE_CTR, counter=ctr_e)
plaintext = raw_input('Enter message to cipher: ')
ciphertext = encryptor.encrypt(plaintext)
print ciphertext
print
# Decryption steps
# Ask user for key: it must be equal to that used for encryption
prompt = 'Input your key. It will padded or truncated at 32 bytes (256 bits).\n-: '
user_keyd = raw_input(prompt)
keyd = (user_keyd + pad)[:32]
# Create counter for decryptor: it is equal to the encryptor, but restarts from the beginning
ctr_d = Counter.new(64, prefix=IV)
# Create decryptor, then decrypt and print decoded text
decryptor = AES.new(keyd, AES.MODE_CTR, counter=ctr_d)
decoded_text = decryptor.decrypt(ciphertext)
print decoded_text

why does it need it to be 16 bytes when my key is 32 bytes
It has to be the same length as the cipher's block size. CTR mode just encrypts the counter and XORs the plaintext with the encrypted counter block.
Notes:
the counter value MUST be unique -- if you EVER use the same counter value to encrypt two different plaintexts under the same key, you just gave away your key.
like an IV, the counter is NOT secret -- just send it along with the ciphertext. If you make the code more complicated by trying to keep it secret, you will probably shoot yourself in the foot.
the counter value need not be unpredictable -- starting with zero and adding one for each block is perfectly fine. But note that if you encrypt multiple messages, you need to keep track of the counter values that have already been consumed, i.e. you need to keep track of how many blocks have already been encrypted with that key (and you can't use the same key in different instances of your program or on different machines).
the plain text can be any length -- CTR mode turns a block cipher into a stream cipher.
Standard disclaimer: Crypto is hard. If you don't understand what you are doing, you will get it wrong.
I just want to store some passwords across sessions.
Use scrypt. scrypt includes encrypt and decrypt which use AES-CTR with a password-derived key.
$ pip install scrypt
$ python
>>> import scrypt
>>> import getpass
>>> pw = getpass.getpass("enter password:")
enter password:
>>> encrypted = scrypt.encrypt("Guido is a space alien.",pw)
>>> out = scrypt.decrypt(encrypted,pw)
>>> out
'Guido is a space alien.'

The initialization vector ("counter") needs to stay the same, just as the key does, between encryption and decryption. It is used so that you can encode the same text a million times, and get different ciphertext each time (preventing some known plaintext attacks and pattern matching / attacks). You still need to use the same IV when decrypting as when encrypting. Usually when you start decrypting a stream, you initialize the IV to the same value that you started with when you started encrypting that stream.
See http://en.wikipedia.org/wiki/Initialization_vector for info on initialization vectors.
Note that os.urandom(16) is not 'deterministic', which is a requirement for counter functions. I suggest you use the increment function, as that is how CTR mode is designed. The initial counter value should be random, but the successive values should be fully predictable from the initial value (deterministic). The initial value may even be taken care of for you (I don't know the details)
About the key, IV, and input sizes, it sounds like the cipher you chose has a block size of 16 bytes. Everything you describe fits that and seems normal to me.