We Keep Coding

Python 3 XChaCha20 test vectors work for encryption but decryption stage fails

I use the following vectors to test XChaCha20 encryption with AEAD by Poly1305 in python:
Vectors:
https://datatracker.ietf.org/doc/html/draft-arciszewski-xchacha-03#appendix-A.3
pycryptodome:
https://pycryptodome.readthedocs.io/en/latest/src/cipher/chacha20_poly1305.html
The drafts use HEX for the test vectors, if you really need to check me convert using this service:
https://www.asciitohex.com/
import json
from base64 import b64encode
from base64 import b64decode
from Crypto.Cipher import ChaCha20_Poly1305
from Crypto.Random import get_random_bytes
#nonce_xchacha20 = get_random_bytes(24)
nonce_xchacha20 = b64decode("QEFCQ0RFRkdISUpLTE1OT1BRUlNUVVZX")
#header = b"header"
header = b64decode("UFFSU8DBwsPExcbH")
plaintext = b"Ladies and Gentlemen of the class of '99: If I could offer you only one tip for the future, sunscreen would be it."
#key = get_random_bytes(32)
key = b64decode("gIGCg4SFhoeIiYqLjI2Oj5CRkpOUlZaXmJmam5ydnp8=")
cipher = ChaCha20_Poly1305.new(key=key, nonce=nonce_xchacha20)
cipher.update(header)
ciphertext, tag = cipher.encrypt_and_digest(plaintext)
jk = [ 'nonce', 'header', 'ciphertext', 'tag' ]
jv = [ b64encode(x).decode('utf-8') for x in (cipher.nonce, header, ciphertext, tag) ]
result = json.dumps(dict(zip(jk, jv)))
print(result)
# We assume that the key was securely shared beforehand
try:
b64 = json.loads(result)
jk = [ 'nonce', 'header', 'ciphertext', 'tag' ]
jv = {k:b64decode(b64[k]) for k in jk}
cipher = ChaCha20_Poly1305.new(key=key, nonce=jv['nonce'])
cipher.update(jv['header'])
plaintext = cipher.decrypt_and_verify(jv['ciphertext'], jv['tag'])
print("The message was: " + plaintext)
except (ValueError, KeyError):
print("Incorrect decryption")
print("sanity check if key values are the same: ")
print(b64encode(jv['nonce']))
print(b64encode(jv['header']))
print(b64encode(jv['ciphertext']))
print(b64encode(jv['tag']))
Why does my decryption stage fail if the test vectors encrypt correctly according to IETF-draft?
{"nonce": "AAAAAFBRUlNUVVZX", "header": "UFFSU8DBwsPExcbH", "ciphertext": "vW0XnT6D1DuVdleUk8DpOVcqFwAlK/rMvtKQLCE5bLtzHH8bC0qmRAvzqC9O2n45rmTGcIxUwhbLlrcuEhO0Ui+Mm6QNtdlFsRtpuYLBu54/P6wrw2lIj3ayODVl0//5IflmTJdjfal2iBL2FcaLE7Uu", "tag": "wIdZJMHHmHlH3q/YeArPSQ=="}
Incorrect decryption
sanity check if key values are the same:
b'AAAAAFBRUlNUVVZX'
b'UFFSU8DBwsPExcbH'
b'vW0XnT6D1DuVdleUk8DpOVcqFwAlK/rMvtKQLCE5bLtzHH8bC0qmRAvzqC9O2n45rmTGcIxUwhbLlrcuEhO0Ui+Mm6QNtdlFsRtpuYLBu54/P6wrw2lIj3ayODVl0//5IflmTJdjfal2iBL2FcaLE7Uu'
b'wIdZJMHHmHlH3q/YeArPSQ=='
When I convert the byte arrays back to base64, they still match the JSON output.
So reading my key values from JSON for decryption was done correctly.
Where is the mistake? I literally use a code example from the site offering pycryptodome and encryption was done correctly. It should decrypt just fine.

The decryption will be done correctly if you replace in the line
jv = [ b64encode(x).decode('utf-8') for x in (cipher.nonce, header, ciphertext, tag) ]
the expression cipher.nonce with nonce_xchacha20. The bug causes an incorrect nonce to be supplied in the JSON.
It seems that cipher.nonce can only be used to determine a randomly generated nonce (a random nonce is generated if no explicit nonce is specified when instantiating the cipher, s. here).
A second (trivial) change is in the line
print("The message was: " + plaintext)
necessary. Here a UTF8 decoding must be performed, i.e. plaintext must be replaced by plaintext.decode('utf8').
In your first post, the AADs were also set incorrectly. But this has been corrected in the meantime.
With these two changes, the code, especially the decryption, works on my machine.

How to resolve AssertionError for converting string characters to bytes using run length encoding?

I have a problem to solve but once I submit my solution the result shows an AssertionError.
I want to be able to convert my string of characters into a byte format using a technique called run-length encoding - https://en.wikipedia.org/wiki/Run-length_encoding
I was doing some research on it and found that what I felt done by me was correct but the solution was apparently not correct on the website and I get this: AssertionError: 'a5b6c4' is not an instance of <class 'bytes'> : compress('') should return bytes
My code:
from collections import OrderedDict
def compress(raw=str)->bytes:
dict=OrderedDict.fromkeys(my_str_as_bytes, 0)
for ch in my_str_as_bytes:
dict[ch] += 1
output = ''
for key,value in dict.items():
output = output + key + str(value)
return output
my_str_as_bytes = "aaaaabbbbbbcccc"
print (bytes(compress(my_str_as_bytes,),encoding='UTF-8'))
The result on my IDE was:
b'a5b6c4'
I'm not sure of what I did is encoding the string and changing it into a byte or not. Any help would be appreciated.
Few of the links I checked:
https://www.tutorialspoint.com/run-length-encoding-in-python
https://www.geeksforgeeks.org/run-length-encoding-python/

ValueError: Ciphertext length must be equal to key size

I know this question was asked a lot, but I still don't really understand how could I debug my code. I was googling this question for over 3 days now, but I just can't find a suitable answer for my code. I want to add the encrypted message with an input but every time I run my code it just gives the
Traceback (most recent call last):
File "decryption.py", line 27, in <module>
label=None
File "/opt/anaconda3/lib/python3.7/site-packages/cryptography/hazmat/backends/openssl/rsa.py", line 357, in decrypt
raise ValueError("Ciphertext length must be equal to key size.")
ValueError: Ciphertext length must be equal to key size.
error. I know its with the label, but I just can't find the answer
Here is my code for the decryption:
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives import serialization
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.asymmetric import padding
with open("private_key.pem", "rb") as key_file:
private_key = serialization.load_pem_private_key(
key_file.read(),
password=None,
backend=default_backend()
)
with open("public_key.pem", "rb") as key_file:
public_key = serialization.load_pem_public_key(
key_file.read(),
backend=default_backend()
)
textInput = str(input(">>> "))
encrypted = textInput.encode()
original_message = private_key.decrypt(
encrypted,
padding.OAEP(
mgf=padding.MGF1(algorithm=hashes.SHA256()),
algorithm=hashes.SHA256(),
label=None
)
)
original_decoded_message = original_message.decode("utf-8")
print(original_decoded_message)
And for the encryption:
from cryptography.hazmat.backends import default_backend
from cryptography.hazmat.primitives.asymmetric import rsa
from cryptography.hazmat.primitives import hashes
from cryptography.hazmat.primitives.asymmetric import padding
from cryptography.hazmat.primitives import serialization
private_key = rsa.generate_private_key(
public_exponent=65537,
key_size=2048,
backend=default_backend()
)
public_key = private_key.public_key()
pem = private_key.private_bytes(
encoding=serialization.Encoding.PEM,
format=serialization.PrivateFormat.PKCS8,
encryption_algorithm=serialization.NoEncryption()
)
with open('private_key.pem', 'wb') as f:
f.write(pem)
pem = public_key.public_bytes(
encoding=serialization.Encoding.PEM,
format=serialization.PublicFormat.SubjectPublicKeyInfo
)
with open('public_key.pem', 'wb') as f:
f.write(pem)
raw_message = str(input(">>> "))
message = raw_message.encode("utf-8")
encrypted = public_key.encrypt(
message,
padding.OAEP(
mgf=padding.MGF1(algorithm=hashes.SHA256()),
algorithm=hashes.SHA256(),
label=None
)
)
print(encrypted)
Im really new to Asymmetric encryption / decryption.

This minor modification in the decryption part of your code made it work for me:
textInput = str(input(">>> "))
# encrypted = textInput.encode() #this is incorrect!
encrypted = eval(textInput)
This is what I get:
For the encryption part:
>>> test message
b'>xB)\xf1\xc5I\xd6\xce\xfb\xcf\x83\xe2\xc5\x8f\xcfl\xb9\x0f\xa2\x13\xa5\xe1\x03\xf7p\xb3\x9c\xeb\r\xc1"\xf2\x17\x8b\xea\t\xed\xb2xG\xb7\r\xa9\xf8\x03eBD\xdd9>\xbe\xd1O\xe2\x9f\xbb\xf9\xff5\x96l\xea\x17FI\x8d\x02\x05\xea\x1dpM\xbb\x04J\xfc\x0c\\\xfe\x15\x07\xaf \x9e\xc2\xf9M\xa4\x1d$\xc3\x99my\xb6\xc5\xad\x97\xd06\xd2\x08\xd3\xe2\xc8H\xca\xd8\xfd{\xe6\xc6\xa3\x18\xeb\xe6\xcc\xc5\x9a\xc8*\xbb\xc1\x8c\x80,\x1f\r#\x9b\x9d\xc5\x91I\xa8\xc01y\xbc\xa73\xd3\x19;\xef\x8a\xfb\xc2\xc4\x9e\xbe\x8f\xeb\x1d\x12\xbd\xe4<\xa0\xbb\x8d\xef\xee\xa3\x89E\x07"m\x1d\xb0\xf3\xd2:y\xd9\xbd\xef\xdf\xc9\xbb\x1b\xd5\x03\x91\xa4l\x8bS\x9e\x80\x14\x90\x18\xc4\x9e\\?\x8eF\x05\xa1H\x9e:\x0c\x96\x8e\xb3E3\x90\xa2\xa1\xd9\x88\xa0<X\x7f\rIP\x00\xbf\xf6\x15\xfb9tW\x17\x9f\xca\x95\xf6|\xd7\x90\xbcp\xe5\xb5,V\x1b\xe9\x90\xf6\x87 v=6'
Now for the decryption part, I use the output:
>>> b'>xB)\xf1\xc5I\xd6\xce\xfb\xcf\x83\xe2\xc5\x8f\xcfl\xb9\x0f\xa2\x13\xa5\xe1\x03\xf7p\xb3\x9c\xeb\r\xc1"\xf2\x17\x8b\xea\t\xed\xb2xG\xb7\r\xa9\xf8\x03eBD\xdd9>\xbe\xd1O\xe2\x9f\xbb\xf9\xff5\x96l\xea\x17FI\x8d\x02\x05\xea\x1dpM\xbb\x04J\xfc\x0c\\\xfe\x15\x07\xaf \x9e\xc2\xf9M\xa4\x1d$\xc3\x99my\xb6\xc5\xad\x97\xd06\xd2\x08\xd3\xe2\xc8H\xca\xd8\xfd{\xe6\xc6\xa3\x18\xeb\xe6\xcc\xc5\x9a\xc8*\xbb\xc1\x8c\x80,\x1f\r#\x9b\x9d\xc5\x91I\xa8\xc01y\xbc\xa73\xd3\x19;\xef\x8a\xfb\xc2\xc4\x9e\xbe\x8f\xeb\x1d\x12\xbd\xe4<\xa0\xbb\x8d\xef\xee\xa3\x89E\x07"m\x1d\xb0\xf3\xd2:y\xd9\xbd\xef\xdf\xc9\xbb\x1b\xd5\x03\x91\xa4l\x8bS\x9e\x80\x14\x90\x18\xc4\x9e\\?\x8eF\x05\xa1H\x9e:\x0c\x96\x8e\xb3E3\x90\xa2\xa1\xd9\x88\xa0<X\x7f\rIP\x00\xbf\xf6\x15\xfb9tW\x17\x9f\xca\x95\xf6|\xd7\x90\xbcp\xe5\xb5,V\x1b\xe9\x90\xf6\x87 v=6'
test message
The problem with your original code is that you are writing the representation of a bytes string as a unicode string (when using print(encrypted)), then encoding it into a bytes object in the decrypt code, and then passing it to the decrypt function. Encoding this string will not yield the original bytes string encrypted.
This example illustrates the problem:
>>> x = bytes(bytearray.fromhex('f3'))
>>> x #A bytes string, similar to encrypt
b'\xf3'
>>> print(x)
b'\xf3'
>>> len(x)
1
>>> str(x) #this is what print(x) writes to stdout
"b'\\xf3'"
>>> print(str(x))
b'\xf3'
>>> len(str(x)) #Note that the lengths are different!
7
>>> x == str(x)
False
>>> str(x).encode() #what you were trying to do
b"b'\\xf3'"
>>> x == str(x).encode()
False
>>> eval(str(x)) #what gets the desired result
b'\xf3'
>>> x == eval(str(x))
True
The thing is that the print function prints the representation of the object rather than the object itself. Essentially, it does this by getting a printable value of inherently unprintable objects by using __repr__ or __str__ methods of that object.
This is the documentation for __str__:
Called by str(object) and the built-in functions format() and print()
to compute the “informal” or nicely printable string representation of
an object. The return value must be a string object.
.
.
The default
implementation defined by the built-in type object calls
object.__repr__().
and __repr__:
Called by the repr() built-in function to compute the “official”
string representation of an object. If at all possible, this should
look like a valid Python expression that could be used to recreate an
object
This "official" representation returned by __repr__ is what allowed me to use the eval function in the decrypt code to solve the problem.
TL;DR:
copying the output of print(encrypted) copies the informal, human readable value returned by encrypted.__str__() or the "official" representation returned by encrypted.__repr__(), which are both in a human readable encoding such as utf-8. They cannot be encoded back(by using utf-8 encoding) to create the original bytes string that they represent.
It is also worth looking into this question from a stackoverflow user, facing the same issue. The answer gives a way to actually encode this representation back into a bytes object if you want. It's worth looking into this because eval should be avoided, it is very unsafe. For that method to work, you would have to remove the leading b' and trailing ' from the string textInput before encoding it.
My final recommendation would be to pass the entire object encrypted between the two programs by either using files with the pickle module, or some form of socket programming. Best to stay away from using print and input to pass data like this between programs to avoid encoding issues.

What is wrong with my decryption function?

import base64
import re
def encrypt(cleartext, key):
to_return = bytearray(len(cleartext))
for i in xrange(len(cleartext)):
to_return[i] = ord(cleartext[i]) ^ ord(key)
return base64.encodestring(str(to_return))
def decrypt(ciphertxt,key):
x = base64.decodestring(re.escape(ciphertxt))
to_return = bytearray(len(x))
for i in xrange(len(x)):
to_return[i] = ord(x[i]) ^ ord(key)
while to_return[i]>127:
to_return[i]-=127
return to_return
When I encrypt bob then use my decrypt function it returns bob. However for longer things like paragraphs that when encrypted, the cipher text contains \ slashes it does not work. I do not get back ascii characters or base64 characters I get back weird chinese characters or square characters. Please any insight to point me in the right direction will help.

As jasonharper said, you're mangling your Base64 data by calling re.escape on it. Once you get rid of that, your code should be fine. I haven't tested it extensively, but it works correctly for me with multi-line text.
You should also get rid of this from your decrypt function:
while to_return[i]>127:
to_return[i]-=127
It won't do anything if the original cleartext is valid ASCII, but it will mess up the decoding if the cleartext does contain bytes > 127.
However, those functions could be a little more efficient.
FWIW, here's a version that works correctly on both Python 2 and Python 3. This code isn't as efficient as it could be on Python 3, due to the compromises made to deal with the changes in text and bytes handling in Python 3.
import base64
def encrypt(cleartext, key):
buff = bytearray(cleartext.encode())
key = ord(key)
buff = bytearray(c ^ key for c in buff)
return base64.b64encode(bytes(buff))
def decrypt(ciphertext, key):
buff = bytearray(base64.b64decode(ciphertext))
key = ord(key)
buff = bytearray(c ^ key for c in buff)
return buff.decode()
# Test
s = 'This is a test\nof XOR encryption'
key = b'\x53'
coded = encrypt(s, key)
print(coded)
plain = decrypt(coded, key)
print(plain)
Python 3 output
b'Bzs6IHM6IHMycyc2ICdZPDVzCxwBczY9MCEqIyc6PD0='
This is a test
of XOR encryption

Django urlsafe base64 decoding with decryption

I'm writing my own captcha system for user registration. So I need to create a suitable URL for receiving generated captcha pictures. Generation looks like this:
_cipher = cipher.new(settings.CAPTCHA_SECRET_KEY, cipher.MODE_ECB)
_encrypt_block = lambda block: _cipher.encrypt(block + ' ' * (_cipher.block_size - len(block) % _cipher.block_size))
#...
a = (self.rightnum, self.animal_type[1])
serialized = pickle.dumps(a)
encrypted = _encrypt_block(serialized)
safe_url = urlsafe_b64encode(encrypted)
But then I'm trying to receive this key via GET request in the view function, it fails on urlsafe_b64decode() with "character mapping must return integer, None or unicode" error:
def captcha(request):
try:
key = request.REQUEST['key']
decoded = urlsafe_b64decode(key)
decrypted = _decrypt_block(decoded)
deserialized = pickle.loads(decrypted)
return HttpResponse(deserialized)
except KeyError:
return HttpResponseBadRequest()
I found that on the output of urlsafe_b64encode there is an str, but GET request returns a unicode object (nevertheless it's a right string). Str() didn't help (it returns decode error deep inside django), and if I use key.repr it works, but decryptor doesn't work with an error "Input strings must be a multiple of 16 in length".
Inside a test file all this construction works perfectly, I can't understand, what's wrong?

The problem is that b64decode quite explicitly can only take bytes (a string), not unicode.
>>> import base64
>>> test = "Hi, I'm a string"
>>> enc = base64.urlsafe_b64encode(test)
>>> enc
'SGksIEknbSBhIHN0cmluZw=='
>>> uenc = unicode(enc)
>>> base64.urlsafe_b64decode(enc)
"Hi, I'm a string"
>>> base64.urlsafe_b64decode(uenc)
Traceback (most recent call last):
...
TypeError: character mapping must return integer, None or unicode
Since you know that your data only contains ASCII data (that's what base64encode will return), it should be safe to encode your unicode code points as ASCII or UTF-8 bytes, those bytes will be equivalent to the ASCII you expected.
>>> base64.urlsafe_b64decode(uenc.encode("ascii"))
"Hi, I'm a string"

I solved the problem!
deserialized = pickle.loads(captcha_decrypt(urlsafe_b64decode(key.encode('ascii'))))
return HttpResponse(str(deserialized))
But still I don't understand, why it didn't work first time.