We Keep Coding

Python Bytes & Lists & Encryption

I'm using Fernet to encrypt my data with this implementation. Let's assume that I have these three data:
data = [fernet.encrypt("Hello".encode()), fernet.encrypt("Stack".encode()), fernet.encrypt("Overflow".encode())]
After this operation, Python automatically converts bytes to string, and I'm writing them to a csv file. When I need to decrypt them like:
fernet.decrypt(data)
It gives me an error like you can only decrypt only bytes etc. I also checked that my data in the csv file is already bytes but string form.
b'gAAAAABiVUw5BzOkOv3VxlV5xa57Iaf0R4dzPbgsrnheAME8uYeslCZfTx9GeyRWe7l9VMM-gdDXiPZ4zsAXoXkG6T1dyXH6EztcqirrPhXX3YCt65_3xXvykVTDPdbEXs51cHvR-3HH'

An end-to-end usage example for encoding, writing to text, reading, and decoding.
The Fernet documentation can be referenced here.
from cryptography.fernet import Fernet
# Auto-generate a secret key.
key = Fernet.generate_key()
f = Fernet(key)
# Encode the string 'Hello' and encrypt.
encoded = f.encrypt('Hello'.encode())
This creates a bytestring (a bytes object) as:
b'gAAAAABiVVOOeO-hUG2QaKCVOyshntpbqVbxnexIVsFr7ttBGmKhHlDeM7jkTCjPPGphZxbh4D15X82pts3hKes12DjzwI8_jQ=='
Write, read and decrypt:
# Write the *decoded* encrypted string to a TXT file.
with open('/tmp/encoded.txt', 'w') as fh:
fh.write(encoded.decode())
# Read the encrypted string from TXT file.
with open('/tmp/encoded.txt') as fh:
encoded = fh.read()
# Encode the string, pass through fernet for decryption,
# and decode the bytes output.
f.decrypt(encoded.encode()).decode()
Output:
'Hello'

fernet.encrypt returns bytes (I assume, you're not being specific which implementation you're using, I'm guessing this one). .decode() them to a string. Then your CSV will contain "gAAA...", not "b'gAAA...'". When reading those again from the CSV, .encode() the string before passing it to fernet.decrypt.
fernet.encrypt returns bytes
bytes.decode() turns bytes into str
CSV wants str
str.encode() turns str into bytes
fernet.decrypt wants bytes

Save bytes in a .txt and read out as bytes later

I have written a small python script which encrypts a message with rsa.
Now I want to save the bytes in a txt to read them later.
But when I use str(...) on it I don't know how to convert the string back.
For example I encrypted "Test" to b'Y\xf8\xbc\xca\x14\x0f\x80\xd3\xc6\xce\xecE\x14\xc1\xaf\xbd\x82\xd24\xcf\x04\xe2\x9a\x81NF\xbeXi\x85\xef\xc4\xbbl\xd3(5\x80\xe4\xde3\x8eC\xd2jR*\xb7.gq\x8c\x8b\xa12\x1a\x10+\xbf\xefHZ\n/'
and saved it as a string.
When I aply bytes(...) on it I get the Error: TypeError: string argument without an encoding.
What can I do in order to do this?

You've saved the Python string representation of a binary byte array (bytestring).
To get the actual bytes back from such a representation, pass it through ast.literal_eval():
>>> import ast
>>> s = r"b'Y\xf8\xbc\xca\x14\x0f\x80\xd3\xc6\xce\xecE\x14\xc1\xaf\xbd\x82\xd24\xcf\x04\xe2\x9a\x81NF\xbeXi\x85\xef\xc4\xbbl\xd3(5\x80\xe4\xde3\x8eC\xd2jR*\xb7.gq\x8c\x8b\xa12\x1a\x10+\xbf\xefHZ\n/'"
>>> b = ast.literal_eval(s)
b'Y\xf8\xbc\xca\x14\x0f\x80\xd3\xc6\xce\xecE\x14\xc1\xaf\xbd\x82\xd24\xcf\x04\xe2\x9a\x81NF\xbeXi\x85\xef\xc4\xbbl\xd3(5\x80\xe4\xde3\x8eC\xd2jR*\xb7.gq\x8c\x8b\xa12\x1a\x10+\xbf\xefHZ\n/'
Better yet, just save the binary bytes to your file without passing through a string:
encrypted_bytes = my_rsa("Test")
with open("encrypted.bin", "wb") as f:
f.write(encrypted_bytes)
# ...
with open("encrypted.bin", "rb") as f:
encrypted_bytes = f.read()
If you really want a "text-safe" format for those bytes, use base64.b64encode() and base64.b64decode().

Base64 encoding issue in Python

I need to save a params file in python and this params file contains some parameters that I won't leave on plain text, so I codify the entire file to base64 (I know that this isn't the most secure encoding of the world but it works for the kind of data that I need to use).
With the encoding, everything works well. I encode the content of my file (a simply txt with a proper extension) and save the file. The problem comes with the decode. I print the text coded before save the file and the text coded from the file saved and there are exactly the same, but for a reason I don't know, the decode of the text of the file saved returns me this error UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8d in position 1: invalid start byte and the decode of the text before save the file works well.
Any idea to resolve this issue?
This is my code, I have tried converting all to bytes, to string, and everything...
params = open('params.bpr','r').read()
paramsencoded = base64.b64encode(bytes(params,'utf-8'))
print(paramsencoded)
paramsdecoded = str(base64.b64decode(str(paramsencoded,'utf-8')),'utf-8')
newparams = open('paramsencoded.bpr','w+',encoding='utf-8')
newparams.write(str(paramsencoded))
newparams.close()
params2 = open('paramsencoded.bpr',encoding='utf-8').read()
print(params2)
paramsdecoded = str(base64.b64decode(str(paramsencoded,'utf-8')),'utf-8')
paramsdecoded = base64.b64decode(str(params2))
print(str(paramsdecoded,'utf-8'))

Your error lies in your handling of the bytes object returned by base64.b64encode(), you called str() on the object:
newparams.write(str(paramsencoded))
That doesn't decode the bytes object:
>>> bytesvalue = b'abc='
>>> str(bytesvalue)
"b'abc='"
Note the b'...' notation. You produced the representation of the bytes object, which is a string containing Python syntax that can reproduce the value for debugging purposes (you can copy that string value and paste it into Python to re-create the same bytes value).
This may not be that easy to notice at first, as base64.b64encode() otherwise only produces output with printable ASCII bytes.
But your decoding problem originates from there, because when decoding the value read back from the file includes the b' characters at the start. Those first two characters are interpreted as Base64 data too; the b is a valid Base64 character, and the ' is ignored by the parser:
>>> bytesvalue = b'hello world'
>>> base64.b64encode(bytesvalue)
b'aGVsbG8gd29ybGQ='
>>> str(base64.b64encode(bytesvalue))
"b'aGVsbG8gd29ybGQ='"
>>> base64.b64decode(str(base64.b64encode(bytesvalue))) # with str()
b'm\xa1\x95\xb1\xb1\xbc\x81\xdd\xbd\xc9\xb1\x90'
>>> base64.b64decode(base64.b64encode(bytesvalue)) # without str()
b'hello world'
Note how the output is completely different, because the Base64 decoding is now starting from the wrong place, as b is the first 6 bits of the first byte (making the first decoded byte a 6C, 6D, 6E or 6F bytes, so m,n, o or p ASCII).
You could properly decode the value (using paramsencoded.decode('ascii') or str(paramsencoded, 'ascii')) but you should't treat any of this data as text.
Instead, open your files in binary mode. Reading and writing then operates with bytes objects, and the base64.b64encode() and base64.b64decode() functions also operate on bytes, making for a perfect match:
with open('params.bpr', 'rb') as params_source:
params = params_source.read() # bytes object
params_encoded = base64.b64encode(params)
print(params_encoded.decode('ascii')) # base64 data is always ASCII data
params_decoded = base64.b64decode(params_encoded)
with open('paramsencoded.bpr', 'wb') as new_params:
newparams.write(params_encoded) # write binary data
with open('paramsencoded.bpr', 'rb') as new_params:
params_written = new_params.read()
print(params_written.decode('ascii')) # still Base64 data, so decode as ASCII
params_decoded = base64.b64decode(params_written) # decode the bytes value
print(params_decoded.decode('utf8')) # assuming the original source was UTF-8
I explicitly use bytes.decode(codec) rather than str(..., codec) to avoid accidental str(...) calls.

Python 3 and base64 encoding of a binary file

I'm new to Python and I do have an issue that is bothering me.
I use the following code to get a base64 string representation of my zip file.
with open( "C:\\Users\\Mario\\Downloads\\exportTest1.zip",'rb' ) as file:
zipContents = file.read()
encodedZip = base64.encodestring(zipContents)
Now, if I output the string it is contained inside a b'' representation. This for me is not necessary and I would like to avoid it. Also it adds a newlines character every 76 characters which is another issue. Is there a way to get the binary content and represent it without the newline characters and trailing and leading b''?
Just for comparison, if I do the following in PowerShell:
$fileName = "C:\Users\Mario\Downloads\exportTest1.zip"
$fileContentBytes = [System.IO.File]::ReadAllBytes($fileName)
$fileContentEncoded = [System.Convert]::ToBase64String($fileContentBytes)
I do get the exact string I'm looking for, no b'' and no \n every 76 chars.

From the base64 package doc:
base64.encodestring:
"Encode the bytes-like object s, which can contain arbitrary binary data, and return bytes containing the base64-encoded data, with newlines (b"\n") inserted after every 76 bytes of output, and ensuring that there is a trailing newline, as per RFC 2045 (MIME)."
You want to use
base64.b64encode:
"Encode the bytes-like object s using Base64 and return the encoded bytes."
Example:
import base64
with open("test.zip", "rb") as f:
encodedZip = base64.b64encode(f.read())
print(encodedZip.decode())
The decode() will convert the binary string to text.

Use b64encode to encode without the newlines and then decode the resulting binary string with .decode('ascii') to get a normal string.
encodedZip = base64.b64encode(zipContents).decode('ascii')

Python - writing a two byte string as a single byte hex character to a binary file

I'm writing a python script that opens a file and reads a two character string value e.g. 32 or 6f. It is then supposed to write that value as a single byte to a binary file as hex e.g. 0x32 or 0x6f.
file_in = open('32.xml')
file_contents = file_in.read()
file_in.close()
file_out = open('testfile', 'wb')
file_out.write(file_contents)
file_out.close()
In this example, 32.xml is a plain text file that contains the string '32'. But the contents of the testfile output file are '32' instead of 0x32 (or just 2).
I've tried all kinds of variations on the write command. I tried the chr() function but that requires converting the string to an int.
file_out.write(chr(int(file_contents)))
That ended up writing the hex value of the string, not what I wanted. It also failed as soon as you had a value containing a-f.
I also tried
file_out.write('\x' + file_contents)
but the python interpreter didn't like that.

You need to interpret the original string as a hexadecimal integer. Hexadecimal is a base-16 notation, so add 16 to the int() call:
file_out.write(chr(int(file_contents, 16)))