We Keep Coding

Python - Base64 - zlib - how to convert encoded Gzip file to Dictionary

When pulling data from a Solar API, a base64 encoded GZip file is sent.
When decoded, the data is received as a string but clearly is a nested dictionary.
Attempting to convert the string to a dictionary is unsuccessful.
I have tried to convert a string output from "json_str" to a dictionary "json_dict" but received an error of:
ValueError: dictionary update sequence element #0 has length 1; 2 is required
My Code:
import base64
import zlib
compressed = #Full byte string attached below
json_str = zlib.decompress(base64.b64decode(data), 16 + zlib.MAX_WBITS).decode('utf-8')
json_dict = dict(json_str)
Link to compressed byte string (.txt file):
Drive Link to code
My Thoughts:
This conversion should be successful as the output string is a nested dictionary as specified by the API reference, maybe I am decoding the string incorrectly from base64?
Any help is much appreciated.
Thanks

How to encode and decode docx file using base64 in python

I am trying to encode docx file and decode/pass it on frontend/UI in streamlit. As of now i knew how to encode/decode strings using base64 but not with docx file.
If any of you guys have any code on how to achieve it. Please do share here.
import base64
import streamlit as st
data = open('/home/lungsang/Desktop/streamlit-practice/content/A0/A0.02-vocab.docx', 'rb').read()
encoded = base64.b64encode(data)
decoded = base64.b64decode(encoded)
st.download_button('Download Here', decoded)
I used the above code but not getting the desired result.
Instead I got collection of .xml file. As shown in the below screenshot
The supposed decoded document should look like this..
If you guys need the docx file that i am trying to encode/decode, here is the link https://docs.google.com/document/d/10zkg1HLDHhZNh83i2tbJqBVMfIsdqW-3/edit

You need to add filename argument to download_button function
import base64
import streamlit as st
data = open("test.docx", "rb").read()
encoded = base64.b64encode(data)
decoded = base64.b64decode(encoded)
st.download_button('Download Here', decoded, "decoded_file.docx")

This is just encoding You Have to Decode
with open('YOUR DATA FILE', 'rb') as binary_file:
binary_file_data = binary_file.read()
base64_encoded_data = base64.b64encode(binary_file_data)
base64_message = base64_encoded_data.decode('utf-8')
print(base64_message)
open the file using open('Your Data file', 'rb'). Note how we passed the 'rb' argument along with the file path - this tells Python that we are reading a binary file. Without using 'rb', Python would assume we are reading a text file.
use the read() method to get all the data in the file into the binary_file_data variable. Similar to how we treated strings, we Base64 encoded the bytes with base64.b64encode and then used the decode('utf-8') on base64_encoded_data to get the Base64 encoded data using human-readable characters.
Executing the code will produce similar output to:
python3 encoding_binary.py

Python Bytes & Lists & Encryption

I'm using Fernet to encrypt my data with this implementation. Let's assume that I have these three data:
data = [fernet.encrypt("Hello".encode()), fernet.encrypt("Stack".encode()), fernet.encrypt("Overflow".encode())]
After this operation, Python automatically converts bytes to string, and I'm writing them to a csv file. When I need to decrypt them like:
fernet.decrypt(data)
It gives me an error like you can only decrypt only bytes etc. I also checked that my data in the csv file is already bytes but string form.
b'gAAAAABiVUw5BzOkOv3VxlV5xa57Iaf0R4dzPbgsrnheAME8uYeslCZfTx9GeyRWe7l9VMM-gdDXiPZ4zsAXoXkG6T1dyXH6EztcqirrPhXX3YCt65_3xXvykVTDPdbEXs51cHvR-3HH'

An end-to-end usage example for encoding, writing to text, reading, and decoding.
The Fernet documentation can be referenced here.
from cryptography.fernet import Fernet
# Auto-generate a secret key.
key = Fernet.generate_key()
f = Fernet(key)
# Encode the string 'Hello' and encrypt.
encoded = f.encrypt('Hello'.encode())
This creates a bytestring (a bytes object) as:
b'gAAAAABiVVOOeO-hUG2QaKCVOyshntpbqVbxnexIVsFr7ttBGmKhHlDeM7jkTCjPPGphZxbh4D15X82pts3hKes12DjzwI8_jQ=='
Write, read and decrypt:
# Write the *decoded* encrypted string to a TXT file.
with open('/tmp/encoded.txt', 'w') as fh:
fh.write(encoded.decode())
# Read the encrypted string from TXT file.
with open('/tmp/encoded.txt') as fh:
encoded = fh.read()
# Encode the string, pass through fernet for decryption,
# and decode the bytes output.
f.decrypt(encoded.encode()).decode()
Output:
'Hello'

fernet.encrypt returns bytes (I assume, you're not being specific which implementation you're using, I'm guessing this one). .decode() them to a string. Then your CSV will contain "gAAA...", not "b'gAAA...'". When reading those again from the CSV, .encode() the string before passing it to fernet.decrypt.
fernet.encrypt returns bytes
bytes.decode() turns bytes into str
CSV wants str
str.encode() turns str into bytes
fernet.decrypt wants bytes

Python gpg decryption not printing non english characters correctly

I am trying to decrypt a json message body having mix of numeric and non english characters. The decrypted string is not showing non English characters properly.
Details:-
1) Input is base64 encoded and gpg encrypted
2) I am using python base64 and gnupg modules to decode and decrypt the message.
The output is displayed as (part of the output due to the data sensitivity):-
{"id":"123","name":"ååéå業é¡
I am expecting the output as
{{"id":"123","name":"豐國業銀"
Here is the python code I am using for the above task:
import json
import os
import base64
import gnupg
gpg = gnupg.GPG()
with open('item2.json', 'r') as file:
json_data = json.load(file)
for value in json_data['items']:
data = value['payload']
print (data)
str_data = base64.b64decode(data)
print (str_data)
decrypted_data = gpg.decrypt(str_data, passphrase=output)
print (decrypted_data)

It's probably caused by a character encoding mismatch. It will also make a difference whether you're using Python 2 or Python 3.
Since Python 2 has finally reached its EOL, I'm going to assume you (and everyone to subsequently read this) need Python 3 code. There are also a number of possible explanations stemming from the way the python-gnupg module accesses the gpg executable.
For both security and ease of use reasons, I recommend instead using the Python bindings module which ships whith the GPGME C API instead.
import json
import os
import base64
import gpg
c = gpg.Context(armor=True)
with open('item2.json', 'r') as file:
json_data = json.load(file)
for value in json_data['items']:
data = value['payload']
print (data)
str_data = base64.b64decode(data.encode())
print (str_data)
decrypted_data, result, verify_result = c.decrypt(str_data)
print (decrypted_data)
Now decrypted_data only contains the plaintext, result contains information about the key(s) the encrypted data is encrypted to, and verify_result contains information about the key(s) the data was signed with, if any.
More details are in the included documentation, a draft copy is available here for convenience.
In all likelihood, however, the cause was the base64.b64decoding method, which needs to process bytes, not str. Which means you either need to encode data (as I have done above), or read in the JSON data as bytes instead of strings.

Convert file stream to base64 python

I have read the file stream of a zip file by the following code:
file = open(source_url, "rb")
data = file.read()
file.close()
byte_arr = base64.b64encode(data)
Now I am trying to call a webservice which accepts base64Binary format of data (byte array written in java). If I send byte_arr to the web-service I get client error:
Fault env:Client: Caught exception while handling request: unexpected element type: expected={http://www.w3.org/2001/XMLSchema}base64Binary, actual={http://www.w3.org/2001/XMLSchema}string
Please suggest why is base64 module not working for me.
type(byte_arr) is still string.
With thanks,
Sandhya

I guess there's nothing wrong with your base64 encoding. It seems like it is not embedded in a correct XML document. Probably the error is when you send your data, maybe you should check that piece of code.