We Keep Coding

Why is Python3 decode/encode duplicating characters?

I am trying to check the subject of emails in foreign languages for automatic tests. I was having issues with encoding so I decided to try and write something that handles the encoding of the subject. In this case it's given to me in base64. Converting this to utf-8 and then decoding it produces this strange double character problem. Here is some code I used to test this:
import base64
ja_str = "こんにちは"
encoded_js = base64.b64encode(ja_str.encode())
print (encoded_js)
print(base64.b64decode(encoded_js).decode())
The result of the above is:
b'44GT44KT44Gr44Gh44Gv'
ここんんににちちはは

Encryption not working properly

i have this piece of code for encryption.
from cryptography.fernet import Fernet
key = Fernet.generate_key()
f = Fernet(key)
token = f.encrypt(b"something cool")
k = f.decrypt(token)
print(k) `
This is the output
b'something cool'
According to the example on the website, that "b" should've gone. I'm very new at this and would like to know or understand how exactly the solution works.
Thanks

That ‘b’ means bytes. So instead of working with strings encryption algorythms are actually using bytes. My experience is that what you give a library (str/bytes/array) it should give you back, which Fernet is doing. I would simply convert the bytes back to a string k.decode(“utf-8”)

The encryption functions are doing what they should: bytes in and bytes out.
Cryptography and encryption work with bytes, not strings or other encoding, decrypt returns bytes. The actual low level decrypt has no idea of encodings, it can't the decryption could be a string, it could be an image, etc.
It is up to the caller to provide encodings in and out that are appropriate to the data being encrypted/decrypted.
As the caller wrap the encryption in a function you write that provides the correct encodings, in this case a string to bytes on encryption and bytes back to a string on decryption.

App Engine ndb StringProperty and string hashes

I'm using PyCrypto for generating secure key hashes. I want to store one or more of the partial keys I generate. Each partial key is in the form
\x0f|4\xcc\x02b\xc3\xf8\xb0\xd8\xfc\xd4\x90VE\xf2
I have an ndb StringProperty() in which I'd lke to store that info. However, it raises a BadValueError saying it expects an UTF-8 encoded string. I tried using str's .encode('uft-8') method but that also raises an error telling me it couldn't encode because bad positioning.
Anyway, my question is, how can I convert that byte string into something I can store in ndb?

Improved Answer:
In this case instead of storing the key as String or Text, you should use a BlobProperty which stores an uninterpreted byte string.
Original Answer:
To convert bytes (strings) to unicode you use the method decode. You also need to use an encoding that preserves the original binary data, which is ISO-8859-1. See ISO-8859-1 encoding and binary data preservation
unicode_key = key.decode('iso-8859-1')
bytes_key = unicode_key.encode('iso-8859-1')
Consider also using A TextProperty instead, as StringProperties are indexed.

How do I get rid of the "u" from a decoded JSON object?

I have a dictionary of dictionaries in Python:
d = {"a11y_firesafety.html":{"lang:hi": {"div1": "http://a11y.in/a11y/idea/a11y_firesafety.html:hi"}, "lang:kn": {"div1": "http://a11y.in/a11ypi/idea/a11y_firesafety.html:kn}}}
I have this in a JSON file and I encoded it using json.dumps(). Now when I decode it using json.loads() in Python I get a result like this:
temp = {u'a11y_firesafety.html': {u'lang:hi': {u'div1': u'http://a11y.in/a11ypi/idea/a11y_firesafety.html:hi'}, u'lang:kn': {u'div1': u'http://a11y.in/a11ypi/idea/a11y_firesafety.html:kn'}}}
My problem is with the "u" which signifies the Unicode encoding in front of every item in my temp (dictionary of dictionaries). How to get rid of that "u"?

Why do you care about the 'u' characters? They're just a visual indicator; unless you're actually using the result of str(temp) in your code, they have no effect on your code. For example:
>>> test = u"abcd"
>>> test == "abcd"
True
If they do matter for some reason, and you don't care about consequences like not being able to use this code in an international setting, then you could pass in a custom object_hook (see the json docs here) to produce dictionaries with string contents rather than unicode.

You could also use this:
import fileinput
fout = open("out.txt", 'a')
for i in fileinput.input("in.txt"):
str = i.replace("u\"","\"").replace("u\'","\'")
print >> fout,str
The typical json responses from standard websites have these two encoding representations - u' and u"
This snippet gets rid of both of them. It may not be required as this encoding doesn't hinder any logical processing, as mentioned by previous commenter

There is no "unicode" encoding, since unicode is a different data type and I don't really see any reason unicode would be a problem, since you may always convert it to string doing e.g. foo.encode('utf-8').
However, if you really want to have string objects upfront you should probably create your own decoder class and use it while decoding JSON.

Radix 64 and encryption

Need to share my problem that is :
A PGP public key server gives me the key in Radix64 format .
And i searching for any method which can encrypt my message using this Radix64 format public key .
any alternate suggestions or documents are welcome .........

exPyCrypto looks good.
This previous SO question addresses Radix64 format specifically for public keys.
To convert the actual base/radix64 encoded characters, see this question:
import base64
decoded_bytes = base64.b64decode(ascii_chars)

You can decode the key by using the base64 module and then encrypt the message.