I have a variable that stores json value. I want to base64 encode it in Python. But the error 'does not support the buffer interface' is thrown. I know that the base64 needs a byte to convert. But as I am newbee in Python, no idea as how to convert json to base64 encoded string.Is there a straight forward way to do it??
In Python 3.x you need to convert your str object to a bytes object for base64 to be able to encode them. You can do that using the str.encode method:
>>> import json
>>> import base64
>>> d = {"alg": "ES256"}
>>> s = json.dumps(d) # Turns your json dict into a str
>>> print(s)
{"alg": "ES256"}
>>> type(s)
<class 'str'>
>>> base64.b64encode(s)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python3.2/base64.py", line 56, in b64encode
raise TypeError("expected bytes, not %s" % s.__class__.__name__)
TypeError: expected bytes, not str
>>> base64.b64encode(s.encode('utf-8'))
b'eyJhbGciOiAiRVMyNTYifQ=='
If you pass the output of your_str_object.encode('utf-8') to the base64 module, you should be able to encode it fine.
Here are two methods worked on python3
encodestring is deprecated and suggested one to use is encodebytes
import json
import base64
with open('test.json') as jsonfile:
data = json.load(jsonfile)
print(type(data)) #dict
datastr = json.dumps(data)
print(type(datastr)) #str
print(datastr)
encoded = base64.b64encode(datastr.encode('utf-8')) #1 way
print(encoded)
print(base64.encodebytes(datastr.encode())) #2 method
You could encode the string first, as UTF-8 for example, then base64 encode it:
data = '{"hello": "world"}'
enc = data.encode() # utf-8 by default
print base64.encodestring(enc)
This also works in 2.7 :)
Here's a function that you can feed a string and it will output a base64 string.
import base64
def b64EncodeString(msg):
msg_bytes = msg.encode('ascii')
base64_bytes = base64.b64encode(msg_bytes)
return base64_bytes.decode('ascii')
Related
I'm using this library to download and decode MMS PDUs:
https://github.com/pmarti/python-messaging
The sample code almost works, except that this method:
mms = MMSMessage.from_data(response)
Is throwing an exception:
TypeError: unsupported operand type(s) for &: 'str' and 'int'
Which seems to obviously be some sort of binary formatting problem.
In the sample code, the HTTP response is passed directly into the from_data method, however in my case it comes through with HTTP headers on it so I'm splitting the response by double CRLF and then passing in just the PDU data:
data = buf.getvalue()
split = data.split("\r\n\r\n");
mms = MMSMessage.from_data(split[1].strip())
This throws an error BUT if I first write the exact same data to a file then use the from_file method it works:
data = buf.getvalue()
split = data.split("\r\n\r\n");
f = open('dump','w+')
f.write(split[1])
f.close()
path = 'dump'
mms = MMSMessage.from_file(path)
I looked in the from_file method, and all it does is load the contents and then pass it into the same method as the from_data method, so the first way should Just Work™.
What I did notice is that the file is opened in binary format, and the content is loaded like this:
data = array.array('B')
with open(filename, 'rb') as f:
data.fromfile(f, num_bytes)
return self.decode_data(data)
So it seems obvious that somehow what I'm passing into the first function is actually a "string representation of binary data" and what's being read from the file is "actual binary data".
I tried using bytearray like this to "binaryfy" the string:
mms = MMSMessage.from_data(bytearray(split[1].strip(), "utf8"))
but that throws the error:
Traceback (most recent call last):
File "decodepdu.py", line 41, in <module>
mms = MMSMessage.from_data(bytearray(split[1].strip(), "utf8"))
UnicodeDecodeError: 'ascii' codec can't decode byte 0x8c in position 0: ordinal not in range(128)
which seems weird because it's using an 'ascii' codec but I specified utf8 encoding.
Anyway at this point I'm in over my head because I'm not really all that familiar with python, so for now I'm just writing the content to a temporary file but I would really rather not.
Any help would be most appreciated!
Okay thanks to Paul M. in the comments, this works:
data = buf.getvalue()
split = data.split("\r\n\r\n");
pdu = array.array('B');
pdu.fromstring(split[1]);
mms = MMSMessage.from_data(pdu);
The following is the string I want to decompress:
eNrNnW1vI9eV598vsN+hoBeDDMxS33vOfajr2DtgU2qbabWoUGr5YbAYFFlFWzv9YKjVsWdfxZnZWQTILOBssjNJJsgEO7OzCyQ2eicZx7En/gbsr7S3SKlVp3SqVaSqLteA1SJVJA/rX/ecc889v7qv/clHDx9E38lPn5w8fvT6ltwWW3/yH/79v3ttJz1LD/Mz/2sUvfbRk1efTN/PH6bRSfb61n7+4flftyL/4kdPXt86/+XVj/zv75+dffDqrVsffvjh9oe4/fj0vVsghLz19r29w8V7XBz78Enm3+X1raenj87f/Un88GR6+vjJ49lZPH388FV/XLw8amthx9KS/EH+MH90Fj1KH+bUmOWxrw6fnD/z+tbZ6dP8xfP3n+SDp6en/rV7j6fpg/z8z+dvvXxz/7EfPMg/OvqLD/IXz5//5f3HJ9M8enjyaDSdPj31X1T4d04/unj09NHk8dNHWZ5tlV7IWXyUTh7k9KCXffbl39Ozs9OTydOz/PyNdvKz9OTBMNuKzvxLXt/yh5w88qfhVqNXv5E/6p+elV77xP/90XtNX+5fu+4nD8/yh4PHWb72Z9/oxfuPz05mJ9P0zF/vT9Z+l9un6aNs7Vfv5E+mpycfFCas/R4H6enZ/uO1X/7W4cGpv55Lr8/y6cnD9EHTNxgvrr4bvsn9RydnN3yL/fym7zB8mL6X3z99sPa5PMwfPPCH39CMb5/9xfDRICVjcoVRdeg92pPB44eFp1n7m7xx+vjpB3dO8gfrX9v38tP38sUFfrO36H+niSCv3XqJ41z88dz5NnHKcvNeeXB6cnaT1+7739b3q/4NjtOuTvvyNYsoehly+fe58g6LJ5ZJwvJxdjKbvXf66vKf9OE6KcX5a5Zvcd1rLj4o/s6Li+S1y9Tj8vssrqPo3LoiWTq/sC5ykNPHH45Os/x0kT1cXCo+7dqKlvH49a353z//7vzr5/9l/q/zz6L5z+dfzH8///X8S//v11vRIuq+vhX74y+iqH+xkFYlaiuqPCSRzkfs7zz4sze+Oc4/ePDNrWgRv3wYGx0O3iwMeRGPCgPm/zD/xfyf5v9j/pNovLvfv793FN0Z253BK9H8f89/Mv/Z/G+j+Q/9IT+b/+P8k8gf/+No/kv/+EfzH/vnPtmKlrHJf0dQAKiNMVvRecApUkznjSuFj9e3YFsnW9GLYHBxzIVrX6Sl/vGFo17mmE98kvmf89PH7wm5PYmn2aPtR/nZrRkmepaAjmeZyuJEpLPYWkxjnSv/3yRNspnc/k8f+Gu67LUvTHjhgxf63LpOV2B0lWVd3Y11NUh0LR62oWv//s4w6ie96Pa9tyLsRXeG/aNoZ3R7b9SLDvr7O/3e/vDwsL8f9ffu7Y770bB3MB4Wv1XEBWe0I+IqfUVcARVxi2PK4krXUNyJnSptpnGSZFlsZpNZnBkNsU2MlVLZDCcpL25hworiIiMulMUF4NV9qZx0mJqWhum94WDc70UDL9CRV7A/GOzuH12VyggqlalItRx2VCpTkco0kwpxOnOTifFPgI0zB/63XCo/DhHATm02AVaqpQkrSqUYqZD4V726VDYhUhUP25DKe9K7O4OrPlKUh1GyLbCqjdg2joizPOhSHLPtZDNxTCZFLv3omZokj8GPnVjCbObdJUxSmQmATLDiLG1YUR3NqKOIOurm4U/T8KdZscarijV4c/d4PNrbPYr6x7t+YN3t740Oe9H+/dvDYryNd999d7SMjW+/woQ9e/Ph5v4/HG6GEVSXBUX5Eq20ICFt8bANre6Mzc4gGs5/NBwtf39FYkUTJbSupCJwNRUx1VQE1tNEGyWVyKbxTM9EPElAxFOp8tgkwuokm+hJVpeKmJU1sYwmhmgibjrItNBUON2WR6wTyyERq8jQKmJJURGrOKYsVvG1uxxASxNWFCthxLJErFoZwGpVCkznDxvIcK//xv7u0TC61x/v7u0NXybI/J/nn/hE/qfFE9E3xgP3zuCPI7s3GEcy8n/5qZ8D/Iik+UYmKERisMjSXwgmtp274vEszQWXx1wK5gMMNoxhk9zoNIc4ddr5Hwhxlvu839oUDGTOK1bj8ezquaBjBEtICJMvU8wKoljxsF3FFuIshCqr4keRsImlqvgYo67GoSsZemX6VTy+EjlWP4/FWLlyIh05kfDSE0kvfdv6pX9wf/9otDyTgzjas3s7gzE5pYkQAvDKKb16obvqjFa5imeSzCldfQ4queKCJNWFVZNfBKlL85Tzhw1i9P4bd196cg/7vfOCQm+vvz8Y9nvHo73j0eUJBunKORNe9SB6cd5KJxYrHgS3IWnmQbJ06jLtHb2ZKYxzhSqeuWkW+1km5DJLlQM2C16asKpMbK2AFAvUGjIBlQkajYdrZFrUApZFgDuj8Y5/vHt41I+Gw97Q/z8YHo1Hu/vRt+7fO9gd9w77/duR84OiJCIinWc6cyVum2rcdpXEVzecZyZpNskgn8QoXBLjxE1jkImOjYWJVJm0aWr5uG1Wj9uSqwlIWhTANVR0VEXXhopvvnPfKzgsTWEWs5XexfMnUvROwP+Pws9rhv3oYDjoe+dX1jEBomOCV3RU1cCRkCkpLKaojUo7gNKL5WKbG4gnSvthCVMXq9yZ6Uylfmo65XVUa4QhrmAgScUA7Oo6GjoaDbThNPd33/Z+8nzGeXj/dn98v3cPoZQt+0BYntTANlZ1wmqevDyG6NRwUjORM61RyFi4aR7nOpPxZDJx8QT1RGUzjTPN6oTr5MmSKx1IUjtI1hhumg433cZwG/Tvjd/p+X/Gw/2Fr4wGIz/s9vq96M2RH3DRYPx2NLwUDS2IG6YQsA0N6z0SjEkmMxcLOctiBbmIhbU+axYZpClMp87ZmurAGgkJVx6QpD4gVxxcUoqkCBCVhw1U6+/d6Ufj0b3d0Uur4S+OiqSykR8NkTTH0VF8eNAf3+1FUpvFk986Ouy9cXTx66WYyugi+JJCuNyGam1Bqu1iRluWdHnUpaZu4UCbaJomszSRzsRikqsYk8zGM2NFnHgn6vMZ6ZRiB+K5EauKytUXJCkwmDVGoqAjUbQxEossc7l40RuPjn2KMuz7v/6f+c/9tPUTbgXrFX7xSrlKGmOuLlthtX5nKvMm3XDZCqZWpdOZjI2yOk59JIxT701jl0xBQqpmNs348Ier1+8kV3+QpAAh10ljJFVTtqHmt6U5GMf3ydzh8O5op7QQZWQ133RX801dzTcrs7EidW5U1JtMrE60H3ECXIzW5rG2dhbnSY7gZgZR1QhVmLCqUFzdQZLCA6wx7AwVyshW8pTlkPOZpJ/b+anDvf67/ue4lE8aZ4DkKcUlV9EpcZU8pThmnTwF8mmeQpbEqU4wTqYij9NcTOOsmBhMMUMrEl6nZPUVDuDKGpLWNdZxj0gHFLYyu/NTugv/WEzeLvWxssgnSvpcCWB+bo0VfaBSHHdN1+kniZPKzGLIwcZuAjrWPnX0+giXTSERsxT4PBJWj17A1UiA1EhArKiPSqwqjaPzhzceR5dy3L47NruxpJJcqd3586ErklRrd01XBWGSAAiZxNM0yWLrf8bG4STO0Uy1FC7337JGktVdG3D1ECD1EEhWl8RQSdpxbRcljwH0Bth7+7DvZ2IHu/ff2B35XFCYnv9he1L4HxftMoO94chPAipivkukLGSpSJnYipTFMWUpTdNGCZdoDcrGJpc+pxAzjLVWIp4IKexEoFA65aUsTFhVSrZTglRF5KqjS7rFpLbysIGUPo/zad78f3nxiqzuJw107UXnYl7KNXizv6wBk7zCmusTQFupYxWPW0jZgKtYAK1YrHqKE5TlBb3lwzZWYs9HQfxH7519c9k4llye2uKJXbJKVNQOyWldPkVWiYonyqtEouGyngGXJ4lE/wRaP/OdeseWWT8kJlM/NIQRlh8Iy09YVSSuXAGkXAF1nX77J9M/zx/UuzZpbEms84cNsoGd3f3D0c06waJqK1i06AW7lPTuMkenY+VqG5iy1bFSaQPDhlNfnAiT5BMZZ5mPVlYaLNKGWSwSmIKdTIVQNWu1ag3vxpUzgJQzQK0pqrDWlkRdPmxD1Lrib9S8+nsuKsk/5GJ1tiJrJSVcHlOWtel6TJqnRVVMxA6tikWWzWILiDH4DN44wCyt6+5bJyXkChpAChrSrSerTwlEafns/GErsi4qid/qv/vuosJY6DZeJBq96PBgND7qv7Hbi/wgHezu7B5GfZ+G9A9LZaq7IBhJua4Kc01XRVNJnTHTXJlpPEt1Hs+mXtLEChUrzF1mZ6lTUtR0VazeAgNcWQNoWSNZX1JHJXXtSLpcbCtWnRfu9si72537g0UXZyWZ9CKPDnb3ouPdwVG5D/fbS1VpMRkatJpV25oaThOmU5lKk/pkcua9b2KKGYLKTWxSlc9ykaUWdGutZsBVQIBUQOz6ETWhETVpSdJi0XQ5VBdFkIv4uahGXtGtMhq7VM7JNBNT71uVRhe7mcriTGUiVsqCmVhMdItNgsjVRIDURGB9BysFGY1SNMpcr5XuYjV7WWKM7vSPR+Ph0XKyFymheyhctNPffWs0iu6Oxj49GlFBK+MQrrbaXKlEgqKprdMNV76dmczMFOPcKhnPJsLFmbWTWEAizDRJJrlCXs01KpHIVVCQVlD0+mpKqqZsR81zofb6+6PD5TrqsoE3kj3t/zeRw9i5i/W54fFwEB1Hx8OqopURurqmsvF0JZu5CULmNc38uPS6YjzL/JxlmtlEJomCSdaiplwJBkkJBnHlEoy2pASzeMh61ZVa5ncG4kp/tXaV/mq86jqrbSaqmpvKbnt5lyasKgxXUEFSUEG3ojBgdbmn6/whO8rW6ZsriiU+H933+cx4N7J7B+NqeyhoVW0PXXfi38JUHVlkhNRTsKZiHx3HeyeP8pdVIaUhVUhp2qhCLkqIlYoibc9hptzVRD6pTLkT021teGnCquJwdRQkdRRcozbsNFHFtdH3UdHj8OXrJ7Boc3vJ+smym7yRQ5pOprlLRKwFpjFqsDFkGmMrIE9naYLeMfGCuDUE4WogSJEPtUaFF2mFt9n61ioVXjKFkua4UuFN7DuDPVrfuLp0DNXgbipdOE3RVIk6z3wSGQurRCycmsWpz3XiLDPGTJxPwid5TX1jjeDO1TeQEiHrJ2xlhOf8YRszpxYWWu4ys2F91TFKV61wVBwjNMzCZ6lOnJqYGNHPqexsKuOpziZx6jKwPgWf+L/WVDhWX2dGrsKBpMKhxEtFjY6O6puslAJVarJaPmxD16O3fGZwNDw4LzZjL+qrnjelFy3nWaPBUf942O9Fx29Fb4z27kTD495xb3dEa1eF6SXveiV7gG2rKt6VZg/Nu4qzdOLFS2Yx5kbGMhM2Vmme++nyxEltZhpVTTeqXWOmzNU4kNQ40K7hXTX1rvq6dM+7xHsHw9H+S5urqitl3olWfai7Koyu1ohdRRjXMOxNtM38+DfxRGU+Gckms1hkUscpSpnaKc7yurCnV68RK66EgaSEoVVdlrjz+GnxngcP/Nl+9PThy+BVZwi82rC18dpp0yJQ/sP8Z/NfRIt/Ppn/o3/i5/MfL1rhfrGsUx2N+/uHw6MemQQXxcYXZeQrrLLRqpzH68WqChHcLibAJcGXx1wKrhpzeXKSpArzJJ4JM4tdplUsE+9unZ5M5SzPRZ6z1calCasKzlU5FGVplG5BcEcFd2EEL7yvd7m7/cX9PO4eHESIr/SiV7RYVpb7h/5qiN68qrcp6622tbgC5TgaT5fHlKGcgoVupPfMCDtTuphjZH56kWA8mSZ5rNMkh9k0Vy5TPJTjVo+niquAKEn1drUBUzgrLxOhi4dtrKBfp2Qh13ljeS8avDl+53Bvd9xbCHyFojWkq8tsY3VWkmwrilQtjymPVmebThNhKtJipd2kJi6mWbGFRPmMd2ZzLVMtka1fLU1YVT2uTKKAqpfccLQqbV15jXb5sA2Ny0MuOro/vj3qndefY4zeOcQ7vYvsNhof+j8t0I9oeO9gvPtuP3pr/PZVpZUjSgt3RWlZVVo4qrRq2GGkpphDmrlYaq+0nTgRZ5hMfRqcCJdMklyajFdarqE0V61RpFqj5BqUozG0EbaVOk1fKrPMeBf0QInT0ZLw7MAwH2bbUIXOjyrlSn50NgUc5TTLJz5Fmhln4jyd2qIHVsYzyCaps6is5RGBpRWrisTe54Pe6EPadVhUS9tgbRttsMs75pzfMKd6vxzjNAYjqpzKvDIijVFp4fMahbGEzM86ZslE596FqplrjahSXCVHUThHmDUmG5JONmTbzXqVyo29/c6AItjJ1VlHJawtj7muOW+dQMTVWpShgahumfraCIQugTJYuHzYxrS8NkksT9HpDL1+gn6wrLwcJCVV7CLNr97GSFBZlgddyqK3bdO5QSIThYktMkSMRT5TsZmYPHZy4qcMfvpmZVJzGyOxhspc8UWR4osWcl2Vi/4DS9sRWuoF86r+cnHHjp8s7udBWDfd8z9M8cP2oju743F/PIzuoNa9i7bAvqlIfDSODx3R2DAaV7jg5UFljRtzAjMzmdrZJE4nM4i1mUxi6SNZ7EOJlTaboZzU3arqejBYNrubQvVWjcubdPowYZa/7y/ununP8BfzX81/P//t/N/mX0fzz+df+l9///yvnn88/3r+m/lnz78//7foTx8+/I/LVx2n/ouLnmtk1rV3GiyZpRdVvEu7fumvwu/OP42e/+Xzv/H//7U381/mf5h//vx70Z/uE2uKxpoGxlx7Z7ySMdIu7mB4ac1/L07D/LP5s7I959bMv/Jn6jN/vn77/Htlu1wz7a6/DdylXQqpWT+e/+H595//lf/kj4vbIT3/QWHed72A3hxv8Vfe3op0xcc1MOr6u59dGpUUdb6SUT+cf7r45HNTvvCm/LfiCW/n18X5qhjUK4rWDUy69v5dJZOspRf533uTfr/49IZGNbu+r719VVk6J4lJf12Mt8VFXZwrb10x7r7wI+4Lb9DflG0xjWy59u5M5ctbaHod/d3818//cv758ux84S373fxTf0lduXqwkSnX33eo5I2QKvXJ4kr+3fyr5z8o1Pru8x/MvyzGXvmENHOJ1960By6tiKk2P/RG/Nob8XnhFyN/kXgzSgbE0Tcu7xH7x82suf5+N1A24Y8enH3z/dNbBYtQ+uBmH3X9PVuglcsyeviwmUEN7j/SrovrNYyaDW6o0aZPaWrW9fePKJulifctfeD8f3pDnvnLuDh1n0aH/d1F8PIC/8Y/5Yd7M2uuvzECtOFZmp6c65H+Zifnp0XGs0hzllfVi8znY+9yFpd75K39YpEZfekjv8+Jmxl4PabeyEAZl3OxS5v+ZXHpf784qc+/18yiBjx2E4ue/1f/sR/7Dy/8sr/KP57/ylvxRXGRRfrPR+838wYNqONmAn5VuObfeks+W4y+3/iLzDvt3ywuLn+1eY/hz9Wzwp0VMxqv4OcrnLMG7C0GDB8NuFNsKXw04CJVyG9+PUKo2vrm14NwKmjgbABxqY0EzgYYktpA4GyA0qhwgbMBBKICBs4GZIPaaOBs0KyvwgbOBq3mKlzgbNBfrTYfOBu0KOuA4aNBU65uKXw0aDfVQcNHg2ZKvZHw0aAhUG8gfDToaNMNw0cpVizDx6KmHs2/WvzycXEWCyfphf6D//3/FkL7c/q74vlmNcMGPV4txBbdg2bmNOhA2mhsadIy08RAjJjY8myd2NKkDSRcbGnQqqA3H1sarNWbgLGlwaq0aSm2NFirNUFjS4NlRbOR2KI4J66SGsOCxRbNOWzl6szqeGqiOX+tRY01nU9NNOedtVz95HQVPjTnrDWsbGB7UxPNOWyNq5+yf34RPgq37C+uL87DyLPzMFKcrU+LhZiP/ekqPHjxh8/mz5qZyXlsrVY3s9O4otmlRV25XUOwuKI5Z68NY86N44rm3Li2Nd8cCjDlSuuAl2bZLcC4x4aLzZzP1kmNFV0vpBrOVWtXY80N2haa5daG89VG1JhjEqysxf/qQpr3yh+umq3kGs4zG1nz4SEcn+FcsYEai1zlWimu2GcLf1FkyUV4r430cvV2AMM5ZYOVG78H8yOGc75GMebc2I8YzoMaXfPNA+Snhm3NMHUGhctPDedxja0xLFh+ajgXbJI6szrOTy3ngo2rsWbt/LSpZpZzwVasfnK6yk8t56atXNnA9ty05dy0hZUtaqe8YTnHbHF1ATtNQy3nr62q7MwcLHxYzqdbzZhz4/BhOW9tTc03b6WBtVnPqGW76myNYZ23sFrOS9ukxpyQTawJ57Gtq7EsWBtrwnnuRNSYFaiRNeG8dSJrjArUyppwDjuBOgG7bmblHHaCdRd6t+2sCeeXkzq/3FlDa8L540RXb1geLD4knNNODGfPjQNEwvnhxNZ99wATjIRtck5qLQo3w3CcK05cnWXBphiO88VO1NrV8RzDcV7YyTpzOi+CO84BO1jj9HQ1y3CcU3a4uoXtTTMc55qdWt2kduYZjnPRTq+hYacTDcd5bmeqO0MHiySO8+7OcvbcOJI4zm+7pLqZczgwRLCciuMMujkaIngMRVQ3UQv49VlaRUjOoha+P4uiCKj7/iHoGMFSKAJrbQoIyAgWRRGqzrZwjIxgsRShay3rGpMRLJgiTJ1B3ZMygiVRhF3jFHUGywiWTRHJ6ja2yMuw6ODC1FWNaietqKEHpVhDyU4Tizr2kHrzkPhIDXxI6UPZFkFSQxRSpLD8/VsoZcpeQx6WxwopV1i2rfNqZg1aWNl0WW2moFkDGlLSUKoN1DRrCMPKTrgqeFmzhjOkoKFUwSubNcRhZaNTFa64ycOFktKFUgWrb/J0oYRar90dsw88Kk69dUh+hOcPJQUQZVsICY8gSsogyrAUCU8hSoohys2AJDyIKCsbom2CJeFZRFnZ10sHnJ+wOKKkPKLUIecnLJEoKZIoN8uN8FCipFRiMxtbnJ+wXKKkYKIMSI/waKJEWEPJbucnyN/xA6t7noeLLyygKCmhKNvCSHhGUVJIUYYlSXhMUVJOUW4GJuFJRUlRRbkJnoSHFSWlFWVApIRHEiVlEmVIqoSnEiXFEuVmwRIeTJSUTJSB2RKeTZQUTmxmVEvxhcUTpcI1lOw2vij+1k3Um4dkSXhIUVbundsWTsJzipKCijIAUcLziZICijIgVMJziZKCiTIkV8IjiZIyibI7tISHECWlEGUjugRao7J5FFFSFlFuDDDhQURJSUQZkjHhQURJSUTZFmbC84SSAoXl79+Zf9H8XfF0nSGd+xcWGpSUGiwbFMC/sC5Y2zqLWvcvrMelWKEMjEXwcKGkdGHZqMD+hYUNJaUNZUgIgScQJUUQZVscAk8XSooXlr9/Z/6FRQklZQllwIZsHiaUlCaULqR/YZlDSaHDskVt+xfD34nU1H1+GP/COl3KF5aNCu1fWJ9MOUMI2cTOo4aSsobQVhs7TxJKihJC2EZ2Hh6UlB6EzbSy8xihpBwhbKKZnUcKJWUKIWA7O08PSooPQsiGdh4glJQghM22tPPkoaToIQRuauepQ0mxQwjY1s5zh9ImayjZbf2NxRAl5RAhZGs7TyBKiiBCW83tPFooKVtY/v6d5a8sTigpTwgBd8fgiUJJkUKQIfNXFiuUlCssW9R2/sryhJICheXPD+L6WKhQUqqwbFTg/JXlECUFESEoPpLwN/dPOItu7l9YtlBSuBAC7y7CcoWSgoWwoQ1GWMhQUsoQNrLHCIsbSsobQshtRli6UFK8EILuNMLChZLShbDhzUZY4lBS5BCC7zfCOnEKGELALUd4wFA6u4aS3eavLJ0oKZ4IQTceEfx2LY6z6OY7cAh+2xVR9/1DbMLBEopACUXYDJ8HLM8IlGeETfB5wFKNQKlGCMjnAYsyAkUZISSfByzCCBRhhM3yecBSjUCpRgjM5wELNgIFGyEgnwcsxQgiWUPJjne1YjeBoWQjBN3eiUUbobIzYms7PEl+Hy1Z9/27qo8AvxttZVPEgEwSsOQiVDZDVAHrI8DyilDZB1F1Vh8Blk6Eys6HgXckApZKhMr+h2pT9RFg0USo7H6og/oX1jVXdj9si98CFjWEyu6GOoB/4fcvpIgh6ID+hd/BkDKGZYO69y81Gx1CnUVt+xd+Z0NKFkJgtAj4PRApgFg2KrB/4XdEpCwihOR3gN8KkRKIYFrzL6x/pVQhhOV3gN/nkIKFsBl+B/hNDyliCJvgd4AFC4GChRCQ3wEWKgQKFUJIfgdYqhAoVQib5XcA+e1pcXUbW3TiLFMIlCmEgPwOsNghoF5DyW7nxyyLCJRFhJD8DrAEIlACEdrid4ClCoFShWDDxhcWLAQKFhKbAsYXFjEEihiWbQsXX1iwEChYCDZgfGGhQqBQYdmg7uMLSxUCpQobnqLO4ovi9yRXq9vYYnxhmUKgTGEzo1qKLyx2CMqsoWS38YXFEoFiiRCS3wKWSwTKJUJb/BawzCFQ5hDC7hQELIcIlEOEzWwWBCyOCBRHhE3sFwQsiQiURISAWwYBCyICBREhCRlfWFgRKKwIm904CFiOESjHCIEhOWBZRqAsIwTcPghYlBG0XUPJbuMLyzwCZR4hJL8HLPAIFHiEtvg9YPlFoPwiBOD3gMUWgWKLEJDfA5ZsBEo2Qkh+D1jEESjiCN3xe8ASjUCJRgjM7wELNQKFGmFj/B6wzCNQ5hFFUP/CumYKPKJozb+w/pXyi+XvH3CrMmBJRqAkY9m2zu/vCyztCJR2LFsU8v6+wGKPQLFHbBl7bGgZ66op9Fi2LND9fYFFHoEij9gqjNnQLtaRU/IRRcBYy4KPQMFHbAXFbGgP67Vtrdfu7P6+wJKNQMlGDMnnAYs2AkUbsS0+D1g+ESifiGv3d4vIf/vIf/3If//oG7dHh4M3G54GFlMEiilia23eLKYIFFMsn4ZWwmhDb8KSi0DJxbJt3YdRFl0Eii6WLQoaRlmKESjFiLiJMMryjUD5xrJlocIoizgCRRwRw4dRlm0EyjYiBgyjLNoIFG1EDBdGWfgRklrn3V0YZZFHoMgjqlbCSEODWIdOOUdsrYubZReBsosYooubZRaBMosYsoubRRaBIosYtIubBRSBAorYYRc3yyICZRExdBc3yyMC5RFxc13cLIUIlEJEHc69IAshAoUQsa0mbmQhRKQQIgZo4kaWPERKHmLAJm5kcUOkuCGGbOJGFjNEihlid03cyFKFSKlCDNzEjSxYiBQsxI01cSPLFCJlCtGEdC8Ja5DlDGrBvTj205K6rx+gBwJZLhApF4ib6eFGlhBESgjiJnq4kaUJkdKEGLCHG1mqEClViCF7uJHFCpFihbjZHm5kQUOkoCEG7uFGlj5ESh9iwB5uZMlDlGYNJTvtgUCWQUTKIKINGF5YBBEpgohttXAjiyAiRRAxbAs3sjQiUhoRN9PCjSyYiBRMxE20cCMLKCIFFDFgCzeyxCJSYhFDtnAjiysixRVxsy3cyDKLSJlFDNzCjSy2iBRbxIAt3MiSjQh2DSW7DS8s7ogUd8QkYHhhGUekjCO21cGNLLeIlFvEsB3cyKKLSNFF3EwHN7IQI1KIETfRwY0suogUXcSAHdzIYotIsUUM2cGNLLeIlFvEzXZwI4ssIkUWMXAHN7LUIlJqEQN2cCMLNiImayjZbXhhaUektCO6gOGFRRyRIo7YVgM3stgiUmwRAzflIosuIkUXy0a10q5iG9rGenEKMZZtW6xXr1XrLVl3r+kKBosvIsUXiZxheh2QJRiREowYsD0fWYIRKcHY7Lpf+qdnRbDxl1Yh7LPmIZjlE5HyiWUzEloWb+gmG9rCumxKJioR0BOy8CJSeFG11WqOLJaIFEssf/3OViFZBhEpg6gCttciix4iRQ+VCLkKybKHSNnDskVtr0KypCFS0lAF3sUEWbQQKVpYNirwKiQLGWJlw0RhQgww1ulWNkoUJuQIY11vZevEskXdDzGWI8TKxollk9oeYyxaiJWtEUWjBRBsbb9aZDlDrGyPWLYq8ChjsUOk2KGSOmy9iCUPkZKH1KiABSMWS0SKJRLjwlWMWDgRKZxITeu6ZMTyiEh5RGLR2jUj3WvqJ1hvToHEpiepQdHIW/3lQtnPvO3PGrsN1sFTjrGhke1tvo0swYiUYGxoVUtlI5ZgREowNhWz27oRizMixRm1CLD1F7L4IlJ8kVjSeTrDgotIwUViUffpDMssImUWiUltpzMso4iUUSQGhPEArD+npCKxKnA6w/KLSPlFrfRmSoEsyogUZSTGdY6uIcs7IuUdiUkh2TVkoUek0COxLhi8hiwhiZSQJKYFoteQxSOR4pHEsFAlXZaSREpJUjG7DjssGokUjaQX/7rJadOLivX5lIik3qsrgg1ZCBIpBKmVadWRyqbzHJaExMTWGhfAkbK+PUlqTQrqSFkyEhNXa104R8qikuhErWmhHCmLTKKTtYaFcqQsXYkO6sXs2pGymCU6rL/4O3akLGaJTtV7r84cKctXoqv4dBtg3seSluhMrSXdXzesH3e21qLu530sWokuqTWp5XmfYlFKdK7WgBBrRYpFLpUQtVaFnfcplsRUouKsk5bTFdnQOGSNg1rjOk9XFItlKoG1JoVMVxQLbSqhaq0Llq4oFt1UQteaFihdUSy0qYSpNSxQuqJYeFMJWy9mx2FHsYCnEkn9xd9tuqJYulMJV++9ukpXFAtzKgpzSmnDLggqFuRUFOSsWBVuRVCxVKeiVCe1LtiSoGL5TkX5zoptHa8JKhbnVBTnpCZ13kiuWJpTUZqz8WnqqpVcsYinoohnUytbTA5ZoFNRoLOpWe2sCyoW6VQU6Wys580XBl+7tZ9/uOOtOczPFk+9duvc5uU/6UP/7Gu3Lo74f1DSNzk=
I have tried zlib:
import zlib
decompressed_data = zlib.decompress(data)
I get the following error:
TypeError: a bytes-like object is required, not 'str'
Then I did:
data = bytes(data, "utf-8")
decompressed_data = zlib.decompress(data)
I get an error again:
Error -3 while decompressing data: incorrect header check
You first need to decode the base64, then zlib decompress:
import zlib, base64
decompressed_data = zlib.decompress(base64.b64decode(data))
Looking at your data, it appears to be UTF-8 encoded XML, so we're almost there:
xml = decompressed_data.decode("utf-8")
So i'm trying to create a very simple program that opens a file, read the file and convert what is in it from hex to base64 using python3.
I tried this :
file = open("test.txt", "r")
contenu = file.read()
encoded = contenu.decode("hex").encode("base64")
print (encoded)
but I get the error:
AttributeError: 'str' object has no attribute 'decode'
I tried multiple other things but always get the same error.
inside the test.txt is :
4B
if you guys can explain me what I do wrong would be awesome.
Thank you
EDIT:
i should get Sw== as output
This should do the trick. Your code works for Python <= 2.7 but needs updating in later versions.
import base64
file = open("test.txt", "r")
contenu = file.read()
bytes = bytearray.fromhex(contenu)
encoded = base64.b64encode(bytes).decode('ascii')
print(encoded)
you need to encode hex string from file test.txt to bytes-like object using bytes.fromhex() before encoding it to base64.
import base64
with open("test.txt", "r") as file:
content = file.read()
encoded = base64.b64encode(bytes.fromhex(content))
print(encoded)
you should always use with statement for opening your file to automatically close the I/O when finished.
in IDLE:
>>>> import base64
>>>>
>>>> with open('test.txt', 'r') as file:
.... content = file.read()
.... encoded = base64.b64encode(bytes.fromhex(content))
....
>>>> encoded
b'Sw=='
just trying to load this JSON file(with non-ascii characters) as a python dictionary with Unicode encoding but still getting this error:
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 90: ordinal not in range(128)
JSON file content = "tooltip":{
"dxPivotGrid-sortRowBySummary": "Sort\"{0}\"byThisRow",}
import sys
import json
data = []
with open('/Users/myvb/Desktop/Automation/pt-PT.json') as f:
for line in f:
data.append(json.loads(line.encode('utf-8','replace')))
You have several problems as near as I can tell. First, is the file encoding. When you open a file without specifying an encoding, the file is opened with whatever sys.getfilesystemencoding() is. Since that may vary (especially on Windows machines) its a good idea to explicitly use encoding="utf-8" for most json files. Because of your error message, I suspect that the file was opened with an ascii encoding.
Next, the file is decoded from utf-8 into python strings as it is read by the file system object. The utf-8 line has already been decoded to a string and is already ready for json to read. When you do line.encode('utf-8','replace'), you encode the line back into a bytes object which the json loads (that is, "load string") can't handle.
Finally, "tooltip":{ "navbar":"Operações de grupo"} isn't valid json, but it does look like one line of a pretty-printed json file containing a single json object. My guess is that you should read the entire file as 1 json object.
Putting it all together you get:
import json
with open('/Users/myvb/Desktop/Automation/pt-PT.json', encoding="utf-8") as f:
data = json.load(f)
From its name, its possible that this file is encoded as a Windows Portugese code page. If so, the "cp860" encoding may work better.
I had the same problem, what worked for me was creating a regular expression, and parsing every line from the json file:
REGEXP = '[^A-Za-z0-9\'\:\.\;\-\?\!]+'
new_file_line = re.sub(REGEXP, ' ', old_file_line).strip()
Having a file with content similar to yours I can read the file in one simple shot:
>>> import json
>>> fname = "data.json"
>>> with open(fname) as f:
... data = json.load(f)
...
>>> data
{'tooltip': {'navbar': 'Operações de grupo'}}
You don't need to read each line. You have two options:
import sys
import json
data = []
with open('/Users/myvb/Desktop/Automation/pt-PT.json') as f:
data.append(json.load(f))
Or, you can load all lines and pass them to the json module:
import sys
import json
data = []
with open('/Users/myvb/Desktop/Automation/pt-PT.json') as f:
data.append(json.loads(''.join(f.readlines())))
Obviously, the first suggestion is the best.
I've got some real problems to encode/decode strings to a specific charset (UTF-8).
My Unicode Object is:
>> u'Valor Econ\xf4mico - Opini\xe3o'
When I call print from python it returns:
>> Valor Econômico - Opinião
When I call .encode("utf-8") from my unicode object to write it to a JSON it returns:
>> 'Valor Econ\xc3\xb4mico - Opini\xc3\xa3o'
What am I doing wrong? What exactly is print() doing that I'm not?
Obs: I'm creating this unicode object from a line of a file.
import codecs
with codecs.open(path, 'r') as local_file:
for line in local_file:
obj = unicode((line.replace(codecs.BOM_UTF8, '')).replace('\n', ''), 'utf-8')
Valor Econ\xc3\xb4mico - Opini\xc3\xa3o is the UTF-8 representation prepared for a non-UTF-8 terminal, probably in the interactive shell. If you were to write this to a file (open("myfile", "wb").write("Valor Econ\xc3\xb4mico - Opini\xc3\xa3o") then you'd have a valid UTF-8 file.
To create Unicode strings from a file, you can use automatic decoding in the io module (Codecs.open() is being deprecated). BOMs will be removed automatically:
import io
with io.open(path, "r", encoding="utf-8") as local_file:
for line in local_file:
unicode_obj = line.strip()
When it comes to creating a JSON response, use the result from json.dumps(my_object). It will return an str with all non-ASCII chars encoded using Unicode codepoints.