I am receiving json data that is base64 encoded and I am supposed to decode it using Python. For most base64 requests, everything works fine, but some requests contains some strange chars after decoding.
The thing is, in Python I am getting an error (Invalid padding), even though the padding is right, but if I try to decode the base64 with the unix command or on https://www.base64decode.org/ website, I get all the data from the base64.
Do you know any work around for this problem, beside calling the unix command from python code?
Thank you!
Related
I am trying to do the first CryptoPals challenge where I have to implement the base64 algorithm. I am using Python.
It has the following advice:
Always operate on raw bytes, never on encoded strings. Only use hex and base64 for pretty-printing.
So say I have a string and I have to convert this into bits. The issue is that I have seen people using UTF-8 or ASCII before converting the string to bytes/bits. (See here)
Should I always proceed with UTF-8? Or there could be issues down the road if my script parses something encoded in ASCII?
So I'm trying to send an email using python but I can't as long as it converts it to ASCII, is there a way around this or do I need to find another function?
File "/usr/lib/python3.6/smtplib.py", line 855, in sendmail
msg = _fix_eols(msg).encode('ascii')
UnicodeEncodeError: 'ascii' codec can't encode character '\u2019' in position 1562: ordinal not in range(128)
Can I get around this or do I have convert? and how would i convert?
Traditionally, SMTP requires the message you submit to be in ASCII. The problem is earlier in the process: The message you are trying to pass in should already have been converted into a proper MIME message when you created it.
Generally, any binary data should be converted to have a Content-Transfer-Encoding: base64 and any non-ASCII text should have Content-Transfer-Encoding: quoted-printable. Then you can safely use non-ASCII bytes and the transfer encoding takes care of transparently converting the payload to ASCII for transport, and the recipient's email software takes care of displaying it as you intended.
Python's email library already knows how to take care of these things. Perhaps you are trying to construct a message manually without actually checking what the specs say? But using the standard library is obviously easier and saves you from a fair bit of learning curve.
For concrete details, see e.g. how to send a email body part through MIMEMultipart
There are now provisions for extending SMTP to handle UTF-8 everywhere, but the error message suggests that your Sendmail is not yet up to the task. (Or perhaps there is an option you can add to its configuration, but that's far outside the scope of this question, and of Stack Overflow.)
I'm receiving frames from a websocket server and I'm not sure how to interpret some of the bytes object because they are mixed with actual words inside them.
I get something like this:
b'\x00\x17\x04\x00\x00\x00\xc0\x05FOCUS\x01\x00\xff\xfc\x00\x05;\xea\x01\x03\xe8\x81'
This one has 'FOCUS' and a ';' in it. I am expecting 'FOCUS' to be part of the payload, but I don't know why it's showing up as is, and not in hex form. Can someone explain what's going on and how I can unpack the rest of the data?
Also, it seems I'm getting the data in reverse order. I think \x81 is supposed to be the first byte of the frame.
I'm using Python 3.6 and the websocket-client lib. Thank you.
I have captured a large amount of data from numerous CSV files. Certain information has been carved out. One section I have carved out is a section that has a large amount of various text formats. Some of these are emotions and other non standard text.
When outputting this data into a HTML format I have had errors. Currently I have the following error:
UnicodeDecodeError: 'charmap' codec can't decode byte 0X90 in Position: character maps to <undefined>.
The program currently stores information into an Array from a String. The Array is then written to a HTML file.
Any idea how to overcome this issue in Python 3.2 or how to implement a Character buffer?
UPDATE
I have tried the comments below and also done more research.
I have used this code to no avail:
MessageArray.append(Message.encode('ascii', 'ignore'))
But I got the error:
TypeError: Cant convert 'bytes' object to str implicitly.
I was able to fix my issue by following #SilverbackNet 's comment. Although this did not fix my overall issue as being able to import and convert raw binary data from a CSV but allowed to ignore the data that was bringing me issues.
Hello I have python script that takes apart an email from a string. I am using the get_payload(decode=True) function from the email class and it works great for pdf's and jpg's but it does not decode bmp files. The file is still encoded base64 when I write it to disk.
Has anyone come across this issue themselves?
OK so I finally found the problem and it was not related to the python mail class at all. I was reading from a named pipe using the .read() function and it was not reading the entire email from the pipe. I had to pass the read function a size argument and then it was able to read the entire email. So ultimately the reason why my bmp file was not decoded is because I had invalid base64 data causing the get_payload() function to not be able to decode the attatchment.