I have a data, let's say
data = [
['header_1', 'header_2'],
['row_1_!', 'row_1_2'],
['row_2_1', 'row_2_2'],
]
I need to send that data as .csv file attachment to email message.
I can not save it as .csv and then attach existing csv - application is working in Googpe App Engine sandbox environment. so no files can be saved.
As I understand, email attachment consists of file name and file encoded as base64.
I tried to make attachment body in the following way:
import csv
if sys.version_info >= (3, 0):
from io import StringIO
else:
from StringIO import StringIO
in_memory_data = StringIO()
csv.writer(inmemory_data).writerows(data)
encoded = base64.b64encode(inmemory_data.getvalue())
But in result I have received by email not valid file 2 columns and 3 rows, but just one string in file (see the picture).
csv_screen
What I'm doing wrong?
I've found out the mistake. I should have been convert it to bytearray instead of encoding to base64:
encoded = bytearray(inmemory_data.getvalue(), "utf-8")
Worked fine that way.
Related
Good Morning,
I have downloaded my *.eml from my Gmail and wanted to extract the content of the email as text.
I used the following codes:
import email
from email import policy
from email.parser import BytesParser
filepath = 'Project\Data\Your GrabPay Wallet Statement for 15 Feb 2022.eml'
fp = open(filepath, 'rb')
msg = BytesParser(policy=policy.default).parse(fp)
text = msg.get_body(preferencelist=('plain')).get_content()
I am unable to extract the content of the email. The length of text is 0.
When I attempted to open the *.eml using Word/Outlook, I could see the content.
When I use a normal file handler to open it:
fhandle = open(filepath)
print(fhandle)
print(fhandle.read())
I get
<_io.TextIOWrapper name='Project\Data\Your GrabPay Wallet Statement
for 15 Feb 2022.eml' mode='r' encoding='cp1252'>
And the contents look something like the one below:
Content-Transfer-Encoding: base64
Content-Type: text/html; charset=UTF-8
PCFET0NUWVBFIGh0bWwgUFVCTElDICItLy9XM0MvL0RURCBYSFRNTCAxLjAgVHJhbnNpdGlvbmFs
Ly9FTiIgImh0dHA6Ly93d3cudzMub3JnL1RSL3hodG1sMS9EVEQveGh0bWwxLXRyYW5zaXRpb25h
bC5kdGQiPgo8aHRtbCB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMTk5OS94aHRtbCI+CjxoZWFk
I might have underestimated the amount of codes needed to extract email body content from *eml to Python.
I do not have access to your email, but I've been able to extract text from an email that I downloaded myself as a .eml from google.
import email
with open('email.eml') as email_file:
email_message = email.message_from_file(email_file)
print(email_message.get_payload())
When working with files it is important to consider using context managers such as I did in my example because it ensures that files are properly cleaned up and file handles are closed when they are no longer needed.
I briefly read over https://docs.python.org/3/library/email.parser.html for additional information on how to achieve the intended goal.
I realised the email is in multipart. So there is a need to get to the specific part, and decode the email. While doing do, it returns a chunk of HTML codes. To strip off the HTML codes and get plain-text, I used html2text.
import email
from email import policy
from email.parser import BytesParser
import html2text
filepath = 'Project\Data\Your GrabPay Wallet Statement for 15 Feb 2022.eml'
with open(filepath) as email_file:
email_message = email.message_from_file(email_file)
if email_message.is_multipart():
for part in email_message.walk():
#print(part.is_multipart())
#print(part.get_content_type())
#print()
message = str(part.get_payload(decode=True))
plain_message = html2text.html2text(message)
print(plain_message)
print()
I was looking for an appropriate way to get a specific image from an eml file, but unfortunately, I always get text data without images!
Here is the code I used but it gives me just text data :
from email.parser import BytesParser
from email import policy
with open(em, 'rb') as fp:
name = fp.name # Get file name
msg = BytesParser(policy=policy.default).parse(fp)
data = msg.get_body(preferencelist=('plain')).get_content()
print(data)
fp.close()
do you find any way to solve this？ I'm eager to know the method
Situation.
I need to create in-memory csv file and send it to bot.
According to this article and bot documentation I assume I can send csv files. Difference: i use another api wrapper: telebot. According to docs it also allows to send binary files.
So I am trying to test this like that:
def test_buf():
import csv
import io
test_data = [[1, 2, 3], ["a", "b", "c"]]
# csv module can write data in io.StringIO buffer only
s = io.StringIO()
csv.writer(s).writerows(test_data)
s.seek(0)
# python-telegram-bot library can send files only from io.BytesIO buffer
# we need to convert StringIO to BytesIO
buf = io.BytesIO()
# extract csv-string, convert it to bytes and write to buffer
buf.write(s.getvalue().encode())
buf.seek(0)
# set a filename with file's extension
buf.name = 'test_data.csv'
return buf
And then
csv_output = test_buf()
bot.send_document(chat_id, csv_output, caption='Caption_text')
And I get an error:
"ApiTelegramException occurred, args=("A request to the Telegram API was unsuccessful. Error code: 400 Description: Bad Request: can't parse entities: Can't find end of the entity starting at byte offset 7",)
What do i do wrong? Maybe I don't fully understand difference between BytesIO and binary format? If it's different how to transform bytesio to binary in-memory so i could send data
For now I can send file but without telebot library:
requests.post(f'https://api.telegram.org/bot{TELEGRAM_TOKEN}/sendDocument',
data={"chat_id": chat_id, "caption": 'caption_text'},
files={'document': csv_output})
So I do not understand what an issue with library. It like I sure that I just don't understand some basic stuff.
I have a Flask view that generates an Excel file (using openpyxl) from some data and it's returned to the user using send_file(). A very simplified version:
import io
from flask import send_file
from openpyxl.workbook import Workbook
#app.route("/download/<int:id>")
def file_download(id):
wb = Workbook()
# Add sheets and data to the workbook here.
file = io.BytesIO()
wb.save(file)
file.seek(0)
return send_file(file, attachment_filename=f"{id}.xlsx", as_attachment=True)
This works fine -- the file downloads and is a valid Excel file. But I'm not sure how to test the file download. So far I have something like this (using pytest):
def test_file_download(test_client):
response = test_client.get("/download/123")
assert response.status_code == 200
assert response.content_type == "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet"
Which passes, but I'd like to test that the (a) the filename used is as expected and (b) that the file... exists? Is an Excel file?
I can access response.get_data(), which is a bytes object, but I'm not sure what to do with it.
To check that the filename used is as expected you could check that the Content-Disposition header is as expected. For example:
assert response.headers['Content-Disposition'] == 'attachment; filename=123.xlsx'
To check "the existance of the file" you could for example check that for some test data it lies within an expected range of size. For example:
assert 3000 <= response.content_length <= 5000
assert 3000 <= len(response.data) <= 5000
Another level of verifying that the Excel file works would be attempting to load the data back into openpyxl and checking if it reports any problems. For example:
from io import BytesIO
from openpyxl import load_workbook
load_workbook(filename=BytesIO(response.data))
Here you risk running into some sort of exception like:
zipfile.BadZipFile: File is not a zip file
Which would indicate that the data contents of the file are invalid as a Excel file.
I have a controller that create a new record for a specific model. This model contains a fields.Binary.
Here's what the controller looks like:
#http.route('/mymodel/create', type='json', method='POST', auth='user')
def create_record(self, **kwargs):
"""
#params:
'field1': string
'field2': int
'binaryField': binary
"""
values = {'my_model_field_1': kwargs.get('field1'),
'my_model_field_2': kwargs.get('field2'),
'my_model_binary_field': kwargs.get('binaryField')}
request.env['my_model'].create(values)
My question is how should I send my file from the remote app connected to the server?
I'll probably have to send it as a string since it's sent in the json format. How do I have to modify my controller to receive it correctly?
I would be grateful for an example of code converting the file in a string that can be sent throught Json. I'll also have to convert it from any language, as I'm building an API, what is the standard that will be recognized by the binary field?
As I said in my comment, you'll probably need to read the file contents as binary, encode it using base64, and then decode the encoded bytes to put it in the JSON.
Python3 snippet for accomplishing this:
import base64
import json
data = {}
# read raw file bytes
with open('filename','rb') as myfile:
file_bytes = myfile.read()
# encode, decode and put it in the JSON
data['file'] = base64.encodebytes(file_bytes).decode('ascii')
# optionally serialize the JSON
serialized_json = json.dumps(data)