We Keep Coding

using python with Sendgrid on Heroku

I am trying to use Sendgrid to send the email on Heroku, but I get the output that I not seen before. I saw the documentation and it says that the helper library supports few versions of python until 3.8 but mine version is 3.9. Is it this cause me get this error?
**Here is my code: **
import os
from sendgrid import SendGridAPIClient
from sendgrid.helpers.mail import Mail
message = Mail(
from_email='hesheitaliabu#gmail.com',
to_emails='hesheitaliabu#gmail.com',
subject='Sending with Twilio SendGrid is Fun',
html_content='<strong>and easy to do anywhere, even with Python</strong>')
try:
sg = SendGridAPIClient('SG.API_KEY')
response = sg.send(message)
# print(response.status_code)
print(response.body)
print(response.headers)
except Exception as e:
print(e.message)
I get an error:
b''
Server: nginx
Date: Mon, 05 Sep 2022 03:53:19 GMT
Content-Length: 0
Connection: close
X-Message-Id: Y19VjElqRe2byQxJrkJUeg
Access-Control-Allow-Origin: https://sendgrid.api-docs.io
Access-Control-Allow-Methods: POST
Access-Control-Allow-Headers: Authorization, Content-Type, On-behalf-of, x-sg-elas-acl
Access-Control-Max-Age: 600
X-No-CORS-Reason: https://sendgrid.com/docs/Classroom/Basics/API/cors.html
Strict-Transport-Security: max-age=600; includeSubDomains

As we've worked out in the comments, the emails are in fact sending successfully. The message you see logged are the response headers from the API because of these lines:
sg = SendGridAPIClient('SG.API_KEY')
response = sg.send(message)
print(response.body)
print(response.headers)
In this case the message is sent and then the program prints out the response body (which is empty for a successfully sent message) and then the response headers, which you are seeing. This is not an error.
Instead, we have worked out that your email is being sent to the spam inbox.
This is a bit harder to fix as it relies on email inbox providers. However I need to point out one thing that's going to make this very difficult to start with.
Your example shows that you are sending the email from your Gmail address to the same Gmail address. Because of this, Gmail knows it did not send the email from that Gmail address and will instantly treat it as suspicious. You are effectively spoofing your own email address. If you have set up single sender verification with SendGrid for that address, all that does is tell SendGrid that you own the email address and aren't trying to spoof just anyone's email address. This is good for testing that you can use the API to send emails.
However, if you want emails to arrive in inboxes and not the spam inbox then your best bet is to send from a domain that you own that you can authenticate with SendGrid. When you authenticate a domain it sets a number of things like SPF and DKIM that show the inboxes you are sending to that you are the rightful owner of the domain and are authenticated to send emails from the domain. This means you are more likely to arrive in the inbox and not spam.
There is much more to pay attention to to ensure your emails stay out of spam, but in this case, I would start with domain authentication.

How to use Mailgun's recipient-variables with Django SMTP mail backend?

How can I properly send batch/bulk/mass emails using MailGun in Django using SMTP protocol?
What I've tried so far?
I am using django.core.mail.backends.smtp.EmailBackend as my EMAIL_BACKEND
and this is the code snippet that I have tried to send the emails.
from django.core.mail import EmailMultiAlternatives
import json
to_emails = [
"mail_1#example.com",
"mail_2#example.com",
"mail_3#example.com",
"mail_4#example.com",
"jerinpetergeorge#gmail.com",
]
mail = EmailMultiAlternatives(
subject="Hey - %recipient.name%",
body="Hey %recipient.name%,\n\nThis is just a batch email test!!!",
from_email="JPG <me#somehost.com>",
to=to_emails,
)
recipient_variables = {
address: {"name": address} for address in to_emails
}
mail.extra_headers["X-Mailgun-Recipient-Variables"] = json.dumps(recipient_variables)
response = mail.send()
print(response)
and I've got the mail as below,
As we can see, the to attribute is filled with all email addresses, which is not what I am expecting.
So, how can I tell the Mailgun/Django to parse my variables properly in order to make the emails looks more personal?
Notes
I prefer to use SMTP protocol
I've tried the REST APIs of Mailgun and it was a success (but, I prefer SMTP)
I found django-anymail and seems it has the feature. But, It also uses the APIs (correct me if I am wrong)
Update-1
Updated the to argument to to="%recipient%" But, got
TypeError: "to" argument must be a list or tuple
Updated the to argument to to=["%recipient%"] But, got
smtplib.SMTPRecipientsRefused: {'=?utf-8?q?=25recipient=25?=': (501, b'Invalid command or cannot parse to address')}

As we can see, the to attribute is filled with all email addresses, which is not what I am expecting.
It is not properly supported with SMTP by Mailgun.
However, relying on the (unintuitive) implementation of BCC in Mailgun, there is a workaround:
mail = EmailMultiAlternatives(
subject="Hey - %recipient.name%",
body="Hey %recipient.name%,\n\nThis is just a batch email test!!!",
from_email="JPG <me#somehost.com>",
# to=to_emails, # Replace this
bcc=to_emails, # with this
)
recipient_variables = {
address: {"name": address} for address in to_emails
}
mail.extra_headers["To"] = "%recipient%" # Add this
mail.extra_headers["X-Mailgun-Recipient-Variables"] = json.dumps(recipient_variables)
Reference: https://stackoverflow.com/questions/37948729/mailgun-smtp-batch-sending-with-recipient-variables-shows-all-recipients-in-to-field
Why does to=["%recipient%"] not work with SMTP?
It's the standard in the protocol.
From https://documentation.mailgun.com/_/downloads/en/latest/pdf/:
SMTP send will error with “cannot parse to address” or “cannot parse from address” if the provided email address fails syntax checks in accordance with RFC5321, RFC5322, RFC6854.
What to do for proper support of Batch Sending with Mailgun?
Use the API.
From https://stackoverflow.com/questions/30787399/laravel-5-sending-group-emails (multiposted to https://laracasts.com/discuss/channels/laravel/sending-email-to-1000s-of-reciepents):
So far, I have created an array of recipient email addresses, sent the email to a webmaster type address, and included the end recipients in BCC
While this works, it's not ideal.
Rather than using Laravel's built in Mail, I elected to use Mailgun's API (specifically batch sending) directly
This also allows me to access unique recipient variables within my email template
(It's not specific to Laravel/PHP, but to SMTP via Mailgun.)
What do you mean by "unintuitive" implementation of BCC in Mailgun?
Mailgun effectively personalises the email for each BCC recipient using recipient-variables.
From https://github.com/mailgun/mailgun-js-boland/issues/89:
the bcc person is receiving the email as it was addressed to them instead of being part of the bcc
This causes a separate issue when you actually want BCC recipients to get the same content.
From https://stackoverflow.com/questions/48887866/bcc-in-mailgun-batch-send-does-not-include-substitutions:
In the copy sent to the bcc address, the recip_vars substitution has not been made.
According to the good people at Mailgun, this is not possible, at least in the current release of the service.

Why is Amazon SES mail simulator bounces not working? (Am I missing headers or something?)

I'm trying to use the amazon SES simulator as documented here: http://docs.aws.amazon.com/ses/latest/DeveloperGuide/mailbox-simulator.html
The body of the message looks like this:
MIME-Version: 1.0
Content-Type: multipart/alternative; charset="utf-8";
boundary="===============123456789=="
Content-Transfer-Encoding: base64
Subject: hello test message!
Reply-To: my_address#my_provider.com
To: complaint#simulator.amazonses.com
Return-Path: my_address#my_provider.com
Bounces-to: my_address#my_provider.com
Errors-to: my_address#my_provider.com
From: my_address#my_provider.com
--===============123456789==
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
Ym9keSB3aXRoIG5vbmNlOiAw
--===============123456789==
MIME-Version: 1.0
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: base64
Ym9keSB3aXRoIG5vbmNlOiAw
--===============123456789==--
I'm sending this as the body, and using the boto3 interface ses_client.send_raw_message.
I'm generating this message body with something like this
from email.mime.text import MIMEText
from email.mime.multipart import MIMEMultipart
import re
msg = MIMEMultipart('alternative')
msg.set_charset('utf8')
text_message='Ym9keSB3aXRoIG5vbmNlOiAw'
p = re.compile('<.*?>')
text_message = 'body with nonce: {}'.format(str(nonce))
text = MIMEText(p.sub('', text_message), 'plain', 'utf8')
html = MIMEText(text_message, 'html', 'utf8')
msg.attach(text)
msg.attach(html)
source = 'my_email#my_provider.com'
msg['Subject'] = 'hello test message!'
msg['Reply-To'] = source
msg['To'] = to_mail
msg['Return-Path'] = source
msg['Bounces-to'] = source
msg['Errors-to'] = source
So I can send emails through SES and it works just fine.
I can also send emails to complaint#simulator.amazonses.com, and that works.
I haven't set up any SNS messaging, but I expected, through all the header fields that I set, to get a bounce back at the desired address. However, nothing happens if i use bounce#simulator.amazonses.com.
This is a list of header fields that amazon acknowledges:
http://docs.aws.amazon.com/ses/latest/DeveloperGuide/header-fields.html
I have also enabled email feedback forwarding, as described here:
http://docs.aws.amazon.com/ses/latest/DeveloperGuide/notifications-via-email.html
Also, for sending the emails I use this logic:
amazon_client = boto3.client('ses', 'region', **credentials)
amazon_client.send_raw_email(
Source=from_mail,
Destinations=to_mails,
RawMessage={'Data': body}
)
According to the page documenting how to receive notifications, I should be getting mails back at the Source address specified - which I do set, and I do have the email feedback forwarding enabled for....
Any ideas what I might be missing?

The headers you are putting in suggest that the real problem is false expectations. Errors-To: will be ignored pretty much by everything you can find, and Return-Path: will be replaced with the envelope sender by the receiving MTA. To control bounces, you need to control the envelope sender (traditionally, with sendmail -f) and what you put in the headers is completely irrelevant.

Solved.
The bounce address was different from the 'From' address. Therefore, the bounces ended up in the SPAM of that other address.
Interestingly, that address had a filter to forward everything to the address I had used as for 'From' address, when sending the email.
So my confusion was caused because I expected to get mail forwarded from that other address, but the bounces never got forwarded.

Emailing script - reply to message Gmail conversation

I have setup an email for my domain to forward emails to a my personal email. I'm writing a simple mailing script to reply to some emails i receive as forwarded email using the domain's email id. I'm using Sendgrid to do that. Whenever i send a response back though, it appears as a new conversation in Gmail inbox. I'm not sure what all params I need to set to make the conversation appear as a single bundle.
here's my sendgrid code:
message = sendgrid.Mail()
message.add_to(email)
message.set_from(my_name + " <" + my_email + ">")
message.set_subject(subject)
message.set_html(body.replace('\n', '<br />'))
message.set_text(body)
status, msg = sg.send(message)
I use the subject as: "Re: " +
and use body from old conversation as:
On Fri, Aug 28, 2015 at 1:00 PM, senders_name <sender's email> wrote:
>
> ...

Thanks to #FSQ suggestion.I was able to make the email conversation attach together. I look up the Message Id from:
Gmail message > right side dropdown > show original
You should see something like:
In-Reply-To: <14f75e5a9b1.3f4.b2d01d#ismtpd-057>
Message-ID: <14f760957fc.5888.11ab5eb#ismtpd-085>
This is your message id. Send this in the header of email as:
message.set_headers({'In-Reply-To': '<14f760957fc.5888.11ab5eb#ismtpd-085>'})
And it works!

Can I get the incoming message by using Mailgun?

A use Mailgun to send b a email, after b receive the email and reply to a.If I want to track the email coming from b, How I can get the email?
Here is the code:
1.sendmail.py
from smtplib import SMTP
import requests
login_name = "postmaster#zzb.mailgun.org"
password = "********"
def send_message_via_smtp():
smtp = SMTP("smtp.mailgun.org", 587)
smtp.login(login_name, password)
smtp.sendmail("zebozhuang#163.com","348284770#qq.com", "Subject:mailgun test \n\n just for test.\n\n")
smtp.quit()
if __name__=="__main__":
send_message_via_smtp()
2.create_route.py
import requests
from werkzeug.datastructures import MultiDict
def create_route():
return requests.post(
"https://api.mailgun.net/v2/routes",
auth=("api", "key-9c4-t2q6fouilngjummvtv1rge7t00f2"),
data=MultiDict([("priority", 1),
("description", "Sample route"),
("expression", "match_recipient('.*#qq.com')"),
("action", "forward('qiyazhuang#gmail.com')"),
("action", "stop()")])
)
I create the route and I run the script sendmail.py.After someone who use email 348284770#qq.com reply to the other who use email zebozhuang#163.com, the Gmail
can not receive the message by using the Mailgun method 'forward'.
Could anyone tell me why?

Your messages are likely being delivered. Check the "Logs" tab of the Mailgun Control Panel.
Do you see any entries that look like this:
Routed: .*#qq.com -> qiyazhuang#gmail.com 'SUBJECT HERE'
The "Routed" prefix means that the message triggered a Route. If you're seeing this, and the next log entry is prefixed with "Delivered", the message is likely being delivered to Gmail without issue. Check your Gmail spam folder if you still don't see the messages in the inbox folder.
Disclaimer: I work for Mailgun Support. :)