We are building a website expecting to serve around 10k users or more.
We want a functionality that each user will have an email address # our domain, however, this email address will not be accessible by the user it will only be used to receive a very rare amount of emails (say it will be published to receive gift cards directed to that user) and we need to notify the user as any gift card is received and do something accordingly at our end
We want to use Google App Engine with Python, and what first came to our minds is that we don't actually create any real mailboxes and we create a single catch-all email that will receive any mail sent to our domain mailboxes even not existed, then we can filter all received emails and map them to our users accounts
Within current Google receive limits (1 email per second) this will work, the main issue is that SPAM mails comes and flood the catch-all mail box with emails that directly reach receive limit and email account gets suspended by Google
We want to apply a smarter solution, we only care for emails that comes from a specific sender domain (say: giftcards.com) and we can drop any other emails
Is there is any reliable service or setup at our side we can use to filter out unwanted emails and only forward emails from giftcards.com (for example) to our main Google apps email?
Related
I wish to setup a subscribe form. The user will fill his/her email address and on submit an email should be sent to the provided email address for verification. The verification email will have a link on which the user can click to verify the email address.
This form will be in a website which I plan to host at Bluehost.
I read this tutorial about how it can be done in Python. Following is a snippet from the tutorial
import smtplib
s = smtplib.SMTP(host='your_host_address_here', port=your_port_here)
s.starttls()
s.login(MY_ADDRESS, PASSWORD)
My question is will I able to make this work on Bluehost ?,
My understanding is that host and port can be obtained as described
in this link.
Also,is there any third-party service that allows me create such subscribe form as described above?
While this could technically work, I highly recommend using a third-party service to send the actual emails. These days, the chance of your email getting delivered if it's being sent from a random Bluehost server is quite low due to its low "email sender reputation".
Snippet from this link explaining it:
An email sender reputation is a score that an Internet Service Provider (ISP) assigns to an organization that sends email. It’s a crucial component of your email deliverability. The higher the score, the more likely an ISP will deliver emails to the inboxes of recipients on their network. If the score falls below a certain threshold, the ISP may send messages to recipients’ spam folders or even reject them outright.
Some email sending services include SendGrid or Amazon SES.
If your website is static, then please go for third party service beacuse it will be difficult for you to handle the backend and integration part. So, look for some third party applications to do it for you for example SendPulse
If your website is dynamic, i.e if its having some backend server part as well, then this email sending part can be done easily depending upon which language you are using at backend. Most languages has support for sending emails.
We have a database of over 200.000 e-mail addresses and associated contacts. I had an idea that if I could find out which e-mail addresses don't exist anymore I could inactivate those contacts, thus keeping a more up to date database. My main goal is not to validate if an e-mail exists. My main goal is to find as many non-exsitent e-mail addresses as possible.
I have based a lot of my research on these answers: How to check if an email address exists without sending an email?
I have tried the python validate_email library, but it was very unreliable. It's also unsafe because you could get banned if you try to validate multiple contacts at the same company any time. It returned False to my active company e-mail, and None to my active gmail as well... so definitely unreliable.
I have tried both DNS with py3dns and MX records. Also the VRFY command. Unfortunately none of these seemed to be reliable since any e-mail server could send a fake response.
Greylisting is also a problem:
There is also an antispam technique called greylisting, which will
cause the server to reject the address initially, expecting a real
SMTP server would attempt a re-delivery some time later. This will
mess up attempts to validate the address.
The idea also occured to my that I could send a dummy e-mail, or two because of greylisting with a bit of delay in between them. I am afraid that this could get me blacklisted after a while, especially if multiple of these contacts work at the same company. Another idea is to do this from randomly generated e-mail addresses and hosts but that is probably not possible. Is there any way I could determine if an e-mail does not exist, preferably in a way that the chance of getting banned is minimal?
I have surfed the internet for a day but didn't find the perfect article which gives the best practices for sending mass emails.
I have configured Amazon SES using django-ses and mails are sending correctly. Now the problem is I don't know how people send mass mail, hide other recipients, which function they use, and what pattern they follow to make sending mass mail efficient and ease.
Also, we are using templates (Django Templates) for mail and below is the best solution I got by mixin all best things I found on the internet:
# 1. Getting queryset of all recipients which will receive mail
# (mine is a little bit different but at the end, it gives queryset of all emails - not list)
subscribers = EmailNotificationSubscriber.objects.all().values_list('user__user_email', flat=True)
# 2. Opening a connection
# 3. [Looping] using `.iterator()` to fetch email one by one from queryset (I think this is to handle the cases where we have an email list of around 10k or even bigger)
# 4. Creating EmailMessage instance and sending an email using `.send()`
# Function to get HTML Message (instance of `EmailMessage`)
def get_html_msg(subject, from_email, to, template_name, ctx, connection=None):
message = get_template(os.path.join(settings.BASE_DIR, 'templates', 'email', template_name)).render(ctx)
msg = EmailMessage(subject, message, from_email, to, connection=connection)
msg.content_subtype = 'html'
return msg
# Function which sends mass mail
def send_mass_mail(subject, qs, mail_template='base.html', ctx=None, fail_silently=True, *args, **kwargs):
from_email = settings.EMAIL_FROM
with get_email_connection() as connection:
for recipient in qs.iterator():
print(f"Sending to recipient: {recipient}")
msg = get_html_msg(subject, from_email, [recipient], mail_template, ctx, connection)
msg.send(fail_silently)
Above is what I am doing:
Getting just emails and not converting it into list so I can use .iterator()
Using a single connection to sending all mails
Hiding other recipient using loop by sending mail one by one
(I will use the same template for all recipient so I will refactor that for performance later)
So, How people send mass mail with Amazon SES? Do they use something else over this? An open-source repo or example would greatly help.
Thanks so much
Edit 1: Removed Emojis
Edit 2: Question Narrowed
according to Your questions i think that firstly, when sending mass mail, You should take into account the reputation of Your SES account which can drop down massively if You decide to buy a recipients email list. In order to achive that, it is good to use Dedicated IP's in SES. Here is link to that: https://docs.aws.amazon.com/ses/latest/DeveloperGuide/dedicated-ip.html
According to Your questions:
The best way would be to include them in bcc header, as You have set, the limit can depends on SMTP server. According to the AWS, it is 50 recipient's per message. So what You can do is You can group all of the mails and make one call with 50 recipients so it would lower Your request amount significantly.
Mass mailing has two approaches. One that, You have already tried(so one by one, and the other one which is considered better -> sending in bulks, as described above). If You would like to check something about for instance bulk sending You can see it there: https://aws.amazon.com/blogs/messaging-and-targeting/introducing-email-templates-and-bulk-sending/ Also, it is good to use only BCC field. You are ensuring Yourself, that there will not be any mistakes in displaying to the recipients email addresses of the other users.
Check that links, that I have already provided.
Yes, it is. You can easily use amazon ses templates, or create Your own ones. Also, the customization, managing Your own domains, configuration sets is really helpful and the documentation is quite good. However there are small drawbacks, and some minor bugs(for instance when You verify the domain, and it doesn't work instantly, there is a possibility that removing and adding it one more time fixes it :) ) However You have to be careful about Your reputation on SES.
Practises:
Don't use TO/CC, instead use BCC
Always monitor Your sending reputation, unless You want Your account to be blocked
Invest in dedicated ips
Block emails, which had already bounced before(Amazon has it's own suppression list, however it is much better to have one more on Your side)
Be aware of the sending quota, that You already have assigned and extend it for Your purposes
Don't buy email list
You can verify recipient's emails address existence before email sending
Be aware of the compliaints
Don't use link shorteners
For users of our Django app that receive their emails through gmail they are finding that emails are getting grouped into conversations that shouldn't be.
I'm not sure what gmail expects in an email to consider it unique enough to not group as a conversation but when I send plain text emails with DIFFERENT subjects using send_mail or even try a multipart/alternative with EmailMultiAlternatives with an html body gmail still assumes they are part of the same conversation.
Obviously this creates confusion when our application sends emails with different subjects and bodies to the same user and they are all grouped and gmail only shows the subject of the first message in the conversation.
I have 100% confirmed by looking at the raw original email messages to make sure the emails are different subjects and bodies.
I just want to know if I can change anything in how django creates the email message so it can play nice with gmail conversations.
I am using python 2.7.4, and can replicate the "issue" with Django 1.4 and 1.5.
Make sure messages on different subjects have a different 'From' address
On GAE, a non-gmail user can create a Google Account using their non-gmail email and log into a google app engine application. However, sending from that email does not appear to work all the time.
For example, suppose foobar#yahoo.com creates a google account and they log in. Then, GAE should be able to send email from foobar#yahoo.com during a user request.
The problem is that this does not appear to work for yahoo email accounts and others. In my experience, only the following users can have email sent on their behalf:
Currently logged in gmail users
Currently logged in users with emails that run on google apps
Administrators
However, you cannot send email on behalf of users with the following email address:
yahoo and hotmail users
.gov or .mil users
most .edu users, although I think some schools use google apps and they work.
If I send from those email addresses, I get the following error:
message.send();
File "/base/python_runtime/python_lib/versions/1/google/appengine/api/mail.py", line 799, in send
raise ERROR_MAP[e.application_error](e.error_detail)
InvalidSenderError: Unauthorized sender
Am I missing something here?
I know there are similar questions out there on this topic but I don't think they nail them.
Your application has no right to originate email from addresses you don't own.
Doing so is called spoofing and there has been a lot of work done in the last 8 years to prevent spoofing: DKIM, SPF...
If you need to send email you need to have your own email address(es)/domain to send the email from for this application and you'll need to handle bounces as well.
Update:
Google Accounts
Google accounts can be created using 3rd party domain email addresses (e.g. hotmail, yahoo, *.edu, etc). There will be an email confirmation step to verify the email address, but there may be no greater relationship than this. The email address is the "username" and there's a password created with Google that has nothing to do with the 3rd party domain.
Google accounts don't have to have a mail service component. You can create a Google account and not have Gmail.
Google email on a 3rd party domain
Google Apps can be run "on" 3rd party domains, this can, but not necessarily include Google email.
Google email could be run on any domain without any other "Apps", they offer email outsourcing which is quite attractive to the Education sector. In this scenario, Google is authoritative for email for that domain.
On behalf of mailing
Google email has the functionality to set up "On Behalf of" emailing. This requires configuration. A confirmation email is sent to the target account. Once created the email is sent using the originating account's email address in the mail envelope, so any delivery status messages (delays, rejections) will be returned to this account. The originating email address will also be in the sender header. The address that you're sending on behalf of will appear in the from header in the messages, but otherwise, with regards to security settings and validation, it's a Google email from Google.
A quick search has returned some tech blog websites that suggest that they are considering retiring this feature in favour of supporting third party SMTP services.
Third party SMTP server
So someone could configure their Google Email account with the SMTP server, username and password of their mail service provider and use it to send email via their Google mail interface through the valid servers of their mail service provider. This email would therefore genuinely "originate" from that domain's infrastructure.
I've not read the documentation for this GAE function to send email. However, I can see that if someone is signed into their Google account and that account has a mail service, then it could be possible to send email from that account using an API.
However, Google will not generate email "From" 3rd party domains for which it has not been assigned authority over. The email would not be valid; it would be "spoofed". It may not comply with a variety of security enhancements, could be used maliciously and would bring them into disrepute.
I hope this makes the situation clearer for you.
If the documentation is lacking in this regard, it could be that those who wrote it, being so familiar with email, might find it hard to imagine someone would think it were possible.
For me it looks like Google has changed something. I've had an application running 1,5 years without problems (and changes) but suddenly on 2011-05-03 sending emails for example on behalf of Yahoo users stopped working.
This change also affected other non-Google (non-developer users).
If foobar#yahoo.com creates a Google account and logs in, you will be able to send on behalf of their Gmail user, but not their Yahoo user. Same for Hotmail. As for .gov, .mil or .edu, there's nothing special about these TLDs. If they are Google Apps domains, you can send from them, otherwise you cannot.
Note, though, that you can specify a Reply-To address when sending mail. This can be any email address whatsoever, so if you use a reply-to address of foobar#yahoo.com on outbound mail and the recipient clicks reply, this should be the address that's populated as the recipient.