I've tried to use smtpd but couldn't get it to work properly (Here's the link to the issue). I only need the server to receive emails and store in database. I've found few like MailGun, and although it looks great it is not free. Any suggestions?
There are so many options, it's an opinion question, and also related to the specific use case. So here are the main options you have.
It's mostly your decision and not "with django". The question is really choosing a mail server for a web application.
From the django point of view it's either local server(faster), or external API. As a developer, sending SMTP is probably easier than external API because django already includes the backend handling. But good email service providers usually have a good clients libraries, and it's not that complex to use their API as well.
Your own email server: Install and maintain your own mail server. Pros: cheaper, a cheap VPS will do, no limits on accounts (useful for testings) and mails. Cons: installing a mail server + auth backend etc is not easy if you didn't installed one before, and your emails can be easily marked as spam. If you select this option, Postfix is a safe choice.
External service: with API. Pros: easier to implement, and less chances to be marked as spammer. Cons: more expensive, possible specifc coding against the specific API
Your mail server + external API: the local server connection is much faster, so django sends the email quickly, and then the server handles the slower sending to an external service
Summary: If you just start something, and need a simple email facility, pick an external service that uses SMTP. You will have something up and running, using the very easy django SMTP utilities, without specific coding to the vendor API. Once you need to scale, do the research and pick a more advanced solution.
Note: usually sending email requires some async queue, that allows django to pass the email sending task and continue with the request, without waiting to the smtp connection. SMTP connections can be slow.
Related
Premise: I am a beginner in search for an easy way to send bug reports from users over sea.
I've made a script for some friends that are living on the other side of the sea (US - EUROPE)... I will like to gather automatic bug reports whenever they happen. So my first idea was to send myself an email with the smtplib module. It works fine when testing home, but as soon as the sender "sends", my email provider (gmail) blocks the connection because of course, its from an "unknown device". I've already enabled "Allow less secure apps" as someone suggested but with no avail.
What I am searching its a simple way of dealing with this.
Yes I could make the script to ignore the error if the email its not being sent, and then go into my google account and enable those devices so at least it will work from the second run..
But it doesn't seem what a programmer would do in this case. I am learning so a solution withing the language is what I am after.
A different provider that has no restriction its also a good start but I tried Yahoo, Live, Yandex but I couldn't make them work. Are there any?
So my question is: how others do? what is the best solution for some one like me?
I've read about sentry or other error/bug tracking but its obviously way too much for want I need
You should certainly not incorporate e.g. Gmail credentials in the code that is remotely executed on devices you do not control, given I understand correctly the Gmail less secure device issue happens as every "user" is running this code and using your credentials. This holds true for any other provider.
Now this won't exactly be simple but one way to go about it would be to create a server side API endpoint that can accept HTTP(s) or any other protocol requests that then will authenticate in a little more secure way on the server side with Gmail.
The concept for emails is:
Bug > Python Script > API call > Email
This could be implemented using Python on the API side (Flask e.g.) using an AWS Lambda Function with Amazon API Gateway, but again that is something to get through and understand by itself which will take a good chunk of time.
You need to touch a lot of concepts, like auth tokens to make this really secure.
Could you elaborate a little on where the code needs to run and if you are willing to try AWS or any other cloud provider, or would have access to an internet connected server ? This makes it easier to provide you with a full example on the solution in a hackish way while I would highlight the problems you could face on the security side.
I understand that this is not the way to go but as for my needs and my level of experience it works for me!
Yandex allows you to send email from different ip so Yandex is the way to go. What I was doing wrong in the first place was to use the wrong port (587 instead of 465)
I'm currently working on a University project that needs to be implemented with a Client - Server model.
I had experiences in the past where I was managing the communication at socket level and that really sucked.
I was wondering if someone could suggest an easy to use python framework that I can use for that purpose.
I don't know what kind of details you may need to answer so I'm just going to describe the project briefly.
Communication should happen over HTTP, possibly HTTPS.
The server does not need to send data back or invoke methods on the clients, it just collects data
Many clients send data concurrently to server, who needs to distinguish the sender, process the data accordingly and put the result in a database.
You can use something like Flask or Django. Both frameworks are fairly easy to implement, Flask is much easier than Django IMO, although Django has a built in authentication layer that you can use, albeit more difficult to implement in a client/server scenario like you need.
I would personally use Flask and JWT (JSON Web Tokens), which will allow you to give a token to each client for authentication with the server, which will also let you differentiate between clients, and you can use HTTPS for your SSL/TLS requirement. It is tons easier to implement this, and although I like django better for what it brings to the table, it is probably overkill to have you learn it for a single assignment.
For Flask with SSL, here is a quick rundown of that.
For JWT with Flask, here is that.
You can use any database system you would like.
If I understood you correctly you can use any web framework in python. For instance, you can use Flask (I use it and I like it). Django is also a popular choice among the python web frameworks. However, you shouldn't be limited to only these two. There are plenty of them out there. Just google for them.
The implementation of the client depends on what kind of communication there will be between the clients and the server - I don't have enough details here. I only know it's unidirectional.
The client can be a browser accessing you web application written in Flask where users send only POST requests to the server. However, even here the communication will bidirectional (the clients need to open the page which means the server sends requests back to the client) and it violates your initial requirement.
Then it can be a specific client written in python sending some particular requests to your server over http/https. For instance, your client can use a requests package to send HTTP requests.
I have already asked a question about IP Authentication here: TastyPie Authentication from the same server
However, I need something more! An IP address could be very easily spoofed.
Scenario: My API (TastyPie) and Client App (in javascript) are on the same server/site/domain. My users don't login. I want to consume my API in my javascript client side.
Question: How can I make sure (authentication) that my AJAX requests are originating from the same server?
I'm using Tatypie. I need to authentication that the requests from the client are being made on the same server/domain etc. I cannot use 'logged in sessions' as my users don't login.
I have looked at private keys and generating a signature but they can viewed in the javascript making that method insecure. If I do it in a way to request a signature form the server (hiding the private key in some python code) anyone can make the same http request to get_signature that my javascript makes, thus defeating the point.
I also tried to have the Django view put the signature in the view eliminating the need to make the get_signature call. This is safe, but means that I have to now refresh the page every time to get a new signature. From a users point of view only the first call to the API would work, after which they need to refresh, again pointless.
I cannot believe I'm the only person with this requirement. This is a common scenario I'm sure. Please help :) An example using custom authentication in Tastypie would be welcome too.
Thanks
Added:
Depending on your infrastructure #dragonx's answer might interest you most.
my 2c
You want to make sure that only if a client visits your website can use the api? Hmm does the bot, robot, crawler fall in the same category with the client then? Or am I wrong? This can be easily exploited in case you really want to secure it really.
I cannot believe I'm the only person with this requirement.
Maybe not, but as you can see you are prone to several attacks to your API and that can be a reason for someone not sharing your design and making security stricter with auth.
EDIT
Since we are talking about AJAX requests what does the IP part has to do with this? The IP will always be the Client's IP! So probably, you want a public API...
I would Go with the tokens/session/cookie part.
I 'd go with a generated token that lasts a little while and a flow described below.
I'd go with a limiter per some time, like Github does. Eg 60 requests per hour per ip or more for registered users
To overcome the problem with the refreshing token I would just do this:
Client visits the site
-> server generates API TOKEN INIT
-> Client gets API TOKEN INIT which is valid only for starting 1 request.
Client makes AJAX Request to API
-> Client uses API TOKEN INIT
-> Server checks against API TOKEN INIT and limits
-> Server accepts request
-> Server passes back API TOKEN
-> Client consumes response data and stores API TOKEN for further usage (Will be stored in browser memory via JS)
Client Starts Comm with the API for a limited amount of time or requests. Notice that you know also the init token date so you can use it to check against the 1st visit on the page.
The 1st token is generated via the server when the client visits.
Then the client uses that token in order to obtain a real one, that lasts for some time or something else as of limitation.
This makes someone actually visit the webpage and then he can access the API for a limit amount of time, requests perhaps etc.
This way you don't need refreshing.
Of course the above scenario could be simplified with only one token and a time limit as mentioned above.
Of course the above scenario is prone to advanced crawlers, etc since you have no authentication.
Of course a clever attacker can grab tokens from server and repeat the steps but, then you already had that that problem from start.
Some extra points
As the comments provided please close writes to the API. You don't want to be a victim of DOS attacks with writes if you have doubts about your implementation(if not use auth) or for extra security
The token scenario as described above can also become more complicated eg by constantly exchanging tokens
Just for reference GAE Cloud storage uses signed_urls for kind of the same purpose.
Hope it helps.
PS. regarding IP spoofing and Defense against spoofing attacks wikipedia says so packet's won't be returned to the attacker:
Some upper layer protocols provide their own defense against IP
spoofing attacks. For example, Transmission Control Protocol (TCP)
uses sequence numbers negotiated with the remote machine to ensure
that arriving packets are part of an established connection. Since the
attacker normally can't see any reply packets, the sequence number
must be guessed in order to hijack the connection. The poor
implementation in many older operating systems and network devices,
however, means that TCP sequence numbers can be predicted.
If it's purely the same server, you can verify requests against 127.0.0.1 or localhost.
Otherwise the solution is probably at the network level, to have a separate private subnet that you can check against. It should be difficult for an attacker to spoof your subnet without being on your subnet.
I guess you're a bit confused (or I am, please correct me). That your JS code is published on the same server as your API does not mean AJAX requests will come from your server. The clients download the JS from your server and execute it, which results in requests to your API sent from the clients, not from the same server.
Now if the above scenario correctly describes your case, what you are probably trying to do is to protect your API from bot scraping. The easiest protection is CAPTCHA, and you can find some more ideas on the Wiki page.
If you are concerned that other sites may make AJAX calls to your API to copy your site functionality, you shouldn't be--AJAX requests can only be sent to the same server as the page the JS is running on, unless it is JSONP.
Short answer: It is not possible to prevent a dedicated attacker.
You have no method of identifying a client other than with the information that they give you. For instance, username/password authentication works under the assumption that only a valid client would be able to provide valid credentials. When someone logs in, all you know is that some person provided those credentials -- you assume that this means that this means that they are a legitimate user.
Let's take a look at your scenario here, as I understand it. The only method you have of authenticating a client is IP Address, a very weak form of authentication. As you stated, this can be easily spoofed, and in with some effort your server's response can be received back to the attacker's original IP address. If this happens, you can't do anything about it. The fact is, if you assume someone from a valid IP address is a valid user, then spoofers and legitimate users are indistinguishable. This is just like if someone steals your password and tries to log in to StackOverflow. To StackOverflow, the attacker and you are indistinguishable, since all they have to go on is the username and password.
You can do fancy things with the client as mentioned in other answers, such as tokens, time limits, etc., but an dedicated attacker would be able to mimic the actions of a legitimate client, and you wouldn't be able to tell them apart because they would both appear to be from valid IP addresses. For instance, in your last example, if I was an attacker looking to make API calls, I would spoof a legitimate IP address, get the signature, and use it to make an API call, just as a legitimate client would.
If your application is critical enough to deem this level of thought into security, you should at least think of implementing something like API tokens, public key encryption, or other authentication methods that are more secure than IP addresses to tell your clients apart from any attackers. Authentication by IP address (or other easily forged tokens like hostname or headers) simply won't cut it.
may be you could achieve this by using Same-origin policy
refer http://en.wikipedia.org/wiki/Same_origin_policy
As suggested by Venkatesh Bachu, Same Origin Policy and http://en.wikipedia.org/wiki/Cross-Origin_Resource_Sharing (CORS) could be used as a solution.
In your API, you can check Origin header and respond accordingly.
Need to check if Origin header can be modified by using extensions like tamper data.
A determined hacker can still snoop by pointing browser to a local proxy server.
If this app server is running on an ordinary web server that has configurable listening IP address, set it to 127.0.0.1. With the TCPServer module, it's like
SocketServer.TCPServer(("127.0.0.1", 12345), TheHandlerClass)
Use netstat command to verify the listening address is correct as "127.0.0.1"
tcp4 0 0 127.0.0.1.12345 *.* LISTEN
This will effectively making any connection originated outside the same host impossible on the TCP level.
There are two general solution types: in-band solutions using normal web server/client mechanisms, that are easy to implement but have limitations; and out-of-band solutions that rely on you to configure something externally, that take a little more work but don't have the same limitations as in-band.
If you prefer an in-band solution, then the typical approach used to prevent cross-site request forgery (XSRF) would work well. Server issues a token with a limited life span; client uses the token in requests; privacy of token is (sort of) assured by using an HTTPS connection. This approach is used widely, and works well unless you are worried about man-in-the-middle attacks that could intercept the token, or buggy browsers that could leak data to other client-side code that's being naughty.
You can eliminate those limitations, if you're motivated, by introducing client certificates. These are kind of the flip side to the SSL certificates we all use on web servers -- they operate the same way, but are used to identify the client rather than the server. Because the certificate itself never goes over the wire (you install it locally in the browser or other client), you don't have the same threats from man-in-the-middle and browser leakage. This solution isn't used much in the wild because it's confusing to set up (very confusing for the typical user), but if you have a limited number of clients and they are under your control, then it could be feasible to deploy and manage this limited number of client certificates. The certificate operations are handled by the browser, not in client code (i.e. not in JavaScript) so your concern about key data being visible in JavaScript would not apply in this scenario.
Lastly, if you want to skip over the client configuration nonsense, use the ultimate out-of-band solution -- iptables or a similar tool to create an application-level firewall that only allows sessions that originate from network interfaces (like local loopback) that you know for certain can't be accessed off the box.
I'm using the xmpppy library to write an XMPP client that can chat with users. It has its own XMPP user account and needs to know if a given user is online. However, the documentation is a bit sparse on how to do this. What would you recommend?
The only solution I've seen thus far is to start up a daemon before the XMPP server starts and monitor all presence messages that are sent out - then a user is said to be online if they've sent the "I'm online"-type message but not the corresponding "I'm logging off" message. However, being new to XMPP in general, I would think there would be a nicer way to do this.
The simple way is to support "subscribe" presence message -- this lets another user check if you're currently present (if they don't already know) by a "subscribe" attempt. Check this useful guide to get started, and the standard for many more important details (esp. on protecting your privacy, if needed, from subscribe requests from user you don't know).
There are basically three ways to connect to an XMPP server: as a client (which you've done), as a component, and as another server. The server-to-server type (s2s) is just a federated connection, very much like how mail servers exchange email with each other.
Alex described how clients keep track of presence. XMPP requires me to approve that you can receive my presence information and vice versa. For your bot this means for you to keep track of who's online the end users need to accept your presence requests. It also means that you can respond to the user's presence requests and keep them informed about if your bot is up or not.
The last way is as a trusted component, and only works if you're running the server. i.e. if you're trying to do this on the jabber.org server, you're out of luck, because you're not running that server. The upsdie is you can have access to the internals of the XMPP server, like pulling lists of everyone who's online. The downside is your component / bot implementation is going to be different for every server implementation.
I'm building my startup and I'm thinking ahead for shared use of services.
So far I want to allow people who have a user account on one app to be able to use the same user account on another app. This means I will have to build an authentication server.
I would like some opinions on how to allow an app to talk to the authentication server. Should I use curl? Should I use Python's http libs? All the code will be in Python.
All it's going to do is ask the authentication server if the person is allowed to use that app and the auth server will return a JSON user object. All authorization (roles and resources) will be app independent, so this app will not have to handle that.
Sorry if this seems a bit newbish; this is the first time I have separated authentication from the actual application.
Assuming you plan to write your own auth client code, it isn't event-driven, and you don't need to validate an https certificate, I would suggest using python's built-in urllib2 to call the auth server. This will minimize dependencies, which ought to make deployment and upgrades easier.
That being said, there are more than a few existing auth-related protocols and libraries in the world, some of which might save you some time and security worries over writing code from scratch. For example, if you make your auth server speak OpenID, many off-the-self applications and servers (including Apache) will have auth client plugins already made for you.
Your question isn't really a programming problem so much as it is an architecture problem. What I would recommend for your specific situation is to setup an LDAP server for authentication, authorization, and accounting (AAA). Then have your applications use that (every language has modules and libraries for LDAP). It is a reliable, secure, proven, and well-known way of handling such things.
Even if you strictly want to enforce HTTP-based authentication it is easy enough to slap an authentication server in front of your LDAP and call it a day. There's even existing code to do just that so you won't have to re-invent the wheel.
There is also CAS that you might wont to look at,