I have been given a task to complete: Deploy my pre-existing Pyramid application onto our EC2 Linux server. I would like to do this with a minimal amount of stress and error, especially considering am I totally new to AWS.
What I have done so far:
Setup the EC2 instance which I can SSH into.
Locally develop my Pyramid application
And, we version control the application with GitHub.
We are using: Pyramid (latest), along with Python 2.7.5 and Postgresql (via SQLAlchemy and Alembic.)
What is a basic, high-level list of steps to ensure that my application is deployed appropriately?
Where, if at all, does something like Elastic Beanstalk come into play?
And, considering my project is currently in a Git repo, what steps or considerations must be taken to accommodate this?
I'm not looking for opinions on how to tweak my setup or anything like that. I am looking for a non-debatable, comprehensible set of steps or considerations to deploy my application in the most basic form. This server is for development purposes only, so I am not looking for a full-blown solution.
I have researched this topic for Django projects, and frankly, I am a bit overwhelmed with the amount of different possible options. I am trying to boil this situation down to its minimal components.
I appreciate the time and help.
Deploying to an EC2 server is just like deploying to any other Linux server.
If you want to put it behind a load balancer, you can do which is fully documented.
You can also deploy to Elastic Beanstalk. Where as EC2 is a normal Linux sever, Beanstalk is more like deploying to an environment, you just push all your git changes into an S3 repo, your app then gets built and deployed onto beanstalk.
Meaning no server setups, no configuration (other than the very basics) and all new changes you push to S3, get built and update each version of your app that may have been launched on beanstalk.
You don't want to host your database server on EC2, use Amazons RDS database server, dead simple and takes about two minutes to setup and configure.
As far as file storage goes, move everything to S3.
EC2 and beanstalk should not be used for any form of storage.
I would suggest to run two instances and use Elastic Load Balancer.
Never run anything important on a single EC2 instance, EC2 instances are not durable, they can suddenly vanish, taking whatever data you had stored on it.
Everything else should work as in Pyramid Cookbook description.
Although there is no specific recipe for deploying a Pyramid application to AWS in the Pyramid Cookbook, there are other recipes that may offer a partial solution.
Related
I need to implement a simple web service in python - it's my first experience with web services and REST APIs so I want to undertand what environment and tools would fit my needs. In my web service, I need to read some data from a database, do some simple logic, and support a GET call from another application (qualtrics).
I read and implemented a simple test web service with python using some useful blogs such as: Building a Basic RestFul API in Python | Codementor
but I need a real server so that I could call the API from external applications.
As I'm looking for a long term solution, I thought that using AWS EC2 instance may be a good solution for a server. I tried to implement it using some guidelines in blogs such as: Deploy a Flask app on AWS EC2 | Codementor
However, as I'm new to this and encountered some implementation/editing errors (e.g. handling of the wsgi file) and as I'm a windows person and the ubuntu stuff are not always easy to get used to, I was wondering what is the best framework for my needs?
Is there any recomended flow in which I'll be able to implement my simple python code and connect it to a small server (either AWS EC2 instance or any other recomended one) in a more convenient way?
Another important note - I will need to run it only from time to time, this web server and web service should not be contantly live (that's why I thought that aws virtual instance would fit best).
To begin, my recommendation would be to look at Elastic Beanstalk, Fargate and API Gateway with Lambda.
You can use Elastic Beanstalk to easliy provision out-of-the-box AWS environment to host your python app in Flask with minimal configurations required:
Deploying a flask application to Elastic Beanstalk.
The other thing to consider would be to develop your python app as a docker container using, e.g., tiangolo/uwsgi-nginx-flask as the base image. This would allow you to easily work with in on your localhost, and then just move your image to AWS for hosting.
You can host it on Fargate to save time on configuring container instances, or on Beanstalk as well which also supports docker.
Yet other choice is to go fully serverless and develop your Python REST api using API Gateway and lambda.
What is the best pratice to deploy a Python Flask website on ec2 Amazon aws?
I saw some tutorial for uwsgi and ngnix...but I don't know if is correct to follow this path.
The easier way would be to go through Elastic Bean Stalk. It does not use uwsgi, instead, it uses Apache with mod_wsgi. This way you can only focus on application instead of worrying about the backend. EB takes care of that for you.
You can find more info about deploying Flask application in BeanStalk :
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create-deploy-python-flask.html
https://github.com/kmrambo/Deploying-a-Flask-or-Dash-application-using-AWS-Elastic-Beanstalk
Note: If you really want to learn, I will suggest to go ahead and do things manually. You can implement Apache + mod_wsgi and you will have a great learning curve.
If your Flask app is simple enough, perhaps Zappa would be an option for you (to deploy your Flask application to AWS Lambda). I currently host several web apps and APIs on Lambda using Zappa and find it very reliable. I still need to use EC2 for a few applications, in particular ones that use Websockets as this requires long-lived connections which don't really work that well in a serverless environment. In these cases I find using the Nginx/Gunicorn stack simple and reliable.
So I am quite the beginner in the world of web development. I have been working on a django application, and now that my application is running pretty well I am staring into the face of the monster that is deployment, and wondering which direction to take.
My application is running Django 1.7 and Python 3.4. I want to use PostgreSQL with Postgis as my database. I was originally thinking of using Amazon AWS for deployment, using their Elastic Beanstalk PaaS service, although now I am wondering if something like Heroku may be better for me as a beginner to all of this. I need my application to scale easily, without me having to dive into too much configuration that I am very likely to mess up.
What I am wondering is which service would be best for a newcomer developer deploying an application running the platforms I mentioned above. Heroku seems like it is a bit easier to use than AWS, but I am concerned that the pricing with Heroku would ramp up more quickly than with AWS. How difficult would it be to initially deploy using one service and migrate to another as I become more familiar with the process of maintaining web applications and/or as my needs change?
Any recommendations or info to help me decide on a service to go with would be very much appreciated.
Can I plug the company I work for, PythonAnywhere? Making Python application deployment easy is pretty much what we're there for :-)
Postgres support (including PosGIS) is currently in beta, but is pretty solid -- if you sign up, just drop us a line and we can enable it for your account.
Deploying Django service using postgres at AWS is as easy as you use local server, AWS is easy and good to use, you just need to confgiure confuguration file of Postgres and server you install at AWS. go to this https://www.digitalocean.com/community/tutorials/how-to-install-and-configure-django-with-postgres-nginx-and-gunicorn if you are using Apache.
I wrote a little bottle web application, played around with setup.py, and I'm ready to deploy.
I had two questions:
How do I deploy this new web app in production? Do I need a a webserver or reverse proxy? Can my main.py be run as a service? How is this typically done?
My web app uses a mongodb backend. Is there a way to package that with my application or to require it / look for it / alert the user when the installer runs? How is this typically done?
Pythonanywhere allows you for free to test/deploy a Bottle app with Pymongo driver. This is one of the webhostings which is python-deployers friendly... If you need something for a serious trafic, then you have to consider a paid programm or your own (virtual/dedicated) hosted server.
I ran my Bottle applications on OpenShift. There's a quickstart template on GitHub for getting a Bottle application online. You could just copy their directory structure and files, add your own, and send it all to the git repository that is created for you. OpenShift uses git when it comes to development, so if you ever make any changes, you just need to commit and push them to the server again.
MongoDB and RockMongo (a GUI for managing your MongoDB database, similar to phpMyAdmin for MySQL) are built in. You get three 'gears' or instances to work with, so the Bottle application will be one, with MongoDB and RockMongo included. You just have to enable them.
In order to redeploy a GAE application, I currently have to install the GAE deployment tools on the system that I am using for deployment. While this process is relatively straight forward, the deployment process is a manual process that does not work from behind a firewall and the deployment tools must be installed on every machine that will be used for updating GAE apps. A more ideal solution would be if I could update a GAE application from another GAE application that I have deployed previously. This would remove the need to have multiple systems configured to deploy apps.
Since the GAE deployment tools are written in Python and the GAE App Engine supports Python, is it possible to modify appcfg.py to work from within GAE? The use case would be to pull a project from GitHub or some other online repository and update one GAE application from another GAE app. If this is not possible, what is the limiting constraint?
Is it possible? Yes. The protocol appcfg uses to update apps is entirely HTTP-based, so there's absolutely no reason you couldn't write an app that's capable of deploying other apps (or redeploying itself - self-modifying code)! You may even be able to reuse large parts of appcfg.py to do it.
Is it easy? Probably not. It's quite likely you'll need to understand a decent chunk of appcfg's internals, and the RPCs it uses to upload new apps - not a trivial undertaking. You'll also need to store your credentials in the app, in all likelihood - though you can use a role account that is and admin only for the apps it's deploying to minimize risk there.
One limiting constraint could be the protocol that the python sdk uses to communicate with the GAE servers. If it only uses HTTP, you might be OK. but if it's anything else, you might be out of luck because you can't open a socket directly from within GAE.
What problem did you have by trying to update behind a firewall?
I've got some, but finally I manage to work around them.
About your question, the constraint is that you cannot write files into a GAE app, so even though you could possibly pull from the VCS you can't write those pulled files.
So you would have to update from outside the GAE in first place.
Anyway every machine that needs to update the GAE should have the SDK anyway just to see if they changes work.
So, If you really want to do this you have two alternatives:
Host your own "updater" site and istall the SDK there, then when you want to update log into your side ( or run a script ) and do the remote update.
Although I don't know Amazon EC2 well, I think you can do pretty much the same thing as op 1 from there.
Finally I think the password to update has to be typed always. ( you could have the SDK of the App engine and modify that, because it is open source )