I need to implement a simple web service in python - it's my first experience with web services and REST APIs so I want to undertand what environment and tools would fit my needs. In my web service, I need to read some data from a database, do some simple logic, and support a GET call from another application (qualtrics).
I read and implemented a simple test web service with python using some useful blogs such as: Building a Basic RestFul API in Python | Codementor
but I need a real server so that I could call the API from external applications.
As I'm looking for a long term solution, I thought that using AWS EC2 instance may be a good solution for a server. I tried to implement it using some guidelines in blogs such as: Deploy a Flask app on AWS EC2 | Codementor
However, as I'm new to this and encountered some implementation/editing errors (e.g. handling of the wsgi file) and as I'm a windows person and the ubuntu stuff are not always easy to get used to, I was wondering what is the best framework for my needs?
Is there any recomended flow in which I'll be able to implement my simple python code and connect it to a small server (either AWS EC2 instance or any other recomended one) in a more convenient way?
Another important note - I will need to run it only from time to time, this web server and web service should not be contantly live (that's why I thought that aws virtual instance would fit best).
To begin, my recommendation would be to look at Elastic Beanstalk, Fargate and API Gateway with Lambda.
You can use Elastic Beanstalk to easliy provision out-of-the-box AWS environment to host your python app in Flask with minimal configurations required:
Deploying a flask application to Elastic Beanstalk.
The other thing to consider would be to develop your python app as a docker container using, e.g., tiangolo/uwsgi-nginx-flask as the base image. This would allow you to easily work with in on your localhost, and then just move your image to AWS for hosting.
You can host it on Fargate to save time on configuring container instances, or on Beanstalk as well which also supports docker.
Yet other choice is to go fully serverless and develop your Python REST api using API Gateway and lambda.
Related
Hey,
I would like to start a small website that will be entirely handled in Python. I will be using the Flask framework for this. So far I had a lot of contact with AWS ECS and ELB service, but I admit, Python itself is still unknown to me. That's why I have a few questions:
1. I understand that from the point of view of a software engineer it is better to separate the backend and frontend - so it is best to create two separate Python projects based on Flask - one will be the API, the other the frontend, right? Generally, both should be separate services in the ECS service I guess.
2. In such configuration do they both have to use some kind of WSGI server, like gunicorn? Is this a good solution to run inside Fargate with multiple vCPU?
3. There are quite a few questions and myths around Nginx for this solution. Until now I assumed that if I use Application Load Balancer it should be enough (after all it also acts as reverse proxy). Is it necessary to use Nginx as a sidecar in ECS, are there any benefits of this? Assuming that using Nginx would be advisable, should it be only for the frontend or also for API?
Thank you really in advance for any supportive advice here - I know that I have asked for a lot of things.
In my consideration
1- if you want to have a microservice concept you can separate you application with front-end and back-end each of them has their freameworks. for front-end you can use Angular, React , Vuejs and so on. Python is backend technology and you can write strong restfull api to communicate with you front-end application
2- if you containerize your application with for example Docker and write Dockerfile for each service witch it is most common in microservice it is okay to run your container with any servers like nginx,apache or WSGI server(i did not work with this) then expose port (if it is needed) to be accessible
3- when you run your service in AWS Fargate it is possible to connect loadbalancer to your service and a service itself run tasks each task actually is one or more container with may be nginx server or something else , if you mean that it is normal to have nginx in your container.
Context: I'm in the process of designing an event-driven Python application. Various stakeholders have tasked me with investigating options for deploying the application using GraphQL endpoints within a serverless environment running on Azure Functions. End goal being that as the underlying data structures grow, we'd like to easily maintain the use-ability and performance of the application over time. Based on below resources it appears this is possible:
(https://azure.microsoft.com/en-us/resources/videos/build-2019-build-scalable-apis-using-graphql-and-serverless/)
(https://azure.microsoft.com/en-us/resources/videos/azure-friday-live-building-serverless-python-apps-with-azure-functions/)
(https://graphene-python.org/)
Question: User requirements dictate that the Azure Functions MUST be for internal use only and cannot be exposed publicly. Reading through the docs below I haven't found any resources on security config options for private endpoints.
https://learn.microsoft.com/en-us/azure/azure-functions/
Private endpoint in Azure
Can someone please point me in the right direction? Are Azure Functions even capable of this? And if they aren't can this be achieved with an alternative like Azure App Service?
Azure Functions have multiple hosting options including Consumption Plan, Premium Plan and App Service Plan.
Out of which, for complete VNET Isolation, App Service Environment is the only way to go as of now since the Private Endpoints for Azure Web Apps is currently in preview.
But note that Azure Functions can be deployed into a Kubernetes cluster as well which could be the better option if you already have a kubernetes cluster to deploy to.
I have a few (python) microservices which serve as the backend of my web application.
I'm willing to deploy the microservices to some cloud platform (either AWS, Azure, etc).
Should I create 1 VM and deploy all the microservices on the same server, with different ports? If I should create 1 VM for each microservice, should they speak with each other through public IP?
Thanks in advance!
The short direct answer for you is YES. You can do that, directly to deploy all microservices on one VM using different ports is feasible. And you can set different public ports for a public IP in the NSG of networking setting on Azure VM.
However, it's not make sense and break the microservice architecture. But per my experience, you can try to use Nginx as reverse proxy or use Docker (even with Kubernetes) to manage yours microservices on a VM in the development or testing stage.
To use Nginx as reverse proxy to expose the same port with different request path to access the backend microservices with different ports. It will help to reduce the cost of migration in future.
Recommended. To use Docker with Kubernetes to deploy different microservices images like using different VMs. These microservices insider container bind the same port, and expose different ports outsider container. Then, to use Nginx as reverse proxy for these containers, too. In future, it will help to migrate them to Azure Kubernetes cluster easily, and get better price than VM.
Hope it helps.
Creating 1 server would kill the purpose for microservices. Services are deployed across
multiple nodes(servers) , more specifically multiple AZs (availability zones)
It also depends if you have stand alone services deployed directly using code behind server or are you using containers ? Because both will have different approaches to deploy and maintain.
As have said #Imran Arshad Deploy some microservice in one server is a bad idea.
If you are in starting phase of a project, you can use the ECS service of AWS with 1-2 ec2 instances. In this way, you can have HA at a low cost.
What is the best pratice to deploy a Python Flask website on ec2 Amazon aws?
I saw some tutorial for uwsgi and ngnix...but I don't know if is correct to follow this path.
The easier way would be to go through Elastic Bean Stalk. It does not use uwsgi, instead, it uses Apache with mod_wsgi. This way you can only focus on application instead of worrying about the backend. EB takes care of that for you.
You can find more info about deploying Flask application in BeanStalk :
http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create-deploy-python-flask.html
https://github.com/kmrambo/Deploying-a-Flask-or-Dash-application-using-AWS-Elastic-Beanstalk
Note: If you really want to learn, I will suggest to go ahead and do things manually. You can implement Apache + mod_wsgi and you will have a great learning curve.
If your Flask app is simple enough, perhaps Zappa would be an option for you (to deploy your Flask application to AWS Lambda). I currently host several web apps and APIs on Lambda using Zappa and find it very reliable. I still need to use EC2 for a few applications, in particular ones that use Websockets as this requires long-lived connections which don't really work that well in a serverless environment. In these cases I find using the Nginx/Gunicorn stack simple and reliable.
I have been given a task to complete: Deploy my pre-existing Pyramid application onto our EC2 Linux server. I would like to do this with a minimal amount of stress and error, especially considering am I totally new to AWS.
What I have done so far:
Setup the EC2 instance which I can SSH into.
Locally develop my Pyramid application
And, we version control the application with GitHub.
We are using: Pyramid (latest), along with Python 2.7.5 and Postgresql (via SQLAlchemy and Alembic.)
What is a basic, high-level list of steps to ensure that my application is deployed appropriately?
Where, if at all, does something like Elastic Beanstalk come into play?
And, considering my project is currently in a Git repo, what steps or considerations must be taken to accommodate this?
I'm not looking for opinions on how to tweak my setup or anything like that. I am looking for a non-debatable, comprehensible set of steps or considerations to deploy my application in the most basic form. This server is for development purposes only, so I am not looking for a full-blown solution.
I have researched this topic for Django projects, and frankly, I am a bit overwhelmed with the amount of different possible options. I am trying to boil this situation down to its minimal components.
I appreciate the time and help.
Deploying to an EC2 server is just like deploying to any other Linux server.
If you want to put it behind a load balancer, you can do which is fully documented.
You can also deploy to Elastic Beanstalk. Where as EC2 is a normal Linux sever, Beanstalk is more like deploying to an environment, you just push all your git changes into an S3 repo, your app then gets built and deployed onto beanstalk.
Meaning no server setups, no configuration (other than the very basics) and all new changes you push to S3, get built and update each version of your app that may have been launched on beanstalk.
You don't want to host your database server on EC2, use Amazons RDS database server, dead simple and takes about two minutes to setup and configure.
As far as file storage goes, move everything to S3.
EC2 and beanstalk should not be used for any form of storage.
I would suggest to run two instances and use Elastic Load Balancer.
Never run anything important on a single EC2 instance, EC2 instances are not durable, they can suddenly vanish, taking whatever data you had stored on it.
Everything else should work as in Pyramid Cookbook description.
Although there is no specific recipe for deploying a Pyramid application to AWS in the Pyramid Cookbook, there are other recipes that may offer a partial solution.