Context: I'm in the process of designing an event-driven Python application. Various stakeholders have tasked me with investigating options for deploying the application using GraphQL endpoints within a serverless environment running on Azure Functions. End goal being that as the underlying data structures grow, we'd like to easily maintain the use-ability and performance of the application over time. Based on below resources it appears this is possible:
(https://azure.microsoft.com/en-us/resources/videos/build-2019-build-scalable-apis-using-graphql-and-serverless/)
(https://azure.microsoft.com/en-us/resources/videos/azure-friday-live-building-serverless-python-apps-with-azure-functions/)
(https://graphene-python.org/)
Question: User requirements dictate that the Azure Functions MUST be for internal use only and cannot be exposed publicly. Reading through the docs below I haven't found any resources on security config options for private endpoints.
https://learn.microsoft.com/en-us/azure/azure-functions/
Private endpoint in Azure
Can someone please point me in the right direction? Are Azure Functions even capable of this? And if they aren't can this be achieved with an alternative like Azure App Service?
Azure Functions have multiple hosting options including Consumption Plan, Premium Plan and App Service Plan.
Out of which, for complete VNET Isolation, App Service Environment is the only way to go as of now since the Private Endpoints for Azure Web Apps is currently in preview.
But note that Azure Functions can be deployed into a Kubernetes cluster as well which could be the better option if you already have a kubernetes cluster to deploy to.
Related
I have python scripts that I want to host as a web app in azure.
I want to keep the function scripts in azure data storage and host them as a python API in azure.
I want some devs to be able to change the scripts in the azure data storage and reflect the changes live without having to deploy.
How can I got about doing this?
Create Function App of Runtime Stack Python supported only in Linux Version in any Hosting Plan through Azure Portal.
Make the Continuous Deployment setting to GitHub under Deployment Center of the function app in the portal like below:
After authorizing, provide your GitHub Repository details in the same section.
Create the Azure Python Functions from VS Code and deploy to the GitHub Repository using Git Clone through Command Palette.
After that, you can deploy the functions to azure function app.
Here after publishing to azure, you'll get the Azure Functions Python Rest API.
I want some devs to be able to change the scripts in the azure data storage and reflect the changes live without having to deploy.
Whenever you or dev's make changes the changes in code through GitHub and commit the changes, then it automatically reflects in the Azure Portal Function App.
For more information, please refer this article and GitHub actions of editing the code/script files.
I am new to AWS with python. I came across boto3 initially, later somone suggested cdk. What is the difference between aws cdk and boto3?
In simple terms, CDK helps you to programmatically create AWS resources(Infrastructure as Code) while boto3 helps you to programmatically access AWS services.
Here is a snippet on CDK and Boto3 from AWS reference links :
CDK:
The AWS Cloud Development Kit (AWS CDK) is an open source software development framework to define your cloud application resources using familiar programming languages. AWS CDK provisions your resources in a safe, repeatable manner through AWS CloudFormation. It also enables you to compose and share your own custom constructs that incorporate your organization's requirements, helping you start new projects faster. (Reference: https://aws.amazon.com/cdk/)
With CDK and Cloudformation, you will get the benefits of repeatable deployment, easy rollback, and drift detection. (Reference: https://aws.amazon.com/cdk/features/)
Boto3:
Boto3 is the Amazon Web Services (AWS) Software Development Kit (SDK) for Python, which allows Python developers to write software that makes use of services like Amazon S3 and Amazon EC2.
(Reference: https://pypi.org/project/boto3/)
Welcome to Stack Overflow and to AWS usage!
boto3 is the python SDK for AWS. It is useful in order for your software to be able to elevate other AWS services.
use case example: your code has to put an object in an S3 bucket (store a file in other words).
aws-cdk is a framework that helps you provision infrastructure in an IaC (Infrastructure as Code) manner.
use case example: describe and provision your application infrastructure (e.g. a lambda function and an S3 bucket).
In many projects you will use both.
you can find an example URL shortener that uses boto3 and aws-cdk here. The URL shortener uses boto3 in order to access a DynamoDB table and aws-cdk in order to provision the whole infrastructure (including the lambda function which uses boto3).
You're creating an application that needs to use AWS services and resources. Should you use cdk or boto-3?
Consider if your application needs AWS services and resources at build time or run time.
Build time: you need the AWS resources to be available IN ORDER TO build the application.
Run time: you need the AWS resources to be available via API call when your application is up and running.
AWS CDK setups the infrastructure your application needs in order to run.
AWS SDK compliments your application to provide business logic and to make services available through your application.
Another point to add is that AWS CDK manages the state of your deployed resources internally, thus allowing you to keep track of what has been deployed and to specify the desired state of your final deployed resources.
On the other hand, if you're using AWS SDK, you have to store and manage the state of the deployed resources (deployed using AWS SDK) yourself.
I am also new to AWS, here is my understanding for relevant AWS services and boto3
AWS Cloud Development Kit (CDK) is a software library, available in different programming languages, to define and provision cloud infrastructure*, through AWS CloudFormation.
Boto3 is a Python software development kit (SDK) to create, configure, and manage AWS services.
AWS CloudFormation is a low-level service to create a collection of relation AWS and third-party resources, and provision and manage them in an orderly and predictable fashion.
AWS Elastic Beanstalk is a high-level service to deploy and run applications in the cloud easily, and sits on top of AWS CloudFormation.
I'm currently building an API in Rails to control devices registered with an Azure IoT Hub.
Now, in the Azure documentation, I only see Python/Node/.Net guides to control the end devices.
I require Rails for certain libraries, and hence used it to build my API. But now I've hit this road block.
Is there no way I can send cloud-to-device messages? I don't see any official Rails support. I only require c2d messaging.
Edit : The only other way I can think of with my limited knowledge is to deploy a small Python/Flask app on the same server and have my Rails code make HTTP requests to it. Is this a bad approach?
I do recommend to use the Azure HttpTrigger Function for your solutions as an API integrator/mediator.
I need to implement a simple web service in python - it's my first experience with web services and REST APIs so I want to undertand what environment and tools would fit my needs. In my web service, I need to read some data from a database, do some simple logic, and support a GET call from another application (qualtrics).
I read and implemented a simple test web service with python using some useful blogs such as: Building a Basic RestFul API in Python | Codementor
but I need a real server so that I could call the API from external applications.
As I'm looking for a long term solution, I thought that using AWS EC2 instance may be a good solution for a server. I tried to implement it using some guidelines in blogs such as: Deploy a Flask app on AWS EC2 | Codementor
However, as I'm new to this and encountered some implementation/editing errors (e.g. handling of the wsgi file) and as I'm a windows person and the ubuntu stuff are not always easy to get used to, I was wondering what is the best framework for my needs?
Is there any recomended flow in which I'll be able to implement my simple python code and connect it to a small server (either AWS EC2 instance or any other recomended one) in a more convenient way?
Another important note - I will need to run it only from time to time, this web server and web service should not be contantly live (that's why I thought that aws virtual instance would fit best).
To begin, my recommendation would be to look at Elastic Beanstalk, Fargate and API Gateway with Lambda.
You can use Elastic Beanstalk to easliy provision out-of-the-box AWS environment to host your python app in Flask with minimal configurations required:
Deploying a flask application to Elastic Beanstalk.
The other thing to consider would be to develop your python app as a docker container using, e.g., tiangolo/uwsgi-nginx-flask as the base image. This would allow you to easily work with in on your localhost, and then just move your image to AWS for hosting.
You can host it on Fargate to save time on configuring container instances, or on Beanstalk as well which also supports docker.
Yet other choice is to go fully serverless and develop your Python REST api using API Gateway and lambda.
Coming from AWS, I am completely new to Azure in general and to Cloud Services spesifically.
I want to write a python application that leverages GPU on Azure in a PaaS architecture (Platform as a Service). The application will hopefully be deployed somewhere central, and then a number of GPU enabled nodes will spin up and run the application until it is done before closing down again.
I want to know, what is the shortest way to accomplish this in Azure?
Is my assumption correct that I will need to use what is called Cloud Services with a worker role, or will I have to create my own infrastructure based on single VMs running in IaaS?
It sounds like you created an application which need to do some general-purpose computing on GPU via Cuda or OpenCL. If so, you need to install GPGPU driver on Azure to support your Python application, so the Azure NC & NV Series VMs are suitable for this scenario like on AWS, as the figure below from here.
Hope it helps. Any concern, please feel free to let me know.