I'm developing a react native application using Expo, it will display different data, preprocessed and cleaned with Python, along with sentiments analysis on tweets regarding a specific topic.
What is the best way to do that? I read about using RESTful API with Flask but after some reading I don't think it will serve this purpose.
Thank you in advance.
You could create an AWS lambda endpoint for sending data and retrieving the processed results. With the free tier of AWS lambda you get "...1M free requests per month and 400,000 GB-seconds of compute time per month."
Seems like it might suit your use-case.
Tutorial you can easily follow along with here: https://www.tutorialspoint.com/aws_lambda/aws_lambda_function_in_python.htm
Some more info on setting up a rest API using lambda:
https://blog.sourcerer.io/full-guide-to-developing-rest-apis-with-aws-api-gateway-and-aws-lambda-d254729d6992
Reference for AWS lambda free tier here:
https://aws.amazon.com/lambda/pricing/
Related
I am trying to get the cost of each and every resource in my azure subscription everyday.
I was thinking to use this link Azure REST API - Where do I find resource costs? but i don't have the resource group names for the subscriptions with me.
At the subscription level, Azure provides and ability to track your spending over your whole infrastructure with "Cost Management + Biling". You can preview your cost analysis and see your burning partner overtime. There's also a new functionality in preview so you can see cost by resource for example.
The above picture is how the cost analysis look like and how you can imagine it.
If you would like to know more: https://learn.microsoft.com/en-us/azure/cost-management-billing/cost-management-billing-overview
EDIT
The above is accessible in Azure portal directly. But I've personally never did it via REST API, but so far I was able to find this and I believe it might help you.
https://learn.microsoft.com/en-us/rest/api/cost-management/
https://learn.microsoft.com/en-us/rest/api/consumption/
https://learn.microsoft.com/en-us/rest/api/consumption/usage-details
I actually wrote a blog post about this:
You can use the Azure Cost Management connector in Power BI Desktop
You can exported daily data in CSV format
I'm building a project using python and grafana where I'd like to generate a certain number of copies of certain grafana dashboards based on certain criteria. I've downloaded the grafanalib library to help me out with that, and I've read through the Generating Dashboards From Code section of the grafanalib website, but I feel like I still need more context to understand how to use this library.
So my first question is, how do I convert a grafana dashboard JSON model into a python friendly format? What method of organization do I use? I saw the dashboard generation function written in the grafanalib documentation, but it looked quite a bit different from how my JSON data is organized. I'd just like some further description of how to do the conversion.
My second question is, once I've converted my grafana JSON into a python format, how do I then get the proper information to send that generated dashboard to my grafana server? I see in the grafanalib documentation the "upload_to_grafana" function used to send the information and it takes in the three parameters (json, server, api_key), and I understand where its getting the json parameter from, but I dont get where the server information or API key are coming from or where that information is found to be input.
This is all being developed on a raspberry pi 4 just to put that out there. I'm working on a personal smart agriculture project as a way to develop my coding abilities further, as I'm self taught. Any help that can be provided to help me in my understanding is most appreciated. Thank you.
create an API key in Grafana configuration ..The secret key that u get while creating is the API key ..Server is localhost:3000 in case of installed grafana
I'm new to Python and am very confused on how to begin this assignment:
Write Python code to connect to Twitter search API at:
https://api.twitter.com/1.1/search/tweets.json
with at least the following parameters:
q for a search topic of your interest;
count to 100 for 100 records
to retrieve twitter data and assign the data to a variable.
I already created the app to access Twitter's API. Thanks.
I had this same issue. According to Twitter, you must register an account to even use basic search features of public tweets and there is no way around it. I followed the guide for the complete setup here and can confirm it works
Edit: the article is for php, but changing it to python shouldn’t be very difficult. It has all the methods outlined in the article
Just as a hint, you might want to use tweepy for this. They have very good documentation and its much easier than trying to reinvent the wheel.
I have a requirement where I have to trigger a dataset in a blob to my python code where processing will happen and then store the processed dataset to the blob? Where should I do it? Any notebooks?
Azure functions dont have an option to write a Python code.
Any help would be appreciated.
Depending on your design, you could create 2 processes. The first one will search the data for whatever should "trigger", then notify the second processes of the "trigger" so it can read and modify the data.
You can work with the blob using python like the examples in azures docs.
https://learn.microsoft.com/en-us/azure/storage/blobs/storage-quickstart-blobs-python
More info from this post:
Azure Blob - Read using Python
from azure.storage.blob import BlockBlobService
block_blob_service = BlockBlobService(account_name='myaccount', account_key='mykey')
block_blob_service.get_blob_to_path('mycontainer', 'myblockblob', 'out-sunset.png')
Azure Functions have support Python in preview, which is based on Linux. You can see the wiki page Azure Functions on Linux Preview to know about it.
Note
Python for Azure Functions is currently in preview. To receive important updates, subscribe to the Azure App Service announcements repository on GitHub.
There are two documents to introduce how to develop Azure Functions using Python.
Create your first Python function in Azure (preview)
Azure Functions Python developer guide
You need to follow the documents above to use Azure CLI to run the command func new and select Blob Trigger to create an Azure Functions for Python to satisfy your requirement.
You should also consider logic apps, which allows you to automate some of the tasks,and includes several actions on data sets. Please add more details to your question to get a more accurate answer to your request.
New Edit:
There is support of Python 3.0 in preview for Azure functions,per the following post
Also, steps on where you can store your code in functions can be found here
In Azure portal ,if one subscription is selected ,the cost analysis can be viewed like the following screenshot
I want to programmatically fetch the information the like the one displayed above may be using using some python SDK API/REST API.
If anybody has any experience/idea on this ,please help.
After going through replies ,I have gone through the Azure Billing Rest API and I am now able to call the Usage Aggrgate and RateCard related Rest APIs.
Following are the results of those REST Calls. .Azure Billing Usage Aggregate Response
Azure Billing Ratecard Response
But honestly speaking ,I still have not figured out how these to would give me detailed view like the cost analysis does where for each resource how much cost associated can be displayed.Actually I am very new to Azure probably that is why I am missing the link some where .
Can somebody give some hint here ?
If you already have the usage and the ratecard data, then you must combine them. Take the meterId of the usage data and get the related ratecard data. The ratecard data contains the MeterRates and the IncludedQuantity which you must take. There are probably multiple meter rates and the included quantity because there are probably different costs per usage (e.g. first 10 calls for free, 3 GB for free, ...). The consumption starts/is reseted at the 14th of the month. That's the reason why you have to read the data from the whole billing period (begins with 14th of each month), because that's the only way how you get the correct consumption.
So, if you are using e.g. Azure Functions and you have a usage of 100.000 units per day and you want the costs from 20th - 30th, then the calculation works as follows:
read data from 14th - 30th. These are 17 days and therefore it used 1.700.000 units.
The first 400.000 are for free = IncludedQuantity (so in this sample the first 4 days). From the 400.001 unit on, you have to take the meter rate (0,0000134928 €) and calculate the costs. 1.300.000 * 0,0000134928 = ~17,54€. Fortunately, the azure functions have only one rate. If the rate changes e.g. after 5.000.000 units, then you also have to take this into account.
If you have the whole costs, then you can filter on your date which is 20.-30. and you will get the result.
That's the short explanation of the calculation. I implemented this calculation in C# and published it as a NuGet package. The source code is on github - probably it helps. It also contains a sample console which you could use to export the data.
Source: https://github.com/codehollow/AzureBillingApi
Blog post: https://codehollow.com/2017/02/using-the-azure-billing-api-to-calculate-the-costs/
I have the same issue. But unluckily, python SDK is too hard to use.
Moreover, you cannot find an availabale sample or example on Google.
So, I choose using restapi rather than python SDK.
With python code, you can do this firstly,
import requests
from azure.common.credentials import ServicePrincipalCredentials
and set headers pyload and url.
headers = {
"Content-Type": "application/json",
"Authorization": <token> }
you can get token through credentials, which generated by client_id,secret,tenant.
credentials.token() will return the token you can use in headers.
you can find restapi in https://learn.microsoft.com/en-us/rest/api/ or use F12 in Chrome when you access the Azure dashboard.
There are the offical documents below for retrieving the billing data using Python SDK or REST API in Python.
For using Python SDK, please see http://azure-sdk-for-python.readthedocs.io/en/latest/resourcemanagementcommerce.html.
For using Billing REST API, please see https://learn.microsoft.com/en-us/rest/api/billing/, and you can try to use Python package requests to get these data.
However, I think you may have know these above, the key is that you need to follow the tutorial Manage access to billing information for Azure using role-based access control to get the role permission via your account admin.
And then you may also need to register a client app to get the client id for Resource Management Authentication if you want to use Service Principal/ADAL (not AD User/Password) in Python SDK or use REST API to do the same authentication as Azure REST API Reference page said. For the authentication topic, you can refer to the content map of Manage Apps to know more if you have been getting some trouble.
Hope it helps. Any concern, please feel free to let me know.