How to copy dashboard to other organisations in Grafana - python

I'm using Grafana to show some data. I have 20 organisations and they all use the same dashboards (Dash1, Dash2, Dash3). Thus I can use the same json data for all dashboards in all organisations.
But I do not want to update it manually every time when I change something, thus I'm trying to create a python script which will do it for me.
I run the script as follows:
python update_dashboards.py Dash1
The python code is as follows:
try:
dashboard_name = sys.argv[1]
response = settings.get_request_with_token(settings.api_url + "search?query=" + dashboard_name)
dashboard = json.loads(response)
if len(dashboard) < 1:
print("There is no dashboard data.")
else:
dashboard_data = dashboard[0]
dashboard_uri = str(dashboard_data["uri"])
dashboard_data = dashboard_api.get_dashboard(dashboard_uri)
// Here I get dashboard details
except IndexError:
print("Please provide dashboard name!")
Thus I give the name of the dashboard that I want to copy to all organisations.
With my code I get successfully the dashboard that I want to copy.
My question is how can I now use this dashboard to copy it to all other organizations?

I'd think to use the API to create new dashboards. You can use requests and create a payload with what you already have in your script. You also need a key to use the API.
payload = {"authorization": "yourkey", "content": thedashboard}
url = yougrafanaaddress/api/dashboards/db
r = requests.post(url, data=payload)

Related

How to get the json value from postman input by the python code?

I'am trying to create a API with normal python without using
any framework. This is my API login page code:
#!C:\python39\python.exe
import cgi
from config import opp
import json
print("Content-type:text/html\r\n\r\n")
z = opp()
jsonval = json.loads(jsonobg)
z.email = jsonval['email']
z.password = jsonval['password']
if z.email != "" and z.password != "":
if not z.login():
print(json.dumps({"status":200, "title":"success",
"message":"loggedin successfully"}))
else:
print(json.dumps({"status":400, "title":"error",
"message":"the user does not extists"}))
else:
print(json.dumps({"status":400, "title":"error",
"message":"you cannot leave the field empty"}))
here I get the connection and login function from the config page. I am using xampp to run this simple api. every thing is working properly. but, if I try to run it in postman software, I don't know how to get the email and password from the postman input and give it to the jsonobj variable to put it in the json.load function. we will give the postman input like '{"email":"example#gmail.com", "password":"pass123"}' but I don't know how to transfer this data to python code. please tell me how to do it.

Databricks API call fails on Azure DevOps pipelines using python script, but run successfully on Postman from local machine

In Azure databricks API I am trying to pull latest changes to main branch in each folder in Databricks repos by using Azure Databricks API. This is where I am refering to:
When I use postman to make the calls by posting requests to the following endpoint, it pulls successfully as shown below:
endpoint:
https://<databricks-workspace>.azuredatabricks.net/api/2.0/repos/<repo-id>
This is the header of the same request:
To explain more, the header is constructed by sending a bearer token, a management token and another field which contains subscription, resource group and databricks workspace as shown below:
/subscriptions/<Azure-subscription-id>/resourceGroups/<resourcegroup-name>/providers/Microsoft.Databricks/workspaces/<databricks-workspace-name>
As shown above it works perfectly well when I call it on my local machine with postman. But when I use the same thing by using Azure DevOps it fails with the error:
Exception: b'{"error_code":"PERMISSION_DENIED","message":"Missing Git provider credentials. Go to User Settings > Git Integration to add your personal access token."}'
Note that following this link I have already generated a PAT token in Azure DevOps and added it to my Service Principal, otherwise it wouldn't have worked on my postman. Still it is giving this error in DevOps pipeline as shown below:
This pipeline is doing exactly what I already did with postman. This pipeline is calling a python script which is constructing the request header and body as shown above in postman. The python code is as below but I am almost sure it is not the python script that is causing the issue as I have used the same method to list repos, get specific repo, create clusters and many more by the same methodology. I think it must be some administrative problem which I cannot pin point.
The python script:
import requests
import os
import json
## Constructing the header request
DBRKS_REQ_HEADERS = {
'Authorization': 'Bearer ' + os.environ['DBRKS_BEARER_TOKEN'],
'X-Databricks-Azure-Workspace-Resource-Id': '/subscriptions/'+ os.environ['DBRKS_SUBSCRIPTION_ID'] +'/resourceGroups/'+ os.environ['DBRKS_RESOURCE_GROUP'] +'/providers/Microsoft.Databricks/workspaces/' + os.environ['DBRKS_WORKSPACE_NAME'],
'X-Databricks-Azure-SP-Management-Token': os.environ['DBRKS_MANAGEMENT_TOKEN']}
TRIGGERING_BRANCH = "\"" + os.environ["TRIGGERING_BRANCH"] + "\""
print("TRIGGERING_BRANCH path is {}".format(TRIGGERING_BRANCH))
## Constructing the body request
body_json = '''
{
"branch": "main"
}
'''
## Checking the request body format is correct
print("Request body in json format:")
print(body_json)
## The prints are only for code tracing
DBRKS_REPOS_LIST_JSON = os.environ["DBRKS_REPOS_LIST"]
print("Type of DBRKS_REPOS_LIST_JSON is {}".format(type(DBRKS_REPOS_LIST_JSON)))
## This section extracts repo Ids from the variable containing repo Ids and branches to later construct url endpoint
str_obj = DBRKS_REPOS_LIST_JSON.replace('[','').replace(']','').replace('(','').replace(')','').replace('\'','').replace(' ','').split(',')
output = {}
str_to_list = [str_obj[i:i+2] for i in range(0, len(str_obj), 2)]
print("str_to_list")
print(str_to_list)
for e in str_to_list:
output[e[0]] = e[1]
print("output")
print(output)
repo_ids_for_main_branch = []
for key, value in output.items():
if value == 'main':
repo_ids_for_main_branch.append(key)
print("repo_ids_for_main_branch")
print(repo_ids_for_main_branch)
## This is the main part which is making the API call like postman:
for repo_id in repo_ids_for_main_branch:
dbrks_pull_repo_url = "https://"+os.environ['DBRKS_INSTANCE']+".azuredatabricks.net/api/2.0/repos/"+str(repo_id)
print("endpoint url is {}".format(dbrks_pull_repo_url))
response = requests.patch(dbrks_pull_repo_url, headers=DBRKS_REQ_HEADERS, data=body_json)
if response.status_code == 200:
print("Branch changed or pulled successfully!")
print(response.status_code)
print(response.content)
print('####################')
else:
print("Couldn't pull or change branch!")
raise Exception(response.content)
All the os variables in the code as passed from Azure DevOps pipeline to the script and I have checked their values by printing and they are all correct.
I would like to know what the root cause problem is and how I can resolve it.
A small operation is there to implement. The major issue was with GIT integration. This can be resolved with the following steps.
Enable support for arbitrary files in Databricks Repos
This operation is needed to implement. Follow the link for the series of steps.

Issue with REST API call from Python to Azure

I am using an HTML form to take in parameters to use for Azure REST API calls.
My Code
#app.route('/storageaccountcreate', methods = ['POST', 'PUT'])
def storageaccountcreate():
name = request.form['storageaccountname']
resourcegroup = request.form['resourcegroup']
subscriptionId = request.form['subscriptionId']
location = request.form['location']
sku = request.form['sku']
keys = [name, resourcegroup, subscriptionId, location, sku]
api_json = {keys: request.form[key] for key in keys}
url = 'https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Storage/storageAccounts/{accountName}?api-version=2019-06-01'
r = requests.put((url),data=(api_json))
print(r.text)
return r.text
I am getting the following error when trying to use this code
Bad Request
The browser (or proxy) sent a request that this server could not understand.
Also the debugging in VSC is showing the following
Photo of error
If you have any info that would help I would greatly appreciate it!
Python doesn't interpolate values in regular strings:
url = 'https://management.azure.com/subscriptions/{subscriptionId}/...'
# ^^^^^^^^^^^^^^^^
Assuming you're using Python 3.6 or later, use an f-string to enable interpolation:
url = f'https://management.azure.com/subscriptions/{subscriptionId}/...'
# ^ here

How to pass extra arguments to Google Pagespeed API service object

I'm using the Google Pagespeed API v4 to extract Pagespeed scores for a large number of URLs. I want to test for both desktop and mobile with the argument strategy within my Pagespeed call. This is possible in the general API call, but I can't find a way to pass these arguments in the batch call.
Minimal working example:
from apiclient.discovery import build
import requests
#define Google API key and API call
google_api_key = "MyKey"
#build service object to call PageSpeed, version 4, with MyKey
ps_service = build('pagespeedonline', 'v4', developerKey = google_api_key)
list_of_urls = [a list of several URLs]
#define function; make list of URLs with column for PageSpeed score
def list_websites(request_id, response, exception):
if exception is not None:
print("This is an exception: " + str(exception) + " " + request_id)
else:
score = response['ruleGroups']['SPEED']['score']
print(score)
#create URL batch
ps_batch = ps_service.new_batch_http_request(callback = list_websites)
service_list = []
for url in list_of_urls:
service_list.append(ps_service.pagespeedapi().runpagespeed(url = url))
for req in service_list:
ps_batch.add(req)
#execute API call by batch
ps_batch.execute()
Although this method works fine for doing batch requests to the API, by default it calculates a speed score based on a desktop view of the said URLs (the analysis strategy is set to desktop), whereas I would also like to receive a score for a batch request in which strategy is set to mobile, to get a score based on the mobile view of the URLs.
My question is, how do I add an extra argument to the build() function in which I can make a distinction between mobile and desktop?
Like this:
for url in list_of_urls:
service_list.append(ps_service.pagespeedapi().runpagespeed(url = url, strategy='mobile'))
Every parameter, used by google can be sent that way

How do I update FB Status using Python & GraphAPI?

How do I update FB Status using Python & GraphAPI? This question has been asked before, but many of the solutions have been deprecated and the requirement of GraphAPI seems to have rendered many solutions irrelevant.
I have fiddled around with the fbpy, Facebook, OAuth, and oauth2 packages, and have looked through their examples, but I still cannot figure out how to get them working. I have no trust in any of the code or the packages I have been using and am wondering if anyone has any definitive solutions that they know will work.
First you need to do is understand login flows. You should understand if you easily want to switch through the different Facebook libraries. Therefore it can have code that is very verbose to code that is very simple based on implementation.
The next thing is that there are different ways to implement handling OAuth and different ways to display and launch your web app in Python. There is no way to authorize without hitting a browser. Otherwise you would have to keep copy pasting the access_token to the code.
Let's say you chose web.py to handle your web app presentation and requests.py to handle the Graph API HTTP calls.
import web, requests
Then setup the URL we want all request to go through
url = (
'/', 'index'
)
Now get your application id, secret and post-login URL you would like to use
app_id = "YOUR_APP_ID"
app_secret = "APP_SECRET"
post_login_url = "http://0.0.0.0:8080/"
This code will have one class index to handle the logic. In this class we want to deal with the authorization code Facebook will return after logging in
user_data = web.input(code=None)
code = user_data.code
From here setup a conditional to check the code
if not code:
# we are not authorized
# send to oauth dialog
else:
# authorized, get access_token
Within the "not authorized" branch, send the user to the dialog
dialog_url = ( "http://www.facebook.com/dialog/oauth?" +
"client_id=" + app_id +
"&redirect_uri=" + post_login_url +
"&scope=publish_stream" )
return "<script>top.location.href='" + dialog_url + "'</script>"
Else we can extract the access_token using the code received
token_url = ( "https://graph.facebook.com/oauth/access_token?" +
"client_id=" + app_id +
"&redirect_uri=" + post_login_url +
"&client_secret=" + app_secret +
"&code=" + code )
response = requests.get(token_url).content
params = {}
result = response.split("&", 1)
for p in result:
(k,v) = p.split("=")
params[k] = v
access_token = params['access_token']
From here you can choose how you want to deal with the call to update the status, for example a form,
graph_url = ( "https://graph.facebook.com/me/feed?" +
"access_token=" + access_token )
return ( '<html><body>' + '\n' +
'<form enctype="multipart/form-data" action="' +
graph_url + ' "method="POST">' + '\n' +
'Say something: ' + '\n' +
'<input name="message" type="text" value=""><br/><br/>' + '\n' +
'<input type="submit" value="Send"/><br/>' + '\n' +
'</form>' + '\n' +
'</body></html>' )
Or using face.py
from facepy import GraphAPI
graph = GraphAPI(access_token)
try:
graph.post(
path = 'me/feed',
message = 'Your message here'
)
except GraphAPI.OAuthError, e:
print e.message
So in the end you can get a slimmed down version like
import web
from facepy import GraphAPI
from urlparse import parse_qs
url = ('/', 'index')
app_id = "YOUR_APP_ID"
app_secret = "APP_SECRET"
post_login_url = "http://0.0.0.0:8080/"
user_data = web.input(code=None)
if not user_data.code:
dialog_url = ( "http://www.facebook.com/dialog/oauth?" +
"client_id=" + app_id +
"&redirect_uri=" + post_login_url +
"&scope=publish_stream" )
return "<script>top.location.href='" + dialog_url + "'</script>"
else:
graph = GraphAPI()
response = graph.get(
path='oauth/access_token',
client_id=app_id,
client_secret=app_secret,
redirect_uri=post_login_url,
code=code
)
data = parse_qs(response)
graph = GraphAPI(data['access_token'][0])
graph.post(path = 'me/feed', message = 'Your message here')
For more info see
* Facebook API - User Feed: http://developers.facebook.com/docs/reference/api/user/#feed
* Publish a Facebook Photo in Python – The Basic Sauce: http://philippeharewood.com/facebook/publish-a-facebook-photo-in-python-the-basic-sauce/
* Facebook and Python – The Basic Sauce: http://philippeharewood.com/facebook/facebook-and-python-the-basic-sauce/
One possible (tested!) solution using facepy:
Create a new application or use an existing one previously created.
Generate a user access token using the Graph API explorer with the status_update extended permission for the application.
Use the user access token created in the previous step with facepy:
from facepy import GraphAPI
ACCESS_TOKEN = 'access-token-copied-from-graph-api-explorer-on-web-browser'
graph = GraphAPI(ACCESS_TOKEN)
graph.post('me/feed', message='Hello World!')
You can try this blog too. It's using fbconsole app.
The code from the blog:
from urllib import urlretrieve
import imp
urlretrieve('https://raw.github.com/gist/1194123/fbconsole.py', '.fbconsole.py')
fb = imp.load_source('fb', '.fbconsole.py')
fb.AUTH_SCOPE = ['publish_stream']
fb.authenticate()
status = fb.graph_post("/me/feed", {"message":"Your message here"})
This is how I got it to work. You absolutely don't need to create any app for this. I'll describe how to post status updates to your profile and to a facebook page of yours.
First, to post a status update to your profile:
Go to https://developers.facebook.com/tools/explorer.
You'll see a textbox with Access Token written before it. Click on the button 'Get Access Token' beside this textbox. It will open a pop up asking you for various permissions for the access token. Basically these permissions define what all you can do through the Graph API using this token. Check the tick boxes beside all the permissions you need one of which will be updating your status.
Now go ahead and install the facepy module. Best way would be to use pip install.
After this pase the following code snippet in any .py file:
from facepy import GraphAPI
access_token = 'YOUR_GENERATED_ACCESS_TOKEN'
apiConnection = GraphAPI(access_token)
apiConnection.post(path='me/feed',
message='YOUR_DESIRED_STATUS_UPDATE_HERE')
Now execute this .py file the standard python way and check your facebook. You should see YOUR_DESIRED_STATUS_UPDATE_HERE posted to your facebook profile.
Next, to do the same thing with a facebook page of yours:
The procedure is almost exactly the same except for generating your access token.
Now you can't use the same access token to post to your facebook page. You need to generate a new one, which might be a little tricky for someone new to the Graph API. Here's what you need to do:
Go to the same developers.facebook.com/tools/explorer page.
Find a dropdown showing 'Graph API Explorer' and click on it. From the dropdown, select your page you want to post updates from. Generate a new access token for this page. The process is described here: . Do not forget to check the manage_pages permission in the extended permissions tab.
Now use this token in the same code as you used earlier and run it.
Go to your facebook page. You should YOUR_DESIRED_STATUS_UPDATE posted to your page.
Hope this helps!

Categories