We Keep Coding

List job runs created through Azure VM managed identity in Databricks

Do we have a solution in python to list job runs that are created using Azure VM managed identity in Azure Databricks.
Appreciate the help!
I am getting http 403 error when using managed identity library in python
from azure.identity import ManagedIdentityCredential
credential = ManagedIdentityCredential()
# Obtain an access token
from azure.identity import DefaultAzureCredential
credentials = DefaultAzureCredential()
access_token = credentials.get_token("management.azure.com/")
headers = { 'Authorization': 'Bearer ' + access_token,
'Content-Type': 'application/json' }
# Set the URL for the Databricks REST API
endpoint url = "databricks_url" + '/api/2.0/clusters/list'
# Make the REST API call to the Databricks endpoint
response = requests.get(url, headers=headers)
print(response.json())

If managed identity isn't added into the Databricks workspace yet, then having only access token isn't enough - you also need to provide an additional access token for accessing Azure management API (the https://management.core.windows.net/ resource URL), and it should be provided as the X-Databricks-Azure-SP-Management-Token header, together with Databricks Workspace Resource ID as the X-Databricks-Azure-Workspace-Resource-Id.
And then you need to generate the access token to the for Databricks workspace resource (2ff814a6-3304-4ab8-85cb-cd0e6f879c1d) that should be sent as bearer token.
This specific scenario is described in the details the official documentation.

How to get Microsoft Graph API token to read mails

I'm trying to retrieve mails from my organization's mailbox, and I can do that via Graph Explorer. However, when I use the same information that I used in Graph Explorer, the generated token returns an error stating '/me request is only valid with delegated authentication flow.' in me/messages endpoint.
So, how can I generate the acceptable token for the /me endpoint?
An example python code or example Postman request would be amazing.

It sounds like the endpoint you're using in Graph Explorer is something like this
https://graph.microsoft.com/v1.0/me/messages
/me is referring to the user signed into Graph Explorer. If you want to read another user's messages you would use
https://graph.microsoft.com/v1.0/users/user#domain.com/messages
When connecting to Graph API as an application with no user interaction, you can never use /me endpoints, as there's no user logged in.
Reference
https://learn.microsoft.com/en-us/graph/api/user-list-messages?view=graph-rest-1.0
Python example to list messages
import requests
def get_messages(access_token, user):
request_url = f"https://graph.microsoft.com/v1.0/users/{user}/messages"
request_headers = {
"Authorization": "Bearer " + access_token
}
result = requests.get(url = request_url, headers = request_headers)
return(result)
msgs = get_messages(access_token = token['access_token'], user = "userPrincipalName#domain.com")
print(msgs.content)
Additional example of obtaining a token, using an app registration and client secret
import msal
def get_token_with_client_secret(client_id, client_secret, tenant_id):
# This function is to obtain a bearer token using the client credentials flow, with a client secret instead of a certificate
# https://docs.microsoft.com/en-us/graph/sdks/choose-authentication-providers?tabs=CS#client-credentials-provider
app = msal.ConfidentialClientApplication(
client_id = client_id,
client_credential = client_secret,
authority = f"https://login.microsoftonline.com/{tenant_id}")
scopes = ["https://graph.microsoft.com/.default"]
token = app.acquire_token_for_client(scopes = scopes)
return(token)

Registering twitter webhook on AWS API Gateway

I'm trying to register a Twitter webhook on an API I create on AWS API Gateway. I was able to set up the CRC portion which provides the appropriate response through a browser however I'm unable to make post requests to the same endpoint. I've been following multiple guides on this including this and this
The code is below that is giving a 403 response:
from urllib.parse import quote_plus
from requests_oauthlib import OAuth1Session
import json
import tweepy
import urllib
CONSUMER_KEY = 'xxxxxxxxxxxxx'
CONSUMER_SECRET = 'xxxxxxxxxxxxx'
ACCESS_TOKEN = 'xxxxxxxxxxxxx'
ACCESS_SECRET = 'xxxxxxxxxxxxx'
twitter = OAuth1Session(CONSUMER_KEY,
client_secret=CONSUMER_SECRET,
resource_owner_key=ACCESS_TOKEN,
resource_owner_secret=ACCESS_SECRET)
webhook_endpoint = urllib.parse.quote_plus("https://#######.execute-api.us-east-2.amazonaws.com/dev/webhook")
url = 'https://api.twitter.com/1.1/account_activity/all/:env_name/'
def register_webhook(url,twitter,webhook_endpoint):
url+=f'webhooks.json?url={webhook_endpoint}'
response = twitter.post(url)
print(response)
def subscribe_to_user_activity(url,twitter):
url+=f'subscriptions.json'
response = twitter.post(url)
print(response)
if __name__ == '__main__':
register_webhook(url,twitter,webhook_endpoint)
subscribe_to_user_activity(url,twitter)```
A couple of things I checked:
I can verify that the account is approved for account activity API, I created and counter checked the dev environment label, Keys were regenerated twice, Testing on Postman as advised here is giving a 200 forbidden response and doesn't work
In postman, I selected Oauth1.0 as the auth type and fill in Consumer Key, Consumer Secret, Access Token, Token Secret values, and provided the URL endpoint in the body. Is the POST request being sent to https://api.twitter.com/1.1/account_activity/all/:my_env_name/webhooks.json?
On the AWS side, the POST method was created on the resource with all default values
Any idea where I could be going wrong? Or a foolproof method of registering a Twitter webhook with AWS API gateway as the listener?

Python (requests library) ETL: Spotify API "Authorization Code Flow" - Request Access Token Problem

Context:
I'm working on a side project to pull data from the Spotify API into a
Microsoft SQL Server database as part of a refreshing ETL job. I need
to use the "Authorization Code Flow" so I can authorize/authenticate
programmatically, so my table will populate each day.
I'm using the Python requests library for this, and I don't want to
make an Object Oriented Solution for this if possible (not my
preference).
Problem:
I'm having trouble getting the Access Token after authenticating.
Looking at similar issues, it's very similar to this one:
Spotify API Authorization Code Flow with Python
.
I'm not sure why I'm getting a Response 400 (Bad Request) from this.
Can someone please advise here?
Code:
# used to to encode byte string from CLIENT_ID : CLIENT_SECRET, then decode for Authentication Header
import base64
# used to make HTTP requests from Spotify API
import requests
# used to access the environment variables
import os
def request_user_authorization():
'''
HTTP GET request to gain access to data (Authorization Code Flow)
HTTP POST request to send the code and receive an Authorization Token (current issue)
https://developer.spotify.com/documentation/general/guides/authorization/code-flow/
'''
# URLs
AUTH_URL = 'https://accounts.spotify.com/authorize'
TOKEN_URL = 'https://accounts.spotify.com/api/token'
BASE_URL = 'https://api.spotify.com/v1'
SPOTIFY_URI = 'https://api.spotify.com/v1/me/player/recently-played'
# sensitive items
CLIENT_ID = os.environ.get('SPOTIFY_CLIENT_ID_ENV')
CLIENT_SECRET = os.environ.get('SPOTIFY_CLIENT_SECRET_ENV')
# make a request to the /authorize endpoint to get an authorization code
user_authorization_code = requests.get(
AUTH_URL, {
'client_id': CLIENT_ID,
'response_type': 'code',
'redirect_uri': SPOTIFY_URI,
'scope': 'user-read-recently-played',
}
)
# Code 200 = "OK"
print(user_authorization_code)
#----------------------------------------------------------#
api_header_string = base64.urlsafe_b64encode((CLIENT_ID + ':' + CLIENT_SECRET).encode('ascii'))
api_headers={
'Content-Type': 'application/x-www-form-urlencoded',
'Authorization': 'Basic %s' % api_header_string.decode('ascii')
}
api_payload = {
'grant_type': 'authorization_code',
'code': user_authorization_code,
'redirect_uri': SPOTIFY_URI,
'client_id': CLIENT_ID,
'client_secret': CLIENT_SECRET
}
#issue here:
# Make a request to the /token endpoint to get an access token
access_token_request = requests.post(url=TOKEN_URL, data=api_payload, headers=api_headers)
# returns <Response [400]>
# https://datatracker.ietf.org/doc/draft-ietf-httpbis-semantics/
# 15.5.1. 400 Bad Request
# The _400 (Bad Request)_ status code indicates that the server cannot
# or will not process the request due to something that is perceived to
# be a client error (e.g., malformed request syntax, invalid request
# message framing, or deceptive request routing).
# print(access_token_request)
#----------------------------------------------------------#
request_user_authorization()

You seem to have misunderstood how the Authorizatuon Code Flow works.
The redirect_uri in this kind of flow is used by the provider api (here spotify) as a callback to give you the authorization code.
The spotify API will call this url with a code parameter that you can use to ask for a token.
Meaning that for this flow to work you need a web server ready to receive requests on the uri that you have given in your code request (and specified when creating your app on the spotify developer portal). You might be better off using the Client Credentials Flow for your use case.
Also you should always use the name of the keywords arguments when using requests.get, requests.post ... It makes the code clearer and the order of the arguments differ for each method so it can get confusing if you don't.

#Speedlulu you're correct, that was the problem.
For anyone in the future reading this question: this is what I learned since posting the question:
What I misunderstood was the flow of data, and that Client Credentials Flow (Application to Spotify only) was the better choice because I don't need to have a "User" portion to this program.
Spotify's Client Credentials Flow Documentation: https://developer.spotify.com/documentation/general/guides/authorization/client-credentials/
# used to access environment variables securely (sensitive data)
import os
# used to encode strings into bytes and back
import base64
# used to convert JSON data into strings
import json
# endpoint that I'm connecting to on Spotify's servers
token_request_url = "https://accounts.spotify.com/api/token"
CLIENT_ID = os.environ.get('SPOTIFY_CLIENT_ID_ENV')
CLIENT_SECRET = os.environ.get('SPOTIFY_CLIENT_SECRET_ENV')
# encode credentials into bytes, then decode into a string for the HTTP POST request to Spotify to authenticate
BASE64_ENCODED_HEADER_STRING = base64.b64encode(bytes(f"{CLIENT_ID}:{CLIENT_SECRET}", "ISO-8859-1")).decode("ascii")
#initializing dictionaries for HTTP POST request
headers = {}
data = {}
headers['Authorization'] = f"Basic {BASE64_ENCODED_HEADER_STRING}"
data['grant_type'] = "client_credentials"
data['json'] = True
data['scope'] = 'user-read-recently-played'
r = requests.post(url=token_request_url, headers=headers, data=data)
# prints the response from the server regarding the access token data (formatted to be easier to read)
print(json.dumps(r.json(), indent=2))
# store the token value in a variable for HTTP GET request
token = r.json()['access_token']
What was unclear is that I first need to POST my request with the credentials to get the token (using the specific URL to do), store the r.json()['access_token'] value in a variable, then use that as part of the following GET request to access my specific data.

"Invalid OAuth access token" on Marketing API in Python

I'm trying to access Facebook Marketing Api in Python, but I encounter some issues with the access token.
I run the following code:
def get_app_access_token(app_id, app_secret):
args = {'grant_type': 'client_credentials',
'client_id': app_id,
'client_secret': app_secret}
f = urllib2.urlopen("https://graph.facebook.com/oauth/access_token?" +
urllib.urlencode(args))
resp = json.loads(f.read())
### resp gives me {"access_token":"xxxxxx|yyyyyy", "token_type":"bearer"}
### so I extract yyyyyy as my access_token
result = resp["access_token"].split('|')[1]
access_token = get_app_access_token(APP_ID, SECRET_KEY)
api = FacebookAdsApi.init(APP_ID, SECRET_KEY, access_token)
account = AdAccount(AD_ACCOUNT)
adsets = account.get_ad_sets(fields=[AdSet.Field.name])
And I get the error "Invalid OAuth access token" on the last line.
Do you know what I did wrong?
Thank you!

I think changing result from result = resp["access_token"].split('|')[1] to result = resp["access_token"]
The Python API can use the format of "xxxxxx|yyyyyy" if you are feeding it to FacebookAdsApi however unless everything is setup properly on the Facebook developer settings for the tokens you get, I am not sure You can get proper access.
Using above I can get to the ad account okay. However my permissions for client tokens didn't allow for access to campaigns and I was using a depreciated API.