We Keep Coding

Python (requests library) ETL: Spotify API "Authorization Code Flow" - Request Access Token Problem

Context:
I'm working on a side project to pull data from the Spotify API into a
Microsoft SQL Server database as part of a refreshing ETL job. I need
to use the "Authorization Code Flow" so I can authorize/authenticate
programmatically, so my table will populate each day.
I'm using the Python requests library for this, and I don't want to
make an Object Oriented Solution for this if possible (not my
preference).
Problem:
I'm having trouble getting the Access Token after authenticating.
Looking at similar issues, it's very similar to this one:
Spotify API Authorization Code Flow with Python
.
I'm not sure why I'm getting a Response 400 (Bad Request) from this.
Can someone please advise here?
Code:
# used to to encode byte string from CLIENT_ID : CLIENT_SECRET, then decode for Authentication Header
import base64
# used to make HTTP requests from Spotify API
import requests
# used to access the environment variables
import os
def request_user_authorization():
'''
HTTP GET request to gain access to data (Authorization Code Flow)
HTTP POST request to send the code and receive an Authorization Token (current issue)
https://developer.spotify.com/documentation/general/guides/authorization/code-flow/
'''
# URLs
AUTH_URL = 'https://accounts.spotify.com/authorize'
TOKEN_URL = 'https://accounts.spotify.com/api/token'
BASE_URL = 'https://api.spotify.com/v1'
SPOTIFY_URI = 'https://api.spotify.com/v1/me/player/recently-played'
# sensitive items
CLIENT_ID = os.environ.get('SPOTIFY_CLIENT_ID_ENV')
CLIENT_SECRET = os.environ.get('SPOTIFY_CLIENT_SECRET_ENV')
# make a request to the /authorize endpoint to get an authorization code
user_authorization_code = requests.get(
AUTH_URL, {
'client_id': CLIENT_ID,
'response_type': 'code',
'redirect_uri': SPOTIFY_URI,
'scope': 'user-read-recently-played',
}
)
# Code 200 = "OK"
print(user_authorization_code)
#----------------------------------------------------------#
api_header_string = base64.urlsafe_b64encode((CLIENT_ID + ':' + CLIENT_SECRET).encode('ascii'))
api_headers={
'Content-Type': 'application/x-www-form-urlencoded',
'Authorization': 'Basic %s' % api_header_string.decode('ascii')
}
api_payload = {
'grant_type': 'authorization_code',
'code': user_authorization_code,
'redirect_uri': SPOTIFY_URI,
'client_id': CLIENT_ID,
'client_secret': CLIENT_SECRET
}
#issue here:
# Make a request to the /token endpoint to get an access token
access_token_request = requests.post(url=TOKEN_URL, data=api_payload, headers=api_headers)
# returns <Response [400]>
# https://datatracker.ietf.org/doc/draft-ietf-httpbis-semantics/
# 15.5.1. 400 Bad Request
# The _400 (Bad Request)_ status code indicates that the server cannot
# or will not process the request due to something that is perceived to
# be a client error (e.g., malformed request syntax, invalid request
# message framing, or deceptive request routing).
# print(access_token_request)
#----------------------------------------------------------#
request_user_authorization()

You seem to have misunderstood how the Authorizatuon Code Flow works.
The redirect_uri in this kind of flow is used by the provider api (here spotify) as a callback to give you the authorization code.
The spotify API will call this url with a code parameter that you can use to ask for a token.
Meaning that for this flow to work you need a web server ready to receive requests on the uri that you have given in your code request (and specified when creating your app on the spotify developer portal). You might be better off using the Client Credentials Flow for your use case.
Also you should always use the name of the keywords arguments when using requests.get, requests.post ... It makes the code clearer and the order of the arguments differ for each method so it can get confusing if you don't.

#Speedlulu you're correct, that was the problem.
For anyone in the future reading this question: this is what I learned since posting the question:
What I misunderstood was the flow of data, and that Client Credentials Flow (Application to Spotify only) was the better choice because I don't need to have a "User" portion to this program.
Spotify's Client Credentials Flow Documentation: https://developer.spotify.com/documentation/general/guides/authorization/client-credentials/
# used to access environment variables securely (sensitive data)
import os
# used to encode strings into bytes and back
import base64
# used to convert JSON data into strings
import json
# endpoint that I'm connecting to on Spotify's servers
token_request_url = "https://accounts.spotify.com/api/token"
CLIENT_ID = os.environ.get('SPOTIFY_CLIENT_ID_ENV')
CLIENT_SECRET = os.environ.get('SPOTIFY_CLIENT_SECRET_ENV')
# encode credentials into bytes, then decode into a string for the HTTP POST request to Spotify to authenticate
BASE64_ENCODED_HEADER_STRING = base64.b64encode(bytes(f"{CLIENT_ID}:{CLIENT_SECRET}", "ISO-8859-1")).decode("ascii")
#initializing dictionaries for HTTP POST request
headers = {}
data = {}
headers['Authorization'] = f"Basic {BASE64_ENCODED_HEADER_STRING}"
data['grant_type'] = "client_credentials"
data['json'] = True
data['scope'] = 'user-read-recently-played'
r = requests.post(url=token_request_url, headers=headers, data=data)
# prints the response from the server regarding the access token data (formatted to be easier to read)
print(json.dumps(r.json(), indent=2))
# store the token value in a variable for HTTP GET request
token = r.json()['access_token']
What was unclear is that I first need to POST my request with the credentials to get the token (using the specific URL to do), store the r.json()['access_token'] value in a variable, then use that as part of the following GET request to access my specific data.

Authorization issues with MS Graph

So I have been using powershell for quite a while now and am somewhat familiar with getting a token from an Azure application using MS Graph configured with application API permissions. I am now attempting to perform the same in a python console application and am getting flummoxed as I constantly get a 400 error. here's the snippet of my code...
import requests
import json
app_id='<appid>'
client_secret='<client secret>'
token_url='https://login.microsoftonline.com/<tenant id>/oauth2 /v2.0/token'
token_data = {
'grant_type': 'client_credentials',
'client_id': app_id,
'client_secret': client_secret,
'resource': 'https://graph.microsoft.com',
'scope':'https://graph.microsoft.com/.default'
}
headers = {'content-type':'application/json'}
token_r = requests.post(token_url, json=token_data)
token = token_r.json().get('access_token')
any ideas?

Figured it out. I needed to add the oAuth2 requests library. See sample code below:
import requests
import json
from requests_oauthlib import OAuth2Session
from oauthlib.oauth2 import BackendApplicationClient
app_id='<app id>'
client_secret='<client Secret>'
token_url='https://login.microsoftonline.com/tennantname.onmicrosoft.com/oauth2/v2.0/token'
scope='https://graph.microsoft.com/.default'
client = BackendApplicationClient(client_id=app_id, scope=scope, grant_type="client_credentials")
session = OAuth2Session(client=client, scope=scope)
# fill access token
token = session.fetch_token(token_url=token_url,client_id=app_id,scope=scope,client_secret=client_secret)

OAuth generates expired bearer token

I've been following the guide for Twitter's 3-legged oauth setup:
https://dev.twitter.com/docs/auth/implementing-sign-twitter
Step 1: Obtaining a request token
For their authentication, step 1 requires making a post request containing the base64 encoded public and secret key.
key = "CONSUMER_KEY"
secret = "CONSUMER_SECRET"
auth = base64.encodestring("%s:%s" % (key, secret)).replace("\n", "")
data = {}
data["grant_type"] = "client_credentials"
headers = {}
headers["Authorization"] = "Basic " + auth
headers["Content-Type"] = "application/x-www-form-urlencoded;charset=UTF-8"
headers["Accept-Encoding"] = "gzip"
response = requests.post("https://api.twitter.com/oauth2/token",
headers=headers, data=data)
This first request returns a valid response code 200 along with an access token. The response looks like this:
{u'access_token': u'AAAAAAAAAAAAAAAAAAAAAHHHHH... ...vncbi', u'token_type': u'bearer'}
Step 2: Redirecting the user
This is where the problem is occurring. According to the docs, the user then just needs to be redirected to the authorization url formatted like this:
https://api.twitter.com/oauth/authenticate?oauth_token=AAAAAAAAAAAAAAAAAAAAAHHHHH... ...vncbi
However when I get to this page I get an error message:
Is there something I missed? The access_token is being generated without an issue. I'm not sure if this message is showing up because I set something up incorrectly earlier in the process. I'm also not sure how to check if the oauth token has expired.

Actually, you have been following https://dev.twitter.com/docs/api/1.1/post/oauth2/token which is quite different, e.g. only used for public resources and not private like status updates. For the three step one checkout https://gist.github.com/ib-lundgren/4487236 or better yet http://twython.readthedocs.org/en/latest/
If you only want to access public resources like user timelines you can do so via the code below.
# OBS: If you want to look at per user details and make status updates
# you want the OAuth1 version. This is only for publicly available
# resources such as user timelines.
from requests_oauthlib import OAuth2Session
from oauthlib.oauth2 import BackendApplicationClient
# Credentials you get from registering a new application
client_id = '<the id you get from github>'
client_secret = '<the secret you get from github>'
# TODO remove
client_id = 'VVq5UniipB5nXFAqtTA'
client_secret = 'PlaHnaSDbeY4eYkv8XiqxS1nzGWyKoq5WYSNjdeaw'
client_id = 'I1Xi7fOeYnA9jabyvGUaZxY20'
client_secret = 'k5PZpINooRpjAfQccGwLUr2ZMEtRJtoX8cKaooHjKewWupxRBG'
token_url = 'https://api.twitter.com/oauth2/token'
client = BackendApplicationClient(client_id)
twitter = OAuth2Session(client_id, client=client)
headers = {
'Content-Type': 'application/x-www-form-urlencoded;charset=UTF-8'
}
twitter.fetch_token(token_url, headers=headers, auth=(client_id, client_secret))
# Only public resources available to this application-only clients.
r = twitter.get('https://api.twitter.com/1.1/statuses/user_timeline.json?count=100&screen_name=twitterapi')
print r.content
Make sure you use the github version of the libraries
pip install git+https://github.com/idan/oauthlib.git
pip install git+https://github.com/requests/requests-oauthlib.git

Google+ Domains API 403 Forbidden

I'm trying to create Circles with the Google+ API, but I'm kinda stuck, this is my code, it was more or less copied from the official API documentation (yes I know it doesn't create Circle, but the issue is the same)
import httplib2
from apiclient.discovery import build
from oauth2client.client import OAuth2WebServerFlow
import json
with open('client_secrets.json', 'r') as f:
json_data = json.load(f)
data = json_data['web']
CLIENT_ID = data['client_id']
CLIENT_SECRET = data['client_secret']
# List the scopes your app requires:
SCOPES = ['https://www.googleapis.com/auth/plus.me',
'https://www.googleapis.com/auth/plus.circles.write']
# The following redirect URI causes Google to return a code to the user's
# browser that they then manually provide to your app to complete the
# OAuth flow.
REDIRECT_URI = 'http://localhost/oauth2callback'
# For a breakdown of OAuth for Python, see
# https://developers.google.com/api-client-library/python/guide/aaa_oauth
# CLIENT_ID and CLIENT_SECRET come from your APIs Console project
flow = OAuth2WebServerFlow(client_id=CLIENT_ID,
client_secret=CLIENT_SECRET,
scope=SCOPES,
redirect_uri=REDIRECT_URI)
auth_uri = flow.step1_get_authorize_url()
# This command-line server-side flow example requires the user to open the
# authentication URL in their browser to complete the process. In most
# cases, your app will use a browser-based server-side flow and your
# user will not need to copy and paste the authorization code. In this
# type of app, you would be able to skip the next 3 lines.
# You can also look at the client-side and one-time-code flows for other
# options at https://developers.google.com/+/web/signin/
print 'Please paste this URL in your browser to authenticate this program.'
print auth_uri
code = raw_input('Enter the code it gives you here: ')
# Set authorized credentials
credentials = flow.step2_exchange(code)
# Create a new authorized API client.
http = httplib2.Http()
http = credentials.authorize(http)
service = build('plusDomains', 'v1', http=http)
from apiclient import errors
try:
people_service = service.people()
people_document = people_service.get(userId='me').execute()
except errors.HttpError, e:
print e.content
My output:
{
"error": {
"errors": [
{
"domain": "global",
"reason": "forbidden",
"message": "Forbidden"
}
],
"code": 403,
"message": "Forbidden"
}
}
I searched for answer, but didn't really find any. On the API console I have Google+ API and
Google+ Domains API services added also my secret and client id are okay (otherwise the whole script would fail sooner). Also the auth is successful, my app's name is shown under https://accounts.google.com/IssuedAuthSubTokens. What did I miss?

The problem lies with your REDIRECT_URI variable. When you are using OAuth 2.0 in a purely server-side flow, the redirect URI MUST be 'urn:ietf:wg:oauth:2.0:oob'.
Try changing the variable like so (and be sure to update your client ID in the API Console):
REDIRECT_URI = 'urn:ietf:wg:oauth:2.0:oob'
Edit: Also, make sure that you are making your API call for a user within a domain. The Google+ Domains API only permits API calls that are restricted to users and content within that domain.

how to get authorization code for Box in python

I was stuck in 'Getting the Access Token' part. In the documentation, I need a 'code'. But how can I get the authorization code? I'm using python.
authorize_url = 'https://www.box.com/api/oauth2/authorize?response_type=code&client_id=MY_CLIENT_ID'
def myRequest(path, method=None, options=None):
response = requests.request(method, path, **options)
return json.dumps(response.json())
code = myRequest(
path=authorize_url,
method='GET',
options={
'headers':{
'response_type': 'code',
'client_id': CLIENT_ID,
}
}
)

You have to use a module to make HTTP requests e.g. requests or urllib and follow the instructions as indicated here.
I have a very simple Flask app set up here: https://github.com/seanrose/box-oauth2-example
The 'code' is returned to your app after you send the user to Box's authorization URL.