After looking at several articles online, StackOverflow, and the Yelp Google Group, I've been unable to figure out the problem to an Invalid Signature error being produced from my Yelp API request.
Here is the exact error:
{'error': {'text': 'Signature was invalid', 'description': 'Invalid signature. Expected signature base string: [some text here with keys]}}
And the code I've written to go along with it:
import rauth
import time
def get_results():
#Obtain these from Yelp's manage access page
consumer_key = ''
consumer_secret = ''
token = ''
token_secret = ''
session = rauth.OAuth1Session(
consumer_key = consumer_key
,consumer_secret = consumer_secret
,access_token = token
,access_token_secret = token_secret)
request = session.get("http://api.yelp.com/v2/search?location=Boston&term=food")
#Transforms the JSON API response into a Python dictionary
data = request.json()
print(data)
session.close()
return data
if __name__=="__main__":
print(get_results())
So what exactly is causing this error? I've done some modifications prior to this attempt, and the previous attempts I made I got similar errors; except one time I only got a "Invalid Signature" error, with no "Expect signature base string" message
There are more steps to authentication as per the docs
Making a Request
Each request must contain the following OAuth protocol parameters:
OAuth Parameter Value
oauth_consumer_key Your OAuth consumer key (from Manage API Access).
oauth_token The access token obtained (from Manage API Access).
oauth_signature_method hmac-sha1
oauth_signature The generated request signature, signed with the oauth_token_secret obtained (from Manage API Access).
oauth_timestamp Timestamp for the request in seconds since the Unix epoch.
oauth_nonce A unique string randomly generated per request.
These parameters may be passed in the HTTP (Authorization) header as URL query keys or in the POST data. Generating the OAuth signature is done by applying the HMAC-SHA1 with the oauth_token_secret. You may view your OAuth consumer key at Manage API Access. OAuth libraries are available to generate these requests.
You are not passing oauth_timestamp which is required or applying the HMAC-SHA1 so you get an Invalid Signature error, it is clearly outlined in the docs above what you need to send.
There is also an actual python yelp api you could use but to make a request you can use the example below based on the request function from the example code: to make a request using oauth2 and requests:
import requests
import oauth2
def request(url, url_params=None):
consumer_key = ""
consumer_secret = ""
token = ""
token_secret =""
url_params = url_params or {}
consumer = oauth2.Consumer(consumer_key, consumer_secret)
oauth_request = oauth2.Request(method="GET", url=url, parameters=url_params)
oauth_request.update(
{
'oauth_nonce': oauth2.generate_nonce(),
'oauth_timestamp': oauth2.generate_timestamp(),
'oauth_token': token,
'oauth_consumer_key': consumer_key
}
)
token = oauth2.Token(token, token_secret)
oauth_request.sign_request(oauth2.SignatureMethod_HMAC_SHA1(), consumer, token)
signed_url = oauth_request.to_url()
print(u'Querying {0} ...'.format(url))
return requests.get(signed_url).json()
Which using your url outputs a whole load of json, the start of which is:
Querying http://api.yelp.com/v2/search?location=Boston&term=food ...
{'region': {'center': {'longitude': -71.05460875, 'latitude': 42.35028894954365}, 'span': {'latitude_delta': 0.0325510910039668, 'longitude_delta': 0.04668455000000904}}, 'total': 8351, 'businesses': [{'name': "Giacomo's Ristorante", 'url': 'http://www.yelp.com/biz/giacomos-ristorante-boston', 'mobile_url': 'http://m.yelp.com/biz/giacomos-ristorante-boston', 'rating_img_url_large': 'http://s3-media2.fl.yelpcdn.com/assets/2/www/img/ccf2b76faa2c/ico/stars/v1/stars_large_4.png', 'phone':
...............................................................
...............................................................
I am not sure if the api supports python 3 but the code above was tested with python3 and python2 and it works fine, to install oauth2 you can simple pip install oauth2 and the same with requests if you don't have it installed.
Another common issue is that the servers time is out of sync. On linux, one can run
sudo ntpdate -s time.nist.gov
Related
I'm trying to register a Twitter webhook on an API I create on AWS API Gateway. I was able to set up the CRC portion which provides the appropriate response through a browser however I'm unable to make post requests to the same endpoint. I've been following multiple guides on this including this and this
The code is below that is giving a 403 response:
from urllib.parse import quote_plus
from requests_oauthlib import OAuth1Session
import json
import tweepy
import urllib
CONSUMER_KEY = 'xxxxxxxxxxxxx'
CONSUMER_SECRET = 'xxxxxxxxxxxxx'
ACCESS_TOKEN = 'xxxxxxxxxxxxx'
ACCESS_SECRET = 'xxxxxxxxxxxxx'
twitter = OAuth1Session(CONSUMER_KEY,
client_secret=CONSUMER_SECRET,
resource_owner_key=ACCESS_TOKEN,
resource_owner_secret=ACCESS_SECRET)
webhook_endpoint = urllib.parse.quote_plus("https://#######.execute-api.us-east-2.amazonaws.com/dev/webhook")
url = 'https://api.twitter.com/1.1/account_activity/all/:env_name/'
def register_webhook(url,twitter,webhook_endpoint):
url+=f'webhooks.json?url={webhook_endpoint}'
response = twitter.post(url)
print(response)
def subscribe_to_user_activity(url,twitter):
url+=f'subscriptions.json'
response = twitter.post(url)
print(response)
if __name__ == '__main__':
register_webhook(url,twitter,webhook_endpoint)
subscribe_to_user_activity(url,twitter)```
A couple of things I checked:
I can verify that the account is approved for account activity API, I created and counter checked the dev environment label, Keys were regenerated twice, Testing on Postman as advised here is giving a 200 forbidden response and doesn't work
In postman, I selected Oauth1.0 as the auth type and fill in Consumer Key, Consumer Secret, Access Token, Token Secret values, and provided the URL endpoint in the body. Is the POST request being sent to https://api.twitter.com/1.1/account_activity/all/:my_env_name/webhooks.json?
On the AWS side, the POST method was created on the resource with all default values
Any idea where I could be going wrong? Or a foolproof method of registering a Twitter webhook with AWS API gateway as the listener?
Context:
I'm working on a side project to pull data from the Spotify API into a
Microsoft SQL Server database as part of a refreshing ETL job. I need
to use the "Authorization Code Flow" so I can authorize/authenticate
programmatically, so my table will populate each day.
I'm using the Python requests library for this, and I don't want to
make an Object Oriented Solution for this if possible (not my
preference).
Problem:
I'm having trouble getting the Access Token after authenticating.
Looking at similar issues, it's very similar to this one:
Spotify API Authorization Code Flow with Python
.
I'm not sure why I'm getting a Response 400 (Bad Request) from this.
Can someone please advise here?
Code:
# used to to encode byte string from CLIENT_ID : CLIENT_SECRET, then decode for Authentication Header
import base64
# used to make HTTP requests from Spotify API
import requests
# used to access the environment variables
import os
def request_user_authorization():
'''
HTTP GET request to gain access to data (Authorization Code Flow)
HTTP POST request to send the code and receive an Authorization Token (current issue)
https://developer.spotify.com/documentation/general/guides/authorization/code-flow/
'''
# URLs
AUTH_URL = 'https://accounts.spotify.com/authorize'
TOKEN_URL = 'https://accounts.spotify.com/api/token'
BASE_URL = 'https://api.spotify.com/v1'
SPOTIFY_URI = 'https://api.spotify.com/v1/me/player/recently-played'
# sensitive items
CLIENT_ID = os.environ.get('SPOTIFY_CLIENT_ID_ENV')
CLIENT_SECRET = os.environ.get('SPOTIFY_CLIENT_SECRET_ENV')
# make a request to the /authorize endpoint to get an authorization code
user_authorization_code = requests.get(
AUTH_URL, {
'client_id': CLIENT_ID,
'response_type': 'code',
'redirect_uri': SPOTIFY_URI,
'scope': 'user-read-recently-played',
}
)
# Code 200 = "OK"
print(user_authorization_code)
#----------------------------------------------------------#
api_header_string = base64.urlsafe_b64encode((CLIENT_ID + ':' + CLIENT_SECRET).encode('ascii'))
api_headers={
'Content-Type': 'application/x-www-form-urlencoded',
'Authorization': 'Basic %s' % api_header_string.decode('ascii')
}
api_payload = {
'grant_type': 'authorization_code',
'code': user_authorization_code,
'redirect_uri': SPOTIFY_URI,
'client_id': CLIENT_ID,
'client_secret': CLIENT_SECRET
}
#issue here:
# Make a request to the /token endpoint to get an access token
access_token_request = requests.post(url=TOKEN_URL, data=api_payload, headers=api_headers)
# returns <Response [400]>
# https://datatracker.ietf.org/doc/draft-ietf-httpbis-semantics/
# 15.5.1. 400 Bad Request
# The _400 (Bad Request)_ status code indicates that the server cannot
# or will not process the request due to something that is perceived to
# be a client error (e.g., malformed request syntax, invalid request
# message framing, or deceptive request routing).
# print(access_token_request)
#----------------------------------------------------------#
request_user_authorization()
You seem to have misunderstood how the Authorizatuon Code Flow works.
The redirect_uri in this kind of flow is used by the provider api (here spotify) as a callback to give you the authorization code.
The spotify API will call this url with a code parameter that you can use to ask for a token.
Meaning that for this flow to work you need a web server ready to receive requests on the uri that you have given in your code request (and specified when creating your app on the spotify developer portal). You might be better off using the Client Credentials Flow for your use case.
Also you should always use the name of the keywords arguments when using requests.get, requests.post ... It makes the code clearer and the order of the arguments differ for each method so it can get confusing if you don't.
#Speedlulu you're correct, that was the problem.
For anyone in the future reading this question: this is what I learned since posting the question:
What I misunderstood was the flow of data, and that Client Credentials Flow (Application to Spotify only) was the better choice because I don't need to have a "User" portion to this program.
Spotify's Client Credentials Flow Documentation: https://developer.spotify.com/documentation/general/guides/authorization/client-credentials/
# used to access environment variables securely (sensitive data)
import os
# used to encode strings into bytes and back
import base64
# used to convert JSON data into strings
import json
# endpoint that I'm connecting to on Spotify's servers
token_request_url = "https://accounts.spotify.com/api/token"
CLIENT_ID = os.environ.get('SPOTIFY_CLIENT_ID_ENV')
CLIENT_SECRET = os.environ.get('SPOTIFY_CLIENT_SECRET_ENV')
# encode credentials into bytes, then decode into a string for the HTTP POST request to Spotify to authenticate
BASE64_ENCODED_HEADER_STRING = base64.b64encode(bytes(f"{CLIENT_ID}:{CLIENT_SECRET}", "ISO-8859-1")).decode("ascii")
#initializing dictionaries for HTTP POST request
headers = {}
data = {}
headers['Authorization'] = f"Basic {BASE64_ENCODED_HEADER_STRING}"
data['grant_type'] = "client_credentials"
data['json'] = True
data['scope'] = 'user-read-recently-played'
r = requests.post(url=token_request_url, headers=headers, data=data)
# prints the response from the server regarding the access token data (formatted to be easier to read)
print(json.dumps(r.json(), indent=2))
# store the token value in a variable for HTTP GET request
token = r.json()['access_token']
What was unclear is that I first need to POST my request with the credentials to get the token (using the specific URL to do), store the r.json()['access_token'] value in a variable, then use that as part of the following GET request to access my specific data.
that is my first try with an API, said API being called OPS.
I would like to get information using the API (OAuth 2) within my python code.
The ressource URL is :
http://ops.epo.org/3.2/rest-services/register/{publication}/{EPODOC}/{EP2814089}/biblio
I also received :
Consumer Key: O220VlTQqAmodifiedsf0YeqgM6c
Consumer Secret Key: swWmodified3edjORU
The documentation states that:
OPS uses the OAuth framework for Authentication and Authorization. At this point in
time, only the “Client Credentials” flow is supported using a Consumer key and
Consumer secret.
The actual steps to follow are:
Step 1: Client converts Consumer key and Consumer secret to
Base64Encode(Consumer key:Consumer secret).
This should be done programmatically using the language you are developing the client
application in. For the purposes of this example, a public website was used to perform
this conversion.
By entering the colon separated Client credentials, an encoded response is generated.
This response is then be used for basic Authentication.
Step 2: Client requests an access token using Basic Authentication, supplying its
Consumer key and Consumer secret with base64Encoding over encrypted HTTPS
connection:
OPS authenticates the client credentials passed in the Authorization header using basic
authentication method.
If credentials are valid, OPS responds with a valid access token.
Step 3: Client accesses OPS resources with access token in authorization header
(bearer tokens) over encrypted HTTPS connection
I tried a few samples of code with requests but, until now, nothing worked.
The client credentials flow is described in the OAuth2 RFC-6749. The client id and secret are base64 encoded in a Basic authentication scheme as described in RFC-7617
You should be able to get a token using Python code like:
import requests
import base64
url = 'https://ops.epo.org/3.2/auth/accesstoken'
data = {"grant_type": "client_credentials"}
creds = base64.b64encode("O220VlTQqAmodifiedsf0YeqgM6c:swWmodified3edjORU".encode())
headers = {'Authorization': 'Basic ' + creds.decode('UTF-8'), 'Content-Type': 'application/x-www-form-urlencoded'}
response = requests.post(url, headers=headers, data=data)
access_token = response.json()["access_token"]
When using the previous response I can obtain a token. (Thanks a lot for your answer)
So I tried :
myUrl = 'http://ops.epo.org/3.2/rest-services/register/publication/EPODOC/EP2814089/biblio'
header = {'PRIVATE-TOKEN': myToken}
response = requests.get(myUrl, headers=header)
print(response.text)
but I obtained a 403 error.
I finally got a specific library to do the job :
EPO OPS Library
But I still don't know how to do it on my own...
I've been following the guide for Twitter's 3-legged oauth setup:
https://dev.twitter.com/docs/auth/implementing-sign-twitter
Step 1: Obtaining a request token
For their authentication, step 1 requires making a post request containing the base64 encoded public and secret key.
key = "CONSUMER_KEY"
secret = "CONSUMER_SECRET"
auth = base64.encodestring("%s:%s" % (key, secret)).replace("\n", "")
data = {}
data["grant_type"] = "client_credentials"
headers = {}
headers["Authorization"] = "Basic " + auth
headers["Content-Type"] = "application/x-www-form-urlencoded;charset=UTF-8"
headers["Accept-Encoding"] = "gzip"
response = requests.post("https://api.twitter.com/oauth2/token",
headers=headers, data=data)
This first request returns a valid response code 200 along with an access token. The response looks like this:
{u'access_token': u'AAAAAAAAAAAAAAAAAAAAAHHHHH... ...vncbi', u'token_type': u'bearer'}
Step 2: Redirecting the user
This is where the problem is occurring. According to the docs, the user then just needs to be redirected to the authorization url formatted like this:
https://api.twitter.com/oauth/authenticate?oauth_token=AAAAAAAAAAAAAAAAAAAAAHHHHH... ...vncbi
However when I get to this page I get an error message:
Is there something I missed? The access_token is being generated without an issue. I'm not sure if this message is showing up because I set something up incorrectly earlier in the process. I'm also not sure how to check if the oauth token has expired.
Actually, you have been following https://dev.twitter.com/docs/api/1.1/post/oauth2/token which is quite different, e.g. only used for public resources and not private like status updates. For the three step one checkout https://gist.github.com/ib-lundgren/4487236 or better yet http://twython.readthedocs.org/en/latest/
If you only want to access public resources like user timelines you can do so via the code below.
# OBS: If you want to look at per user details and make status updates
# you want the OAuth1 version. This is only for publicly available
# resources such as user timelines.
from requests_oauthlib import OAuth2Session
from oauthlib.oauth2 import BackendApplicationClient
# Credentials you get from registering a new application
client_id = '<the id you get from github>'
client_secret = '<the secret you get from github>'
# TODO remove
client_id = 'VVq5UniipB5nXFAqtTA'
client_secret = 'PlaHnaSDbeY4eYkv8XiqxS1nzGWyKoq5WYSNjdeaw'
client_id = 'I1Xi7fOeYnA9jabyvGUaZxY20'
client_secret = 'k5PZpINooRpjAfQccGwLUr2ZMEtRJtoX8cKaooHjKewWupxRBG'
token_url = 'https://api.twitter.com/oauth2/token'
client = BackendApplicationClient(client_id)
twitter = OAuth2Session(client_id, client=client)
headers = {
'Content-Type': 'application/x-www-form-urlencoded;charset=UTF-8'
}
twitter.fetch_token(token_url, headers=headers, auth=(client_id, client_secret))
# Only public resources available to this application-only clients.
r = twitter.get('https://api.twitter.com/1.1/statuses/user_timeline.json?count=100&screen_name=twitterapi')
print r.content
Make sure you use the github version of the libraries
pip install git+https://github.com/idan/oauthlib.git
pip install git+https://github.com/requests/requests-oauthlib.git
We're using Rauth to connect to various OAuth 1 APIs. It works fine for a single request, but trying to do 2 or more requests against the given session results in 401 not authorized errors from the APIs.
Twitter API example:
import requests
from rauth import OAuth1Service
from rauth import OAuth1Session
consumer_key = {the consumer key}
consumer_secret = {the consumer secret}
access_token = {the access token}
access_token_secret = {the access token secret}
oauth_service = OAuth1Service(consumer_key = consumer_key,
consumer_secret = consumer_secret)
oauth_session = oauth_service.get_session(token = (access_token, access_secret))
url = 'https://api.twitter.com/1.1/statuses/home_timeline.json'
params = {'include_rts': 'true'}
r = oauth_session.get(url, params=params) # THIS WORKS
r = oauth_session.get(url, params=params) # THIS RETURNS 401 ERROR
This happens on both Twitter and LinkedIn APIs. How do we execute multiple requests against a single OAuth1Session object?
VERSIONS:
rauth==0.5.4
requests==1.1.0
UPDATE:
Strangely, if the params argument is not included then multiple requests can be made- but once params are included, even if it is an empty dict, we get 401s.
Example 1:
r = oauth_session.get(url) # THIS WORKS
r = oauth_session.get(url) # THIS WORKS
Example 2:
r = oauth_session.get(url, params={}) # THIS WORKS
r = oauth_session.get(url, params={}) # THIS RETURNS 401 ERROR
Carrying over from the comments, using session.get(..., header_auth=True) should do the trick. It's hard to say exactly why it doesn't work without this, but for the record, header-based authentication is preferred by the spec and given Twitter's position, I wouldn't be surprised if they also prefer it as a provider.
A quick search reveals dozens upon dozens of reports of their API failing where it ostensibly should work and one remedy is to prefer header authentication. From what I can tell, rauth is signing appropriately, so perhaps this is something to do with the way the provider is showing preference and handling non-header authenticated requests.
Update
It looks like either rauth or Requests was not properly handling params. It's odd because the signature base string and oauth_signature seemed to be correct, in that they were appropriately different on each respective request and the data they operated on seemed to checkout. So it seems like it should have validated the request.
At any rate, to correct this, we need to deepcopy elements of the request parameters that are mutable types, e.g. dictionaries. I've got a patch that should correct this, so you should be able to use this without header_auth. However, header authentication is the preferred method so I would still recommend it.