I'm using the requests-oauthlib library to get a request token from an OAuth (v1) provider.
oauth = OAuth1Session(CONSUMER_KEY, client_secret=CONSUMER_SECRET,
signature_method=SIGNATURE_HMAC,
signature_type=SIGNATURE_TYPE_AUTH_HEADER)
resp = oauth.fetch_request_token(url=REQUEST_TOKEN_URL)
I'd like to send a custom User-Agent header with the request token fetch request and include some contact information in case there are ever any problems with my script. Can this be done?
It's possible to pass in a client class to the OAuth1Session constructor. From the docblock in the relevant file:
"""
:param client_class: A subclass of `oauthlib.oauth1.Client` to use with
`requests_oauthlib.OAuth1` instead of the default
"""
Within the oauthlib.oauth1.Client class, the _render(self, request, formencode=False, realm=None) method appears responsible for preparing the request. Since unrelated headers don't impact the base string that the request signature is created from, adding a new header/changing an existing User-Agent header shouldn't cause the signature to change in any way.
As such, we can create a custom client class, override the _render method and call the implementation in the parent class once we've added our header:
class CustomClient(Client):
def _render(self, request, formencode=False, realm=None):
request.headers['User-Agent'] = "FooClient/1.0"
return super()._render(request, formencode, realm)
The code that instantiates OAuth1Session then simply needs to reference the above class:
oauth = OAuth1Session(CONSUMER_KEY, client_secret=CONSUMER_SECRET,
signature_method=SIGNATURE_HMAC,
signature_type=SIGNATURE_TYPE_AUTH_HEADER, client_class=CustomClient)
Related
I am using this code to get data from Twitter API.
The code works, but I cannot understand how.
Specifically, I cannot understand how the auth=bearer_oauth argument works, since I am passing a function. And how the function works, since I am calling it without its argument.
Sorry if this is too basic, but I could not find an answer.
import requests
bearer_token = "AAA"
api_url = "https://api.twitter.com/2/tweets/search/recent"
def bearer_oauth(r):
r.headers["Authorization"] = f"Bearer {bearer_token}"
return r
def connect_to_endpoint(url, params):
response = requests.get(url, auth=bearer_oauth)
return response
query_params = {'query': 'test'}
json_response = connect_to_endpoint(api_url, query_params)
The bearer_oauth function is just setting the request's authorization header to the bearer token before the request is sent.
The code you provided essentially has the same functionality as this:
headers = {"Authorization": f"Bearer {bearer_token}"
requests.get(url, headers=headers)
After you send the request, Twitter's server parses the authorization header and checks that the bearer token you supplied is valid and has access to the requested resources.
As for why your specific code works, bearer_oauth is an authentication handler that gets attached to the request. The handler gets called when the request is constructed. You don't need to pass the request object because the handler is part of it already.
If you're curious about the implementation, I'd read the internal code here. It looks like the request object is passed to the handler, which then modifies the request (in this case, by setting the authorization header), and then returns the modified request object back to the internal function preparing the request. Then, all of the modified request object's attributes are copied:
# Allow auth to make its changes.
r = auth(self)
# Update self to reflect the auth changes.
self.__dict__.update(r.__dict__)
Since __dict__ is an internal dictionary that holds all the attributes of a single object, everything that was changed about the request object in the handler function will be copied and included in the request before it is sent.
I would need some of your help adapting Authlib with Django.
I'm trying to develop a Django app using OpenId and Authlib to connect my users and facing an issue with the access token, the issue invalid_client occurs. Using Postman I found out that the OpenId provider needs some parameters in the Header like 'Content-Length' or 'Host'.
When the Header param is defined in client.py, it works like a charm. However, I'd like to pass the custom header from views.py (mostly to avoid defining the Host directly in the package), but authorize_access_token doesn't allow multiple arguments,
def auth(request):
token = oauth.customprovider.authorize_access_token(request)
Maybe the "Compliance Fix for non Standard" feature might help, but I wasn't able to adapt it for Django and the Header parameter
https://docs.authlib.org/en/stable/client/oauth2.html#compliance-fix-oauth2
from authlib.common.urls import add_params_to_uri, url_decode
def _non_compliant_param_name(url, headers, data):
params = {'site': 'stackoverflow'}
url = add_params_to_uri(url, params)
return url, headers, body
def _fix_token_response(resp):
data = dict(url_decode(resp.text))
data['token_type'] = 'Bearer'
data['expires_in'] = int(data['expires'])
resp.json = lambda: data
return resp
session.register_compliance_hook(
'protected_request', _non_compliant_param_name)
session.register_compliance_hook(
'access_token_response', _fix_token_response)
Does anyone know a way to pass a custom Header to Authlib or defining it using the Compliance Fix and Django?
I had to do this recently for a provider that required an Authorization header added to the the refresh token. Here is the code I used.
Add the register_compliance_hook inside the function that is called using the compliance_fix argument when initializing the service.
def _compliance_fixes(session):
def _add_header_refresh(url, headers, body):
headers.update({'Authorization': "Basic " + self.secret_client_key})
return url, headers, body
session.register_compliance_hook('refresh_token_request', _add_header_refresh)
oauth = OAuth()
oauth.register("oauth-service", compliance_fix=_compliance_fixes)
Consider an http request using an OAuth token. The access token needs to be included in the header as bearer. However, if the token is expired, another request needs to be made to refresh the token and then try again. So the custom Retry object will look like:
s = requests.Session()
### token is added to the header here
s.headers.update(token_header)
retry = OAuthRetry(
total=2,
read=2,
connect=2,
backoff_factor=1,
status_forcelist=[401],
method_whitelist=frozenset(['GET', 'POST']),
session=s
)
adapter = HTTPAdapter(max_retries=retry)
s.mount('http://', adapter)
s.mount('https://', adapter)
r = s.post(url, data=data)
The Retry class:
class OAuthRetry(Retry):
def increment(self, method, url, *args, **kwargs):
# refresh the token here. This could be by getting a reference to the session or any other way.
return super(OAuthRetry, self).increment(method, url, *args, **kwargs)
The problem is that after the token is refreshed, HTTPConnectionPool is still using the same headers to make the request after calling increment. See: https://github.com/urllib3/urllib3/blob/master/src/urllib3/connectionpool.py#L787.
Although the instance of the pool is passed in increment, changing the headers there will not affect the call since it is using a local copy of the headers.
This seems like a use case that should come up frequently for the request parameters to change in between retries.
Is there a way to change the request headers in between two subsequent retries?
No, in current version of Requests(2.18.4) and urllib3(1.22).
Retrys is finally handled by openurl in urllib3. And by trace the code of the whole function, there is not a interface to change headers between retrys.
And dynamically changing headers should not be considered as a solution. From the doc:
headers – Dictionary of custom headers to send, such as User-Agent, If-None-Match, etc. If None, pool headers are used. If provided, these headers completely replace any pool-specific headers.
headers is a param passed to the function. And there is no guarantee that it will not be copy after passed. Although in current version of urllib3, openurl does not copy headers, any solution based on changing headers is considered hacky, since it's based on the implementation but not the documentation.
One work around
Interrupt a function and edit some verible it's using is very dangerous.
Instead of injecting something into urllib3, one simple solution is that check the response status and try again if needed.
r = s.post(url, data=data)
if r.status_code == 401:
# refresh the token here.
r = s.post(url, data=data)
Why does the original approach not work?
Requests copy the header in prepare_headers before sending it to urllib3. So urllib3 use the copy created before editing when retrying.
I am using Google App Engine's Protocol RPC library. I want to get the headers for a request and check that a certain header exists. I can't figure out how to get the requests headers?
The code basically looks like this:
class MyService(remote.Service):
#remote.method(MyRequest, MyResponse)
def my_request(self, request):
# TODO: Check that header exists in request
The passed in request object is of the type 'MyRequest' and doesn't have any header information attached to it.
There is a special method initialize_request_state that allows you to access all of the requests headers.
class MyService(remote.Service):
def initialize_request_state(self, state):
self.headers = state.headers
#remote.method(MyRequest, MyResponse)
def my_request(self, request):
logging.debug(self.headers)
I need a library that can only format a correct HTTP response (without creating a web server because I already have this one).
I have to pass a body of an http response and a content type to this library.
The following function does not work properly if I try to send AJAX - jQuery does not find any response. However if I type a corresponding URL in the URL string of browser then a page is displayed.
def response( data ):
return "HTTP/1.0 200 OK\r\nContent-Type:application/json\r\nConnection:close\r\n\r\n{0}\r\n".format( data )
Details. The data variable is string of json. I also use the SocketServer class, call self.request.sendall( result_response ) in child's handle() function.
# the 'request handler' class
class StateRequestHandler( SocketServer.BaseRequestHandler ):
def handle( self ):
...
self.request.sendall( response( some_json ) )
# the configured server class
class StateServer( SocketServer.ThreadingMixIn, SocketServer.TCPServer ):
pass
Solved. The reason was in cross-domain requests.
Try the following:
Although the HTTP 1.0 and 1.1 spec say that a white space is optional between the header name and header value, the specs do say "a single SP[ace] is preferred".
Ensure that you do not have any Cross Domain issues so ensure your HTML page is served from exactly the domain,scheme and port as your JSON response (Unless you've correctly configured CORS.
Use Chrome Dev Tools / Firebug / Fiddler to see what request they're making and check the Javascript console.
Use Wireshark to see exactly what's on the wire.