We Keep Coding

Accomplishing Oauth2.0 authorization with refresh token through Python (Google API service creation)

I'm trying to access Google API services through a headless Linux server using Oauth2. I read through all the answers on this post: How do I authorise an app (web or installed) without user intervention? but none of them showed how to use the refresh token to generate an access token in python. pinnoyyid had a javascript example (https://stackoverflow.com/a/19766913/15713034) that went something like this:
function get_access_token_using_saved_refresh_token() {
// from the oauth playgroundfunction get_access_token_using_saved_refresh_token() {
// from the oauth playground
const refresh_token = "1/0PvMAoF9GaJFqbNsLZQg-f9NXEljQclmRP4Gwfdo_0";
// from the API console
const client_id = "559798723558-amtjh114mvtpiqis80lkl3kdo4gfm5k.apps.googleusercontent.com";
// from the API console
const client_secret = "WnGC6KJ91H40mg6H9r1eF9L";
// from https://developers.google.com/identity/protocols/OAuth2WebServer#offline
const refresh_url = "https://www.googleapis.com/oauth2/v4/token";
let refresh_request = {
body:`grant_type=refresh_token&client_id=${encodeURIComponent(client_id)}&client_secret=${encodeURIComponent(client_secret)}& refresh_token=${encodeURIComponent(refresh_token)}`;,
method: "POST",
headers: new Headers({
'Content-Type': 'application/x-www-form-urlencoded'
})
}
JavaScript isn't really my best language, but I could decipher they were sending a POST request to the google server. So I tried to recreate the request in Python with the requests package:
import requests
result = requests.post("https://www.googleapis.com/oauth2/v4/token", body={'grant_type':'refresh-token', 'client_id':client_id, 'client_secret':client_secret, 'refresh_token': refresh_token}, headers={'Content-Type': 'application/x-www-form-urlencoded'})
And when I look at result it shows it has a 200 status code (success) but when I try to examine the response, there's nothing easy to read and I can't parse the result in JSON to get the access token. The other approach I tried was to spin up a Flask server using Google's suggested code: https://developers.google.com/identity/protocols/oauth2/web-server#python_5 but that doesn't work either because when I try to return the credentials from one of the functions (object that contains the access code) that won't return JSON no matter what. I'd prefer the post request method since it is cleaner and uses less code. Thanks!

In Python, one approach is to use requests-oauthlib to perform the Backend Application Flow. This is useful when you don't have a front-end to redirect someone to, in order to approve fetching a token.

This website (https://community.atlassian.com/t5/Bitbucket-questions/Refresh-Tokens-using-Python-requests/qaq-p/1213162) says solution could be something like this:
import requests
auth = ("<consumer_id>", "<consumer_secret>")
params = {
"grant_type":"refresh_token",
"refresh_token":"<your_refresh_token_here>"
}
url = "https://www.googleapis.com/oauth2/v4/token"
ret = requests.post(url, auth=auth, data=params) #note data=params, not params=params

Since none of the solutions above worked, I had to finally just give up and use a service account.

FlowExchangeError thrown when getting access token OAuth via Google

I want to add 'sign-in via GMail' functionality to a website. I create login.html and project.py to process the response.
I add a button to login.html:
function renderButton() {
gapi.signin2.render('my-signin2', {
'scope': 'profile email',
'width': 240,
'height': 50,
'longtitle': true,
'theme': 'dark',
'onsuccess': signInCallback,
'onfailure': signInCallback
});
};
I have a callBack function. In the browser console, I can see that the response contains access_token, id_token (what is the difference?), and my user profile details (name, email, etc) so the request itself must have succeeded, however, error function is called because the response returned by my gconnect handler is 401:
function signInCallback(authResult) {
var access_token = authResult['wc']['access_token'];
if (access_token) {
// Hide the sign-in button now that the user is authorized
$('#my-signin2').attr('style', 'display: none');
// Send the one-time-use code to the server, if the server responds, write a 'login successful' message to the web page and then redirect back to the main restaurants page
$.ajax({
type: 'POST',
url: '/gconnect?state={{STATE}}',
processData: false,
data: access_token,
contentType: 'application/octet-stream; charset=utf-8',
success: function(result)
{
....
},
error: function(result)
{
if (result)
{
// THIS CASE IS EXECUTED, although authResult['error'] is undefined
console.log('Logged in successfully as: ' + authResult['error']);
} else if (authResult['wc']['error'])
{
....
} else
{
....
}//else
}//error function
});//ajax
};//if access token
};//callback
The code that handles the ajax request to Google throws FlowExchangeError when trying to get credentials = oauth_flow.step2_exchange(code) :
#app.route('/gconnect', methods=['POST'])
def gconnect():
if request.args.get('state') != login_session['state']:
response = make_response(json.dumps('Invalid state parameter.'), 401)
response.headers['Content-Type'] = 'application/json'
return response
# Obtain authorization code
code = request.data
try:
# Upgrade the authorization code into a credentials object
oauth_flow = flow_from_clientsecrets('client_secrets.json', scope='')
oauth_flow.redirect_uri = 'postmessage'
##### THROWS EXCEPTION HERE #####
credentials = oauth_flow.step2_exchange(code)
except FlowExchangeError:
response = make_response(
json.dumps('Failed to upgrade the authorization code.'), 401)
response.headers['Content-Type'] = 'application/json'
return response
# Check that the access token is valid.
access_token = credentials.access_token
url = ('https://www.googleapis.com/oauth2/v1/tokeninfo?access_token=%s'
% access_token)
h = httplib2.Http()
result = json.loads(h.request(url, 'GET')[1])
# If there was an error in the access token info, abort.
if result.get('error') is not None:
response = make_response(json.dumps(result.get('error')), 500)
response.headers['Content-Type'] = 'application/json'
return response
# Verify that the access token is used for the intended user.
gplus_id = credentials.id_token['sub']
if result['user_id'] != gplus_id:
response = make_response(
json.dumps("Token's user ID doesn't match given user ID."), 401)
response.headers['Content-Type'] = 'application/json'
return response
# Verify that the access token is valid for this app.
if result['issued_to'] != CLIENT_ID:
response = make_response(
json.dumps("Token's client ID does not match app's."), 401)
print "Token's client ID does not match app's."
response.headers['Content-Type'] = 'application/json'
return response
stored_access_token = login_session.get('access_token')
stored_gplus_id = login_session.get('gplus_id')
if stored_access_token is not None and gplus_id == stored_gplus_id:
response = make_response(json.dumps('Current user is already connected.'),
200)
response.headers['Content-Type'] = 'application/json'
return response
# Store the access token in the session for later use.
login_session['access_token'] = credentials.access_token
login_session['gplus_id'] = gplus_id
# Get user info
userinfo_url = "https://www.googleapis.com/oauth2/v1/userinfo"
params = {'access_token': credentials.access_token, 'alt': 'json'}
answer = requests.get(userinfo_url, params=params)
data = answer.json()
login_session['username'] = data['name']
login_session['picture'] = data['picture']
login_session['email'] = data['email']
output = ''
output += '<h1>Welcome, '
output += login_session['username']
return output
I have checked client_secrets.json I got from Google API, and it seems ok, do I need to renew it?
{"web":{"client_id":"blah blah blah.apps.googleusercontent.com","project_id":"blah","auth_uri":"https://accounts.google.com/o/oauth2/auth","token_uri":"https://oauth2.googleapis.com/token","auth_provider_x509_cert_url":"https://www.googleapis.com/oauth2/v1/certs","client_secret":"blah client secret","redirect_uris":["http://localhost:1234"],"javascript_origins":["http://localhost:1234"]}}
Why does credentials = oauth_flow.step2_exchange(code) fail?
This is my first time implenting this, and I am learning Web and OAuth on-the-go, all the concepts are hard to grasp at once. I am also using the Udacity OAuth course but their code is old and does not work. What might I be missing here?

You need to follow Google Signin for server side apps which describes thoroughly how the authorization code flow works, and the interactions between frontend, backend and user.
On server side, you use oauth_flow.step2_exchange(code) which expects an authorization code whereas you are sending an access token. Sending an access token here is not part of the authorization code flow or one-time-code flow as explained in the link above :
Your server exchanges this one-time-use code to acquire its own access
and refresh tokens from Google for the server to be able to make its
own API calls, which can be done while the user is offline. This
one-time code flow has security advantages over both a pure
server-side flow and over sending access tokens to your server.
If you want to use this flow, you need to use auth2.grantOfflineAccess() in the frontend :
auth2.grantOfflineAccess().then(signInCallback);
so that when the user clicks on the button it will return an authorization code + access token :
The Google Sign-In button provides both an access token and an
authorization code. The code is a one-time code that your server can
exchange with Google's servers for an access token.
You only need the authorization code if you want your server to access Google services on behalf of your user
From this tutorial it gives the following example that should work for you (with some modification on your side) :
<html itemscope itemtype="http://schema.org/Article">
<head>
<script src="//ajax.googleapis.com/ajax/libs/jquery/1.8.2/jquery.min.js"></script>
<script src="https://apis.google.com/js/client:platform.js?onload=start" async defer></script>
<script>
function start() {
gapi.load('auth2', function() {
auth2 = gapi.auth2.init({
client_id: 'YOUR_CLIENT_ID.apps.googleusercontent.com',
// Scopes to request in addition to 'profile' and 'email'
//scope: 'additional_scope'
});
});
}
</script>
</head>
<body>
<button id="signinButton">Sign in with Google</button>
<script>
$('#signinButton').click(function() {
auth2.grantOfflineAccess().then(signInCallback);
});
</script>
<script>
function signInCallback(authResult) {
if (authResult['code']) {
// Hide the sign-in button now that the user is authorized, for example:
$('#signinButton').attr('style', 'display: none');
// Send the code to the server
$.ajax({
type: 'POST',
url: 'http://example.com/storeauthcode',
// Always include an `X-Requested-With` header in every AJAX request,
// to protect against CSRF attacks.
headers: {
'X-Requested-With': 'XMLHttpRequest'
},
contentType: 'application/octet-stream; charset=utf-8',
success: function(result) {
console.log(result);
// Handle or verify the server response.
},
processData: false,
data: authResult['code']
});
} else {
// There was an error.
}
}
</script>
</body>
</html>
Note that the answer above suppose that you want to use authorization code flow / one-time code flow as it was what you've implemented server side.
It's also possible to just send the access token as you did (eg leave the client side as is) and remove the "Obtain authorization code" part :
# Obtain authorization code
code = request.data
try:
# Upgrade the authorization code into a credentials object
oauth_flow = flow_from_clientsecrets('client_secrets.json', scope='')
oauth_flow.redirect_uri = 'postmessage'
##### THROWS EXCEPTION HERE #####
credentials = oauth_flow.step2_exchange(code)
except FlowExchangeError:
response = make_response(
json.dumps('Failed to upgrade the authorization code.'), 401)
response.headers['Content-Type'] = 'application/json'
return response
instead :
access_token = request.data
but doing this wouldn't be the authorization code flow / one-time-code flow anymore
You've asked what was the difference between access_token and id_token :
an access token is a token that gives you access to a resource, in this case Google services
an id_token is a JWT token that is used to identify you as a Google user - eg an authenticated user, it's a token usually checked server side (the signature and the fields of the JWT are checked) in order to authenticate a user
The id_token will be useful server-side to identify the connected user. Checkout the Python example in step 7 :
# Get profile info from ID token
userid = credentials.id_token['sub']
email = credentials.id_token['email']
Note that there are other flow where the website send the id_token to the server, the server checks it, and authenticates the user(server doesn't care about access token/refresh token in this flow). In the case of authorization code, flow only the temporary code is shared between frontend and backend.
One more thing is about refresh_token which are token that are used to generate other access_token. Access tokens have limited lifetime (1 hour). Using grantOfflineAccess generate a code that will give you an access_token + refresh_token the first time user authenticates. It belongs to you if you want to store this refresh_token for accessing Google services in the background, it depends on your needs

Not persistent connection for flutter/dart http.Client()

I have a running django-server that works with sessions. A simple example from my views.py that should be enough to reproduce my problem is given here:
def test(request):
print("Session objects(")
for k,v in request.session.items():
print(k,v)
print(")")
request.session["a"] = "b"
So this does just print everything in the current session and after that saving some dummy-data in the session. If I do access this via my browser the first time the output is
Session objects(
)
so the session is empty just like expected. Then after refreshing the site the output is:
Session objects(
a b
)
also as expected, so everything seems to work just fine.
But now I want to use the site with my flutter app. For that I used the flutter packacke import 'package:http/http.dart' as http like this:
var client = http.Client();
String host = ...; // just the ip:port to my host
void my_request() async {
var response = await client.get(host + "/path/to/test/");
response = await client.get(host + "/path/to/test/");
}
So everything this should do is requesting my site twice just like i did before in the browser manually. But now my server just logges twice:
Session objects(
)
So obviously the client has a not persistent connection where the session is not preserved. But according to the doc https://pub.dev/packages/http this should work
If you're making multiple requests to the same server, you can keep open a persistent connection by using a Client rather than making one-off requests
is this a problem with my flutter/dart app or is the problem on my server? Is it maybe a big in the flutter package?
note: I first thought this could be a problem with csrf-authentication so deactivated it on my server, but this doesn't change anything ...

You don't need a 3rd-party library. It's a fairly small amount of code. So, after a first authorized request, server will respond with a cookie in the response headers, under the key set-cookie. The useful information is in the value of set-cookie key, from the beginning, to the 1st occurrence of the ; character (more on this later). For example, the value of set-cookie might look like this:
sessionid=asfweggv7bet4zw11wfpb4u415yx; expires=Fri, 06 Nov 2020 11:14:40 GMT;
You need to save it and use every time in your next authorized requests.
To save it, I created a method, which you should call after the first authorized response. You can call it after every response (if you have generic response handling), since it won't mess with the existing cookie if the server didn't send a new one.***
import 'package:flutter_secure_storage/flutter_secure_storage.dart';
const kCookie = 'my_fancy_cookie';
// ...
void _storeCookie(http.Response response) async {
String rawCookie = response.headers['set-cookie'];
if (rawCookie != null) {
int index = rawCookie.indexOf(';');
String cookie = (index == -1) ? rawCookie : rawCookie.substring(0, index);
await FlutterSecureStorage().write(key: kCookie, value: cookie);
}
}
And then before I send my request, I add the cookie to headers:
// endpoint, payload and accessToken are defined earlier
cookie = await FlutterSecureStorage().read(key: kCookie);
http.Response response = await http.post(
endpoint,
body: json.encode(payload),
headers: {
'Content-Type': 'application/json',
'Accept': 'application/json',
'Authorization': 'Bearer ' + accessToken,
'cookie': cookie,
}
);
Remember to clear the cookie from secure storage after logout. :)
*** - Servers might change the session id (to reduce things like clickjacking that we don't need to consider yet), so it's good to keep extracting the cookie from every response.

The browser is saving/sending the Django session cookie. The flutter app http requests are not.
"Persistent" connection does not mean it will handle cookies for you. You will need to do that yourself or find a third-party library that does it.

CSRF token not set in Django with Python Request

I'm trying to send a POST request to a Django view from an ordinary Python script using Python-Request. The django view is not #login_required, so the only thing i need to send, other than my JSON data, is a CSRF token, here is what i tried:
token = session.get('http://127.0.0.1:8000/myview/view')
data = json.dumps({'test': 'value'})
session.post('http://127.0.0.1:8000/myview/myview',
data={
'csrfmiddlewaretoken': token,
'data': data})
The django view should just receive the Request and print it to my console:
def myview(request):
if request.method == 'POST':
data = request.POST.get('data')
print(json.loads(data))
print('received.')
response = HttpResponse(get_token(request))
return response
The problem with my current code is that my console will throw a log: WARNING - Forbidden (CSRF token missing or incorrect.). I cannot use #csrf_exempt, since i need this to be as safe as possible. Any advice? Thanks in advance!

Why might a user encounter a CSRF validation failure after logging in?
For security reasons, CSRF tokens are rotated each time a user logs in. Any page
with a form generated before a login will have an old, invalid CSRF token and need to be reloaded. This might happen if a user uses the back button after a login or if they log in a different browser tab.
This also goes for cookies. After you log in, django will send a new csrf cookie to the client. This will be stored in client.cookies and replaces the old one. The django server does not keep any record of the old token, so that's why you get the "CSRF token missing or incorrect." response.
You can access the new token from request.cookies['csrftoken'] as before.
import requests
LOGIN_URL = 'http://127.0.0.1:8000/myview/view'
request = requests.session()
request.get(LOGIN_URL)
# Retrieve the CSRF token first
csrftoken = request.cookies['csrftoken']
r1 = request.post(LOGIN_URL, headers={'X-CSRFToken': csrftoken},
allow_redirects=False))
new_csrftoken = r1.cookies['csrftoken']
data = json.dumps({'test': 'value'})
payload = {'csrfmiddlewaretoken': new_csrftoken,'data':data }
In fact, you can just use the client cookie directly. This would have avoided this bug in the first place. Requests keeps track of cookies for you when you use requests.session().
try :
r2 = request.post('http://127.0.0.1:8000/myview/myview', data=payload, headers={'X-CSRFToken': r1.cookies['crsftoken']})
except :
print('error expected')

How to get Authentication header to persist

I am trying to setup token based authentication in python. using Django/DRF. but this is more about http in general i think.
When users put in username/password I return to them their token via JSON.
The client then can post the token in HTTP Header for me to check.
My problem is I want the token to persist in the header automatically, just like cookies.
When the server says "set-cookie" to browser (Chrome/FF), the browser will automatically send up the cookie without me actually doing anything. Is there something I can do with this token?
I have tried storing it in header: "Authorization", but the browser didn't return it. Is there something like "Set-Authorization"?
thanks

No, only cookies stored in browser persistently. All other headers are transient by HTTP protocol definition.