I've created a simple Python/flask website which connects to a Google Calendar via the Google Calendar API. I followed the instructions at https://developers.google.com/calendar/api/quickstart/python, and it works well...
The whole thing is internal to my company.
I copied the credentials.json and token.json along with my website to an internal device I'm using as a webserver. Again, all works fine.
Except, at some stage, I am under the impression that my credentials will expire, and the web server will need to re-authenticate with my personal google account.
How do I get around this?
Thanks
The reason your refresh token is expiring is that your app is still in the testing phase. Go to google cloud console under the consent screen and set it to production. Your tokens will stop expiring.
web app vs installed app
You should note is the sample you are following states
and uses the code for an installed application
flow = InstalledAppFlow.from_client_secrets_file(
'credentials.json', SCOPES)
So by following this example you are not creating a web application you are creating an installed application.
Using flask you can design the following scheme.
You should have a /auth endpoint that generates authorization_url that you can redirect to perform authorization in Google API.
You should have a /callback endpoint that handles requests when your authorization in Google API is completed. In this callback, you can store your credentials in a flask session.
Before making events request, you should check whether your stored credentials are still valid. If not, you should call /auth again.
In GCP app console, when you create credentials for your app you should choose "web application".
from flask import Flask, redirect, request, url_for
from google_auth_oauthlib.flow import Flow
app = Flask(__name__)
app.secret_key = os.environ.get("SECRET_KEY")
app.config["SESSION_TYPE"] = "filesystem"
# this callback URL should match one saved in GCP app console "Authorized redirection URIs" section
CALLBACK_URL = os.environ.get("CALLBACK_URL") # you can use `url_for('callback')` instead
API_CLIENT_ID = os.environ.get("API_CLIENT_ID")
API_CLIENT_SECRET = os.environ.get("API_CLIENT_SECRET")
SCOPES = ["https://www.googleapis.com/auth/calendar"]
class CalendarClient:
API_SERVICE = "calendar"
API_VERSION = "v3"
def __init__(self, client_id: str, client_secret: str, scopes: Sequence[str]):
self._client_id = client_id
self._client_secret = client_secret
self._scopes = scopes
self._client_config = {
"web": {
"client_id": client_id,
"client_secret": client_secret,
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token",
}
}
def get_flow(self, callback_url: str) -> Flow:
return Flow.from_client_config(
self._client_config, self._scopes, redirect_uri=callback_url
)
def get_auth_url(self, callback_url: str) -> str:
flow = self.get_flow(callback_url)
auth_url, _ = flow.authorization_url(
access_type="offline", include_granted_scopes="true"
)
return auth_url
def get_credentials(self, code: str, callback_url: str) -> Credentials:
flow = self.get_flow(callback_url)
flow.fetch_token(code=code)
return flow.credentials
#app.route("/callback")
def callback():
credentials = client.get_credentials(
code=request.args.get("code"),
callback_url=CALLBACK_URL,
)
session["credentials"] = {
"token": credentials.token,
"refresh_token": credentials.refresh_token,
"token_uri": credentials.token_uri,
"client_id": credentials.client_id,
"client_secret": credentials.client_secret,
"scopes": credentials.scopes,
}
return credentials.to_json()
#app.route("/auth")
def auth():
return redirect(client.get_auth_url(CALLBACK_URL))
Full codebase: https://github.com/jorzel/flask-google-calendar
Related
Libs: dj-rest-auth + allauth
I. I'm trying to interact with google API with credentials that I use to obtain internal access token. I managed to obtain both code and token but can't find how to use them with google API. The last page I found inapplicable is https://github.com/googleapis/google-api-python-client/blob/main/docs/oauth.md but probably I'm missing some things.
Here's the view I'm trying to use google API in:
class CreateGoogleDoc(GenericAPIView):
...
def get(self, request):
token = request.query_params['token']
module_dir = os.path.dirname(__file__) # get current directory
file_path = os.path.join(module_dir, 'client_secret.json')
flow = Flow.from_client_secrets_file(
file_path,
scopes=SCOPES,
redirect_uri='https://example.com'
)
credentials = service_account.Credentials.from_service_account_file(file_path, scopes=SCOPES)
service = build('docs', 'v1', credentials=credentials)
document = service.documents().create().execute()
return Response([document['documentId']])
II. While I tried to swap code to internal access token class I got another error:
Error retrieving access token: `{ "error": "invalid_request", "error_description": "You can't sign in to this app because it doesn't comply with Google's OAuth 2.0 policy for keeping apps secure. You can let the app developer know that this app doesn't comply with one or more Google validation rules."}`
Here's a view that I'm using for swapping:
GoogleLogin(SocialLoginView):
adapter_class = GoogleOAuth2Adapter
callback_url = 'http://localhist:8000/dj-rest-auth/google/'
client_class = OAuth2Client
Thanks!
Offering a workaround
If you already have a token from the GET response, why are you trying to get credentials from a service account file? Probably there is some wrong configuration there, but if you already have the access token, you can just use it like below and avoid the whole service account token fetching.
from google.oauth2.credentials import Credentials
# ...
def get(self, request):
token = request.query_params['token']
credentials = Credentials(token)
service = build('docs', 'v1', credentials=credentials)
document = service.documents().create().execute()
return Response([document['documentId']])
I'm trying to set up a deployment street in Google Cloud Build. To do this, I want to:
Run unit test
Deploy to Cloud Run without traffic
Run integration tests
Migrate traffic in Cloud Run
I've got this mostly set up, but my integration tests include a couple of calls to Cloud Run to validate that authenticated calls return 200 and unauthenticated return 401. The thing I'm having difficulties with is to make signed requests from Cloud Build. When deploying by hand and running integration tests, they work, but not from Cloud Build.
Ideally, I would like to use the Cloud Build Service Account for invoking Cloud Run like I usually do in AWS, but I can't find how to get access to that from the Cloud Runner. So instead, I retrieve a credentials file from Secret Manager. This credentials file is from a newly created Service Account with Cloud Run Invoker role:
steps:
- name: gcr.io/cloud-builders/gcloud
id: get-github-ssh-secret
entrypoint: 'bash'
args: [ '-c', 'gcloud secrets version access latest --secret=name-of-secret > /root/service-account/credentials.json' ]
volumes:
- name: 'service-account'
path: /root/service-account
...
- name: python:3.8.7
id: integration-tests
entrypoint: /bin/sh
args:
- '-c'
- |-
if [ $_STAGE != "prod" ]; then
python -m pip install -r requirements.txt
python -m pytest test/integration --disable-warnings ;
fi
volumes:
- name: 'service-account'
path: /root/service-account
For the integration tests, I've created a class called Authorizer and I have __get_authorized_header_for_cloud_build and __get_authorized_header_for_cloud_build2 as attempts:
import json
import time
import urllib
from typing import Optional
import google.auth
import requests
from google import auth
from google.auth.transport.requests import AuthorizedSession
from google.oauth2 import service_account
import jwt
class Authorizer(object):
cloudbuild_credential_path = "/root/service-account/credentials.json"
# Permissions to request for Access Token
scopes = ["https://www.googleapis.com/auth/cloud-platform"]
def get_authorized_header(self, receiving_service_url) -> dict:
auth_header = self.__get_authorized_header_for_current_user() \
or self.__get_authorized_header_for_cloud_build(receiving_service_url)
return auth_header
def __get_authorized_header_for_current_user(self) -> Optional[dict]:
credentials, _ = auth.default()
auth_req = google.auth.transport.requests.Request()
credentials.refresh(auth_req)
if hasattr(credentials, "id_token"):
authorized_header = {"Authorization": f'Bearer {credentials.id_token}'}
auth_req.session.close()
print("Got auth header for current user with auth.default()")
return authorized_header
def __get_authorized_header_for_cloud_build2(self, receiving_service_url) -> dict:
credentials = service_account.Credentials.from_service_account_file(
self.cloudbuild_credential_path, scopes=self.scopes)
auth_req = google.auth.transport.requests.Request()
credentials.refresh(auth_req)
return {"Authorization": f'Bearer {credentials.token}'}
def __get_authorized_header_for_cloud_build(self, receiving_service_url) -> dict:
with open(self.cloudbuild_credential_path, 'r') as f:
data = f.read()
credentials_json = json.loads(data)
signed_jwt = self.__create_signed_jwt(credentials_json, receiving_service_url)
token = self.__exchange_jwt_for_token(signed_jwt)
return {"Authorization": f'Bearer {token}'}
def __create_signed_jwt(self, credentials_json, run_service_url):
iat = time.time()
exp = iat + 3600
payload = {
'iss': credentials_json['client_email'],
'sub': credentials_json['client_email'],
'target_audience': run_service_url,
'aud': 'https://www.googleapis.com/oauth2/v4/token',
'iat': iat,
'exp': exp
}
additional_headers = {
'kid': credentials_json['private_key_id']
}
signed_jwt = jwt.encode(
payload,
credentials_json['private_key'],
headers=additional_headers,
algorithm='RS256'
)
return signed_jwt
def __exchange_jwt_for_token(self, signed_jwt):
body = {
'grant_type': 'urn:ietf:params:oauth:grant-type:jwt-bearer',
'assertion': signed_jwt
}
token_request = requests.post(
url='https://www.googleapis.com/oauth2/v4/token',
headers={
'Content-Type': 'application/x-www-form-urlencoded'
},
data=urllib.parse.urlencode(body)
)
return token_request.json()['id_token']
So when running locally, the __get_authorized_header_for_current_user is being used and works. When running in Cloud Build, __get_authorized_header_for_cloud_build is used. But even when temporarily disabling __get_authorized_header_for_current_user and let cloudbuild_credential_path reference to a json-file on my local pc, it keep getting 401s. Even when I give the service account from the credentials-file Owner rights. Another attempt is __get_authorized_header_for_cloud_build where I try to get the token more by myself instead of a package, but still 401.
For completeness, the integration test look somewhat like this:
class NameOfViewIntegrationTestCase(unittest.TestCase):
base_url = "https://**.a.run.app"
name_of_call_url = base_url + "/name-of-call"
def setUp(self) -> None:
self._authorizer = Authorizer()
def test_name_of_call__authorized__ok_result(self) -> None:
# Arrange
url = self.name_of_call_url
# Act
response = requests.post(url, headers=self._authorizer.get_authorized_header(url))
# Arrange
self.assertTrue(response.ok, msg=f'{response.status_code}: {response.text}')
Any idea what I'm doing wrong here? Let me know if you need any clarification on something. Thanks in advance!
Firstly, your code is too complex. If you want to leverage the Application Default Credential (ADC) according with the runtime environment, only these lines are enough
from google.oauth2.id_token import fetch_id_token
from google.auth.transport import requests
r = requests.Request()
print(fetch_id_token(r,"<AUDIENCE>"))
On Google Cloud Platform, the environment service account will be used thanks to the metadata server. On your local environment, you need to set the environment variable GOOGLE_APPLICATION_CREDENTIALS with as value the path of the service account key file
Note: you can generate id_token only with service account credential (on GCP or on your environment), it's not possible with your user account
The problem here, it's that doesn't work on Cloud Build. I don't know why, but it's not possible to generate an id_token with the Cloud Build metadata server. So, I wrote an article on this with a possible workaround
I am setting up RBAC with Airflow, and testing locally to start. I have provisioned an AWS Cognito User Group via the console. Additionally, I have a webserver_config.py file I have mounted to my Airflow docker container to set up OAuth with RBAC.
Relevant section in my webserver_config.py file:
COGNITO_URL = os.getenv('COGNITO_URL')
CONSUMER_KEY = os.getenv('COGNITO_CLIENT_KEY')
SECRET_KEY = os.getenv('COGNITO_CLIENT_SECRET')
# When using OAuth Auth, uncomment to setup provider(s) info
# Google OAuth example:
OAUTH_PROVIDERS = [{
'name':'AWS Cognito',
'whitelist': ['#company.com'], # optional
'token_key':'access_token',
'icon':'fa-amazon',
'remote_app': {
'base_url': os.path.join(COGNITO_URL, 'oauth2/idpresponse'),
# 'base_url': COGNITO_URL,
'request_token_params':{
'scope': 'email profile'
},
'access_token_url': os.path.join(COGNITO_URL, 'oauth2/token'),
'authorize_url': os.path.join(COGNITO_URL, 'oauth2/authorize'),
'request_token_url': None,
'consumer_key': CONSUMER_KEY,
'consumer_secret': SECRET_KEY,
}
}]
Variables are as follows:
COGNITO_URL: The domain name I have created in the "App Integration" section of my user pool
COGNITO_CLIENT_KEY: The app client id for my app in the "App Clients" section of my user pool
COGNITO_CLIENT_SECRET: The app client secret for my app in the "App Clients" section of my user pool
In the Cognito UI, I have the following settings for my App Client:
enter image description here
Basically, I have set the endpoints as they should be on my local machine when testing. I have fiddled with both the http://localhost:8083/oauth2/idpresponse and http://localhost:8083/admin (normal home page for Airflow) routes and received the same error.
I think that the issue is that the URI the client is trying to request and the URI specified do not match. I tried following the advice at https://stackoverflow.com/a/53602884/13717098, but when I extracted that URI and saved it in the Cognito console, I continue to get the same error. I am looking for help identifying the URI needed. The request I've identified per the linked post is: /oauth2/authorize?response_type=code&client_id=269vguq386076suj80vpq4ctmj&redirect_uri=http%3A%2F%2Flocalhost%3A8083%2Foauth-authorized%2FAWS%2520Cognito&scope=email+profile&state=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJuZXh0IjpbImh0dHA6Ly9sb2NhbGhvc3Q6ODA4My9ob21lIl19.CcuxpZyuVIqW0GtnNL219Xkg1IftE0tzFiVilR6b4us I would appreciate any help with identifying the URI and/or its associated patterns.
Edited for spacing.
Flask builder library uses the name of the config object as value in redirect_uri.
Set callback value to: http://localhost:8083/oauth-authorized/AWS%20Cognito instead of http://localhost:8080/oauth2/idresponse in AWS Cognito client. This should solve the redirection issue.
The real problem will start for userinfo endpoint as AWS cognito uses OpenID auth pattern.
aws-cognito-client
EDIT
AWS Cognito has oauth2/userinfo endpoint for receiving user information. To retrieve the userinfo, you're supposed to send openid scope along with your request. Following is my webserver_config.py.
from airflow.www_rbac.security import AirflowSecurityManager
from flask_appbuilder.security.manager import AUTH_OAUTH
import os
import json
class CognitoSecurity(AirflowSecurityManager):
def oauth_user_info(self, provider, response=None):
if provider == "aws_cognito":
me = self.appbuilder.sm.oauth_remotes[provider].get("userInfo")
data = json.loads(me.raw_data)
print("User info from aws_cognito: {0}".format(data))
return {"username": data.get("username"), "email": data.get("email")}
else:
return {}
AUTH_TYPE = AUTH_OAUTH
AUTH_USER_REGISTRATION = True
AUTH_USER_REGISTRATION_ROLE = "Admin"
COGNITO_URL = ""
CONSUMER_KEY = ""
SECRET_KEY = ""
OAUTH_PROVIDERS = [{
'name':'aws_cognito',
'whitelist': ['#positsource.com'], # optional
'token_key':'access_token',
'url': COGNITO_URL,
'icon': 'fa-amazon',
'remote_app': {
'base_url': os.path.join(COGNITO_URL, 'oauth2/idpresponse'),
'request_token_params': {
'scope': 'email profile openid'
},
'access_token_url': os.path.join(COGNITO_URL, 'oauth2/token'),
'authorize_url': os.path.join(COGNITO_URL, 'oauth2/authorize'),
'request_token_url': None,
'consumer_key': CONSUMER_KEY,
'consumer_secret': SECRET_KEY,
}
}]
SECURITY_MANAGER_CLASS = CognitoSecurity
This should get the airflow webserver working with AWS cognito. Roles and permissions management can be done by you.
I am writing some code to move files over to OneDrive (enterprise account). My app is authenticated in Azure AD and should have the correct accesses (Files.ReadWrite.All in MS Graph, Sites.ReadWrite.All in Office365 SPO and User.Read in Azure AD).
The code to receive the app token works fine:
import msal
client_id = 'dc185bb*************6bcda94'
authority_host_uri = 'https://login.microsoftonline.com'
discovery_uri = 'https://api.office.com/discovery/'
client_secret = 'VsY7vV**************ToiA0='
tenant = '4a6*********************65079'
authority_uri = authority_host_uri + '/' + tenant
scopes=['https://graph.microsoft.com/.default']
app = msal.ConfidentialClientApplication(
client_id=client_id, authority=authority_uri,
client_credential=client_secret)
result = app.acquire_token_for_client(scopes=scopes)
print(result)
However, when I try to use this token with the OneDrive SDK library it seems like I am not able pass it through:
def __init__(self, http_provider, client_id=None, scopes=None, access_token=None, session_type=None, loop=None,
auth_server_url=None, auth_token_url=None):
"""Initialize the authentication provider for authenticating
requests sent to OneDrive
Args:
http_provider (:class:`HttpProviderBase<onedrivesdk.http_provider_base>`):
The HTTP provider to use for all auth requests
client_id (str): Defaults to None, the client id for your
application
scopes (list of str): Defaults to None, the scopes
that are required for your application
access_token (str): Defaults to None. Not used in this implementation.
The above is from the auth_provider.py part of the onedrivesdk, and clearly states the access_token is not used in the implementation.
Is there another way around this? Or other libraries to use?
You could try to use this Authentication of OneDrive for Business.
import onedrivesdk
from onedrivesdk.helpers import GetAuthCodeServer
from onedrivesdk.helpers.resource_discovery import ResourceDiscoveryRequest
redirect_uri = 'http://localhost:8080'
client_id = your_client_id
client_secret = your_client_secret
discovery_uri = 'https://api.office.com/discovery/'
auth_server_url='https://login.microsoftonline.com/common/oauth2/authorize'
auth_token_url='https://login.microsoftonline.com/common/oauth2/token'
http = onedrivesdk.HttpProvider()
auth = onedrivesdk.AuthProvider(http,
client_id,
auth_server_url=auth_server_url,
auth_token_url=auth_token_url)
auth_url = auth.get_auth_url(redirect_uri)
code = GetAuthCodeServer.get_auth_code(auth_url, redirect_uri)
auth.authenticate(code, redirect_uri, client_secret, resource=discovery_uri)
# If you have access to more than one service, you'll need to decide
# which ServiceInfo to use instead of just using the first one, as below.
service_info = ResourceDiscoveryRequest().get_service_info(auth.access_token)[0]
auth.redeem_refresh_token(service_info.service_resource_id)
client = onedrivesdk.OneDriveClient(service_info.service_resource_id + '/_api/v2.0/', auth, http)
Upload an Item:
returned_item = client.item(drive='me', id='root').children['newfile.txt'].upload('./path_to_file.txt')
For more examples, you can refer to this link.
I'm trying to create Circles with the Google+ API, but I'm kinda stuck, this is my code, it was more or less copied from the official API documentation (yes I know it doesn't create Circle, but the issue is the same)
import httplib2
from apiclient.discovery import build
from oauth2client.client import OAuth2WebServerFlow
import json
with open('client_secrets.json', 'r') as f:
json_data = json.load(f)
data = json_data['web']
CLIENT_ID = data['client_id']
CLIENT_SECRET = data['client_secret']
# List the scopes your app requires:
SCOPES = ['https://www.googleapis.com/auth/plus.me',
'https://www.googleapis.com/auth/plus.circles.write']
# The following redirect URI causes Google to return a code to the user's
# browser that they then manually provide to your app to complete the
# OAuth flow.
REDIRECT_URI = 'http://localhost/oauth2callback'
# For a breakdown of OAuth for Python, see
# https://developers.google.com/api-client-library/python/guide/aaa_oauth
# CLIENT_ID and CLIENT_SECRET come from your APIs Console project
flow = OAuth2WebServerFlow(client_id=CLIENT_ID,
client_secret=CLIENT_SECRET,
scope=SCOPES,
redirect_uri=REDIRECT_URI)
auth_uri = flow.step1_get_authorize_url()
# This command-line server-side flow example requires the user to open the
# authentication URL in their browser to complete the process. In most
# cases, your app will use a browser-based server-side flow and your
# user will not need to copy and paste the authorization code. In this
# type of app, you would be able to skip the next 3 lines.
# You can also look at the client-side and one-time-code flows for other
# options at https://developers.google.com/+/web/signin/
print 'Please paste this URL in your browser to authenticate this program.'
print auth_uri
code = raw_input('Enter the code it gives you here: ')
# Set authorized credentials
credentials = flow.step2_exchange(code)
# Create a new authorized API client.
http = httplib2.Http()
http = credentials.authorize(http)
service = build('plusDomains', 'v1', http=http)
from apiclient import errors
try:
people_service = service.people()
people_document = people_service.get(userId='me').execute()
except errors.HttpError, e:
print e.content
My output:
{
"error": {
"errors": [
{
"domain": "global",
"reason": "forbidden",
"message": "Forbidden"
}
],
"code": 403,
"message": "Forbidden"
}
}
I searched for answer, but didn't really find any. On the API console I have Google+ API and
Google+ Domains API services added also my secret and client id are okay (otherwise the whole script would fail sooner). Also the auth is successful, my app's name is shown under https://accounts.google.com/IssuedAuthSubTokens. What did I miss?
The problem lies with your REDIRECT_URI variable. When you are using OAuth 2.0 in a purely server-side flow, the redirect URI MUST be 'urn:ietf:wg:oauth:2.0:oob'.
Try changing the variable like so (and be sure to update your client ID in the API Console):
REDIRECT_URI = 'urn:ietf:wg:oauth:2.0:oob'
Edit: Also, make sure that you are making your API call for a user within a domain. The Google+ Domains API only permits API calls that are restricted to users and content within that domain.