We Keep Coding

Web application using Google Calendar API

I've created a simple Python/flask website which connects to a Google Calendar via the Google Calendar API. I followed the instructions at https://developers.google.com/calendar/api/quickstart/python, and it works well...
The whole thing is internal to my company.
I copied the credentials.json and token.json along with my website to an internal device I'm using as a webserver. Again, all works fine.
Except, at some stage, I am under the impression that my credentials will expire, and the web server will need to re-authenticate with my personal google account.
How do I get around this?
Thanks

The reason your refresh token is expiring is that your app is still in the testing phase. Go to google cloud console under the consent screen and set it to production. Your tokens will stop expiring.
web app vs installed app
You should note is the sample you are following states
and uses the code for an installed application
flow = InstalledAppFlow.from_client_secrets_file(
'credentials.json', SCOPES)
So by following this example you are not creating a web application you are creating an installed application.

Using flask you can design the following scheme.
You should have a /auth endpoint that generates authorization_url that you can redirect to perform authorization in Google API.
You should have a /callback endpoint that handles requests when your authorization in Google API is completed. In this callback, you can store your credentials in a flask session.
Before making events request, you should check whether your stored credentials are still valid. If not, you should call /auth again.
In GCP app console, when you create credentials for your app you should choose "web application".
from flask import Flask, redirect, request, url_for
from google_auth_oauthlib.flow import Flow
app = Flask(__name__)
app.secret_key = os.environ.get("SECRET_KEY")
app.config["SESSION_TYPE"] = "filesystem"
# this callback URL should match one saved in GCP app console "Authorized redirection URIs" section
CALLBACK_URL = os.environ.get("CALLBACK_URL") # you can use `url_for('callback')` instead
API_CLIENT_ID = os.environ.get("API_CLIENT_ID")
API_CLIENT_SECRET = os.environ.get("API_CLIENT_SECRET")
SCOPES = ["https://www.googleapis.com/auth/calendar"]
class CalendarClient:
API_SERVICE = "calendar"
API_VERSION = "v3"
def __init__(self, client_id: str, client_secret: str, scopes: Sequence[str]):
self._client_id = client_id
self._client_secret = client_secret
self._scopes = scopes
self._client_config = {
"web": {
"client_id": client_id,
"client_secret": client_secret,
"auth_uri": "https://accounts.google.com/o/oauth2/auth",
"token_uri": "https://oauth2.googleapis.com/token",
}
}
def get_flow(self, callback_url: str) -> Flow:
return Flow.from_client_config(
self._client_config, self._scopes, redirect_uri=callback_url
)
def get_auth_url(self, callback_url: str) -> str:
flow = self.get_flow(callback_url)
auth_url, _ = flow.authorization_url(
access_type="offline", include_granted_scopes="true"
)
return auth_url
def get_credentials(self, code: str, callback_url: str) -> Credentials:
flow = self.get_flow(callback_url)
flow.fetch_token(code=code)
return flow.credentials
#app.route("/callback")
def callback():
credentials = client.get_credentials(
code=request.args.get("code"),
callback_url=CALLBACK_URL,
)
session["credentials"] = {
"token": credentials.token,
"refresh_token": credentials.refresh_token,
"token_uri": credentials.token_uri,
"client_id": credentials.client_id,
"client_secret": credentials.client_secret,
"scopes": credentials.scopes,
}
return credentials.to_json()
#app.route("/auth")
def auth():
return redirect(client.get_auth_url(CALLBACK_URL))
Full codebase: https://github.com/jorzel/flask-google-calendar

Making authorised requests from Google Cloud Build

I'm trying to set up a deployment street in Google Cloud Build. To do this, I want to:
Run unit test
Deploy to Cloud Run without traffic
Run integration tests
Migrate traffic in Cloud Run
I've got this mostly set up, but my integration tests include a couple of calls to Cloud Run to validate that authenticated calls return 200 and unauthenticated return 401. The thing I'm having difficulties with is to make signed requests from Cloud Build. When deploying by hand and running integration tests, they work, but not from Cloud Build.
Ideally, I would like to use the Cloud Build Service Account for invoking Cloud Run like I usually do in AWS, but I can't find how to get access to that from the Cloud Runner. So instead, I retrieve a credentials file from Secret Manager. This credentials file is from a newly created Service Account with Cloud Run Invoker role:
steps:
- name: gcr.io/cloud-builders/gcloud
id: get-github-ssh-secret
entrypoint: 'bash'
args: [ '-c', 'gcloud secrets version access latest --secret=name-of-secret > /root/service-account/credentials.json' ]
volumes:
- name: 'service-account'
path: /root/service-account
...
- name: python:3.8.7
id: integration-tests
entrypoint: /bin/sh
args:
- '-c'
- |-
if [ $_STAGE != "prod" ]; then
python -m pip install -r requirements.txt
python -m pytest test/integration --disable-warnings ;
fi
volumes:
- name: 'service-account'
path: /root/service-account
For the integration tests, I've created a class called Authorizer and I have __get_authorized_header_for_cloud_build and __get_authorized_header_for_cloud_build2 as attempts:
import json
import time
import urllib
from typing import Optional
import google.auth
import requests
from google import auth
from google.auth.transport.requests import AuthorizedSession
from google.oauth2 import service_account
import jwt
class Authorizer(object):
cloudbuild_credential_path = "/root/service-account/credentials.json"
# Permissions to request for Access Token
scopes = ["https://www.googleapis.com/auth/cloud-platform"]
def get_authorized_header(self, receiving_service_url) -> dict:
auth_header = self.__get_authorized_header_for_current_user() \
or self.__get_authorized_header_for_cloud_build(receiving_service_url)
return auth_header
def __get_authorized_header_for_current_user(self) -> Optional[dict]:
credentials, _ = auth.default()
auth_req = google.auth.transport.requests.Request()
credentials.refresh(auth_req)
if hasattr(credentials, "id_token"):
authorized_header = {"Authorization": f'Bearer {credentials.id_token}'}
auth_req.session.close()
print("Got auth header for current user with auth.default()")
return authorized_header
def __get_authorized_header_for_cloud_build2(self, receiving_service_url) -> dict:
credentials = service_account.Credentials.from_service_account_file(
self.cloudbuild_credential_path, scopes=self.scopes)
auth_req = google.auth.transport.requests.Request()
credentials.refresh(auth_req)
return {"Authorization": f'Bearer {credentials.token}'}
def __get_authorized_header_for_cloud_build(self, receiving_service_url) -> dict:
with open(self.cloudbuild_credential_path, 'r') as f:
data = f.read()
credentials_json = json.loads(data)
signed_jwt = self.__create_signed_jwt(credentials_json, receiving_service_url)
token = self.__exchange_jwt_for_token(signed_jwt)
return {"Authorization": f'Bearer {token}'}
def __create_signed_jwt(self, credentials_json, run_service_url):
iat = time.time()
exp = iat + 3600
payload = {
'iss': credentials_json['client_email'],
'sub': credentials_json['client_email'],
'target_audience': run_service_url,
'aud': 'https://www.googleapis.com/oauth2/v4/token',
'iat': iat,
'exp': exp
}
additional_headers = {
'kid': credentials_json['private_key_id']
}
signed_jwt = jwt.encode(
payload,
credentials_json['private_key'],
headers=additional_headers,
algorithm='RS256'
)
return signed_jwt
def __exchange_jwt_for_token(self, signed_jwt):
body = {
'grant_type': 'urn:ietf:params:oauth:grant-type:jwt-bearer',
'assertion': signed_jwt
}
token_request = requests.post(
url='https://www.googleapis.com/oauth2/v4/token',
headers={
'Content-Type': 'application/x-www-form-urlencoded'
},
data=urllib.parse.urlencode(body)
)
return token_request.json()['id_token']
So when running locally, the __get_authorized_header_for_current_user is being used and works. When running in Cloud Build, __get_authorized_header_for_cloud_build is used. But even when temporarily disabling __get_authorized_header_for_current_user and let cloudbuild_credential_path reference to a json-file on my local pc, it keep getting 401s. Even when I give the service account from the credentials-file Owner rights. Another attempt is __get_authorized_header_for_cloud_build where I try to get the token more by myself instead of a package, but still 401.
For completeness, the integration test look somewhat like this:
class NameOfViewIntegrationTestCase(unittest.TestCase):
base_url = "https://**.a.run.app"
name_of_call_url = base_url + "/name-of-call"
def setUp(self) -> None:
self._authorizer = Authorizer()
def test_name_of_call__authorized__ok_result(self) -> None:
# Arrange
url = self.name_of_call_url
# Act
response = requests.post(url, headers=self._authorizer.get_authorized_header(url))
# Arrange
self.assertTrue(response.ok, msg=f'{response.status_code}: {response.text}')
Any idea what I'm doing wrong here? Let me know if you need any clarification on something. Thanks in advance!

Firstly, your code is too complex. If you want to leverage the Application Default Credential (ADC) according with the runtime environment, only these lines are enough
from google.oauth2.id_token import fetch_id_token
from google.auth.transport import requests
r = requests.Request()
print(fetch_id_token(r,"<AUDIENCE>"))
On Google Cloud Platform, the environment service account will be used thanks to the metadata server. On your local environment, you need to set the environment variable GOOGLE_APPLICATION_CREDENTIALS with as value the path of the service account key file
Note: you can generate id_token only with service account credential (on GCP or on your environment), it's not possible with your user account
The problem here, it's that doesn't work on Cloud Build. I don't know why, but it's not possible to generate an id_token with the Cloud Build metadata server. So, I wrote an article on this with a possible workaround

Calendar events not fetching | Microsoft Graph API | Using premium personal outlook accounts

I am trying to fetch the calendar events from my personal premium outlook account. It is not a work account.
I have a office 365 subscription. Using the same, I have setup Azure actuve directory where I have added my python web application, granted all the required permissions to the app. While accessing the API via web app I am able to fetch profile details, users and all, but unable to get data related to events, calendar etc.
I am getting this error - "message": "The tenant for tenant guid \u00*******b5d-*9-4b-b1-c5c***2ec8\u0027 does not exist."
I looked at many solutions on msdn and also on stackoverflow, but everyone told to get a premium account which I did but still the issue is not resolved.
Please help resolving the same. Thankyou in advance :)
I am attaching the copy of my app.config file for your reference.
import os
CLIENT_SECRET = "client secret key"
AUTHORITY = "https://login.microsoftonline.com/tenant id"
CLIENT_ID = "client id"
REDIRECT_PATH = "/getAToken"
ENDPOINT =ENDPOINT = 'https://graph.microsoft.com/v1.0/users/{my id}/events
# I also tried 'ENDPOINT = ' 'https://graph.microsoft.com/v1.0/users/{my id}/calendar/events''
SCOPE = ["User.ReadBasic.All"]
SESSION_TYPE = "filesystem" # So token cache will be stored in server-side session

use 'Oauth' class of python to pass all your token and ADD details like client id, client secret etc.
Something like this -
(Note config files contains all my details mentioned above.)
OAUTH = OAuth(APP)
MSGRAPH = OAUTH.remote_app(
'microsoft',
consumer_key=config.CLIENT_ID,
consumer_secret=config.CLIENT_SECRET,
request_token_params={'scope': config.SCOPES},
base_url=config.RESOURCE + config.API_VERSION + '/',
request_token_url=None,
access_token_method='POST',
access_token_url=config.AUTHORITY_URL + config.TOKEN_ENDPOINT,
authorize_url=config.AUTHORITY_URL + config.AUTH_ENDPOINT)
my config file :
CLIENT_ID = 'put here'
CLIENT_SECRET = 'put here'
REDIRECT_URI = 'http://localhost:5000/login/authorized'
AUTHORITY_URL = 'https://login.microsoftonline.com/common'
AUTH_ENDPOINT = '/oauth2/v2.0/authorize'
TOKEN_ENDPOINT = '/oauth2/v2.0/token'
RESOURCE = 'https://graph.microsoft.com/'
API_VERSION = 'v1.0'
SCOPES = ['User.Read', 'Mail.Send', 'Files.ReadWrite','Calendars.Read', 'Calendars.ReadWrite']
now you can call the get events like this :
eventResponse = MSGRAPH.get('me/events',headers=request_headers()) #request_headers() return all the requeried headers
print(eventResponce.data)

AWS Custom Federation Broker: calling federation endpoint error 400 python

I'm trying to creating a URL that enables federated users to access the AWS Management Console following the
[officlal documentation][1]. I'm using Cognito with [enhanced authflow][2] in order to authenticate user with username and password. This is the code:
################## 1. LOGIN ####################
cognito = boto3.client('cognito-idp', aws_access_key_id='', aws_secret_access_key='')
response = cognito.initiate_auth(
ClientId = app_client_id,
AuthFlow = 'USER_PASSWORD_AUTH',
AuthParameters = {
"USERNAME": username,
"PASSWORD": password
},
ClientMetadata = { 'UserPoolId': user_pool_id }
)
id_token = response['AuthenticationResult']['IdToken']
################## 2. GET ID ####################
cognito_identity = boto3.client('cognito-identity', aws_access_key_id='', aws_secret_access_key='', region_name=region)
response = cognito_identity.get_id(
IdentityPoolId = identity_pool_id,
Logins = {
'cognito-idp.{}.amazonaws.com/{}'.format(region, user_pool_id) : id_token
}
)
identity_id = response['IdentityId']
################## 3. RETRIEVE CREDENTIALS ####################
response = cognito_identity.get_credentials_for_identity(
IdentityId = identity_id,
Logins = {
'cognito-idp.{}.amazonaws.com/{}'.format(region, user_pool_id) : id_token
}
)
access_key_id = response['Credentials']['AccessKeyId']
secret_key = response['Credentials']['SecretKey']
session_token = response['Credentials']['SessionToken']
For the next step (assume role and call federation endpoint) i'm not using the example in the official documentation linked above because it use boto rather than boto3. This is the code:
sts_boto_3 = boto3.client('sts', aws_access_key_id = access_key_id,
aws_secret_access_key = secret_key,
aws_session_token = session_token,
region_name = region)
response = sts_boto_3.assume_role(
RoleArn = role,
RoleSessionName = role_session_name,
)
session_id = response['Credentials']['AccessKeyId']
session_key = response['Credentials']['SecretAccessKey']
session_token = response['Credentials']['SessionToken']
session_string = '{{"sessioId" : "{}" , "sessionKey": "{}", "sessionToken" : "{}"}}'.format(session_id, session_key, session_token)
req_url = 'https://signin.aws.amazon.com/federation?Action=getSigninToken&SessionDuration={}&Session={}'.format(3600, urllib.quote_plus(session_string))
r = requests.get(req_url)
print r
The result is
<Response [503]>
What i'm wrong?
[EDIT]
There wasn't an error in session_string (sessioId instead of sessionId)
session_string = '{{"sessionId" : "{}" , "sessionKey": "{}", "sessionToken" : "{}"}}'.format(session_id, session_key, session_token)
Now the response is 400 BAD REQUEST
<Response [400]>

I've added a full example of how to set up credentials and construct a URL that gives federated users direct access to the AWS Management Console on GitHub.
Here's the salient part of the code that constructs the URL:
def construct_federated_url(assume_role_arn, session_name, issuer, sts_client):
"""
Constructs a URL that gives federated users direct access to the AWS Management
Console.
1. Acquires temporary credentials from AWS Security Token Service (AWS STS) that
can be used to assume a role with limited permissions.
2. Uses the temporary credentials to request a sign-in token from the
AWS federation endpoint.
3. Builds a URL that can be used in a browser to navigate to the AWS federation
endpoint, includes the sign-in token for authentication, and redirects to
the AWS Management Console with permissions defined by the role that was
specified in step 1.
For more information, see Enabling Custom Identity Broker Access to the AWS Console
[https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_enable-console-custom-url.html]
in the AWS Identity and Access Management User Guide.
:param assume_role_arn: The role that specifies the permissions that are granted.
The current user must have permission to assume the role.
:param session_name: The name for the STS session.
:param issuer: The organization that issues the URL.
:param sts_client: A Boto3 STS instance that can assume the role.
:return: The federated URL.
"""
response = sts_client.assume_role(
RoleArn=assume_role_arn, RoleSessionName=session_name)
temp_credentials = response['Credentials']
print(f"Assumed role {assume_role_arn} and got temporary credentials.")
session_data = {
'sessionId': temp_credentials['AccessKeyId'],
'sessionKey': temp_credentials['SecretAccessKey'],
'sessionToken': temp_credentials['SessionToken']
}
aws_federated_signin_endpoint = 'https://signin.aws.amazon.com/federation'
# Make a request to the AWS federation endpoint to get a sign-in token.
# The requests.get function URL-encodes the parameters and builds the query string
# before making the request.
response = requests.get(
aws_federated_signin_endpoint,
params={
'Action': 'getSigninToken',
'SessionDuration': str(datetime.timedelta(hours=12).seconds),
'Session': json.dumps(session_data)
})
signin_token = json.loads(response.text)
print(f"Got a sign-in token from the AWS sign-in federation endpoint.")
# Make a federated URL that can be used to sign into the AWS Management Console.
query_string = urllib.parse.urlencode({
'Action': 'login',
'Issuer': issuer,
'Destination': 'https://console.aws.amazon.com/',
'SigninToken': signin_token['SigninToken']
})
federated_url = f'{aws_federated_signin_endpoint}?{query_string}'
return federated_url

Google+ Domains API 403 Forbidden

I'm trying to create Circles with the Google+ API, but I'm kinda stuck, this is my code, it was more or less copied from the official API documentation (yes I know it doesn't create Circle, but the issue is the same)
import httplib2
from apiclient.discovery import build
from oauth2client.client import OAuth2WebServerFlow
import json
with open('client_secrets.json', 'r') as f:
json_data = json.load(f)
data = json_data['web']
CLIENT_ID = data['client_id']
CLIENT_SECRET = data['client_secret']
# List the scopes your app requires:
SCOPES = ['https://www.googleapis.com/auth/plus.me',
'https://www.googleapis.com/auth/plus.circles.write']
# The following redirect URI causes Google to return a code to the user's
# browser that they then manually provide to your app to complete the
# OAuth flow.
REDIRECT_URI = 'http://localhost/oauth2callback'
# For a breakdown of OAuth for Python, see
# https://developers.google.com/api-client-library/python/guide/aaa_oauth
# CLIENT_ID and CLIENT_SECRET come from your APIs Console project
flow = OAuth2WebServerFlow(client_id=CLIENT_ID,
client_secret=CLIENT_SECRET,
scope=SCOPES,
redirect_uri=REDIRECT_URI)
auth_uri = flow.step1_get_authorize_url()
# This command-line server-side flow example requires the user to open the
# authentication URL in their browser to complete the process. In most
# cases, your app will use a browser-based server-side flow and your
# user will not need to copy and paste the authorization code. In this
# type of app, you would be able to skip the next 3 lines.
# You can also look at the client-side and one-time-code flows for other
# options at https://developers.google.com/+/web/signin/
print 'Please paste this URL in your browser to authenticate this program.'
print auth_uri
code = raw_input('Enter the code it gives you here: ')
# Set authorized credentials
credentials = flow.step2_exchange(code)
# Create a new authorized API client.
http = httplib2.Http()
http = credentials.authorize(http)
service = build('plusDomains', 'v1', http=http)
from apiclient import errors
try:
people_service = service.people()
people_document = people_service.get(userId='me').execute()
except errors.HttpError, e:
print e.content
My output:
{
"error": {
"errors": [
{
"domain": "global",
"reason": "forbidden",
"message": "Forbidden"
}
],
"code": 403,
"message": "Forbidden"
}
}
I searched for answer, but didn't really find any. On the API console I have Google+ API and
Google+ Domains API services added also my secret and client id are okay (otherwise the whole script would fail sooner). Also the auth is successful, my app's name is shown under https://accounts.google.com/IssuedAuthSubTokens. What did I miss?

The problem lies with your REDIRECT_URI variable. When you are using OAuth 2.0 in a purely server-side flow, the redirect URI MUST be 'urn:ietf:wg:oauth:2.0:oob'.
Try changing the variable like so (and be sure to update your client ID in the API Console):
REDIRECT_URI = 'urn:ietf:wg:oauth:2.0:oob'
Edit: Also, make sure that you are making your API call for a user within a domain. The Google+ Domains API only permits API calls that are restricted to users and content within that domain.