We Keep Coding

Python: googleapiclient Returning "insufficient authentication scopes" When Attempting to Create Secondary Calendar with Refreshed Credentials

While working with the Python googleapiclient module, I am noticing some interesting behavior that I have not been able to track down.
The Issue:
When I receive a user's credentials from Google after successfully following the OAuth2.0 flow, I am able to programmatically create a secondary calendar for the user and add events to that calendar. After these credentials expire, I am able to refresh the credentials using the offline/refresh token and can continue to lookup and add more events to the secondary calendar that was created. However, I am no longer able to create any additional secondary calendars after the credentials have been refreshed. When I attempt to do so, I receive the following error:
googleapiclient.errors.HttpError: <HttpError 403 when requesting https://www.googleapis.com/calendar/v3/calendars?alt=json returned "Request had insufficient authentication scopes.">
The authorization scopes that I am using throughout the entire process are:
SCOPES = ['https://www.googleapis.com/auth/calendar.app.created',
'https://www.googleapis.com/auth/calendar.calendarlist.readonly']
If, after receiving this error, I then have the user successfully go through the OAuth2.0 flow again, I am able to use those credentials to create another secondary calendar as expected. That being said, this is not ideal for my application as the user's credentials expire after a few hours and it will often be days before the application needs to create new events/calendars.
Troubleshooting:
I have tried passing my developer key to the googleapiclient.discovery.build function in addition to the user's credentials to give it an additional authentication method, however when I do, I then receive the following error:
googleapiclient.errors.HttpError: <HttpError 401 when requesting https://www.googleapis.com/calendar/v3/calendars?key=[REDACTED]&alt=json returned "Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.">
Researching this issue, I have found a number of sources that suggest deleting the token.pickle and refreshing the credentials.json and running through the process again. I do not use either of these files but instead store the pickled data in a DB and keep the app credentials in the environment. I have cleared out the user's credentials and made sure that my app credentials are correct.
Coding Example:
# Check to see if the credentials have expired
if not client_creds.valid:
# If the credentials are expired and can be refreshed,
# then refresh the credentials
if client_creds.expired and client_creds.refresh_token:
client_creds.refresh(Request())
retStatus = 0
# Use the refreshed credentials to build a Google Calendar v3 service
service = build("calendar", "v3",
credentials=client_creds,
developerKey=os.environ["GCAL_APIKEY"])
# Create a body for the new calendar
newCalBody = {
"summary": "Example Summary",
"description": "Example Description"
}
# Attempt to create the secondary calendar for the user
created_calendar = service.calendars().insert(body=newCalBody).execute()
# At this step, the 40x error will occur
Any help would be appreciated! Thanks in advance!

If you check the documentation for Calendar.linsert you will notice that it requires the following scope.
The error message
Request had insufficient authentication scopes
Means that your applcaiton does not have the consent of the user to prform this action beouse you have only requested the following scopes
https://www.googleapis.com/auth/calendar.app.created
https://www.googleapis.com/auth/calendar.calendarlist.readonly
Fix your scopes and then make sure that you reauthorize the user, the consent screen must popup.

"Unauthorized WebHook callback channel" Google Api response even if Heroku web application domain was verified

I have recently deployed python web application to Heroku platform that takes advantage of Google-calendar API. I was able to gain events list from calendar using OAuth 2.0 credentials but web application domain verification isn't required in such cases in general. In opposite was with push-notification feature that requires either domain verification and HTTPS request sent to Google API to activate it.
When I try to execute the HTTPS request using watch metod
with code:
body={
"id": "<specified_uuid_read_from_file>",
"type": "web_hook",
"address": "https://<heroku_application_id>.herokuapp.com"
}
events_result = service.events().watch(calendarId="primary", body=body).execute()
I get this response:
googleapiclient.errors.HttpError: <HttpError 401 when requesting https://www.googleapis.com/calendar/v3/calendars/primary/events/watch?alt=json returned "Unauthorized WebHook callback channel: https://<heroku_application_id>.herokuapp.com">
I have already done:
Verified domain https://<heroku_application_id>.herokuapp.com via Google Search Console by uploading html file.
Added this domain to Google Api "Domain verification" tab.
I would like to know what could be other possible resason than domain verification issues resulted from inappropriate Google authoriztion process and how to fix it?
Thank you in advance.
EDIT:
Eventually I was able to gain access by renewing Google API credentials. Everything works fine for now. Nevertheless I can't still explain what could be actual problem which caused this error.

Getting 401 Client Error when using firebase

I'm new to firebase and I'm trying to update some data in an existing project but I'm getting the following error: 401 Client Error: Unauthorized for url. So how to fix this problem and thank you! I just want to mention also that I have been added to the project, so I'm not the owner.
this is my code:
from firebase import firebase
firebase = firebase.FirebaseApplication("the http path", None) # None bcz we are testing
firebase.put("/esco-lebanon/device-configs/atest-dev", "brightness", 50)
print("Updated")

According to the documentation, a 401 error means one of the following:
The auth token has expired.
The auth token used in the request is invalid.
Authenticating with an access_token failed.
The request violates your Firebase Realtime Database Rules.
The understanding here is that your client code needs to correctly identify a Firebase Authentication user with a token in access_token, and that user account must have access to read the data in the database according to its security rules. Since you haven't provided a token, your access is coming in anonymously, so you could only query data where security rules allow universal access.
If you haven't investigated using authentication in your request, you should read the documentation about that.

Insufficient Permission Error 403 : Google Drive Upload (Python)

I am trying to access Google Drive using the Drive API Version 3 (Python). Listing the files seems to work fine. I get insufficient Permission error when I try to upload a file.
I changed My scope to give full permission to my script
SCOPES = 'https://www.googleapis.com/auth/drive'
Below is the block that I use to create the file
file_metadata = {
'name': 'Contents.pdf',
'mimeType': 'application/vnd.google-apps.file'
}
media = MediaFileUpload('Contents.pdf',
mimetype='application/vnd.google-apps.file',
resumable=True)
file = service.files().create(body=file_metadata,
media_body=media,
fields='id').execute()
print ('File ID: %s' % file.get('id'))
I get this error message:
ResumableUploadError: HttpError 403 "Insufficient Permission"
I am not sure what is wrong here.

I think that your script works fine. From the error you show, I thought the requirement of reauthorize of access token and refresh token. So please try a following flow.
When you authorize using client_secret.json, a credential JSON file is created. At the default Quickstart, it is created in .credentials of your home directory.
For your current situation, please delete your current the credential JSON file which is not client_secret.json, and reauthorize by launching your script. The default file name of Quickstart is drive-python-quickstart.json.
By this, scope of https://www.googleapis.com/auth/drive is reflected to access token and refresh token, and they are used for uploading process. When the error occurs even if this flow is done, please confirm whether Drive API is enabled at API console, again.
If this was not useful for you, I'm sorry.

Maybe you already have a file with the same name there?

Getting user info with Cloud Endpoints (using other API Endpoints)

I'm trying to setup endpoints api (with google app engine, python), but I'm having some trouble getting user profile info. API is working, I can create entities through API Explorer on my localhost.
My goal is to allow user to register for my app by providing just an email, and authorizing the app to get the reset of the info from their profile. I have this endpoints method:
#User.method(http_method="POST",
auth_level=endpoints.AUTH_LEVEL.REQUIRED,
allowed_client_ids=[
endpoints.API_EXPLORER_CLIENT_ID
],
scopes=[
'https://www.googleapis.com/auth/userinfo.email',
'https://www.googleapis.com/auth/userinfo.profile',
'https://www.googleapis.com/auth/plus.me',
],
user_required=True,
request_fields=('email',),
response_fields=('id',),
name="register",
path="users")
def UserRegister(self, instance):
logging.info(os.getenv( 'HTTP_AUTHORIZATION' ))
# 'Beared __TOKEN__'
logging.info(endpoints.users_id_token._get_token(None))
# '__TOKEN__'
instance.put()
return instance
This works fine, I receive authorization token and user is created in datastore, but I can't figure out how to get the profile info. If I enter the token in OAuth2 API (through API Explorer):
POST https://www.googleapis.com/oauth2/v2/tokeninfo?access_token=__TOKEN__
I get token info with some data I need { "user_id": "__ID__", "verified_email": true, ...}, and if I use user_id in +API:
GET https://www.googleapis.com/plus/v1/people/__ID__
I can get the rest of the data I need (name, image, etc).
What do I need to do to achieve this in my UserRegister() method? I'd prefer to return just entity ID and do the rest of registration asynchronously, but that's another issue, I'll figure it out (; Just need some guidance how to call other endpoints from my code...
EDIT:
I've managed to figure out how to call other APIs (code on Gist), now only have one issue with Plus API:
I did some queries and eventually got anonymous quota error. Then I added key parameter and set it to WEB_CLIENT_ID or SERVICE_ACCOUNT:
WEB_CLIENT_ID is OAuth2 Client ID (type: Web Application) from console.developers.google.com/apis/credentials,
SERVICE_ACCOUNT is default App Engine service account - MY_APP#appspot.gserviceaccount.com...
and now I'm getting following error:
HttpError: <HttpError 400 when requesting https://www.googleapis.com/plus/v1/people/__VALID_USER_ID__?key=__WEB_CLIENT_ID__or__SERVICE_ACCOUNT__&alt=json returned "Bad Request">
When I use +API explorer I get results as expected:
REQUEST:
https://www.googleapis.com/plus/v1/people/__VALID_USER_ID__?key={YOUR_API_KEY}
RESPONSE:
200 OK + json data for user...
Anyone knows why is this happening?

Why am I getting BadRequest response?
Problem with BadRequest was that I didn't send authorization token... I did try to send it as access_token, but seams like +api docs are outdated - it should be oauth_token. When I included this parameter issue was resolved:
build('plus', 'v1').people().get(userId=user_id, key=SERVICE_ACCOUNT, oauth_token=token).execute()
HINT: Use http://localhost:8001/_ah/api/discovery/v1/apis/, and discoveryRestUrl property it has to see real properties of your API - this is where I found the answer.
oauth_token can be obtained like this:
token = os.getenv('HTTP_AUTHORIZATION').split(" ")[1]
# or like in my question:
token = endpoints.users_id_token._get_token(None)
I'd suggest HTTP_AUTHORIZATION variable, because users_id_token docs state that it's a:
Utility library for reading user information from an id_token.
This is an experimental library that can temporarily be used to extract
a user from an id_token. The functionality provided by this library
will be provided elsewhere in the future.
How to call other API Endpoints?
This is also an answer to my first question:
from googleapiclient.discovery import build
service = build('plus', 'v1')
request = service.people().get(userId=user_id, key=SERVICE_ACCOUNT, oauth_token=token)
response = request.execute()
data = dict(self.response.POST)
Code that worked for me is here.
NOTE: WEB_CLIENT_ID obtained from https://console.developers.google.com/apis/credentials (OAuth2 Client ID of type Web Application) will NOT work in this case. I had to use SERVICE_ACCOUNT - I didn't try to generate one through console, default service account I got from App Engine worked fine.
...things are much clearer now that I got this working. Hope it will help someone else (;