We Keep Coding

Google OAuth client is using the wrong project_id from the json file- Python

My Python (3.6.7) code uses oauth2client to access Google Photos APIs. It successfully authenticates, but when it tries to access the Google Photos albums, it seems to be using the username as the project_id.
from __future__ import print_function
from apiclient.discovery import build
from httplib2 import Http
from oauth2client import file, client, tools
# Setup the Photo v1 API
SCOPES = 'https://www.googleapis.com/auth/photoslibrary.readonly'
store = file.Storage('credentials.json')
creds = store.get()
if not creds or creds.invalid:
flow = client.flow_from_clientsecrets('scripts/client_id.json', SCOPES)
creds = tools.run_flow(flow, store)
service = build('photoslibrary', 'v1', http=creds.authorize(Http()))
# Call the Photo v1 API
results = service.albums().list(
pageSize=10, fields="nextPageToken,albums(id,title)").execute()
items = results.get('albums', [])
if not items:
print('No albums found.')
else:
print('Albums:')
for item in items:
print('{0} ({1})'.format(item['title'].encode('utf8'), item['id']))
When executing the above code, it prompts me the auth page. When I successfully authenticate, it shows me the following error:
HttpError 403 when requesting {URL} returned "Photos Library API has not been used in project 123456 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/photoslibrary.googleapis.com/overview?project=123456 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.">
Interestingly, the number in bold 123456 (obviously changed) is actually the first part of the client_id found in the client_id.json
But the project_id looks something like this: test1-235515
So what I got from this error is that the oauth2client client is passing the client_id instead of the project_id. So even though I have enabled the Photos API, it will never access it correctly.
Please help with this error. How can I manually change the project_id?

The project ID is different from the project number. You will be able to see both in your Google Cloud Console configuration. See this documentation for more on how to identify your projects [1].
A single Google Cloud project can have many different OAuth client IDs configured. See this documentation for information about creating OAuth client credentials [2]. You should be only have to make sure that the client you created belongs to the project for which you have enabled APIs. Going to the URL provided in the error message should take you to the right configuration page.
[1] https://cloud.google.com/resource-manager/docs/creating-managing-projects#identifying_projects
[2] https://support.google.com/cloud/answer/6158849?hl=en

how to obtain GCR access token with python / listing docker images

The access token im getting with gcloud auth print-access-token is obviously a different access token than the one i can get with some basic python code:
export GOOGLE_APPLICATION_CREDENTIALS=/the-credentials.json
from oauth2client.client import GoogleCredentials
credentials = GoogleCredentials.get_application_default()
credentials.get_access_token()
What i am trying to do is get a token that would work with:
curl -u _token:<mytoken> https://eu.gcr.io/v2/my-project/my-docker-image/tags/list
I'd prefer not to install gcloud utility as a dependency for my app, hence my tries to obtain the access token progrmatically via oath google credentials

I know this is a very old question, but I just got faced with the exact same problem of requiring an ACCESS_TOKEN in Python and not being able to generate it, and managed to make it work.
What you need to do is to use the variable credentials.token, except it won't exist once you first create the credentials object, returning None. In order to generate a token, the credentials must be used by a google.cloud library, which in my case was done by using the googleapiclient.discovery.build method:
sqladmin = googleapiclient.discovery.build('sqladmin', 'v1beta4', credentials=credentials)
response = sqladmin.instances().get(project=PROJECT_ID, instance=INSTANCE_ID).execute()
print(json.dumps(response))
After which the ACCESS_TOKEN could be properly generated using
access_token = credentials.token
I've also tested it using google.cloud storage as a way to test credentials, and it also worked, by just trying to access a bucket in GCS through the appropriate Python library:
from google.oauth2 import service_account
from google.cloud import storage
PROJECT_ID = your_project_id_here
SCOPES = ['https://www.googleapis.com/auth/sqlservice.admin']
SERVICE_ACCOUNT_FILE = '/path/to/service.json'
credentials = service_account.Credentials.from_service_account_file(
SERVICE_ACCOUNT_FILE, scopes=SCOPES)
try:
list(storage.Client(project=PROJECT_ID, credentials=credentials).bucket('random_bucket').list_blobs())
except:
print("Failed because no bucket exists named 'random_bucket' in your project... but that doesn't matter, what matters is that the library tried to use the credentials and in doing so generated an access_token, which is what we're interested in right now")
access_token = credentials.token
print(access_token)

So I think there are a few questions:
gcloud auth print-access-token vs GoogleCredentials.get_application_default()
gcloud doesn't set application default credentials by default anymore when performing a gcloud auth login, so the access_token you're getting from gcloud auth print-access-token is going to be the one corresponding to the used you used to login.
As long as you follow the instructions to create ADC's for a service account, that account has the necessary permissions, and the environment from which you are executing the script has access to the ENV var and the adc.json file, you should be fine.
How to make curl work
The Docker Registry API specifies that a token exchange should happen, swapping your Basic auth (i.e. Authorization: Basic base64(_token:<gcloud_access_token>)) for a short-lived Bearer token. This process can be a bit involved, but is documented here under "How to authenticate" and "Requesting a Token". Replace auth.docker.io/token with eu.gcr.io/v2/token and service=registry.docker.io with service=eu.gcr.io, etc. Use curl -u oauth2accesstoken:<mytoken> here.
See also: How to list images and tags from the gcr.io Docker Registry using the HTTP API?
Avoid the question entirely
We have a python lib that might be relevant to your needs:
https://github.com/google/containerregistry

Modifying Google Calendar API to accept environment variables rather than a JSON file for Heroku

I'm pushing a Django site (python 2.7) to Heroku, however I'm getting the error:
raise InvalidClientSecretsError('File not found: "%s"' % filename)
oauth2client.clientsecrets.InvalidClientSecretsError: File not found: "calendar_secret.json"
When running a cron job, and that's because my file calendar_secret.json isn't in my repo. I purposefully didn't push it so that the data isn't known for security reasons.
My question is about how to modify Google's default code found here: https://developers.google.com/google-apps/calendar/quickstart/python, specifically the get_credentials method in order to use environment variables in either JSON or in the method directly so that I don't have to upload my calendar secrets.
Here's Google's code:
def get_credentials(self):
"""Gets valid user credentials from storage.
If nothing has been stored, or if the stored credentials are invalid,
the OAuth2 flow is completed to obtain the new credentials.
Returns:
Credentials, the obtained credential.
"""
home_dir = os.path.expanduser('~')
credential_dir = os.path.join(home_dir, '.credentials')
if not os.path.exists(credential_dir):
os.makedirs(credential_dir)
credential_path = os.path.join(credential_dir, 'json-file.json')
store = Storage(credential_path)
credentials = store.get()
if not credentials or credentials.invalid:
flow = client.flow_from_clientsecrets(CLIENT_SECRET_FILE, SCOPES)
flow.user_agent = APPLICATION_NAME
if flags:
credentials = tools.run_flow(flow, store, flags)
else: # Needed only for compatibility with Python 2.6
credentials = tools.run(flow, store)
print('Storing credentials to ' + credential_path)
return credentials
Which works great. But I'd love to either modify this method to use environment variables in JSON.
This is another take on the question: Environment Variables in Json file, however the answer isn't something that makes sense here.
Thanks in advance!

Figured it out! All you'll have to do is replace the flow code to the following:
flow = client.OAuth2WebServerFlow(client_id=YOUR-SAVED-ENV-ID-HERE,
client_secret=YOUR-SAVED-ENV-SECRET-HERE,
scope='https://www.googleapis.com/auth/calendar',
redirect_uris='YOUR-URIS-HERE')
More information here: https://developers.google.com/api-client-library/python/guide/aaa_oauth

Creating active users via Google Directory API

I am the admin for my Google domain and I am trying to automate provisioning of new users. I first attempted to do so using a command line tool called GAM.
https://github.com/jay0lee/GAM
While this tool works, at least half of the users I create are flagged for unusual activity and are created as suspended. I have tried setting the flags in a variety of combinations but am getting the same result.
I also tried to use the Directory API directly by writing a python program but all new users are still being created as suspended users. Here is my program. Can anyone shed some light on this? Thank you.
from __future__ import print_function
import httplib2
import os
from apiclient import discovery
import oauth2client
from oauth2client import client
from oauth2client import tools
try:
import argparse
flags = argparse.ArgumentParser(parents=[tools.argparser]).parse_args()
except ImportError:
flags = None
SCOPES = 'https://www.googleapis.com/auth/admin.directory.user'
CLIENT_SECRET_FILE = 'client_secret.json'
APPLICATION_NAME = 'Directory API Python Quickstart'
def get_credentials():
home_dir = os.path.expanduser('~')
credential_dir = os.path.join(home_dir, '.credentials')
if not os.path.exists(credential_dir):
os.makedirs(credential_dir)
credential_path = os.path.join(credential_dir,
'admin-directory_v1-python-quickstart.json')
store = oauth2client.file.Storage(credential_path)
credentials = store.get()
if not credentials or credentials.invalid:
flow = client.flow_from_clientsecrets(CLIENT_SECRET_FILE, SCOPES)
flow.user_agent = APPLICATION_NAME
if flags:
credentials = tools.run_flow(flow, store, flags)
else: # Needed only for compatibility with Python 2.6
credentials = tools.run(flow, store)
print('Storing credentials to ' + credential_path)
return credentials
def create_user():
credentials = get_credentials()
http = credentials.authorize(httplib2.Http())
service = discovery.build('admin', 'directory_v1', http=http)
userinfo = {'primaryEmail': 'jane#mydomain.com',
'name': { 'givenName': 'Jane', 'familyName': 'Smith' },
'password': '34gjklre304iojlo24j2kl3kdlj',}
service.users().insert(body=userinfo).execute()
if __name__ == '__main__':
create_user()
main()

Take note of the following statement from Directory API about Creating accounts:
If the Google account has purchased mail licenses, the new user
account is automatically assigned a mailbox. This assignment may take
a few minutes to be completed and activated.
So it may take sometime. Also note from the user resources that:
A new account is automatically suspended by Google until the initial
administrator login which activates the account. After the account is
activated, the account is no longer suspended.
So after creating the new account, try logging it in to officially activate it.

This issue ended up being attributed to testing on a free trial domain. Using a trial domain, both GAM and the python program above created some suspended users and some active users. However, when I tried both methods on my paid full version google domain I was able to consistently create all active users.

Upload files to a specific Google Drive folder using python

I'm creating a simple web app that saves some files in my Google Drive using the Auth2.0 authentication. Everything works well when I use the code below, but I struggle with some things here. The first one is to save the content in a different specified folder. The code below save the files in the root folder and I can't figure out why.The upload_folder_id evidently has the good value, that I omitted here.
from apiclient.discovery import build
from httplib2 import Http
from oauth2client import file, client, tools
try:
import argparse
flags = argparse.ArgumentParser(parents=[tools.argparser]).parse_args()
except ImportError:
flags = None
SCOPES = 'https://www.googleapis.com/auth/drive.file'
store = file.Storage('storage.json')
creds = store.get()
if not creds or creds.invalid:
flow = client.flow_from_clientsecrets('client_secret.json', SCOPES)
creds = tools.run_flow(flow, store, flags) \
if flags else tools.run(flow, store)
DRIVE = build('drive', 'v3', http=creds.authorize(Http()))
upload_folder_id = <my_folder_id>
FILES = (('hello.txt', None)) # More files will be added after
for filename, mimeType in FILES:
metadata = {'name': filename}
if mimeType:
metadata['mimeType'] = mimeType
''' I tried all this possibilities too
metadata['parentId'] = upload_folder_id
metadata['folderId'] = upload_folder_id
'''
metadata['parents'] = [{'id': upload_folder_id}]
res = DRIVE.files().create(body=metadata, media_body=filename).execute()
if res:
print('Uploaded "%s" (%s)' % (filename, res['mimeType']))
The second is the authentication step. The first time that I connected the application, I was redirected to a google web page where I grant the API access. Is there a way to do this without going through the web browser to validate the access? (It's fine doing that for my local tests but I'll not be able to do this in the production server).

The code below save the files in the root folder and I can't figure out why.The upload_folder_id evidently has the good value, that I omitted here.
parents is supposed to be a string array, so there's no need for the id property to be there; just specify the parent folderId and I think you're good to go
Is there a way to do this without going through the web browser to validate the access?
The consent page is critical for the authentication process of the user, I don't think scraping can be a solution here. An alternative I can think of is for you to use Service Accounts, which is a server-to-server implementation so that the users don't need to login. The drawback here though is for the user doesn't own the data, the service account does.