We Keep Coding

Google Drive API for Python: how to create credential?

I was writing a Python script to automate uploading some files to Google Drive. Since I'm still a newbie Python programmer and this is an exercise as much as anything else, I started following the Google Quickstart and decided to use their quickstart.py as a basis on which to base my own script. In the part where it talks about how to create credentials for your Python script, it refers to the "Create credentials" link, at https://developers.google.com/workspace/guides/create-credentials
I follow the link, get into one of my Google Cloud projects, and try to set up the OAuth consent screen, using an "Internal" project, as they tell you... but I can't. Google says:
“Because you’re not a Google Workspace user, you can only make your
app available to external (general audience) users. ”
So I try to create an "External" project, and then proceed to create a new client ID, using a Desktop application. Then I download the JSON credentials and put them in the same folder as my Python script, as "credentials.json". I then execute the Python script in order to authenticate it: the browser opens, I log into my Google account, give it my permissions... and then the browser hangs, because it's redirecting to a localhost URL and obviously my little Python script isn't listening in my computer at all.
I believe they must have changed this recently, because a year ago I started following the same Python tutorial and could create credentials without problems, but the Google Drive API docs haven't been updated yet. So... how do I create credentials for a Python script now?
EDIT: adding here the source code for my script. As I said, it's very similar to Google's "quickstart.py":
from __future__ import print_function
import pickle
import os.path
from googleapiclient.discovery import build
from google_auth_oauthlib.flow import InstalledAppFlow
from google.auth.transport.requests import Request
from googleapiclient.errors import HttpError
# If modifying these scopes, delete the file token.pickle.
SCOPES = ['https://www.googleapis.com/auth/drive.metadata', 'https://www.googleapis.com/auth/drive']
def main():
"""Shows basic usage of the Drive v3 API.
Prints the names and ids of the first 10 files the user has access to.
"""
creds = None
# The file token.pickle stores the user's access and refresh tokens, and is
# created automatically when the authorization flow completes for the first
# time.
if os.path.exists('token_myappname.pickle'):
with open('token_myappname.pickle', 'rb') as token:
creds = pickle.load(token)
# If there are no (valid) credentials available, let the user log in.
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
flow = InstalledAppFlow.from_client_secrets_file(
'credentials.json', SCOPES)
creds = flow.run_local_server(port=0)
# Save the credentials for the next run
with open('token_myappname.pickle', 'wb') as token:
pickle.dump(creds, token)
service = build('drive', 'v3', credentials=creds)
# Call the Drive v3 API
results = service.files().list(
pageSize=10, fields="nextPageToken, files(id, name)").execute()
items = results.get('files', [])
if not items:
print('No files found.')
else:
#print(items[0])
print('Files:')
for item in items:
#print (item)
print(u'{0} {1} {2}'.format(item['name'], item['owners'], item['parents']))

I propose you to use a service account to access to the Drive.
For that, you need to share the drive (or the folder) with the service account email. And then use this code
from googleapiclient.discovery import build
import google.auth
SCOPES = ['https://www.googleapis.com/auth/drive.metadata', 'https://www.googleapis.com/auth/drive']
def main():
credentials, project_id = google.auth.default(scopes=SCOPES)
service = build('drive', 'v3', credentials=credentials)
# Call the Drive v3 API
results = service.files().list(
q=f"'1YJ6gMgACOqVVbcgKviJKtVa5ITgsI1yP' in parents",
pageSize=10, fields="nextPageToken, files(id, name, owners, parents)").execute()
items = results.get('files', [])
if not items:
print('No files found.')
else:
#print(items[0])
print('Files:')
for item in items:
#print (item)
print(u'{0} {1} {2}'.format(item['name'], item['owners'], item['parents']))
If you run your code on Google Cloud, in a compute engine instance for example, you need to customize the VM with the service account that you authorized in your drive. (Don't use the compute engine default service account, else you will need extra configuration on your VM)
If you run your script outside GCP, you need to generate a service account key file and to store it on your local server. Then, create an environment variable GOOGLE_APPLICATION_CREDENTIALS that reference the full path of the stored key file.

Aside from the other solution posted by Guillaume Blaquiere, I also found another one on my own, which I wanted to post here in case it's helpful. All I had to do is to... erm, actually read the code I was copying and pasting, in particular this line:
creds = flow.run_local_server(port=0)
I checked Google's documentation outside of the Quickstart and found in the following: https://google-auth-oauthlib.readthedocs.io/en/latest/reference/google_auth_oauthlib.flow.html
It turns out, the example code was opening a local port in my computer to listen to the request, and it wasn't working probably due to the "port 0" part, or some other network problem.
So the workaround I found was to use a different auth method found in the docs:
creds = flow.run_console()
In this case, you paste manually in the command line the auth code given to you by Google. I just tried it, and have my credentials happily stored in my local pickle file.

unable to copy a google slide file using google drive api

I want to copy an existing template ppt present in my google drive. Then I want to change the placeholder text to some other text.
here is what I am trying.
from google.oauth2 import service_account
import googleapiclient.discovery
SCOPES = (
'https://www.googleapis.com/auth/drive',
'https://www.googleapis.com/auth/presentations',
)
SERVICE_ACCOUNT_FILE = 'cred.json'
credentials = service_account.Credentials.from_service_account_file(
SERVICE_ACCOUNT_FILE, scopes=SCOPES)
SLIDES = discovery.build('slides', 'v1', credentials=credentials)
DRIVE = discovery.build('drive', 'v3', credentials=credentials)
TMPLFILE = 'title slide template'
rsp = DRIVE.files().list(q="name='%s'" % TMPLFILE).execute().get('files')[0]
print(rsp)
DATA = {'name': 'Google Slides API template DEMO'}
print('** Copying template %r as %r' % (rsp['name'], DATA['name']))
DECK_ID = DRIVE.files().copy(body=DATA, fileId=rsp['id']).execute().get('id')
print(DECK_ID)
print('** Replacing placeholder text')
reqs = [
{'replaceAllText': {
'containsText': {'text': '{{text}}'},
'replaceText': final_til[0]
}},
]
SLIDES.presentations().batchUpdate(body={'requests': reqs},
presentationId=DECK_ID).execute()
print('DONE')
But it is not working. I don't get any error. everything works fine but I don't see the new ppt.
Output:
{'kind': 'drive#file', 'id': '15mVjkrT7PkckKetK_q9aYRVxaDcwDdHpAh7xjrAWB6Q', 'name': 'title slide template', 'mimeType': 'application/vnd.google-apps.presentation'} <--- rsp
** Copying template 'title slide template' as 'Google Slides API template DEMO'
11O97tySSNaboW6YRVD62Q7HLs8aVuS2pWyLYXImdSec <-- DECK_ID
** Replacing placeholder text
DONE
If I change
SLIDES.presentations().batchUpdate(body={'requests': reqs},
presentationId=DECK_ID).execute()
to
SLIDES.presentations().batchUpdate(body={'requests': reqs},
presentationId=rsp.get('id')).execute()
then it does replace the text but in my template file which I don't want.
Why is this happening?

I believe your current situation and goal as follows.
From your script,
You are using googleapis for python.
You have already been able to use Drive API and Slides API using the service account.
an existing template ppt present is a Google Slides which is not the PowerPoint file.
Modification points:
From your script and But it is not working. I don't get any error. everything works fine but I don't see the new ppt., I understood that you might want to see the Google Slides copied by the service account at your Google Drive.
When the Google Slides is copied by the service account, the copied Google Slides is put to the Drive of the service account. The Drive of service account is different from your Google Drive. By this, you cannot see the copied Google Slides on your Drive. I thought that this might be the reason of your issue.
In order to see the Google Slides copied by the service account at your Google Drive, for example, the following workarounds can be used.
Share the copied Google Slides with your email of Google account.
In this case, you can see the shared file at the shared folder.
At first, it creates new folder in your Google Drive and share the folder with the email of the service account. And when the Google Slides is copied, it sets the shared folder as the destination folder.
Workaround 1:
In this workaround, it shares the copied Google Slides with your email of Google account. When this is reflected to your script, it becomes as follows.
Modified script:
In this case, new permission is created to the copied file using "Permissions: create".
From:
print(DECK_ID)
print('** Replacing placeholder text')
To:
print(DECK_ID)
permission = {
'type': 'user',
'role': 'writer',
'emailAddress': '####gmail.com' # <--- Please set the email of your Google account.
}
DRIVE.permissions().create(fileId=DECK_ID, body=permission).execute()
print('** Replacing placeholder text')
Workaround 2:
In this workaround, the Google Slides is copied to the shared folder in your Google Drive. Before you use this script, please create new folder and share the folder with the email of service account. When this is reflected to your script, it becomes as follows.
Modified script:
In this case, the metadata is added to the request body of DRIVE.files().copy().
From:
DATA = {'name': 'Google Slides API template DEMO'}
To:
DATA = {'name': 'Google Slides API template DEMO', 'parents': ['###']}
Please set the folder ID of shared folder to ###.
References:
Permissions: create
Files: copy

#Tanaike's answer is great, but there is one other option too:
Account Impersonation
credentials = service_account.Credentials.from_service_account_file(
SERVICE_ACCOUNT_FILE, scopes=SCOPES)
delegated_credentials = credentials.with_subject(<email>)
DRIVE = build('drive','v3', credentials = delegated_credentials)
Here is a good overview: Using OAuth 2.0 for Server to Server Applications, specifically this section goes through the code.
Remember to set Domain Wide Delegation in both the GCP console and the Admin console.
The project initialized in the GCP Cloud console has also been granted scopes from within the Admin console > Security > API Controls > Domain wide delegation > Add new
The first thing the script does is build the credentials using from_service_account_file:
credentials = service_account.Credentials.from_service_account_file(
SERVICE_ACCOUNT_FILE, scopes=SCOPES)
Then it builds the delegated credentials, that is, the user to be impersonated:
delegated_credentials = credentials.with_subject('<EMAIL>')
From there it can build the service as normal. You can save to the user's drive as if it were the user doing it themselves.
References
Service Accounts
Using OAuth 2.0 for Server to Server Applications

How to get access to shared files in a domain with a service account using the Google Drive API?

I have been trying to access some simple information on Google Shared Drive files from a Python 3.7 script:
The last time a Google Sheets file on a shared drive was modified.
I have created a service account in the GCP Drive API menu and it can access/edit/etc Google Sheets without any problem the via the Sheets API.
However, when I use the same service account for the Drive API, it does not return any info on files outside its own folder (which contains only one file: "Getting Started"). The account has access to all Cloud APIs, has Domain-wide Delegation with all scopes related to Drive API included in the API control menu in GSuite.
The email address of the service account has been properly added to all folders in the shared drive.
Any idea? Basically all I need is to know when is the last time a sheet was modified by any given user.
secret_cred_file = ...
SCOPES = ['https://www.googleapis.com/auth/drive']
credentials = service_account.Credentials.from_service_account_file(secret_cred_file, scopes=SCOPES)
service = discovery.build('drive', 'v3', credentials=credentials)
results = service.files().list(pageSize=10, fields="nextPageToken, files(id, name,modifiedTime)").execute()
items = results.get('files', [])
PS: I have seen this: Getting files from shared folder but it does not help

I was able to list shared drive files without impersonating a user by adding some parameters to the list method as stated on google documentation:
Implement shared drive support
Shared drives follow different organization, sharing, and ownership models from My Drive. If your app is going to create and manage files on shared drives, you must implement shared drive support in your app. To begin, you need to include the supportsAllDrives=true query parameter in your requests when your app performs these operations:
files.get, files.list, files.create, files.update, files.copy, files.delete, changes.list, changes.getStartPageToken, permissions.list, permissions.get, permissions.create, permissions.update, permissions.delete
Search for content on a shared drive
Use the files.list method to search for shared drives. This section covers shared drive-specific fields in the files.list method. To search for shared drive, refer to Search for files and folders.
The files.list method contains the following shared drive-specific fields and query modes:
driveId — ID of shared drive to search.
includeItemsFromAllDrives — Whether shared drive items should be included in results. If not present or set to false, then shared drive items are not returned.
corpora — Bodies of items (files/documents) to which the query applies. Supported bodies are user, domain, drive, and allDrives. Prefer user or drive to allDrives for efficiency.
supportsAllDrives — Whether the requesting application supports both My Drives and shared drives. If false, then shared drive items are not included in the response.
Example
service.files().list(includeItemsFromAllDrives=True, supportsAllDrives=True, pageSize=10, fields="nextPageToken, files(id, name,modifiedTime)").execute()
It is nice to remember that the folder or files needs to be shared with the service account.

You need to impersonate your users.
It is not possible to make an API call to get all the files in your domain in one go.
In the Service Accounts article it says:
Service accounts are not members of your Google Workspace domain, unlike user accounts. For example, if you share assets with all members in your Google Workspace domain, they will not be shared with service accounts...This doesn't apply when using domain-wide delegation, because API calls are authorized as the impersonated user, not the service account itself.
So unfortunately you can't just share a file with a service account. To get all the files in your domain you would need to:
Impersonate an admin account and get a list of all the users.
Impersonate each user and make Drive API request for each.
Here is a good quick start for the Python Library, specifically this section
Remember to set permissions in both the GCP console and the Admin console though it seems like you have done this correctly.
Example script
from google.oauth2 import service_account
from googleapiclient.discovery import build
def main():
SCOPES = ['https://www.googleapis.com/auth/drive.metadata.readonly',
'https://www.googleapis.com/auth/admin.directory.user.readonly']
SERVICE_ACCOUNT_FILE = 'credentials.json'
credentials = service_account.Credentials.from_service_account_file(
SERVICE_ACCOUNT_FILE, scopes=SCOPES)
# Admin SDK to get users
admin_delegated_credentials = credentials.with_subject('[ADMIN_EMAIL]')
admin_service = build(
'admin',
'directory_v1',
credentials=admin_delegated_credentials
)
admin_results = admin_service.users().list(customer='my_customer', maxResults=10,
orderBy='email').execute()
users = admin_results.get('users', [])
if not users:
print('No users in the domain.')
else:
for user in users:
print(u'{0} ({1})'.format(user['primaryEmail'],
user['name']['fullName']))
# Drive to get files for each user
delegated_credentials = credentials.with_subject(user['primaryEmail'])
drive_service = build(
'drive',
'v3',
credentials=delegated_credentials
)
drive_results = drive_service.files().list(
pageSize=10,
fields="nextPageToken, files(id, name,modifiedTime)"
).execute()
items = drive_results.get('files', [])
if not items:
print('No files found.')
else:
print('Files:')
for item in items:
print(u'{0} ({1})'.format(item['name'],
item['id']))
if __name__ == '__main__':
main()
Explanation
This script has two scopes:
'https://www.googleapis.com/auth/drive.metadata.readonly'
'https://www.googleapis.com/auth/admin.directory.user.readonly'
The project initialized in the GCP Cloud console has also been granted these scopes from within the Admin console > Security > API Controls > Domain wide delegation > Add new
The first thing the script does is build the credentials using from_service_account_file:
credentials = service_account.Credentials.from_service_account_file(
SERVICE_ACCOUNT_FILE, scopes=SCOPES)
Then it builds the delegated credentials, that is, the user to be impersonated:
admin_delegated_credentials = credentials.with_subject('[ADMIN_EMAIL]')
From there it can build the service as normal. It gets a list of the users, loops through the users and lists their files. You could adapt this to your needs.
References
Service Accounts
Using OAuth 2.0 for Server to Server Applications

Google sheets with python

I have more than 100 google sheets that are shared with a lot of people. I am trying to remove inactive people from the access list. Is there a way in python to extract the list of people who have contributed to the google sheet from the version history?
I used gspread library to access the sheet but not sure how to get the list of contributing users.
from oauth2client.service_account import ServiceAccountCredentials
from googleapiclient.discovery import build
scope = ['https://www.googleapis.com/auth/drive.activity.readonly']
creds = ServiceAccountCredentials.from_json_keyfile_name('accessAPI.json', scopes = scope)
drive_service = build('driveactivity', 'v2', credentials=creds)
edit_activities = drive_service.activity().query(body={"filter":"detail.action_detail_case:EDIT",
"itemName":"items/xyz",
"consolidationStrategy":"legacy"}).execute()
# Call the People API
scope = ['https://www.googleapis.com/auth/contacts.readonly']
creds = ServiceAccountCredentials.from_json_keyfile_name('accessAPI.json', scopes = scope)
service = build('people', 'v1', credentials=creds)
results = service.people().get(resourceName='people/1080745054',personFields='names,emailAddresses').execute()
Running a people ID through people API gives back the below result. It doesn't contain the email address
{'resourceName': 'people/1080745054',
'etag': '%EgcBAj0JPjcuGgQBAgUH'}
Is the output being truncated?

Approach
Using Python you can achieve this behavior passing through the Drive API and the People API.
Get EDIT activity for your google sheets using Drive Activities API
Get editors people ids from the actors object resource in the Drive Activities API response body.
Get editors email addresses from People API with the editors people ids.
List all file permissions on you google sheets with Drive API Permissions resource endpoint.
Update the permission if the user is not in the editor list according to your logic.
Here is the proposed script in pseudocode:
loop your_google_sheets:
edit_activities = drive_service.activites().query(filter="detail.action_detail_case:EDIT", itemName="items/"+your_google_sheets.id)
editors = edit_activities.get(actors_ids)
loop editors:
editors_emails += people_service.people().get(resourceName=editors.personName, personFields="emailAddresses")
file_permissions = drive_service.permissions().list(fileId=your_google_sheets.id)
loop file_permissions:
update_if_not_editor(editors_email, file_permissions.id) # Implement your own logic
References
People API get
Drive Activities API query
Drive API Permissions list
Drive API Permissions update

Google OAuth client is using the wrong project_id from the json file- Python

My Python (3.6.7) code uses oauth2client to access Google Photos APIs. It successfully authenticates, but when it tries to access the Google Photos albums, it seems to be using the username as the project_id.
from __future__ import print_function
from apiclient.discovery import build
from httplib2 import Http
from oauth2client import file, client, tools
# Setup the Photo v1 API
SCOPES = 'https://www.googleapis.com/auth/photoslibrary.readonly'
store = file.Storage('credentials.json')
creds = store.get()
if not creds or creds.invalid:
flow = client.flow_from_clientsecrets('scripts/client_id.json', SCOPES)
creds = tools.run_flow(flow, store)
service = build('photoslibrary', 'v1', http=creds.authorize(Http()))
# Call the Photo v1 API
results = service.albums().list(
pageSize=10, fields="nextPageToken,albums(id,title)").execute()
items = results.get('albums', [])
if not items:
print('No albums found.')
else:
print('Albums:')
for item in items:
print('{0} ({1})'.format(item['title'].encode('utf8'), item['id']))
When executing the above code, it prompts me the auth page. When I successfully authenticate, it shows me the following error:
HttpError 403 when requesting {URL} returned "Photos Library API has not been used in project 123456 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/photoslibrary.googleapis.com/overview?project=123456 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.">
Interestingly, the number in bold 123456 (obviously changed) is actually the first part of the client_id found in the client_id.json
But the project_id looks something like this: test1-235515
So what I got from this error is that the oauth2client client is passing the client_id instead of the project_id. So even though I have enabled the Photos API, it will never access it correctly.
Please help with this error. How can I manually change the project_id?

The project ID is different from the project number. You will be able to see both in your Google Cloud Console configuration. See this documentation for more on how to identify your projects [1].
A single Google Cloud project can have many different OAuth client IDs configured. See this documentation for information about creating OAuth client credentials [2]. You should be only have to make sure that the client you created belongs to the project for which you have enabled APIs. Going to the URL provided in the error message should take you to the right configuration page.
[1] https://cloud.google.com/resource-manager/docs/creating-managing-projects#identifying_projects
[2] https://support.google.com/cloud/answer/6158849?hl=en