We Keep Coding

Retrieve files from google drive folder based on search term

I am quite new to working in google drive and I am well aware that i can't ask stackoverflow the complete example of the below scenario, however if you can direct me to something similar it would be really helpful. I am quite stuck and couldn't move forward.
I have uploaded the contents of 7-8 gb of pdf files which includes pdf, docx, ppt etc in google drive. My concern is to list all the files that contain the term queried by user. For instance, if i want to search 'computer vision using google drive api' then the results should contain the list of files that contain the term 'computer vision' .
The above scenario is possible when i type something in google drive search box and below is the screen shot.
When i type machine learning, i get list of files. How to retrieve the same results by programatically. I have read the documentation of google drive api and came across the syntac 'fulltext contains term' but then i don't know how to use it.

As you correctly said, an easy way to do this is to use the q parameter of the request, along with the fullText contains X operator. Below you can see an adaptation of the Python Quickstart from the reference that uses this feature:
from __future__ import print_function
import pickle
import os.path
from googleapiclient.discovery import build
from google_auth_oauthlib.flow import InstalledAppFlow
from google.auth.transport.requests import Request
# If modifying these scopes, delete the file token.pickle.
SCOPES = ['https://www.googleapis.com/auth/drive.metadata.readonly']
def main():
"""Shows basic usage of the Drive v3 API.
Prints the names and ids of the first 10 files the user has access to.
"""
creds = None
# The file token.pickle stores the user's access and refresh tokens, and is
# created automatically when the authorization flow completes for the first
# time.
if os.path.exists('token.pickle'):
with open('token.pickle', 'rb') as token:
creds = pickle.load(token)
# If there are no (valid) credentials available, let the user log in.
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
flow = InstalledAppFlow.from_client_secrets_file(
'credentials.json', SCOPES)
creds = flow.run_local_server(port=0)
# Save the credentials for the next run
with open('token.pickle', 'wb') as token:
pickle.dump(creds, token)
service = build('drive', 'v3', credentials=creds)
# Call the Drive v3 API
results = service.files().list(
pageSize=1000, fields="nextPageToken, files(id, name)", q="fullText contains 'computer vision'").execute()
items = results.get('files', [])
if not items:
print('No files found.')
else:
print('Files:')
for item in items:
print(u'{0} ({1})'.format(item['name'], item['id']))
if __name__ == '__main__':
main()
Notice the q parameter upon calling the service.files().list() method.
Reference
Google Drive API - Search for Files
Python Drive API v3 reference - list()

Downloading Google Documents using api

this is shareable link link to file
id = 1wzCjl51u131v1KBgpbiKLJs8DPPakhXCFosfYjp7BY0
so manage downloads documentation.
file_id = '11wzCjl51u131v1KBgpbiKLJs8DPPakhXCFosfYjp7BY0'
request = drive_service.files().export_media(fileId=file_id,
mimeType='application/pdf')
fh = io.BytesIO()
downloader = MediaIoBaseDownload(fh, request)
done = False
while done is False:
status, done = downloader.next_chunk()
print ("Download %d%%." % int(status.progress() * 100))
i saved the file as dd.py, run using f5 and got this error
line 2, in
request = drive_service.files().export_media(fileId=file_id,
NameError: name 'drive_service' is not defined

First off you cant be sure that the id you find in the shareable link is in fact the id of the file on google drive this is not always the case. Actually I have never known this to be the case
drive_service' is not defined
Second you need create the drive service and be authenticated before you can run that code. You should try following the python quickstart
from __future__ import print_function
from googleapiclient.discovery import build
from httplib2 import Http
from oauth2client import file, client, tools
# If modifying these scopes, delete the file token.json.
SCOPES = 'https://www.googleapis.com/auth/drive.metadata.readonly'
def main():
"""Shows basic usage of the Drive v3 API.
Prints the names and ids of the first 10 files the user has access to.
"""
store = file.Storage('token.json')
creds = store.get()
if not creds or creds.invalid:
flow = client.flow_from_clientsecrets('credentials.json', SCOPES)
creds = tools.run_flow(flow, store)
service = build('drive', 'v3', http=creds.authorize(Http()))
# Call the Drive v3 API
results = service.files().list(
pageSize=10, fields="nextPageToken, files(id, name)").execute()
items = results.get('files', [])
if not items:
print('No files found.')
else:
print('Files:')
for item in items:
print('{0} ({1})'.format(item['name'], item['id']))
if __name__ == '__main__':
main()

google-drive-sdk export Daily Limit unauthenticated use

I am trying to download/export a file according to the v3 example published by google. I am getting the "Daily Limit for Unauthenticated Use Exceeded. Continued use requires signup." error.
I have searched here and elsewhere and all of the links suggest I am missing setting up the credentials. However, I am building on top of the basic quickstart example and am able to list out the contents of my drive folder in this same application. And yes, I have changed the requested scope from drive.metadata.readonly to drive.readonly to support downloading. What am I missing?
from __future__ import print_function
from apiclient.discovery import build
from httplib2 import Http
from oauth2client import file, client, tools
import io
from googleapiclient.http import MediaIoBaseDownload
# Setup the Drive v3 API
SCOPES = 'https://www.googleapis.com/auth/drive.readonly'
store = file.Storage('credentials.json')
creds = store.get()
if not creds or creds.invalid:
flow = client.flow_from_clientsecrets('client_secret.json', SCOPES)
creds = tools.run_flow(flow, store)
drive_service = build('drive', 'v3', http=creds.authorize(Http()))
# Call the Drive v3 API to list first 10 items (this works)
# example from google.
results = drive_service.files().list(
pageSize=10, fields="nextPageToken, files(id, name)").execute()
items = results.get('files', [])
if not items:
print('No files found.')
else:
print('Files:')
for item in items:
print('{0} ({1})'.format(item['name'], item['id']))
# Try to download the first item (it's a google doc I can edit, this FAILS)
# code pretty much lifted from google
file_id = items[0]['id']
print (file_id)
request = drive_service.files().export_media(fileId=file_id,
mimeType='application/pdf')
fh = io.BytesIO()
downloader = MediaIoBaseDownload(fh, request)
done = False
while done is False:
status, done = downloader.next_chunk()
print ( "Download %d%%." % int(status.progress() * 100) )

Found it. The example from Google was caching the credentials file (credentials.json). When I originally ran the example the scoped permissions were not for drive.readonly, but for drive.metadata.readonly. I think when I changed them the request was not longer valid.
I deleted and credentials.json and re-ran the script (and re-approved the credentials request on my browser) and it was successful. I also ended up using the following to store the data as the BytesIO wasn't actually writing to disk.
data = drive_service.files().export(fileId=file_id,
mimeType='application/pdf').execute()
f = open('MyFile.pdf','wb')
f.write(data)
f.close()

Getting WebViewLinks with Google Drive

I've just started trying to use the Google Drive API. Using the quickstart guide I set up the authentication, I can print a list of my files and I can even make copies. All that works great, however I'm having trouble trying to access data from a file on Drive. In particular, I'm trying to get a WebViewLink, however when I call .get I receive only a small dictionary that has barely any of the file's metadata. The documentation makes it look like all the data should just be there by default but it's not appearing. I couldn't find any way to flag for requesting any additional information.
credentials = get_credentials()
http = credentials.authorize(httplib2.Http())
service = discovery.build('drive', 'v3', http=http)
results = service.files().list(fields="nextPageToken, files(id, name)").execute()
items = results.get('files', [])
if not items:
print('No files found.')
else:
print('Files:')
for item in items:
print(item['name'], item['id'])
if "Target File" in item['name']:
d = service.files().get(fileId=item['id']).execute()
print(repr(d))
This is the output of the above code: (the formatting is my doing)
{u'mimeType': u'application/vnd.google-apps.document',
u'kind': u'drive#file',
u'id': u'1VO9cC8mGM67onVYx3_2f-SYzLJPR4_LteQzILdWJgDE',
u'name': u'Fix TVP Licence Issues'}
For anyone confused about the code there is some missing that's just the basic get_credentials function from the API's quickstart page and some constants and imports. For completeness, here's all that stuff, unmodified in my code:
from __future__ import print_function
import httplib2
import os
from apiclient import discovery
import oauth2client
from oauth2client import client
from oauth2client import tools
SCOPES = 'https://www.googleapis.com/auth/drive'
CLIENT_SECRET_FILE = 'client_secret.json'
APPLICATION_NAME = 'Drive API Python Quickstart'
try:
import argparse
flags = argparse.ArgumentParser(parents=[tools.argparser]).parse_args()
except ImportError:
flags = None
def get_credentials():
"""Gets valid user credentials from storage.
If nothing has been stored, or if the stored credentials are invalid,
the OAuth2 flow is completed to obtain the new credentials.
Returns:
Credentials, the obtained credential.
"""
home_dir = os.path.expanduser('~')
credential_dir = os.path.join(home_dir, '.credentials')
if not os.path.exists(credential_dir):
os.makedirs(credential_dir)
credential_path = os.path.join(credential_dir,
'drive-python-quickstart.json')
store = oauth2client.file.Storage(credential_path)
credentials = store.get()
if not credentials or credentials.invalid:
flow = client.flow_from_clientsecrets(CLIENT_SECRET_FILE, SCOPES)
flow.user_agent = APPLICATION_NAME
if flags:
credentials = tools.run_flow(flow, store, flags)
else: # Needed only for compatibility with Python 2.6
credentials = tools.run(flow, store)
print('Storing credentials to ' + credential_path)
return credentials
So what's missing, how can I get the API to return all that extra meta data that's just not appearing right now?

You are very close. With the newer version of the Drive API v3, to retrieve other metadata properties, you will have to add the fields parameter to specify additional properties to include in a partial response.
In your case, since you are looking to retrieve the WebViewLinkproperty your request should look something similar to this:
results = service.files().list(
pageSize=10,fields="nextPageToken, files(id, name, webViewLink)").execute()
To display your items from the response:
for item in items:
print('{0} {1} {2}'.format(item['name'], item['id'], item['webViewLink']))
I also suggest try it out with the API Explorer so you can view what additional metadata properties you would like to display on your response.
Good Luck and Hope this helps ! :)

You explicitly request only the id and name fields in your files.list call. Add webViewLink to the list to results = service.files().list(fields="nextPageToken, files(id, name, webViewLink)").execute(). To retrieval all metadata files/* should be used. For more information about this performance optimizations see Working with partial resources in the Google Drive docs.

I have written a custom function to help with getting a sharable web link given a file/folder id. More information can be gotten here
def get_webViewLink_by_id(spreadsheet_id):
sharable_link_response = drive_service.files().get( fileId=spreadsheet_id, fields='webViewLink').execute()
return(sharable_link_response['webViewLink'])
print(get_webViewLink_by_id(spreadsheet_id = '10Ik3qXK4wseva20lNGUKUTBzKoywaugi6XOmRUoP-4A'))

Looking for example using MediaFileUpload

Does anyone know where I can find complete sample code for uploading a local file and getting contents with MediaFileUpload?
I really need to see both the HTML form used to post and the code to accept it. I'm pulling my hair out and so far only getting partial answers.

I found this question while trying to figure out where the heck "MediaFileUpload" came from in the Google API examples, and I eventually figured it out. Here is a more complete code example that I used to test things with Python 2.7.
You need a JSON credentials file for this code to work. This is the credentials file you get from your Google app / project / thing.
You also need a file to upload, I'm using "test.html" here in the example.
from oauth2client.service_account import ServiceAccountCredentials
from apiclient.discovery import build
from apiclient.http import MediaFileUpload
#Set up a credentials object I think
creds = ServiceAccountCredentials.from_json_keyfile_name('credentials_from_google_app.json', ['https://www.googleapis.com/auth/drive'])
#Now build our api object, thing
drive_api = build('drive', 'v3', credentials=creds)
file_name = "test"
print "Uploading file " + file_name + "..."
#We have to make a request hash to tell the google API what we're giving it
body = {'name': file_name, 'mimeType': 'application/vnd.google-apps.document'}
#Now create the media file upload object and tell it what file to upload,
#in this case 'test.html'
media = MediaFileUpload('test.html', mimetype = 'text/html')
#Now we're doing the actual post, creating a new file of the uploaded type
fiahl = drive_api.files().create(body=body, media_body=media).execute()
#Because verbosity is nice
print "Created file '%s' id '%s'." % (fiahl.get('name'), fiahl.get('id'))
A list of valid Mime Types to use in the "body" hash is available at
https://developers.google.com/drive/v3/web/mime-types
A list of valid mimetype strings for the MediaFileUpload (they'll attempt to convert your file to whatever you put here):
https://developers.google.com/drive/v3/web/integrate-open#open_files_using_the_open_with_contextual_menu

Python 2.7, resumable upload.
https://github.com/googleapis/google-api-python-client/blob/master/docs/media.md
from __future__ import print_function
import pickle
import os.path
from googleapiclient.http import MediaFileUpload
from googleapiclient.discovery import build
from google_auth_oauthlib.flow import InstalledAppFlow
from google.auth.transport.requests import Request
# If modifying these scopes, delete the file token.pickle.
SCOPES = ['https://www.googleapis.com/auth/drive']
def main():
"""Shows basic usage of the Drive v3 API.
Prints the names and ids of the first 10 files the user has access to.
"""
creds = None
# The file token.pickle stores the user's access and refresh tokens, and is
# created automatically when the authorization flow completes for the first
# time.
if os.path.exists('token.pickle'):
with open('token.pickle', 'rb') as token:
creds = pickle.load(token)
# If there are no (valid) credentials available, let the user log in.
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
flow = InstalledAppFlow.from_client_secrets_file(
'credentials.json', SCOPES)
creds = flow.run_local_server(port=0)
# Save the credentials for the next run
with open('token.pickle', 'wb') as token:
pickle.dump(creds, token)
service = build('drive', 'v3', credentials=creds)
media = MediaFileUpload(
'big.jpeg',
mimetype='image/jpeg',
resumable=True
)
request = service.files().create(
media_body=media,
body={'name': 'Big', 'parents': ['<your folder Id>']}
)
response = None
while response is None:
status, response = request.next_chunk()
if status:
print("Uploaded %d%%." % int(status.progress() * 100))
print("Upload Complete!")
if __name__ == '__main__':
main()

You won't need to post JSON yourself, the client library handles that for you.
We provide full code samples already which can be found here: https://github.com/gsuitedevs/python-samples
Also you could check the file.insert reference documentation which contains a Python sample: https://developers.google.com/drive/v2/reference/files/insert
If this does not answer what you want perhaps you could explain in more details what you want to achieve and your architecture currently in place.

I want to provide additional information on uploading to a specific drive folder. I am providing the example from my AWS Lambda with Python 3.7
Note:
You need the folder ID for your desired location. You can find this by going to your drive folder and looking for the ID in the URL in the Browser.
For example in this URL, https://drive.google.com/drive/u/0/folders/1G91IKgQqI9YgNj8Odc8SIOPHrWOjdvOO, 1G91IKgQqI9YgNj8Odc8SIOPHrWOjdvOO would be your ID.
You need to provide your service account email access to the folder to be accessed. The service account email is found in the IAM section of your Google Cloud account. Add access to your folder by going to it in Drive, clicking the "i" icon on the top right, clicking details, then manage access.
You also need the JSON file associated with the service account. Find/create this in the Service Accounts section in Google Cloud IAM on the KEYS tab. The file contains the private key for your project. Store it where your code can access.
I'm not sure which dependencies you need to install but I think
pydrive installed them all for me: pip3 install pydrive
from apiclient.discovery import build
from google.oauth2 import service_account
from googleapiclient.http import MediaFileUpload
# This provides what authority the service account has as well as the location of the JSON file containing the private key.
scopes = ['https://www.googleapis.com/auth/drive']
service_account_file = 'path/to/service_account.json'
# Create the credentials object for the service account
credentials = service_account.Credentials.from_service_account_file(service_account_file, scopes=scopes)
drive = build('drive', 'v3', credentials=credentials)
# Create the metadata for the file and upload it to the drive folder. Supply the corresponding MIME type for your file. The parents parameter is very important, this is where you supply the ID you found for your drive folder.
body = {'name': 'testfile.txt', 'mimeType': 'text/plain', 'parents': ["theStringForTheDriveFolder"]}
media = MediaFileUpload('path/to/testfile.txt', mimeType='text/plain')
drive.files().create(body=body, media_body=media).execute()
Here's the documentation I followed:
How to use service accounts to call google APIs: https://developers.google.com/identity/protocols/oauth2/service-account#python
Documentation for the Drive API V3 to upload files:
https://developers.google.com/drive/api/v3/reference/files/create
Documentation for the Google-Python API client:
https://github.com/googleapis/google-api-python-client/blob/main/docs/oauth.md
How to perform various upload types with the Drive API:
https://developers.google.com/drive/api/guides/manage-uploads#simple
How to create and use Service Accounts, including generating the JSON
file/private keys:
https://developers.google.com/identity/protocols/oauth2/service-account