This is my first time posting here so i apologize if i have not posted in the correct format!
from google.auth.transport.requests import Request
from google.oauth2.credentials import Credentials
from google_auth_oauthlib.flow import InstalledAppFlow
from googleapiclient.discovery import build
from googleapiclient.errors import HttpError
# If modifying these scopes, delete the file token.json.
SCOPES = ['https://www.googleapis.com/auth/drive.metadata.readonly']
def main():
"""Shows basic usage of the Drive v3 API.
Prints the names and ids of the first 10 files the user has access to.
"""
creds = None
# The file token.json stores the user's access and refresh tokens, and is
# created automatically when the authorization flow completes for the first
# time.
if os.path.exists('token.json'):
creds = Credentials.from_authorized_user_file('token.json', SCOPES)
# If there are no (valid) credentials available, let the user log in.
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
flow = InstalledAppFlow.from_client_secrets_file(
'credentials.json', SCOPES)
creds = flow.run_local_server(port=0)
# Save the credentials for the next run
with open('token.json', 'w') as token:
token.write(creds.to_json())
try:
service = build('drive', 'v3', credentials=creds)
# Call the Drive v3 API
results = service.files().list(
pageSize=10, fields="nextPageToken, files(id, name)").execute()
items = results.get('files', [])
if not items:
print('No files found.')
return
print('Files:')
for item in items:
print(u'{0} ({1})'.format(item['name'], item['id']))
except HttpError as error:
# TODO(developer) - Handle errors from drive API.
print(f'An error occurred: {error}')
if __name__ == '__main__':
main()
I'm trying to create a simple python script that does the following.
1.Import the necessary Libraries.
2.Connect/Authenticate to Google drive.
3.Check a folder on google drive for file names and then upload all files that are not on the drive yet from a
specific folder on computer to the checked folder on google drive.
4.repeat every hour, If no new files found then restart loop.
this is what i tried so far but i get errors when running the script, The issue is it cannot find the credentials.json file.
The credentials.json file is in the same folder as the settings.yaml and the Setup.py
I was expecting to be directed to a login page when running the script.
To be fair i know very very little about python and the most experience i have with coding would be from my old modded minecraft days using lua. on computercraft. hahah thanks for your help in advance.**
Related
headers = {"Authorization": "Bearer " + ACCESS_TOKEN}
folder=Folder_id // google drive folder Id
para = {"title": assignment_file_name,
"parents": [{"id": "root"}, {'id': folder}]}
files = {
"data": ("metadata", json.dumps(para), "application/json; charset=UTF-8"),
"file": assignment_file.stream.read()
}
response = requests.get("https://drive.google.com/drive/folders/"+Folder_id,
headers= headers, files=files)
I want to upload file fetched from requests to google drive folder.
But From where to get this ACCESS_TOKEN in the headers variable?
The access token is the result of the Oauth2 request. When you want to access private user data you must first ask for the users consent to your application accessing their data. To do this we use Oauth2. If you are interested in knowing how it works i have a video on it Understanding Oauth2 with curl..
Rather then coding all this manually you should consider using the Google Api python client library. As shown in the official Python quickstart for Drive api will walk you though how to create credentials (be sure to enable drive api) for your application and then how to request access of the user.
from __future__ import print_function
import os.path
from google.auth.transport.requests import Request
from google.oauth2.credentials import Credentials
from google_auth_oauthlib.flow import InstalledAppFlow
from googleapiclient.discovery import build
from googleapiclient.errors import HttpError
# If modifying these scopes, delete the file token.json.
SCOPES = ['https://www.googleapis.com/auth/drive.metadata.readonly']
def main():
"""Shows basic usage of the Drive v3 API.
Prints the names and ids of the first 10 files the user has access to.
"""
creds = None
# The file token.json stores the user's access and refresh tokens, and is
# created automatically when the authorization flow completes for the first
# time.
if os.path.exists('token.json'):
creds = Credentials.from_authorized_user_file('token.json', SCOPES)
# If there are no (valid) credentials available, let the user log in.
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
flow = InstalledAppFlow.from_client_secrets_file(
'credentials.json', SCOPES)
creds = flow.run_local_server(port=0)
# Save the credentials for the next run
with open('token.json', 'w') as token:
token.write(creds.to_json())
try:
service = build('drive', 'v3', credentials=creds)
# Call the Drive v3 API
results = service.files().list(
pageSize=10, fields="nextPageToken, files(id, name)").execute()
items = results.get('files', [])
if not items:
print('No files found.')
return
print('Files:')
for item in items:
print(u'{0} ({1})'.format(item['name'], item['id']))
except HttpError as error:
# TODO(developer) - Handle errors from drive API.
print(f'An error occurred: {error}')
if __name__ == '__main__':
main()
I just start learning the python scripting and I created a script using pydrive and the function is uploading all files from local folder (linux OS) to google drive but I'm planning to modify the script for my automation and add the function that can upload only the most recent file added to the local folder with no reuploading of all the files inside the folder, may I know if this is possible with python script alone?
Thank you in advance!
You dont need to use pydrive. You can use the Google api python client library directly. As far as i know pydrive does use the client library internally. There's a starter example here
Quick start python
from __future__ import print_function
import os.path
from google.auth.transport.requests import Request
from google.oauth2.credentials import Credentials
from google_auth_oauthlib.flow import InstalledAppFlow
from googleapiclient.discovery import build
from googleapiclient.errors import HttpError
# If modifying these scopes, delete the file token.json.
SCOPES = ['https://www.googleapis.com/auth/drive.metadata.readonly']
def main():
"""Shows basic usage of the Drive v3 API.
Prints the names and ids of the first 10 files the user has access to.
"""
creds = None
# The file token.json stores the user's access and refresh tokens, and is
# created automatically when the authorization flow completes for the first
# time.
if os.path.exists('token.json'):
creds = Credentials.from_authorized_user_file('token.json', SCOPES)
# If there are no (valid) credentials available, let the user log in.
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
flow = InstalledAppFlow.from_client_secrets_file(
'credentials.json', SCOPES)
creds = flow.run_local_server(port=0)
# Save the credentials for the next run
with open('token.json', 'w') as token:
token.write(creds.to_json())
try:
service = build('drive', 'v3', credentials=creds)
# Call the Drive v3 API
results = service.files().list(
pageSize=10, fields="nextPageToken, files(id, name)").execute()
items = results.get('files', [])
if not items:
print('No files found.')
return
print('Files:')
for item in items:
print(u'{0} ({1})'.format(item['name'], item['id']))
except HttpError as error:
# TODO(developer) - Handle errors from drive API.
print(f'An error occurred: {error}')
if __name__ == '__main__':
main()
Manage uploads
file_metadata = {'name': 'photo.jpg'}
media = MediaFileUpload('files/photo.jpg', mimetype='image/jpeg')
file = drive_service.files().create(body=file_metadata,
media_body=media,
fields='id').execute()
print 'File ID: %s' % file.get('id')
For example, in my Google Drive, I have a directory called raw_pdf, is it possible to list all the files in that directory using Google Drive API?
Using the Q paramater which is part of files.list allows you to do a Files search
parents in Whether the parents collection contains the specified ID.
by sending something like
parents in 1234
where 1234 is equal to the file id of your raw_pdf directory
I recommend following the official Python quick start example which shows how to authenticate your application and how to use file.list you will just need to then add the q parameter to the request.
from __future__ import print_function
import pickle
import os.path
from googleapiclient.discovery import build
from google_auth_oauthlib.flow import InstalledAppFlow
from google.auth.transport.requests import Request
# If modifying these scopes, delete the file token.pickle.
SCOPES = ['https://www.googleapis.com/auth/drive.metadata.readonly']
def main():
"""Shows basic usage of the Drive v3 API.
Prints the names and ids of the first 10 files the user has access to.
"""
creds = None
# The file token.pickle stores the user's access and refresh tokens, and is
# created automatically when the authorization flow completes for the first
# time.
if os.path.exists('token.pickle'):
with open('token.pickle', 'rb') as token:
creds = pickle.load(token)
# If there are no (valid) credentials available, let the user log in.
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
flow = InstalledAppFlow.from_client_secrets_file(
'credentials.json', SCOPES)
creds = flow.run_local_server(port=0)
# Save the credentials for the next run
with open('token.pickle', 'wb') as token:
pickle.dump(creds, token)
service = build('drive', 'v3', credentials=creds)
# Call the Drive v3 API
results = service.files().list(
pageSize=10, fields="nextPageToken, files(id, name)").execute()
items = results.get('files', [])
if not items:
print('No files found.')
else:
print('Files:')
for item in items:
print(u'{0} ({1})'.format(item['name'], item['id']))
if __name__ == '__main__':
main()
I am quite new to working in google drive and I am well aware that i can't ask stackoverflow the complete example of the below scenario, however if you can direct me to something similar it would be really helpful. I am quite stuck and couldn't move forward.
I have uploaded the contents of 7-8 gb of pdf files which includes pdf, docx, ppt etc in google drive. My concern is to list all the files that contain the term queried by user. For instance, if i want to search 'computer vision using google drive api' then the results should contain the list of files that contain the term 'computer vision' .
The above scenario is possible when i type something in google drive search box and below is the screen shot.
When i type machine learning, i get list of files. How to retrieve the same results by programatically. I have read the documentation of google drive api and came across the syntac 'fulltext contains term' but then i don't know how to use it.
As you correctly said, an easy way to do this is to use the q parameter of the request, along with the fullText contains X operator. Below you can see an adaptation of the Python Quickstart from the reference that uses this feature:
from __future__ import print_function
import pickle
import os.path
from googleapiclient.discovery import build
from google_auth_oauthlib.flow import InstalledAppFlow
from google.auth.transport.requests import Request
# If modifying these scopes, delete the file token.pickle.
SCOPES = ['https://www.googleapis.com/auth/drive.metadata.readonly']
def main():
"""Shows basic usage of the Drive v3 API.
Prints the names and ids of the first 10 files the user has access to.
"""
creds = None
# The file token.pickle stores the user's access and refresh tokens, and is
# created automatically when the authorization flow completes for the first
# time.
if os.path.exists('token.pickle'):
with open('token.pickle', 'rb') as token:
creds = pickle.load(token)
# If there are no (valid) credentials available, let the user log in.
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
flow = InstalledAppFlow.from_client_secrets_file(
'credentials.json', SCOPES)
creds = flow.run_local_server(port=0)
# Save the credentials for the next run
with open('token.pickle', 'wb') as token:
pickle.dump(creds, token)
service = build('drive', 'v3', credentials=creds)
# Call the Drive v3 API
results = service.files().list(
pageSize=1000, fields="nextPageToken, files(id, name)", q="fullText contains 'computer vision'").execute()
items = results.get('files', [])
if not items:
print('No files found.')
else:
print('Files:')
for item in items:
print(u'{0} ({1})'.format(item['name'], item['id']))
if __name__ == '__main__':
main()
Notice the q parameter upon calling the service.files().list() method.
Reference
Google Drive API - Search for Files
Python Drive API v3 reference - list()
$ pip3 list | grep googl
google-api-python-client 1.7.9
google-auth 1.6.3
google-auth-httplib2 0.0.3
google-auth-oauthlib 0.4.0
I can successfully list files shared to me. But I get "File not found" error when I try to download an existing file by its id. How to download a file by its id?
Script to list files
from __future__ import print_function
import pickle
import os.path
import io
import sys
from googleapiclient.discovery import build
from googleapiclient.http import MediaIoBaseDownload
from google_auth_oauthlib.flow import InstalledAppFlow
from google.auth.transport.requests import Request
# If modifying these scopes, delete the file token.pickle.
SCOPES = ['https://www.googleapis.com/auth/drive.metadata.readonly']
TOKEN_FILE = 'tockenRead.pickle'
def main():
"""Shows basic usage of the Drive v3 API.
Prints the names and ids of the first 10 files the user has access to.
"""
creds = None
# The file token.pickle stores the user's access and refresh tokens, and is
# created automatically when the authorization flow completes for the first
# time.
if os.path.exists(TOKEN_FILE):
with open(TOKEN_FILE, 'rb') as token:
creds = pickle.load(token)
# If there are no (valid) credentials available, let the user log in.
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
flow = InstalledAppFlow.from_client_secrets_file(
'credentials.json', SCOPES)
creds = flow.run_local_server()
# Save the credentials for the next run
with open(TOKEN_FILE, 'wb') as token:
pickle.dump(creds, token)
service = build('drive', 'v3', credentials=creds)
# Call the Drive v3 API
results = service.files().list(
q="mimeType != 'application/vnd.google-apps.folder'",
pageSize=10,
fields="nextPageToken, files(id, name)"
).execute()
items = results.get('files', [])
if not items:
print('No files found.')
else:
print('Files:')
for item in items:
print(u'{0} ({1})'.format(item['name'], item['id']))
if __name__ == '__main__':
main()
Result
$ python3 list_files.py
Files:
20140810_125633.mp4 (1SwYm5Z1zPczZnDulmsbA9wrEJ-JT-hwE)
Getting started (0B3K2QXOGSOFRc3RhcnRlcl9maWxl)
Script to download file with id 1SwYm5Z1zPczZnDulmsbA9wrEJ-JT-hwE
from __future__ import print_function
import pickle
import os.path
import io
import sys
from googleapiclient.discovery import build
from googleapiclient.http import MediaIoBaseDownload
from google_auth_oauthlib.flow import InstalledAppFlow
from google.auth.transport.requests import Request
# If modifying these scopes, delete the file token.pickle.
SCOPES = ['https://www.googleapis.com/auth/drive.file']
TOKEN_FILE = 'tokenWrite.pickle';
def downloadFile(driveService, fileId):
request = driveService.files().get_media(fileId=fileId)
fh = io.BytesIO()
downloader = MediaIoBaseDownload(fh, request)
done = False
while done is False:
status, done = downloader.next_chunk()
print ("Download %d%%." % int(status.progress() * 100))
def main():
"""Shows basic usage of the Drive v3 API.
Prints the names and ids of the first 10 files the user has access to.
"""
creds = None
# The file token.pickle stores the user's access and refresh tokens, and is
# created automatically when the authorization flow completes for the first
# time.
if os.path.exists(TOKEN_FILE):
with open(TOKEN_FILE, 'rb') as token:
creds = pickle.load(token)
# If there are no (valid) credentials available, let the user log in.
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
flow = InstalledAppFlow.from_client_secrets_file(
'credentials.json', SCOPES)
creds = flow.run_local_server()
# Save the credentials for the next run
with open(TOKEN_FILE, 'wb') as token:
pickle.dump(creds, token)
service = build('drive', 'v3', credentials=creds)
downloadFile(service, '1SwYm5Z1zPczZnDulmsbA9wrEJ-JT-hwE')
if __name__ == '__main__':
main()
Error
$ python3 download_files.py
Please visit this URL to authorize this application: https://accounts.google.com/o/oauth2/auth?response_type=code&client_id=619229308650-91gkhdgo7v0jbt6df1phahmq868eb7gd.apps.googleusercontent.com&redirect_uri=http%3A%2F%2Flocalhost%3A8080%2F&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive.file&state=4mP9kgVJQD4ETOu5JjIRQFBLcyViAG&access_type=offline&code_challenge=ybCzMgZ2SOXdrpZZYn1dq9nSJk8wMtLo7Deg_Xix9So&code_challenge_method=S256
Traceback (most recent call last):
File "download_files.py", line 52, in <module>
main()
File "download_files.py", line 49, in main
downloadFile(service, '1SwYm5Z1zPczZnDulmsbA9wrEJ-JT-hwE')
File "download_files.py", line 21, in downloadFile
status, done = downloader.next_chunk()
File "/usr/local/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 130, in positional_wrapper
return wrapped(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/googleapiclient/http.py", line 705, in next_chunk
raise HttpError(resp, content, uri=self._uri)
googleapiclient.errors.HttpError: <HttpError 404 when requesting https://www.googleapis.com/drive/v3/files/1SwYm5Z1zPczZnDulmsbA9wrEJ-JT-hwE?alt=media returned "File not found: 1SwYm5Z1zPczZnDulmsbA9wrEJ-JT-hwE.">
How about this answer?
Reason of issue:
When I saw your scripts, I noticed that the scope of below script are different from the above script. I think that this is the reason of your issue.
At above script, https://www.googleapis.com/auth/drive.metadata.readonly is used. On the other hand, at the below script, https://www.googleapis.com/auth/drive.file is used.
The official document says about the scope of https://www.googleapis.com/auth/drive.file as follows.
View and manage Google Drive files and folders that you have opened or created with this app
This means that when your script uploads a file using the scope of https://www.googleapis.com/auth/drive.file, you can retrieve the file using the scope. But for example, when the file is manually upload to the Google Drive, the file cannot be downloaded by the scope of https://www.googleapis.com/auth/drive.file, even when the file is shared with you.
In order to download the file, how about the following workarounds?
Workaround 1:
You use the scope of https://www.googleapis.com/auth/drive or https://www.googleapis.com/auth/drive.readonly instead of https://www.googleapis.com/auth/drive.file.
Workaround 2:
If you are required to use the scope of https://www.googleapis.com/auth/drive.file, it uploads the file using the scope of https://www.googleapis.com/auth/drive.file. By this, the file can be downloaded by the scope.
Note:
When you change the scopes, please remove the file of tokenWrite.pickle and authorize the scopes again and create new tokenWrite.pickle. By this, you can use new scopes. Please be careful this.
Reference:
Scopes of Drive API, v3
If I misunderstood your question and this was not the direction you want, I apologize.