We Keep Coding

Trying to create a simple python script. Keep getting errors

This is my first time posting here so i apologize if i have not posted in the correct format!
from google.auth.transport.requests import Request
from google.oauth2.credentials import Credentials
from google_auth_oauthlib.flow import InstalledAppFlow
from googleapiclient.discovery import build
from googleapiclient.errors import HttpError
# If modifying these scopes, delete the file token.json.
SCOPES = ['https://www.googleapis.com/auth/drive.metadata.readonly']
def main():
"""Shows basic usage of the Drive v3 API.
Prints the names and ids of the first 10 files the user has access to.
"""
creds = None
# The file token.json stores the user's access and refresh tokens, and is
# created automatically when the authorization flow completes for the first
# time.
if os.path.exists('token.json'):
creds = Credentials.from_authorized_user_file('token.json', SCOPES)
# If there are no (valid) credentials available, let the user log in.
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
flow = InstalledAppFlow.from_client_secrets_file(
'credentials.json', SCOPES)
creds = flow.run_local_server(port=0)
# Save the credentials for the next run
with open('token.json', 'w') as token:
token.write(creds.to_json())
try:
service = build('drive', 'v3', credentials=creds)
# Call the Drive v3 API
results = service.files().list(
pageSize=10, fields="nextPageToken, files(id, name)").execute()
items = results.get('files', [])
if not items:
print('No files found.')
return
print('Files:')
for item in items:
print(u'{0} ({1})'.format(item['name'], item['id']))
except HttpError as error:
# TODO(developer) - Handle errors from drive API.
print(f'An error occurred: {error}')
if __name__ == '__main__':
main()
I'm trying to create a simple python script that does the following.
1.Import the necessary Libraries.
2.Connect/Authenticate to Google drive.
3.Check a folder on google drive for file names and then upload all files that are not on the drive yet from a
specific folder on computer to the checked folder on google drive.
4.repeat every hour, If no new files found then restart loop.
this is what i tried so far but i get errors when running the script, The issue is it cannot find the credentials.json file.
The credentials.json file is in the same folder as the settings.yaml and the Setup.py
I was expecting to be directed to a login page when running the script.
To be fair i know very very little about python and the most experience i have with coding would be from my old modded minecraft days using lua. on computercraft. hahah thanks for your help in advance.**

How to get Access Token for uploading files on google drive

headers = {"Authorization": "Bearer " + ACCESS_TOKEN}
folder=Folder_id // google drive folder Id
para = {"title": assignment_file_name,
"parents": [{"id": "root"}, {'id': folder}]}
files = {
"data": ("metadata", json.dumps(para), "application/json; charset=UTF-8"),
"file": assignment_file.stream.read()
}
response = requests.get("https://drive.google.com/drive/folders/"+Folder_id,
headers= headers, files=files)
I want to upload file fetched from requests to google drive folder.
But From where to get this ACCESS_TOKEN in the headers variable?

The access token is the result of the Oauth2 request. When you want to access private user data you must first ask for the users consent to your application accessing their data. To do this we use Oauth2. If you are interested in knowing how it works i have a video on it Understanding Oauth2 with curl..
Rather then coding all this manually you should consider using the Google Api python client library. As shown in the official Python quickstart for Drive api will walk you though how to create credentials (be sure to enable drive api) for your application and then how to request access of the user.
from __future__ import print_function
import os.path
from google.auth.transport.requests import Request
from google.oauth2.credentials import Credentials
from google_auth_oauthlib.flow import InstalledAppFlow
from googleapiclient.discovery import build
from googleapiclient.errors import HttpError
# If modifying these scopes, delete the file token.json.
SCOPES = ['https://www.googleapis.com/auth/drive.metadata.readonly']
def main():
"""Shows basic usage of the Drive v3 API.
Prints the names and ids of the first 10 files the user has access to.
"""
creds = None
# The file token.json stores the user's access and refresh tokens, and is
# created automatically when the authorization flow completes for the first
# time.
if os.path.exists('token.json'):
creds = Credentials.from_authorized_user_file('token.json', SCOPES)
# If there are no (valid) credentials available, let the user log in.
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
flow = InstalledAppFlow.from_client_secrets_file(
'credentials.json', SCOPES)
creds = flow.run_local_server(port=0)
# Save the credentials for the next run
with open('token.json', 'w') as token:
token.write(creds.to_json())
try:
service = build('drive', 'v3', credentials=creds)
# Call the Drive v3 API
results = service.files().list(
pageSize=10, fields="nextPageToken, files(id, name)").execute()
items = results.get('files', [])
if not items:
print('No files found.')
return
print('Files:')
for item in items:
print(u'{0} ({1})'.format(item['name'], item['id']))
except HttpError as error:
# TODO(developer) - Handle errors from drive API.
print(f'An error occurred: {error}')
if __name__ == '__main__':
main()

Is it possible to sync or upload the file from google drive without copying the whole thing in the folder using python?

I just start learning the python scripting and I created a script using pydrive and the function is uploading all files from local folder (linux OS) to google drive but I'm planning to modify the script for my automation and add the function that can upload only the most recent file added to the local folder with no reuploading of all the files inside the folder, may I know if this is possible with python script alone?
Thank you in advance!

You dont need to use pydrive. You can use the Google api python client library directly. As far as i know pydrive does use the client library internally. There's a starter example here
Quick start python
from __future__ import print_function
import os.path
from google.auth.transport.requests import Request
from google.oauth2.credentials import Credentials
from google_auth_oauthlib.flow import InstalledAppFlow
from googleapiclient.discovery import build
from googleapiclient.errors import HttpError
# If modifying these scopes, delete the file token.json.
SCOPES = ['https://www.googleapis.com/auth/drive.metadata.readonly']
def main():
"""Shows basic usage of the Drive v3 API.
Prints the names and ids of the first 10 files the user has access to.
"""
creds = None
# The file token.json stores the user's access and refresh tokens, and is
# created automatically when the authorization flow completes for the first
# time.
if os.path.exists('token.json'):
creds = Credentials.from_authorized_user_file('token.json', SCOPES)
# If there are no (valid) credentials available, let the user log in.
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
flow = InstalledAppFlow.from_client_secrets_file(
'credentials.json', SCOPES)
creds = flow.run_local_server(port=0)
# Save the credentials for the next run
with open('token.json', 'w') as token:
token.write(creds.to_json())
try:
service = build('drive', 'v3', credentials=creds)
# Call the Drive v3 API
results = service.files().list(
pageSize=10, fields="nextPageToken, files(id, name)").execute()
items = results.get('files', [])
if not items:
print('No files found.')
return
print('Files:')
for item in items:
print(u'{0} ({1})'.format(item['name'], item['id']))
except HttpError as error:
# TODO(developer) - Handle errors from drive API.
print(f'An error occurred: {error}')
if __name__ == '__main__':
main()
Manage uploads
file_metadata = {'name': 'photo.jpg'}
media = MediaFileUpload('files/photo.jpg', mimetype='image/jpeg')
file = drive_service.files().create(body=file_metadata,
media_body=media,
fields='id').execute()
print 'File ID: %s' % file.get('id')

Authorize google api requests to list users

I want to get list of users google api but face an issue
My steps are:
Created Service account with domain wide delegation and listed scopes (same as in the script)
Downloaded json file with private key
On execution of the next script
import google.auth
import google.auth.transport.requests
from google.oauth2 import service_account
import requests
credentials = service_account.Credentials.from_service_account_file('key.json', scopes=[
'https://www.googleapis.com/auth/admin.directory.user',
'https://www.googleapis.com/auth/admin.directory.group',
'https://www.googleapis.com/auth/admin.directory.group.member',
'https://www.googleapis.com/auth/admin.directory.user.security',
'https://www.googleapis.com/auth/admin.directory.user.readonly'
])
auth_req = google.auth.transport.requests.Request()
refresh = credentials.refresh(auth_req)
response = requests.get('https://www.googleapis.com/admin/directory/v1/users?domain=domain.com',
headers={'Authorization': f'Bearer {credentials.token}'})
Response is:
{
"error": {
"code": 403,
"message": "Not Authorized to access this resource/api",
"errors": [
{
"message": "Not Authorized to access this resource/api",
"domain": "global",
"reason": "forbidden"
}
]
}
}

I ended up with the next solution
console.cloud.google.com => APIs & Services > Credentials
Create Credentials > OAuth Client ID > Desktop APp
Saved key as 'credentials.json'
The rest id described here
from __future__ import print_function
import os.path
from googleapiclient.discovery import build
from google_auth_oauthlib.flow import InstalledAppFlow
from google.auth.transport.requests import Request
from google.oauth2.credentials import Credentials
# If modifying these scopes, delete the file token.json.
SCOPES = ['https://www.googleapis.com/auth/admin.directory.user']
#
def main():
"""Shows basic usage of the Admin SDK Directory API.
Prints the emails and names of the first 10 users in the domain.
"""
creds = None
# The file token.json stores the user's access and refresh tokens, and is
# created automatically when the authorization flow completes for the first
# time.
if os.path.exists('token.json'):
creds = Credentials.from_authorized_user_file('token.json', SCOPES)
# If there are no (valid) credentials available, let the user log in.
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
print('Refreshing token')
creds.refresh(Request())
else:
flow = InstalledAppFlow.from_client_secrets_file(
'python_admin.json', SCOPES)
creds = flow.run_local_server(port=0)
# Save the credentials for the next run
with open('token.json', 'w') as token:
token.write(creds.to_json())
service = build('admin', 'directory_v1', credentials=creds)
# Call the Admin SDK Directory API
print('Getting the first 10 users in the domain')
service_users = service.users()
results = service_users.list(customer='my_customer', maxResults=10, orderBy='email').execute()
users = results.get('users', [])
service_users.insert()
if not users:
print('No users in the domain.')
else:
print('Users:')
for user in users:
print(u'{0} ({1})'.format(user['primaryEmail'],
user['name']['fullName']))
if __name__ == '__main__':
main()
For the first time I had to authorize request in the browser, after that access and refresh token could be used for next requests. Except manual authorization for the first time - works fine

Unable to download file by its id from Google Drive

$ pip3 list | grep googl
google-api-python-client 1.7.9
google-auth 1.6.3
google-auth-httplib2 0.0.3
google-auth-oauthlib 0.4.0
I can successfully list files shared to me. But I get "File not found" error when I try to download an existing file by its id. How to download a file by its id?
Script to list files
from __future__ import print_function
import pickle
import os.path
import io
import sys
from googleapiclient.discovery import build
from googleapiclient.http import MediaIoBaseDownload
from google_auth_oauthlib.flow import InstalledAppFlow
from google.auth.transport.requests import Request
# If modifying these scopes, delete the file token.pickle.
SCOPES = ['https://www.googleapis.com/auth/drive.metadata.readonly']
TOKEN_FILE = 'tockenRead.pickle'
def main():
"""Shows basic usage of the Drive v3 API.
Prints the names and ids of the first 10 files the user has access to.
"""
creds = None
# The file token.pickle stores the user's access and refresh tokens, and is
# created automatically when the authorization flow completes for the first
# time.
if os.path.exists(TOKEN_FILE):
with open(TOKEN_FILE, 'rb') as token:
creds = pickle.load(token)
# If there are no (valid) credentials available, let the user log in.
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
flow = InstalledAppFlow.from_client_secrets_file(
'credentials.json', SCOPES)
creds = flow.run_local_server()
# Save the credentials for the next run
with open(TOKEN_FILE, 'wb') as token:
pickle.dump(creds, token)
service = build('drive', 'v3', credentials=creds)
# Call the Drive v3 API
results = service.files().list(
q="mimeType != 'application/vnd.google-apps.folder'",
pageSize=10,
fields="nextPageToken, files(id, name)"
).execute()
items = results.get('files', [])
if not items:
print('No files found.')
else:
print('Files:')
for item in items:
print(u'{0} ({1})'.format(item['name'], item['id']))
if __name__ == '__main__':
main()
Result
$ python3 list_files.py
Files:
20140810_125633.mp4 (1SwYm5Z1zPczZnDulmsbA9wrEJ-JT-hwE)
Getting started (0B3K2QXOGSOFRc3RhcnRlcl9maWxl)
Script to download file with id 1SwYm5Z1zPczZnDulmsbA9wrEJ-JT-hwE
from __future__ import print_function
import pickle
import os.path
import io
import sys
from googleapiclient.discovery import build
from googleapiclient.http import MediaIoBaseDownload
from google_auth_oauthlib.flow import InstalledAppFlow
from google.auth.transport.requests import Request
# If modifying these scopes, delete the file token.pickle.
SCOPES = ['https://www.googleapis.com/auth/drive.file']
TOKEN_FILE = 'tokenWrite.pickle';
def downloadFile(driveService, fileId):
request = driveService.files().get_media(fileId=fileId)
fh = io.BytesIO()
downloader = MediaIoBaseDownload(fh, request)
done = False
while done is False:
status, done = downloader.next_chunk()
print ("Download %d%%." % int(status.progress() * 100))
def main():
"""Shows basic usage of the Drive v3 API.
Prints the names and ids of the first 10 files the user has access to.
"""
creds = None
# The file token.pickle stores the user's access and refresh tokens, and is
# created automatically when the authorization flow completes for the first
# time.
if os.path.exists(TOKEN_FILE):
with open(TOKEN_FILE, 'rb') as token:
creds = pickle.load(token)
# If there are no (valid) credentials available, let the user log in.
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
flow = InstalledAppFlow.from_client_secrets_file(
'credentials.json', SCOPES)
creds = flow.run_local_server()
# Save the credentials for the next run
with open(TOKEN_FILE, 'wb') as token:
pickle.dump(creds, token)
service = build('drive', 'v3', credentials=creds)
downloadFile(service, '1SwYm5Z1zPczZnDulmsbA9wrEJ-JT-hwE')
if __name__ == '__main__':
main()
Error
$ python3 download_files.py
Please visit this URL to authorize this application: https://accounts.google.com/o/oauth2/auth?response_type=code&client_id=619229308650-91gkhdgo7v0jbt6df1phahmq868eb7gd.apps.googleusercontent.com&redirect_uri=http%3A%2F%2Flocalhost%3A8080%2F&scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fdrive.file&state=4mP9kgVJQD4ETOu5JjIRQFBLcyViAG&access_type=offline&code_challenge=ybCzMgZ2SOXdrpZZYn1dq9nSJk8wMtLo7Deg_Xix9So&code_challenge_method=S256
Traceback (most recent call last):
File "download_files.py", line 52, in <module>
main()
File "download_files.py", line 49, in main
downloadFile(service, '1SwYm5Z1zPczZnDulmsbA9wrEJ-JT-hwE')
File "download_files.py", line 21, in downloadFile
status, done = downloader.next_chunk()
File "/usr/local/lib/python3.7/site-packages/googleapiclient/_helpers.py", line 130, in positional_wrapper
return wrapped(*args, **kwargs)
File "/usr/local/lib/python3.7/site-packages/googleapiclient/http.py", line 705, in next_chunk
raise HttpError(resp, content, uri=self._uri)
googleapiclient.errors.HttpError: <HttpError 404 when requesting https://www.googleapis.com/drive/v3/files/1SwYm5Z1zPczZnDulmsbA9wrEJ-JT-hwE?alt=media returned "File not found: 1SwYm5Z1zPczZnDulmsbA9wrEJ-JT-hwE.">

How about this answer?
Reason of issue:
When I saw your scripts, I noticed that the scope of below script are different from the above script. I think that this is the reason of your issue.
At above script, https://www.googleapis.com/auth/drive.metadata.readonly is used. On the other hand, at the below script, https://www.googleapis.com/auth/drive.file is used.
The official document says about the scope of https://www.googleapis.com/auth/drive.file as follows.
View and manage Google Drive files and folders that you have opened or created with this app
This means that when your script uploads a file using the scope of https://www.googleapis.com/auth/drive.file, you can retrieve the file using the scope. But for example, when the file is manually upload to the Google Drive, the file cannot be downloaded by the scope of https://www.googleapis.com/auth/drive.file, even when the file is shared with you.
In order to download the file, how about the following workarounds?
Workaround 1:
You use the scope of https://www.googleapis.com/auth/drive or https://www.googleapis.com/auth/drive.readonly instead of https://www.googleapis.com/auth/drive.file.
Workaround 2:
If you are required to use the scope of https://www.googleapis.com/auth/drive.file, it uploads the file using the scope of https://www.googleapis.com/auth/drive.file. By this, the file can be downloaded by the scope.
Note:
When you change the scopes, please remove the file of tokenWrite.pickle and authorize the scopes again and create new tokenWrite.pickle. By this, you can use new scopes. Please be careful this.
Reference:
Scopes of Drive API, v3
If I misunderstood your question and this was not the direction you want, I apologize.