I'm currently trying to download a file from Google Drive using PyDrive, but am only able to download the file to the same location as my Python program. Is there a way to specify the file's download location? This is how I am downloading files currently.
if file1['title'] == file_name:
file2 = drive.CreateFile({'id': file1['id']})
print('Downloading file %s from Google Drive' % file2['title'])
file2.GetContentFile(file_name) # Save Drive file as a local file
Try following:
if file1['title'] == file_name:
location = "Path/where/you/want/to/save/"
"""
if you are in linux and want to save it to documents try following
location = "~/Documents/"
the last forward slash i.e. '/' is important
"""
full_path = location + file_name
file2 = drive.CreateFile({'id': file1['id']})
print('Downloading file %s from Google Drive' % file2['title'])
file2.GetContentFile(full_path)
Related
I've wrote the following code to upload a file into blob storage using Python:
blob_service_client = ContainerClient(account_url="https://{}.blob.core.windows.net".format(ACCOUNT_NAME),
credential=ACCOUNT_KEY,
container_name=CONTAINER_NAME)
blob_service_client.upload_blob("my_file.txt", open("my_file.txt", "rb"))
this works fine. I wonder how can I upload the entire folder with all files and sub folders in it while keeping the structure of my local folder intact?
After reproducing from my end I could able to achieve your requirement using os module. Below is the complete code that worked for me.
dir_path = r'<YOUR_LOCAL_FOLDER>'
for path, subdirs, files in os.walk(dir_path):
for name in files:
fullPath=os.path.join(path, name)
print("FullPath : "+fullPath)
file=fullPath.replace(dir_path,'')
fileName=file[1:len(file)];
print("File Name :"+fileName)
# Create a blob client using the local file name as the name for the blob
blob_service_client = ContainerClient(account_url=ACCOUNT_URL,
credential=ACCOUNT_KEY,
container_name=CONTAINER_NAME)
print("\nUploading to Azure Storage as blob:\n\t" + fileName)
blob_service_client.upload_blob(fileName, open(fullPath, "rb"))
Below is the folder structure in my local.
├───g.txt
├───h.txt
├───Folder1
├───z.txt
├───y.txt
├───Folder2
├───a.txt
├───b.txt
├───SubFolder1
├───c.txt
├───d.txt
RESULTS:
I have this script made with big help from #Tanaike. It is doing two things:
Delete old files in Google Drive folder by listing my files in local folder (CSVtoGD), checking names of that files and delete files in Google Drive named the same.
Uploads new csv files to Google Drive
I have a problem with deleting old files in Google Drive. The script is uploading new files and delete old files in GD but If in my local folder (CSVtoGD) there is new file which has never been uploaded to Google Drive I receive error:
HttpError: <HttpError 404 when requesting https://www.googleapis.com/upload/drive/v3/files?fields=id&alt=json&uploadType=multipart returned "File not found: 19vrbvaeDqWcxFGwPV82APWYTmBMEn-hi.". Details: "[{'domain': 'global', 'reason': 'notFound', 'message': 'File not found: 19vrbvaeDqWcxFGwPV82APWYTmBMEn-hi.', 'locationType': 'parameter', 'location': 'fileId'}]">
Script:
import gspread
import os
from googleapiclient.discovery import build
from googleapiclient.http import MediaFileUpload
gc = gspread.oauth(credentials_filename='/users/user/credentials.json')
service = build("drive", "v3", credentials=gc.auth)
folder_id = '19vrbvaeDqWcxFGwPV82APWYTmBME'
def getSpreadsheetId(filename, filePath):
q = "name='" + filename + "' and mimeType='application/vnd.google-apps.spreadsheet' and trashed=false"
res = service.files().list(q=q, fields="files(id)", corpora="allDrives", includeItemsFromAllDrives=True, supportsAllDrives=True).execute()
items = res.get("files", [])
if not items:
print("No files found.")
file_metadata = {
"name": filename,
"parents": [folder_id],
"mimeType": "application/vnd.google-apps.spreadsheet",
}
media = MediaFileUpload(filePath + "/" + filename + ".csv")
file = service.files().create(body=file_metadata, media_body=media, fields="id").execute()
id = file.get("id")
print("File was uploaded. The file ID is " + id)
exit()
return items[0]["id"]
filePath = '/users/user/CSVtoGD'
os.chdir(filePath)
files = os.listdir()
for filename in files:
fname = filename.split(".")
if fname[1] == "csv":
oldSpreadsheetId = getSpreadsheetId(fname[0], filePath)
print(oldSpreadsheetId)
sh = gc.del_spreadsheet(oldSpreadsheetId)
sh = gc.create(fname[0], folder_id)
content = open(filename, "r").read().encode("utf-8")
gc.import_csv(sh.id, content)
I was trying to make it work but none of my efforts work, so I thing I have to change the script to just simply delete all files which are in my Google Drive folder and then, upload the new files, but reading docs of Google API I really can't see where to start as the files ned to be deleted by checking their ID in google drive so I have to somehow list the files ID's and then delete all of them.
Simple in two words: How to delete ALL files in Google Drive folder and then upload new files from local folder (CSVtoGD) - uploading part of the script works great I have problem with deleting old files.
I believe your goal is as follows.
After all files in the folder were deleted, you want to upload the file.
From your showing script, you want to delete Google Spreadsheet files in the folder.
In this case, how about the following modification?
Modified script:
import gspread
import os
from googleapiclient.discovery import build
from googleapiclient.http import MediaFileUpload
gc = gspread.oauth(credentials_filename='/users/user/credentials.json')
service = build("drive", "v3", credentials=gc.auth)
folder_id = '19vrbvaeDqWcxFGwPV82APWYTmBME'
def deleteAllFilesInFolder():
q = "'" + folder_id + "' in parents and mimeType='application/vnd.google-apps.spreadsheet' and trashed=false"
res = service.files().list(q=q, fields="files(id)", corpora="allDrives", includeItemsFromAllDrives=True, supportsAllDrives=True, pageSize=1000).execute()
for e in res.get("files", []):
gc.del_spreadsheet(e["id"])
def uploadCSV(filename, filePath):
file_metadata = {
"name": filename,
"parents": [folder_id],
"mimeType": "application/vnd.google-apps.spreadsheet",
}
media = MediaFileUpload(filePath + "/" + filename + ".csv")
file = service.files().create(body=file_metadata, media_body=media, fields="id", supportsAllDrives=True).execute()
id = file.get("id")
print("File was uploaded. The file ID is " + id)
filePath = '/users/user/CSVtoGD'
os.chdir(filePath)
files = os.listdir()
delete = False
for filename in files:
fname = filename.split(".")
if fname[1] == "csv":
if not delete:
deleteAllFilesInFolder()
delete = True
uploadCSV(fname[0], filePath)
When this script is run, first, all Spreadsheet files in the folder are deleted. And then, the CSV files are uploaded as Google Spreadsheet.
Note:
In this modified script, all Spreadsheet files in the folder are completely deleted. Please be careful about this. So, I would like to recommend testing using the sample Spreadsheet files.
If you want to delete all files including except for Spreadsheet, please modify q = "'" + folder_id + "' in parents and mimeType='application/vnd.google-apps.spreadsheet' and trashed=false" to q = "'" + folder_id + "' in parents and trashed=false".
I have a problem witch was hard to write in the title. I have this script with a lot of help from #Tanaike . This script is doing basically two things:
Deletes files from Google Drive folder by filenames which are in local folder CSVtoGD (using spreadsheet ID's)
then:
Upload list of CSV from local folder "CSVtoGD" to Google Drive folder
I have a big problem now and can not work it out. The script is deleting old files in google drive when there are the same filenames in CSVtoGD. When I add new file to local folder CSVtoGD, there is a error "list index out of range" and I got printed "No files found" like in the script. I was trying to make some modification but it was blind shoots. What I want this script to do is to delete from Google Drive folder ONLY files which are in local CSVtoGD folder and work on with rest of the files in CSVtoGD (just upload them). Anyone have some answer to that? Thank you :)
import gspread
import os
from googleapiclient.discovery import build
gc = gspread.oauth(credentials_filename='/users/user/credentials.json')
service = build("drive", "v3", credentials=gc.auth)
def getSpreadsheetId(filename):
q = "name='" + filename + "' and mimeType='application/vnd.google-apps.spreadsheet' and trashed=false"
res = service.files().list(q=q, fields="files(id)", corpora="allDrives", includeItemsFromAllDrives=True, supportsAllDrives=True).execute()
items = res.get("files", [])
if not items:
print("No files found.")
exit()
return items[0]["id"]
os.chdir('/users/user/CSVtoGD2')
files = os.listdir()
for filename in files:
fname = filename.split(".")
if fname[1] == "csv":
folder_id = '1z_pUvZyt5AoTNy-aKCKLmlNjdR2OPo'
oldSpreadsheetId = getSpreadsheetId(fname[0])
#print(oldSpreadsheetId)
sh = gc.del_spreadsheet(oldSpreadsheetId)
**# IF there are the same filenames in CSVtoGD folder on my Mac
#and the same filenames on Google Drive folder,
#those lines works well.
#Problem is when there are new files in CSVtoGD local folder on Mac.**
sh = gc.create(fname[0], folder_id)
content = open(filename, "r").read().encode("utf-8")
gc.import_csv(sh.id, content)
I believe your goal is as follows.
For example, when sample.csv is existing on your local PC and a Spreadsheet of sample is existing in your Google Drive, you want to delete the Spreadsheet of sample from your Google Drive.
When sample1.csv is existing on your local PC and the Spreadsheet of sample1 is NOT existing in your Google Drive, you want to upload sample1.csv to Google Drive.
In this case, how about the following modification?
Modified script:
import gspread
import os
from googleapiclient.discovery import build
from googleapiclient.http import MediaFileUpload
gc = gspread.oauth(credentials_filename='/users/user/credentials.json')
service = build("drive", "v3", credentials=gc.auth)
folder_id = '1z_pUvZyt5AoTNy-aKCKLmlNjdR2OPo' # Please set the folder ID you want to upload the file.
def getSpreadsheetId(filename, filePath):
q = "name='" + filename + "' and mimeType='application/vnd.google-apps.spreadsheet' and trashed=false"
res = service.files().list(q=q, fields="files(id)", corpora="allDrives", includeItemsFromAllDrives=True, supportsAllDrives=True).execute()
items = res.get("files", [])
if not items:
print("No files found.")
file_metadata = {
"name": filename,
"parents": [folder_id],
"mimeType": "application/vnd.google-apps.spreadsheet",
}
media = MediaFileUpload(filePath + "/" + filename + ".csv")
file = service.files().create(body=file_metadata, media_body=media, fields="id").execute()
id = file.get("id")
print("File was uploaded. The file ID is " + id)
exit()
return items[0]["id"]
filePath = '/users/user/CSVtoGD2'
os.chdir(filePath)
files = os.listdir()
for filename in files:
fname = filename.split(".")
if fname[1] == "csv":
oldSpreadsheetId = getSpreadsheetId(fname[0], filePath)
print(oldSpreadsheetId)
sh = gc.del_spreadsheet(oldSpreadsheetId)
sh = gc.create(fname[0], folder_id)
content = open(filename, "r").read().encode("utf-8")
gc.import_csv(sh.id, content)
When this script is run, the above flow is run.
Note:
In this modification, the CSV file is uploaded as a Google Spreadsheet. From your question, I thought that this might be your expected result. But, if you want to upload the CSV file as the CSV file, please remove "mimeType": "application/vnd.google-apps.spreadsheet", from file_metadata.
If an error related to the scope, please add the scope of https://www.googleapis.com/auth/drive and authorize the scopes again and test it again.
Reference:
Upload file data
I have created a Python script that pulls files with a .xlsx format from a folder on my computer and uploads the file to a specific folder in my Google Drive. This is using the pydrive package in Python. The script runs with no issues, and the files are uploaded as expected. However, for some reason, when the uploaded Google Drive file is downloaded and re-opened, Excel gives the following error message:
Excel cannot open the file...because the file format or file extension is not valid. Verify that the file has not been corrupted and that the file extension matches the format of the file.
When I open the file directly on my computer, it opens fine with no issues. When I manually drag/upload the file into the Google Drive folder, and then re-download the file, it opens with no problem. The transformation seems to be coming from my Python script (see below).
Can anyone provide any help here? I have been trying different things and I keep getting the same result. I can provide more information if necessary.
Updated to add full Python Script:
from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive
import glob,os,shutil
import datetime
os.chdir(os.path.dirname(os.path.abspath(__file__)))
gauth = GoogleAuth()
#gauth.LocalWebserverAuth()
# Try to load saved client credentials
gauth.LoadCredentialsFile("mycreds.txt")
if gauth.credentials is None:
# Authenticate if they're not there
gauth.LocalWebserverAuth()
elif gauth.access_token_expired:
# Refresh them if expired
gauth.Refresh()
else:
# Initialize the saved creds
gauth.Authorize()
# Save the current credentials to a file
gauth.SaveCredentialsFile("mycreds.txt")
drive = GoogleDrive(gauth)
#Google Drive Folder ID
fid = '[FOLDER ID PLACEHOLDER]'
#Check to see if today's folder is created on PC
date = datetime.date.today()
today = date.strftime('%Y-%m-%d')
starting_folder = '[FOLDER PLACEHOLDER]'
if not os.path.exists(starting_folder + "/" + today):
os.makedirs(starting_folder + "/" + today)
destination_folder = starting_folder + "/" + today
#Change directory to the folder where the bulk bid tools are stored
os.chdir("[FOLDER PLACEHOLDER]")
for file in glob.glob("*.xlsx"):
try:
print(file)
with open(file,"r") as f:
fn = os.path.basename(f.name)
file_drive = drive.CreateFile({'parents':[{'kind':"drive#parentReference",'id':fid}],'title':fn})
file_drive.Upload()
print("The file: " + fn + " has been uploaded.")
shutil.move(starting_folder + "/" + fn,destination_folder + "/" + fn)
except:
pass
print("All files have been uploaded")
You are not actually passing the information of the file to the request. Meaning, you are not really uploading the actual "bytes" of this file to Drive. Just creating an empty drive file with the same name in the determined folder.
If you look at the documentation for pyDrive you can see that after calling of CreateFile they use SetContentFile.
Copied from the documentation you can see an example like so:
file2 = drive.CreateFile()
file2.SetContentFile('hello.png') # This line needs to be added to your code
# with the name of the file in your computer
file2.Upload()
# Also check the mimetype afterwards to check the file has been correctly uploaded
print('Created file %s with mimeType %s' % (file2['title'],
file2['mimeType']))
# Created file hello.png with mimeType image/png
Also from the comments you say you are still running python2 code. Take into consideration that python 2 is "dead" and there will not be more security updates, neither development/support. You should really be considering change as a lot of package and modules are also dropping (or will start to do so) python 2 support.
More information of this issue in Sunsetting Python 2.
How to upload csv file to Dropbox with Python
I tried all the examples in this post bellow, neither works
upload file to my dropbox from python script
I am getting error:
FileNotFoundError: [Errno 2] No such file or directory: 'User\pb\Automation\test.csv'
My username: pb
Folder name: Automation
file name: test.csv
import pathlib
import dropbox
import re
# the source file
folder = pathlib.Path("User/pb/Automation") # located in folder
filename = "test.csv" # file name
filepath = folder / filename # path object, defining the file
# target location in Dropbox
target = "Automation" # the target folder
targetfile = target + filename # the target path and file name
# Create a dropbox object using an API v2 key
token = ""
d = dropbox.Dropbox(token)
# open the file and upload it
with filepath.open("rb") as f:
# upload gives you metadata about the file
# we want to overwite any previous version of the file
meta = d.files_upload(f.read(), targetfile, mode=dropbox.files.WriteMode("overwrite"))
# create a shared link
link = d.sharing_create_shared_link(targetfile)
# url which can be shared
url = link.url
# link which directly downloads by replacing ?dl=0 with ?dl=1
dl_url = re.sub(r"\?dl\=0", "?dl=1", url)
print (dl_url)
FileNotFoundError: [Errno 2] No such file or directory: 'User\\pb\\Automation\\test.csv'
The error message is indicating that you're supplying a local path of 'User\pb\Automation\test.csv' but nothing was found at that path on your local filesystem.
Based on the path format, it looks like you're on macOS, but you have the wrong path for accessing your home folder. The path should start with "/", and the home folders are located under "Users" (not "User"), so your folder definition should probably be:
folder = pathlib.Path("/Users/pb/Automation")
Or, use pathlib.Path.home() to automatically expand the home folder for you:
pathlib.Path.home() / "Automation"