We Keep Coding

How to download file from google drive folder?

I have a script that gets a list of files from google drive
from pydrive.drive import GoogleDrive
gauth = GoogleAuth()
gauth.LoadCredentialsFile("mycreds.txt")
gauth.LoadCredentialsFile("mycreds.txt")
if gauth.credentials is None:
gauth.LocalWebserverAuth()
elif gauth.access_token_expired:
gauth.Refresh()
else:
gauth.Authorize()
gauth.SaveCredentialsFile("mycreds.txt")
gauth.LocalWebserverAuth()
drive = GoogleDrive(gauth)
folder = "1CNtWRS005fkX6vlZowZiXYITNGifXPKS"
file_list = drive.ListFile({'q': f"'{folder}' in parents"}).GetList()
for file in file_list:
print(file['title'])
-> 1.txt
It receives data only from its disk, but I need the script to receive a list of files from a folder to which it has access - "available to me". I have a folder ID, but if I substitute it in the folder field, nothing happens

I think gdown could help you.
pip install gdown
Then could try something like this:
import gdown
id = "folderId..."
gdown.download_folder(id=id, quiet=True, use_cookies=False)

Add member(s) to Google Drive Shared Drive with Python

How can I add a member to a Google Drive's Shared Drive?
I am working on a Colab and tried to search but nothing useful came up.
I tried using PyDrive but:
I can't select the whole Shared Drive, only a folder
I can't update permissions on the folder even tought I can do it through the google drive web gui
This is what I'm doing now (error 400):
from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive
from google.colab import auth
from oauth2client.client import GoogleCredentials
from google.colab import drive as drv
drv.mount('/gdrive', force_remount=True)
auth.authenticate_user()
gauth = GoogleAuth()
gauth.credentials = GoogleCredentials.get_application_default()
drive = GoogleDrive(gauth)
%cd /gdrive/Shared\ drives/
file_list = drive.ListFile({'q': " title = 'FolderTitleOrSharedDriveTitle'"}).GetList()
for file1 in file_list:
print('title: %s, id: %s' % (file1['title'], file1['id']))
file = file1
print(file['title'])
file.GetPermissions()
new_permission = {
'type': 'user',
'value': 'usertogrant#access.com',
'role': 'reader'
}
permission = file.auth.service.permissions().insert(fileId=file['id'], body=new_permission, supportsAllDrives=True).execute(http=file.http)

How to be able to import every file in a folder in google drive?

I read this article which was about how we can import files in a google drive to our google colab environment. For each file we go through these steps as the article says:
1 - Get a shareable link
2 - then we extract the id section of the link.
3 - after that we use this code to be able to import
from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive
from google.colab import auth
from oauth2client.client import GoogleCredentials
auth.authenticate_user()
gauth = GoogleAuth()
gauth.credentials = GoogleCredentials.get_application_default()
drive = GoogleDrive(gauth)
downloaded = drive.CreateFile({'id':"your_file_ID"})
downloaded.GetContentFile('your_file_name.csv')
So then I can write import file.py. I wanted to ask that is there any way that I could get the access to main folder and start importing like from shared_file.subfolder.some_module import func1, class1?
What I really need is to do authentication just once and avoid doing all the steps above for each file in a folder. Even automating above steps can help.
Thanks

If the folder is in your own Google Drive, it's easier. Otherwise, you can add that folder to your Google Drive first (it won't take your space quota).
Then you can mount it with
from google.colab import drive
drive.mount('gdrive')
Now, you can access that folder, by changing the current directory.
import os
os.chdir("/content/gdrive/My Drive/that_folder")
Now you can import your_library.py easily, because it's in the current directory.
from your_library import *

google drive api to upload all pdfs to google drive

I am using the pydrive to upload pdf files to my google drive folder. I am wanting to send all *pdf files in a local folder at once with this code but not sure where to go from here? Should I use glob? If so I would like to see an example, please.
working code that sends 1 file to the designated google drive folder:
from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive
gauth = GoogleAuth()
gauth.LocalWebserverAuth()
drive = GoogleDrive(g_login)
folder_id = 'google_drive_id_goes_here'
f = drive.CreateFile({'title': 'testing_pdf',
'mimeType': 'application/pdf',
'parents': [{'kind': 'drive#fileLink', 'id':folder_id}]})
f.SetContentFile('/Users/Documents/python/google_drive/testing.pdf')
f.Upload()

You cant upload files at once. Create file with the API is a single thing and pydrive as no mechanism for uploading more then one .
Your going to have to put this in a loop and upload each file as you go.
import os
directory = 'the/directory/you/want/to/use'
for filename in os.listdir(directory):
if filename.endswith(".txt"):
f = open(filename)
lines = f.read()
print (lines[10])
continue
else:
continue

Where is dumped file in Google Colab?

When I wrote this code in google colab:
import pickle
x=10;
output = open('data.pkl', 'wb')
pickle.dump(x,output)
x is saved and also in another window in Google Colab I can access this file and read it but I don't know where is the file. Does anybody know where is it?

It’s in the current directory. You can also download it back to your local machine with
from google.colab import files
files.download(‘data.pkl’)

You can upload it to your Google drive:
from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive
from google.colab import auth
from oauth2client.client import GoogleCredentials
# 1. Authenticate and create the PyDrive client.
auth.authenticate_user()
gauth = GoogleAuth()
gauth.credentials = GoogleCredentials.get_application_default()
drive = GoogleDrive(gauth)
# get the folder id where you want to save your file
file = drive.CreateFile({'parents':[{u'id': folder_id}]})
file.SetContentFile('data.pkl')
file.Upload()
This code basically fetches the data.pkl from the cloud VM and upload it permanently to your Google Drive under a specific folder.
If you choose not to specify a folder, the file will be uploaded under the root of your Google Drive.

You can save and read the dumped file anywhere in your google drive folder.
import gc
import pickle
from google.colab import drive
drive.mount('/content/drive', force_remount=True)
pick_insert = open('drive/My Drive/data.pickle','wb')
pickle.dump(data, pick_insert)
pick_insert.close()
pick_read = open('drive/My Drive/data.pickle','rb')
data = pickle.load(pick_read)
pick_read.close()

Saved dump then can be loaded from the same dir as below,
dump(stories, open('review_dataset.pkl', 'wb'))
stories = load(open('review_dataset.pkl', 'rb'))

In my case, I was trying to access the pickle files in a sub-directory (data) under the . directory.
The data directory has 2 pickle files generated from the pre-processing step.
So I tried #korakot suggestion in the comments, and it worked fine!. That what I did so far.
# connect your colab with the drive
from google.colab import drive
drive.mount('/content/drive')
# list the directories in the home directory
import os
os.listdir('.')
# move the sub-directory (data) into google-drive
mv /content/data/ /content/drive/MyDrive/

You can obtain the pkl file using the following statements
from google.colab import files files
files.download("model.pkl")
Not only pkl you can retrieve other format of data also by changing the extension

you can save your pkl file by inputting this instead:
import pickle
from google.colab import drive
drive.mount('/content/drive')
x=10;
output = open('/content/drive/MyDrive/Colab Notebooks/data.pkl', 'wb')
pickle.dump(x,output)
and open it using this code:
import pickle
from google.colab import drive
drive.mount('/content/drive')
x = pickle.load(open('/content/drive/MyDrive/Colab Notebooks/data.pkl', 'rb'))
it worked for me :)