I have generated multiple files and stored them in a directory on Google Drive.
I want to retrieve the File IDs of those files such that I can use them in my python script further down-stream in the following way:
col='image'
df[col] = f'https://drive.google.com/file/d/'+df['ID']+'
Where df['ID'] contains the required File IDs in my pd.DataFrame.
As I am very inexperienced in this regard I would like to ask whether it is possible to:
retrieve these File IDs in an easy way e.g. via a bash command (similar to ls) or directly via python without using the Drive API.
alternatively, store the File IDs while generating the files in my Colab notebook.
Thanks in advance for the help! :-)
Related
I have a google sheet with a column of URLs to other google sheets, each of which could either be a native google sheet or an excel file uploaded to the Drive. All the files are stored in my work Google Drive, so the share link provides access for anyone within the company.
My research revealed several ways to access individual Google Drive files or all files in a particular directory, but I'm hoping to find a way to access hundreds of other file URLs and read each of them (and their tabs) to a separate pandas dataframe.
I could go through the process of creating shortcuts for each of the files to a folder on my drive and go the "pull in everything from this directory" route, but before I subject myself to the tedium I thought I'd put myself out there and ask.
i am new in the world of programming and python. I have a small flask app that consists of a form that ask users to upload a certain document. That document i then store into a variable in my python script called
file_upload = request.files['filename']
I want to send this same file storage into a folder i have in google drive.
The problem i am having is that every tutorial that i see is uploading files from a path in their computers, but i havent seen one that doesnt use a folder path.
(btw i am using GOOGLE APP ENGINE which doesnt allow me to store the file first to a directory and then to google drive)
Good Afternoon from Germany, everybody!
Google Colab and I seem to have divergent opinions on what is possible or not...
I just want a way to access the contents of all other cells from within a cell.
My use case is, that I want to POST the contents of a current Colab Notebook to an external server for grading with minimal user interaction (just running the cell).
So my question is: Is there a way to access the code cells of a Colab NB programmatically?
I saw this answer for Jupyter NB, but it does not work in Google Colab as the Jupyter JS-Variable is not available. The variable google.colab seems to not provide the same functionality, or am I missing something?
Google Colab seems to sandbox each cell in its one iframe, so I cannot query the contents of other cells via JS:
%%js
document.getElementsByClassName('cell')
When run in a cell this just leads to an empty HTMLCollection, when run in the Developer Tools of my browser I get the correct results.
Am I missing something here? Is there a way to escape the sandbox or access the current NB contents within a Python cell?
Thanks in Advance for your help!
Sure, here's a complete example:
https://colab.research.google.com/drive/1mXuyMsPEXFU4ik9EGLBiWUuNfztf7J6_
The key bit is this:
# Obtain the notebook JSON as a string
from google.colab import _message
notebook_json_string = _message.blocking_request('get_ipynb', request='', timeout_sec=5)
Reproducing the executed example from the notebook:
Probably not an ideal solution - but one option may be to read the notebook file directly instead of trying to access the cells internally. The following code mounts the user's google drive in the notebook so that you can read its contents as a file:
from pathlib import Path
from google.colab import drive
# Mount the users google drive into the notebook (takes the
# user to an auth flow for access).
drive.mount('/content/drive')
# Read the contents of the notebook file to POST to grading server.
base = Path('/content/drive/MyDrive/Colab Notebooks')
notebook_path = 'notebook.ipynb'
with open(base / notebook_path) as infile:
notebook_contents = infile.read()
Finding the path of the notebook seems to be tricky, but if it has a standard name you could search for it using base.rpath('*.ipynb') and present the user with some options of files to submit.
I am working with the Google Sheets API in Python but am having trouble finding out how to save a newly created Spreadsheet to a specific folder. As of now all it does is populate my drive without the ability to indicate an end location. Any idea how to do this in Python?
I have found help with other languages but the python documentation is much different. Any insight would be great...
Is there any way we can load direct excel file into BigQuery, instead of converting to CSV.
I get the files every days in excel format and need to load into BigQuery. Right now converting into CSV manually and loading into BigQuery.
Planning to schedule the job.
If not possible to load the excel files directly into BigQuery then I need to write a process(Python) to convert into CSV before loading into BigQuery.
Please let me know if any better options are there.
Thanks,
I think you could achieve above in a few clicks, without any code.
You need to use Google Drive and external (federated) tables.
1) You could upload manually you excel files to Google Drive or synchronise them
2) In Google Drive Settings find:
"**Convert uploads** [x] Convert uploaded files to Google Docs editor format"
and check it.
To access above option go to https://drive.google.com/drive/my-drive, click on the Gear settings icon and then choose Settings.
Now you excel files will be accessible by Big Query
3) Last part: https://cloud.google.com/bigquery/external-data-drive
You could access you excel file by URI: https://cloud.google.com/bigquery/external-data-drive#drive-uri and then create table manually using above uri.
You could do last step also by API.