If I set up a Google Sheets API instance and a Google Drive API instance and then connect to the Google Sheet using the credentials key from a python script (application) on my desktop. This script performs basic CRUD operations.
My question:
Is this connection secure? In other words does the data travel over the Internet plain text or encrypted?
If not secure...how can I ensure the data securely travels from python script to Google Sheets?
I have searched for API data integrity but no luck finding out if the connection to the API need TLA or SSL.
The network calls are secure since they must be using https calls and thus are not transported in plane text. Post a link to the python script if you want someone to check here.
You do have to trust google with all your data.
Related
I am trying to access a Google Sheet stored in my Drive through the Google Sheets REST API.
This will just be a Python script without any user interaction. How can I authenticate my request using something like an access key or a service account?
I understand the concept of generating access keys or creating a service account in my Google Cloud console. But, I don't quite understand how the Sheet in my Drive can be associated with it.
I would like to know the steps I should follow in order to accomplish this. For instance, how can I send a request to this API endpoint?
GET https://sheets.googleapis.com/v4/spreadsheets/{spreadsheetId}
Note: I want to do this using the REST API. I do not want to use a Python API that has already been developed. So, I simply want to hit the above endpoint using maybe the requests package.
Google does not permit API only access to Google (Workspace?) documents.
See Authorizing Requests
API keys authenticate programs.
OAuth is used to authenticate users and Google requires that users authenticate requests when access user data stored in Workspace documents.
Domain-wide Delegation enables the use of a Service Account to operate on behalf of users in situations such as this but it is only available for (paid) Workspace accounts.
I'm unsure how to refer to the free and paid (Workspace) versions.
This is what I want to achieve:
Ask the user to authorize the collection of their data on a Google Analytics 4 property (or Universal Analytics but I would rather not)
Programmatically retrieve and store the data every n-hours
I was able to do (1) client-side by asking for authorization with google's OAUTH2 and making a call to Reporting API v4 https://developers.google.com/analytics/devguides/reporting/core/v4 using gapi on the front-end.
However, I'm not sure how to do it on a schedule without user interaction. I've searched Google's API docs and I believe there's a way to do it in python https://developers.google.com/analytics/devguides/reporting/core/v4/quickstart/service-py but I am currently limited to Node and the browser. I guess I could make a server in python that does the data fetching and connects with the Node application, but that's yet another layer of complications that I'm trying to avoid. Is there a way to do everything in Node?
GCP APIs are all documented in a way which allows everyone to generate client libraries in a variety of languages, including node.js. The documentation for the node.js client for Analytics Reporting is here.
For the question of how to schedule this on GCP, I would recommend you to use Cloud Scheduler. This will hit an endpoint running on Cloud Run, which will do the actual work. Alternatively, if you already have a service running somewhere else, you can simply add the required endpoints there and point Cloud Scheduler to it.
The overall design which I would suggest you goes something like this:
Build a site which takes the user through the OAUTH2 login process,
requesting the relevant Google Analytics Reporting API scopes
required to make the request.
Store the obtained credentials in their user database.(preferably
Firestore in Datastore mode)
Set up a Cloud Run service (or anything else), with two endpoints
Iteration endpoint: Iterate through the list of users and add tasks
to Cloud Tasks to hit the download endpoint for each one.
Download endpoint: Takes a user ID (e.g. as a query parameter) and
performs the download for this user. You will need to load the
credentials for the user from the database and use this to access the
reporting API.
Store the downloaded data in the desired location, e.g. Cloud
Storage, Firestore, Cloud SQL, etc.
Set up Cloud Scheduler to hit the iteration endpoint at the desired
frequency.
For the GCP services mentioned above, basically everything other than Analytics, you may use the "cloud" clients for node.js, which are available here
Note : The question you have asked is a very broad question and this answer is just a suggestion. You may think about other designs whichever works best for you.
For the Google Drive Python API, in all the tutorials I have seen, they require
users to create a project in their Google Dashboard, before obtaining a client ID and a client secret json file. I've been researching both the default Google Drive API and the pydrive module.
Is there a way for users to simply login to their Google Account with username and password,
without having to create a project? So once they login to their Google Account, they are free to
access all files in their Google Drive?
It's not possible to use the Drive API without creating a GCP project for the application. Otherwise Google has no idea what application is requesting access, and what scope of account access it should have.
Using simply a username and password to log in is not possible. You need to create a project and use OAuth.
it might be possible using some pysimplegui hackery or just simply modifying the code of a python based browser but in most cases it is not practical
except if you need to automate something (like renaming files ) that would take 1 hour in a place where you do not have access to GCP
Preface: I work at an agency with access to many clients google analytics accounts. I am working to set up an API to pipe all the clients data to a data warehouse.When I try to use the python API library, I created a service account that needs to be added as a user by each client. This can take a while depending on the client.
Question: why is it that if I wanted to use something like stitch data, that they can access all of my client's data without having a service account of theirs added to each individual client? How does the singer-tap used for this work to avoid this service account problem?
Is there any way using OAuth to download all content of a google bucket directory to a local directory.
I found two ways using (get request object) from storage API and gsutil. But since API uses direct name downloading I have to first parse all the list of bucket content and then send get request and then download it. I find gsutil more convenient but for this, I have to hard code details for the credential.
Basically, i am developing a client related application where I have to download the big query table data to the client local server
Can anyone help me for this
Unless your application knows ahead of time the object names that you want to download, you'll need to perform a list followed by GETs for each object.
You can use the gcloud-python client library to do this. You can configure your client application has the OAuth2 credentials and the library should handle the rest of the necessary authentication for you. See the documentation here for the basics of authentication, and [here](https://googlecloudplatform.github.io/google-cloud-python/stable/storage-blobs.html for interacting with Google Cloud Storage objects.