We Keep Coding

How to get Google OAUTH Token without using Gcloud Command Line

I am currently using the following code to get the OAUTH Token
command = 'gcloud auth print-access-token'
result = str(subprocess.Popen(command, universal_newlines=True, shell=True, stdout=subprocess.PIPE,
stderr=subprocess.PIPE).communicate())
The result variable has the OAUTH Token. This technique uses my current logged in gcloud config.
However, I am looking out for a way to get the OAUTH Token without using command line.
I am using this OAUTH Token to make CDAP calls to get the Google Dataflow Pipeline Execution Details.
I checked some google blogs. This is the one I think should try but it asks to create consent screen and it will require one time activity to provide consent to the scopes defined and then it should work.
Google Document
Shall I follow steps in above document and check OR is there any other way we can get the OAUTH Token?
Is there a way to get authentication done by service account instead of google user account and get the OAUTH Token?

For automated process, service account is the recommended way. You can use the google-oauth library for this. You can generate an access token like this
# With default credential (your user account or the Google Cloud Component service account.
# Or with the service account key file defined in the GOOGLE_APPLICATION_CREDENTIALS env var -> for platform outside GCP)
credentials, project_id = google.auth.default(scopes=["https://www.googleapis.com/auth/cloud-platform"])
# With service account key file (not recommended)
# credentials = service_account.Credentials.from_service_account_file('service-account.json',
# scopes=["https://www.googleapis.com/auth/cloud-platform"])
from google.auth.transport import requests
credentials.refresh(requests.Request())
print(credentials.token)
However, if you want to call Google cloud APIs, I recommend you to use authorized request object
Here an example of BigQuery call. You can use service account key file to generate your credential as in my previous example.
base_url = 'https://bigquery.googleapis.com'
credentials, project_id = google.auth.default(scopes=['https://www.googleapis.com/auth/cloud-platform'])
project_id = 'MyProjectId'
authed_session = AuthorizedSession(credentials)
response = authed_session.request('GET', f'{base_url}/bigquery/v2/projects/{project_id}/jobs')
print(response.json())
EDIT
When you want to use Google APIs, a service account key file is not needed (and I recommend you to not use it) on your computer and on GCP component. The Application Default Credential is always sufficient.
When you are in your local environment, you must run the command gcloud auth application-default login. With this command, you will register your personal account as default credential when you run locally your app. (of course, you need to have your user account email authorized on the component that you call)
When you are on GCP environment, each component have a default service account (or you can specify one with you configure your component). Thanks to the component "identity", you can use the default credential. (of course, you need to have the service account email authorized on the component that you call)
ONLY when you run an app automatically and outside GCP, you need a service account key file (for example, in your CI/CD other that Cloud Build, or in an app deployed on other Cloud Provider or on premise)
Why service account key file is not recommended? It's at least my recommendation because this file is ..... a file!! That's the problem. You have a way to authenticate a service account in a simple file: you have to store it securely (it's a secret and an authentication method!!), you can copy it, you can send it by email, you can even commit it in a public GIT repository... In addition, Google recommend to rotate them every 90 days, so it's a nightmare to rotate, to trace and to manage

How can I trigger an HTTP Cloud Function from within Google App Engine?

I have a Flask app in Google App Engine Standard Environment Python, and I also have a Cloud Function with an HTTP trigger which accepts a JSON body including the URL of a file. The CF downloads the file at that URL then saves it to a GCS bucket. The GAE service account has Cloud Function Invoker permissions, yet when using urlfetch.fetch() in my GAE code to trigger CF, the App Engine code gets a 403 Forbidden error unless I make the CF trigger callable by anyone.
How do I successfully call/trigger CF's from GAE in Python? I assume the answer is one of these:
Set IAM permissions on GAE service account to {enlighten me here}
Add authentication headers in urlfetch.fetch() like so {different enlightenment}
Make CF triggerable from anywhere, but hard code some secret key so the CF code itself handles authentication.

It's well documented here: Cloud Functions Authentication
In short you have to provide your service account credentials in the authentication header.
To get your credentials use the Google Auth Client library. If you are testing from local you should create a service account JSON and load it to the environment variable GOOGLE_APPLICATION_CREDENTIALS but on App Engine it will work from scratch.
After you have gotten your token, pass it as an auth header like so:
auth_req = google.auth.transport.requests.Request()
auth_token = google.oauth2.id_token.fetch_id_token(auth_req, cloud_function_url)
response = requests.post(cloud_function_url, json=payload, headers={"Authorization" : f"Bearer {auth_token}"})

Access denied for service account when accessing IAP-protected application

Trying to let a python command line to access(HTTP GET) IAP-protected application.
With https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/iap/make_iap_request.py
But 403 err occurs Exception: Service account xxx#xxxxxxxxx-production.iam.gserviceaccount.com does not have permission to access the IAP-protected application.
Following methods have been tried, but still issue persists as above
1. remove/create the service account
2. re-create the json key file
3. grant the service account project owner permission
if __name__ == '__main__':
os.environ[
"GOOGLE_APPLICATION_CREDENTIALS"] = '/Users/foo/bar.json'
print make_iap_request('https://foo.com/', 'xxxxtheidforthatiapservice.apps.googleusercontent.com')

I think you need to follow the Authenticating from a service account procedure:
Use an OpenID Connect (OIDC) token to authenticate a service account
to a Cloud IAP-secured resource.
Add the service account to the access list for the Cloud IAP-secured project.
Generate a JWT-based access token. This uses a target_audience additional claim that requires a client ID. To find
your client ID, follow the steps below:
a. Go to the Cloud IAP page.
b. Find the resource you want to access, then click More > Edit
OAuth Client.
edit OAuth client on the More menu
c. On the Credentials page that appears, note the client ID.
Request an OIDC token for the Cloud IAP-secured client ID.
Include the OIDC token in an Authorization: Bearer header to make the authenticated request to the Cloud IAP-secured resource.

I faced the same problem.
You have to run the script authenticated as a service account used to access the IAP-protected application programmatically. You can do so either by running script directly from VM which uses that service account as default, or you will need to download SA credentials and do it locally Obtaining an OIDC token from a local service account key file
Besides that, you also need to assign the IAP-secured Web App User role to the desired Service Account for that particular IAP-protected app.
After that, rerun the script.

Unauthorized error in GAE SDK, but it works once deployed

I am running this code in a small example:
from google.cloud import storage
from google.appengine.api import app_identity
class TestB(base_handler.TRNHandler):
#...
def post(self):
client = storage.Client()
bucket_name = os.environ.get('BUCKET_NAME',
app_identity.get_default_gcs_bucket_name())
bucket = client.get_bucket(bucket_name)
#...
If I deploy this code everything works as expected. But when I run it locally (SDK), I get an error: Unauthorized: 401 Invalid Credentials. What's happening and how can I fix it?

I've got a pretty strong guess, although I can't be sure without seeing your exact logs and whatnot.
The google.cloud library is smart about authorization. It uses a thing called "application default credentials." If you run that code on App Engine or on a GCE instance, the code will be able to figure out which service account is associated with that instance and authorize itself with the credentials of that account.
However, when you run the program locally, the library has no way of knowing which credentials to use, and so it just makes calls anonymously. Your bucket probably hasn't granted anonymous users access (which is good), and so the call fails with a 401.
You can, however, register credentials locally with the gcloud command:
$> gcloud auth application-default login
Run that, and the library will use whatever credentials you've used to log in for a while. Alternatively, you could also make sure that the environment variable GOOGLE_APPLICATION_CREDENTIALS points to a service account's JSON key file.
There's a bunch of documentation on exactly how Application Default Credentials pick a credential.
Alternately, if you'd prefer to specify auth right in the program, you can do that too:
storage = Storage.from_service_account_json('/path/to/key_file.json')

python3 upload files to ondrive or sharepoint?

Anyone know if this is possible?
I just want to automate dropping some documents into my onedrive for business account.
I tried
import onedrivesdk
from onedrivesdk.helpers import GetAuthCodeServer
from onedrivesdk.helpers.resource_discovery import ResourceDiscoveryRequest
redirect_uri = 'http://localhost:8080'
client_id = 'appid'
client_secret = 'mysecret'
discovery_uri = 'https://api.office.com/discovery/'
auth_server_url='https://login.live.com/oauth20_authorize.srf?scope=wl.skydrive_update'
#auth_server_url='https://login.microsoftonline.com/common/oauth2/authorize',
auth_token_url='https://login.microsoftonline.com/common/oauth2/token'
http = onedrivesdk.HttpProvider()
auth = onedrivesdk.AuthProvider(http,
client_id,
auth_server_url=auth_server_url,
auth_token_url=auth_token_url)
auth_url = auth.get_auth_url(redirect_uri)
code = GetAuthCodeServer.get_auth_code(auth_url, redirect_uri)
auth.authenticate(code, redirect_uri, client_secret, resource=resource)
# If you have access to more than one service, you'll need to decide
# which ServiceInfo to use instead of just using the first one, as below.
service_info = ResourceDiscoveryRequest().get_service_info(auth.access_token)[0]
auth.redeem_refresh_token(service_info.service_resource_id)
client = onedrivesdk.OneDriveClient(service_info.service_resource_id + '/_api/v2.0/', auth, http)
I registered an APP and got a secret and id. But when I ran this I got scope is invalid errors. Plus it tries to launch a webpage which isn't great for a command line kinda environment. I think this SDK might be outdated as well because originally this script had login.microsoftonline, but that wasn't reachable so I changed it to login.live.com.

I wrote this sample code you posted. You replaced the auth_server_URLwith the authentication URL for Microsoft Account authentication, which can only be used to access OneDrive (the consumer product). You need to continue using the login.microsoftonline.com URL to log into your OneDrive for Business account.
You are correct that this pops up a dialog. However, you can write a little supporting code so that only happens the first time you log into a particular app. Follow these steps (assuming you are using the default implementation of AuthProvider:
Use the sample code above up through the line auth.redeem_refresh_token()
The AuthProvider will now have a Session object, which caches the credentials of the current user and session. Use AuthProvider.save_session() to save the credentials for later.
Next time you start your app, use AuthProvider.load_session() and AuthProvider.refresh_token() to retrieve the previous session and refresh the auth token. This will all be headless.
Take note that the default implementation of SessionBase (found here) uses Pickle and is not safe for product use. Make sure to create a new implementation of Session if you intend to deploy this app to other users.

Onerive's website shows "Not Yet" on "OneDrive SDK for Python" to "OneDrive for Business"
https://dev.onedrive.com/SDKs.htm
The github sample codes did not work for me either, it tried to popup a window of authentication, but IE can not find the address:
http://('https//login.microsoftonline.com/common/oauth2/authorize',)?redirect_uri=http%3A%2F%2Flocalhost%3A8080&client_id=034xxxx9-9xx8-4xxf-bexx-1bc5xxxxbd0c&response_type=code
or removed all the "-" in client id
http://('https//login.microsoftonline.com/common/oauth2/authorize',)?redirect_uri=http%3A%2F%2Flocalhost%3A8080&client_id=034xxxx99xx84xxfbexx1bc5xxxxbd0c&response_type=code
Either way, I got the same result, IE did not show the popup with a line "This page can’t be displayed"