I have a Python code that was using SendGrid API to send emails, but now I want to migrate to Google in order to send business emails. It also runs in Docker containers.
I followed Gmail Python Quickstart in order to use Gmail API in my Python code and the problem is that when trying to send email, it shows an authorization link in Docker logs in order to get token, etc.
Is there a way to complete authorization in the background without any further interaction or use an API key just like SendGrid to programmatically authenticate your application?
I am a service provider and want to send emails such as reset password links, confirmation code, etc. automatically; the code is deployed on a Linux host. I have access to workspace account, and I already have verified my domain.
The tutorial you are following is designed for an installed application. Hence the InstalledAppFlow.
It states it at the top of the file.
Authorization credentials for a desktop application. To learn how to create credentials for a desktop application, refer to Create credentials.
This means when your code runs it is going to pop up the consent screen on the machine the code is running on, in this instance Docker.
flow = InstalledAppFlow.from_client_secrets_file(
'credentials.json', SCOPES)
You need to create it using a web application so that your users can consent to your application accessing their data and their Gmail account.
Automated send emails service account option.
You were not clear as who you are sending emails for. As you are using send grid, it implies to me this is some kind of automated system. Which would mean that you are trying to send emails on behalf of a Gmail account that you control.
In that case you would most often want to use a service account. Service accounts allow for server-to-server interaction between Google APIs. However, service accounts will only work with Gmail if this is a Google workspace Gmail account and you can set up domain-wide delegation.
There is an example in the documentation. Just change it to Gmail scopes. The key point is the create_delegated which must be a user on your domain.
from googleapiclient.discovery import build
from oauth2client.service_account import ServiceAccountCredentials
# Email of the Service Account
SERVICE_ACCOUNT_EMAIL = '<some-id>#developer.gserviceaccount.com'
# Path to the Service Account's Private Key file
SERVICE_ACCOUNT_PKCS12_FILE_PATH = '/path/to/<public_key_fingerprint>-privatekey.p12'
def create_directory_service(user_email):
"""Build and returns an Admin SDK Directory service object authorized with the service accounts
that act on behalf of the given user.
Arguments:
user_email: The email of the user. Needs permissions to access the Admin APIs.
Returns:
Admin SDK directory service object.
"""
credentials = ServiceAccountCredentials.from_p12_keyfile(
SERVICE_ACCOUNT_EMAIL,
SERVICE_ACCOUNT_PKCS12_FILE_PATH,
'notasecret',
scopes=['https://www.googleapis.com/auth/admin.directory.user'])
credentials = credentials.create_delegated(user_email)
return build('admin', 'directory_v1', credentials=credentials)
Standard Gmail solution
You can run your application once, and then when you place it in the Docker container, make sure that you include the token.json file that was created this is the file that contains the credentials that grant the application access to your account.
If you open it you will find an access token and a refresh token within. The refresh token will give your application the ability to request a new access token whenever it needs one.
Related
I am currently using the following code to get the OAUTH Token
command = 'gcloud auth print-access-token'
result = str(subprocess.Popen(command, universal_newlines=True, shell=True, stdout=subprocess.PIPE,
stderr=subprocess.PIPE).communicate())
The result variable has the OAUTH Token. This technique uses my current logged in gcloud config.
However, I am looking out for a way to get the OAUTH Token without using command line.
I am using this OAUTH Token to make CDAP calls to get the Google Dataflow Pipeline Execution Details.
I checked some google blogs. This is the one I think should try but it asks to create consent screen and it will require one time activity to provide consent to the scopes defined and then it should work.
Google Document
Shall I follow steps in above document and check OR is there any other way we can get the OAUTH Token?
Is there a way to get authentication done by service account instead of google user account and get the OAUTH Token?
For automated process, service account is the recommended way. You can use the google-oauth library for this. You can generate an access token like this
# With default credential (your user account or the Google Cloud Component service account.
# Or with the service account key file defined in the GOOGLE_APPLICATION_CREDENTIALS env var -> for platform outside GCP)
credentials, project_id = google.auth.default(scopes=["https://www.googleapis.com/auth/cloud-platform"])
# With service account key file (not recommended)
# credentials = service_account.Credentials.from_service_account_file('service-account.json',
# scopes=["https://www.googleapis.com/auth/cloud-platform"])
from google.auth.transport import requests
credentials.refresh(requests.Request())
print(credentials.token)
However, if you want to call Google cloud APIs, I recommend you to use authorized request object
Here an example of BigQuery call. You can use service account key file to generate your credential as in my previous example.
base_url = 'https://bigquery.googleapis.com'
credentials, project_id = google.auth.default(scopes=['https://www.googleapis.com/auth/cloud-platform'])
project_id = 'MyProjectId'
authed_session = AuthorizedSession(credentials)
response = authed_session.request('GET', f'{base_url}/bigquery/v2/projects/{project_id}/jobs')
print(response.json())
EDIT
When you want to use Google APIs, a service account key file is not needed (and I recommend you to not use it) on your computer and on GCP component. The Application Default Credential is always sufficient.
When you are in your local environment, you must run the command gcloud auth application-default login. With this command, you will register your personal account as default credential when you run locally your app. (of course, you need to have your user account email authorized on the component that you call)
When you are on GCP environment, each component have a default service account (or you can specify one with you configure your component). Thanks to the component "identity", you can use the default credential. (of course, you need to have the service account email authorized on the component that you call)
ONLY when you run an app automatically and outside GCP, you need a service account key file (for example, in your CI/CD other that Cloud Build, or in an app deployed on other Cloud Provider or on premise)
Why service account key file is not recommended? It's at least my recommendation because this file is ..... a file!! That's the problem. You have a way to authenticate a service account in a simple file: you have to store it securely (it's a secret and an authentication method!!), you can copy it, you can send it by email, you can even commit it in a public GIT repository... In addition, Google recommend to rotate them every 90 days, so it's a nightmare to rotate, to trace and to manage
I am using a serverless Python function for retrieving G Suite logs. I have done the following to authenticate against Admin SDK:
1) Created a project
2) Created a service account with the Project->Owner role
3) Create a private key (JSON) file
4) Enabled domain-wide delegation
5) Enabled the Admin SDK
6) Delegate domain-wide authority via 'One or More API Scopes' by authorizing my service account with https://www.googleapis.com/auth/admin.reports.audit.readonly
Now, when running my authorized application I use 1) the username (privileged user's email address) with access to the created service account and 2) the private key (JSON) file for the service account.
When authorizing my application, I use ServiceAccountCredentials from the oauth2client.service_account package.
The Question:
When running my application, I can't authorize against G Suite unless I use the username (email address) of a privileged user. When I substitute this email address for the service account's email address, I can't retrieve G Suite logs.
Is it possible to configure the service account to allow authorization via the service account email address? If so, how can I do that?
I have found the answer to the question. A service account's email address cannot be used.
The proper steps for retrieving G Suite logs is to either 1) use or 2) create a user with the access rights for reviewing the logs.
If a user must be created, this can be done at the user's console on from Google Admin console. After, the appropriate role(s) can be created at Google IAM. Finally, these roles can be applied to the user at the Google Admin console.
Trying to let a python command line to access(HTTP GET) IAP-protected application.
With https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/iap/make_iap_request.py
But 403 err occurs Exception: Service account xxx#xxxxxxxxx-production.iam.gserviceaccount.com does not have permission to access the IAP-protected application.
Following methods have been tried, but still issue persists as above
1. remove/create the service account
2. re-create the json key file
3. grant the service account project owner permission
if __name__ == '__main__':
os.environ[
"GOOGLE_APPLICATION_CREDENTIALS"] = '/Users/foo/bar.json'
print make_iap_request('https://foo.com/', 'xxxxtheidforthatiapservice.apps.googleusercontent.com')
I think you need to follow the Authenticating from a service account procedure:
Use an OpenID Connect (OIDC) token to authenticate a service account
to a Cloud IAP-secured resource.
Add the service account to the access list for the Cloud IAP-secured project.
Generate a JWT-based access token. This uses a target_audience additional claim that requires a client ID. To find
your client ID, follow the steps below:
a. Go to the Cloud IAP page.
b. Find the resource you want to access, then click More > Edit
OAuth Client.
edit OAuth client on the More menu
c. On the Credentials page that appears, note the client ID.
Request an OIDC token for the Cloud IAP-secured client ID.
Include the OIDC token in an Authorization: Bearer header to make the authenticated request to the Cloud IAP-secured resource.
I faced the same problem.
You have to run the script authenticated as a service account used to access the IAP-protected application programmatically. You can do so either by running script directly from VM which uses that service account as default, or you will need to download SA credentials and do it locally Obtaining an OIDC token from a local service account key file
Besides that, you also need to assign the IAP-secured Web App User role to the desired Service Account for that particular IAP-protected app.
After that, rerun the script.
I'm writing a small python script that will retrieve a list of my Google Contacts (using the Google Contacts API) and will randomly suggest one person for me to contact (good way to automate keeping in touch with friends!)
This is just a standalone script that I plan to schedule on a cron job. The problem is that Google seems to require OAuth2 style authentication, where the user (me) has to approve the access and then the app receives an authorization token I can then use to query the user's (my) contacts.
Since I'm only accessing my own data, is there a way to "pre-authorize" myself? Ideally I'd love to be able to retrieve some authorization token and then I'd run the script and pass that token as an environment variable
AUTH_TOKEN=12345 python my_script.py
That way it doesn't require user input/interaction to authorize it one time.
The implementation you're describing invokes the full "three-legged" OAuth handshake, which requires explicit user consent. If you don't need user consent, you can instead utilize "two-legged" OAuth via a Google service account, which is tied to an application, rather than a user. Once you've granted permission to your service account to access your contacts, you can use the oauth2client ServiceAccountCredentials class to directly access GData without requiring user consent.
Here's the two-legged authentication example from the Google service account documentation:
import json
from httplib2 import Http
from oauth2client.service_account import ServiceAccountCredentials
from apiclient.discovery import build
scopes = ['https://www.googleapis.com/auth/sqlservice.admin']
credentials = ServiceAccountCredentials.from_json_keyfile_name(
'service-account.json', scopes)
sqladmin = build('sqladmin', 'v1beta3', credentials=credentials)
response = sqladmin.instances().list(project='examinable-example-123').execute()
print response
I am trying to check if the user has installed my marketplace app
from oauth2client.client import SignedJwtAssertionCredentials
import json
from httplib2 import Http
from apiclient.discovery import build
fname = 'my-creds.json'
scope = [
'https://www.googleapis.com/auth/appsmarketplace.license'
]
applicationId='12345678'
customerId='mydomain.com'
json_key = json.load(open(fname))
credentials = SignedJwtAssertionCredentials(json_key['client_email'], json_key['private_key'], scope)
http_auth = credentials.authorize(Http())
appsmarket = build('appsmarket', 'v2', http=http_auth)
response = appsmarket.customerLicense().get(applicationId=applicationId, customerId=customerId).execute()
print response
I have created a service account in the same project as my app in Google console.
But I keep getting
googleapiclient.errors.HttpError: <HttpError 403 when requesting https://www.googleapis.com/appsmarket/v2/customerLicense/471879773528/ashish%40noodletools.com?alt=json returned "Not authorized to access the application ID">
Do you add service account to your domain's authorized API clients in security section?
Look at Delegating domain-wide authority to the service account here: https://developers.google.com/identity/protocols/OAuth2ServiceAccount
To delegate domain-wide authority to a service account, an administrator of the Google Apps domain must complete the following steps:
Go to your Google Apps domain’s Admin console.
Select Security from the list of controls. If you don't see Security listed, select More controls from the gray bar at the bottom of the page, then select Security from the list of controls. If you can't see the controls, make sure you're signed in as an administrator for the domain.
Select Show more and then Advanced settings from the list of options.
Select Manage API client access in the Authentication section.
In the Client Name field enter the service account's Client ID.
In the One or More API Scopes field enter the list of scopes that your application should be granted access to.
Click Authorize.
Your application now has the authority to make API calls as users in your domain (to "impersonate" users). When you prepare to make authorized API calls, you specify the user to impersonate.