We Keep Coding

Sending automated emails using Gmail API with Python and OAuth authentication

I have a Python code that was using SendGrid API to send emails, but now I want to migrate to Google in order to send business emails. It also runs in Docker containers.
I followed Gmail Python Quickstart in order to use Gmail API in my Python code and the problem is that when trying to send email, it shows an authorization link in Docker logs in order to get token, etc.
Is there a way to complete authorization in the background without any further interaction or use an API key just like SendGrid to programmatically authenticate your application?
I am a service provider and want to send emails such as reset password links, confirmation code, etc. automatically; the code is deployed on a Linux host. I have access to workspace account, and I already have verified my domain.

The tutorial you are following is designed for an installed application. Hence the InstalledAppFlow.
It states it at the top of the file.
Authorization credentials for a desktop application. To learn how to create credentials for a desktop application, refer to Create credentials.
This means when your code runs it is going to pop up the consent screen on the machine the code is running on, in this instance Docker.
flow = InstalledAppFlow.from_client_secrets_file(
'credentials.json', SCOPES)
You need to create it using a web application so that your users can consent to your application accessing their data and their Gmail account.
Automated send emails service account option.
You were not clear as who you are sending emails for. As you are using send grid, it implies to me this is some kind of automated system. Which would mean that you are trying to send emails on behalf of a Gmail account that you control.
In that case you would most often want to use a service account. Service accounts allow for server-to-server interaction between Google APIs. However, service accounts will only work with Gmail if this is a Google workspace Gmail account and you can set up domain-wide delegation.
There is an example in the documentation. Just change it to Gmail scopes. The key point is the create_delegated which must be a user on your domain.
from googleapiclient.discovery import build
from oauth2client.service_account import ServiceAccountCredentials
# Email of the Service Account
SERVICE_ACCOUNT_EMAIL = '<some-id>#developer.gserviceaccount.com'
# Path to the Service Account's Private Key file
SERVICE_ACCOUNT_PKCS12_FILE_PATH = '/path/to/<public_key_fingerprint>-privatekey.p12'
def create_directory_service(user_email):
"""Build and returns an Admin SDK Directory service object authorized with the service accounts
that act on behalf of the given user.
Arguments:
user_email: The email of the user. Needs permissions to access the Admin APIs.
Returns:
Admin SDK directory service object.
"""
credentials = ServiceAccountCredentials.from_p12_keyfile(
SERVICE_ACCOUNT_EMAIL,
SERVICE_ACCOUNT_PKCS12_FILE_PATH,
'notasecret',
scopes=['https://www.googleapis.com/auth/admin.directory.user'])
credentials = credentials.create_delegated(user_email)
return build('admin', 'directory_v1', credentials=credentials)
Standard Gmail solution
You can run your application once, and then when you place it in the Docker container, make sure that you include the token.json file that was created this is the file that contains the credentials that grant the application access to your account.
If you open it you will find an access token and a refresh token within. The refresh token will give your application the ability to request a new access token whenever it needs one.

Python Generating an IAM authentication token boto3.session

I am trying to use the documentation on https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.IAMDBAuth.Connecting.Python.html. Right now I am stuck at session = boto3.session(profile_name='RDSCreds'). What is profile_name and how do I find that in my RDS?
import sys
import boto3
ENDPOINT="mysqldb.123456789012.us-east-1.rds.amazonaws.com"
PORT="3306"
USR="jane_doe"
REGION="us-east-1"
os.environ['LIBMYSQL_ENABLE_CLEARTEXT_PLUGIN'] = '1'
#gets the credentials from .aws/credentials
session = boto3.Session(profile_name='RDSCreds')
client = session.client('rds')
token = client.generate_db_auth_token(DBHostname=ENDPOINT, Port=PORT, DBUsername=USR, Region=REGION)

session = boto3.Session(profile_name='RDSCreds')
profile_name here means the name of the profile you have configured to use for your aws cli.
usually when you run aws configure it creates a default profile.But sometime users want to manage aws cli with another account credentials or amange request for another region so they configure separate profile. docs for creating configuring multiple profiles
aws configure --profile RDSCreds #enter your access keys for this profile
in case if you think you have already created RDSCreds profile to check that profile less ~/.aws/config
the documentation which you have mentioned for rds using boto3 also says "The code examples use profiles for shared credentials. For information about the specifying credentials, see Credentials in the AWS SDK for Python (Boto3) documentation."

How to get Google OAUTH Token without using Gcloud Command Line

I am currently using the following code to get the OAUTH Token
command = 'gcloud auth print-access-token'
result = str(subprocess.Popen(command, universal_newlines=True, shell=True, stdout=subprocess.PIPE,
stderr=subprocess.PIPE).communicate())
The result variable has the OAUTH Token. This technique uses my current logged in gcloud config.
However, I am looking out for a way to get the OAUTH Token without using command line.
I am using this OAUTH Token to make CDAP calls to get the Google Dataflow Pipeline Execution Details.
I checked some google blogs. This is the one I think should try but it asks to create consent screen and it will require one time activity to provide consent to the scopes defined and then it should work.
Google Document
Shall I follow steps in above document and check OR is there any other way we can get the OAUTH Token?
Is there a way to get authentication done by service account instead of google user account and get the OAUTH Token?

For automated process, service account is the recommended way. You can use the google-oauth library for this. You can generate an access token like this
# With default credential (your user account or the Google Cloud Component service account.
# Or with the service account key file defined in the GOOGLE_APPLICATION_CREDENTIALS env var -> for platform outside GCP)
credentials, project_id = google.auth.default(scopes=["https://www.googleapis.com/auth/cloud-platform"])
# With service account key file (not recommended)
# credentials = service_account.Credentials.from_service_account_file('service-account.json',
# scopes=["https://www.googleapis.com/auth/cloud-platform"])
from google.auth.transport import requests
credentials.refresh(requests.Request())
print(credentials.token)
However, if you want to call Google cloud APIs, I recommend you to use authorized request object
Here an example of BigQuery call. You can use service account key file to generate your credential as in my previous example.
base_url = 'https://bigquery.googleapis.com'
credentials, project_id = google.auth.default(scopes=['https://www.googleapis.com/auth/cloud-platform'])
project_id = 'MyProjectId'
authed_session = AuthorizedSession(credentials)
response = authed_session.request('GET', f'{base_url}/bigquery/v2/projects/{project_id}/jobs')
print(response.json())
EDIT
When you want to use Google APIs, a service account key file is not needed (and I recommend you to not use it) on your computer and on GCP component. The Application Default Credential is always sufficient.
When you are in your local environment, you must run the command gcloud auth application-default login. With this command, you will register your personal account as default credential when you run locally your app. (of course, you need to have your user account email authorized on the component that you call)
When you are on GCP environment, each component have a default service account (or you can specify one with you configure your component). Thanks to the component "identity", you can use the default credential. (of course, you need to have the service account email authorized on the component that you call)
ONLY when you run an app automatically and outside GCP, you need a service account key file (for example, in your CI/CD other that Cloud Build, or in an app deployed on other Cloud Provider or on premise)
Why service account key file is not recommended? It's at least my recommendation because this file is ..... a file!! That's the problem. You have a way to authenticate a service account in a simple file: you have to store it securely (it's a secret and an authentication method!!), you can copy it, you can send it by email, you can even commit it in a public GIT repository... In addition, Google recommend to rotate them every 90 days, so it's a nightmare to rotate, to trace and to manage

Calling onedrive methods with graph api

I've purchased MS Office and they give me a space on onedrive.
Now I want to operate with that via ms graph API in my python console application (I used python onedrivesdk before but now it is said that it is deprecated).
I have registered an application and can see it via Azure AD on portal.azure.com.
Currently I'm trying to interact with my onedrive like this:
tenant_id = 'xxxxxx-xxxx-xxxx-xxxx-xxxxxxxxx'
authority = f'https://login.microsoftonline.com/{tenant_id}'
scope = 'https://graph.microsoft.com/.default'
app = msal.ConfidentialClientApplication(self.client_id, authority=authority, client_credential=self.client_secret)
result = app.acquire_token_silent(["https://graph.microsoft.com/.default"], account=None)
if not result:
logging.info("No suitable token exists in cache. Let's get a new one from AAD.")
result = app.acquire_token_for_client(scopes=["https://graph.microsoft.com/.default"])
authToken = result['access_token']
#### Make a call to the graph API
graphResponse = requests.get('https://graph.microsoft.com/beta/me/drive',headers={'Authorization':f'Bearer {authToken}'})
if graphResponse.status_code != 200:
print('Error code: ', graphResponse.status_code)
print(graphResponse.text)
I successfully get an access token, but when I try to call /me/drive
I get status_code = 400 with
Current authenticated context is not valid for this request. This
occurs when a request is made to an endpoint that requires user
sign-in. For example, /me requires a signed-in user. Acquire a token
on behalf of a user to make requests to these endpoints. Use the
OAuth 2.0 authorization code flow for mobile and native apps and the
OAuth 2.0 implicit flow for single-page web apps
I've add permissions for the application on the portal via "API permission -> Add permission", but I'm unable to grant admin consent (In another ms account I have full fledged azure subscription where I'm an admin) because I'm not an admin. But who is admin of this account my MS office assigned to?

According to the code you provided, you use OAuth 2.0 client credentials flow to complete Azure AD auth and get access token. The access token required by service principal. We cannot use the access token to call /me/drive endpoint. We just can use the access token to call /users/<UserObjectIdOrUserPrincipalName}>/drive endpoint. For more details, please refer to the document
So if you want to call /me/drive endpoint, I suggest you use the OAuth 2.0 authorization code flow. Regarding how to implement it in your application, please refer to the sample.

Access denied for service account when accessing IAP-protected application

Trying to let a python command line to access(HTTP GET) IAP-protected application.
With https://github.com/GoogleCloudPlatform/python-docs-samples/blob/master/iap/make_iap_request.py
But 403 err occurs Exception: Service account xxx#xxxxxxxxx-production.iam.gserviceaccount.com does not have permission to access the IAP-protected application.
Following methods have been tried, but still issue persists as above
1. remove/create the service account
2. re-create the json key file
3. grant the service account project owner permission
if __name__ == '__main__':
os.environ[
"GOOGLE_APPLICATION_CREDENTIALS"] = '/Users/foo/bar.json'
print make_iap_request('https://foo.com/', 'xxxxtheidforthatiapservice.apps.googleusercontent.com')

I think you need to follow the Authenticating from a service account procedure:
Use an OpenID Connect (OIDC) token to authenticate a service account
to a Cloud IAP-secured resource.
Add the service account to the access list for the Cloud IAP-secured project.
Generate a JWT-based access token. This uses a target_audience additional claim that requires a client ID. To find
your client ID, follow the steps below:
a. Go to the Cloud IAP page.
b. Find the resource you want to access, then click More > Edit
OAuth Client.
edit OAuth client on the More menu
c. On the Credentials page that appears, note the client ID.
Request an OIDC token for the Cloud IAP-secured client ID.
Include the OIDC token in an Authorization: Bearer header to make the authenticated request to the Cloud IAP-secured resource.

I faced the same problem.
You have to run the script authenticated as a service account used to access the IAP-protected application programmatically. You can do so either by running script directly from VM which uses that service account as default, or you will need to download SA credentials and do it locally Obtaining an OIDC token from a local service account key file
Besides that, you also need to assign the IAP-secured Web App User role to the desired Service Account for that particular IAP-protected app.
After that, rerun the script.