We Keep Coding

Invalid response 405 when using airflow GoogleBaseHook in python

I am running airflow in docker. I create an airflow Google Cloud connection through web server GUI on my localhost:8080. In python code, I want to read data in Gsheet to with GoogleBaseHook as:
import json
from googleapiclient.discovery import build
from airflow.providers.google.common.hooks.base_google import GoogleBaseHook
from oauth2client.service_account import ServiceAccountCredentials
gcp_hook = GoogleBaseHook(gcp_conn_id="spx-service-account")
cred_dict = json.loads(gcp_hook._get_field('keyfile_dict'))
SCOPES = ['https://www.googleapis.com/auth/spreadsheets', 'https://www.googleapis.com/auth/drive']
creds = ServiceAccountCredentials.from_json_keyfile_dict(cred_dict, scopes=SCOPES)
service = build('sheets', 'v4', credentials=creds, cache_discovery=False)
sheet = service.spreadsheets()
result = sheet.values().get(spreadsheetId="1yN3atY6NG7PfY8yNcNAiVqURra8WtQJWCKXc-ccymk0", range="Sheet1!A1:B4").execute()['values']
But executing the code above shows Failed to retrieve access token. 405 Not Allowed
Error:
Does anyone know what causes this error and how I can solve it? Thank you.

Oops, it ends up to be something wrong with my keyfile JSON. I have no idea why those token_uri is encoded when I downloaded it from my Gmail (tech team sent it to us via email) and opened it with my windows laptop. If your credential is correct, the code below should work:
import json
from googleapiclient.discovery import build
from airflow.providers.google.common.hooks.base_google import GoogleBaseHook
from oauth2client.service_account import ServiceAccountCredentials
gcp_hook = GoogleBaseHook(gcp_conn_id="spx-service-account")
cred_dict = json.loads(gcp_hook._get_field('keyfile_dict'))
SCOPES = ['https://www.googleapis.com/auth/spreadsheets', 'https://www.googleapis.com/auth/drive']
creds = ServiceAccountCredentials.from_json_keyfile_dict(cred_dict, scopes=SCOPES)
service = build('sheets', 'v4', credentials=creds, cache_discovery=False)
sheet = service.spreadsheets()
result = sheet.values().get(spreadsheetId="1yN3atY6NG7PfY8yNcNAiVqURra8WtQJWCKXc-ccymk0", range="Sheet1!A1:B4").execute()['values']

Unrecognized arguments using oauth2 and Google APIs

I'm using the Google API services in some scripts and having some problems. This error is something weird, but here we go.
I have a script that is listing my Google Drive files.
from apiclient import discovery
from httplib2 import Http
from oauth2client import file, client, tools
SCOPES = 'https://www.googleapis.com/auth/drive.readonly.metadata'
store = file.Storage('storage.json')
creds = store.get()
if not creds or creds.invalid:
flow = client.flow_from_clientsecrets('client_secret.json', SCOPES)
creds = tools.run_flow(flow, store)
DRIVE = discovery.build('drive', 'v3', http=creds.authorize(Http()))
files = DRIVE.files().list().execute().get('files', [])
for f in files:
print(f['name'], f['mimeType'],f['id'])
It works perfectly, I download the client_secret.json from Google API and save it in the same folder, then I start the script to check everythings is ok.
Then I start to edit my file to change the way how I execute it and don't read a file, instead, call the script and send the client_id and client_secret strings to the script, and the final version is this:
import sys
from apiclient import discovery
from httplib2 import Http
from oauth2client import file, client, tools
# ID and SECRET arguments
client_id = sys.argv[1]
client_secret = sys.argv[2]
SCOPES = 'https://www.googleapis.com/auth/drive.readonly.metadata'
def listFiles(drive):
"""Receive the service and list the files"""
files = drive.files().list().execute().get('files', [])
for f in files:
print(f['name'], f['mimeType'],f['id'])
def main():
store = file.Storage('storage.json')
creds = store.get()
if not creds or creds.invalid:
flow = client.OAuth2WebServerFlow(client_id, client_secret, SCOPES)
creds = tools.run_flow(flow, store, tools.argparser.parse_args())
DRIVE = discovery.build('drive', 'v3', http=creds.authorize(Http()))
listFiles(DRIVE)
if __name__ == "__main__":
main()
The first time that I start this new version of the script it works, because the script in the old version has created the storage.json file.
Then I move my new version script to another folder or machine (where the storage.json file doesn't exists) to check if it works and then I get this:
$ python3 drive_list.py asdasdasdsa jijfkljflksdjflksdj
/usr/local/lib/python3.4/dist-packages/oauth2client/_helpers.py:255: UserWarning: Cannot access storage.json: No such file or directory
warnings.warn(_MISSING_FILE_MESSAGE.format(filename))
usage: drive_list.py [--auth_host_name AUTH_HOST_NAME]
[--noauth_local_webserver]
[--auth_host_port [AUTH_HOST_PORT [AUTH_HOST_PORT ...]]]
[--logging_level {DEBUG,INFO,WARNING,ERROR,CRITICAL}]
drive_list.py: error: unrecognized arguments: asdasdasdsa jijfkljflksdjflksdj
The warning about the storage.json file is normal and appears in both script versions, is part of oauth2client.
This is the curios part, why the arguments are recognized when the storage.json file exists (ONLY created reading the client_secret.json)? if the first time that the script start it creates the file.
Is really weird this error, and I'm just trying to find what is happening exactly.
If someone can help me, I will be really thankful.

This is happening because you are importing the oauth2client.tools module.
To work correctly this module relies on the standard argparse module. In case you don't know, this standard module is used to write user-friendly command-line interfaces with easy management of command line arguments. This does not get along with your usage of sys.argv[1] and sys.argv[2] arguments.
To work around this, you can add new arguments to the command line like in the sample below. With this modification you would then run the tool like this
python3 drive_list.py -ci "your_client_id" -cs "your_client_secret"
Here is your code slightly modified to add the new command line arguments:
import argparse
from apiclient import discovery
from httplib2 import Http
from oauth2client import file, client, tools
# ID and SECRET arguments as new command line parameters
# Here is where you extend the oauth2client.tools startnd arguments
tools.argparser.add_argument('-ci', '--client-id', type=str, required=True, help='The client ID of your GCP project')
tools.argparser.add_argument('-cs', '--client-secret', type=str, required=True,
help='The client Secret of your GCP project')
SCOPES = 'https://www.googleapis.com/auth/drive.readonly.metadata'
def list_files(drive):
"""Receive the service and list the files"""
files = drive.files().list().execute().get('files', [])
for f in files:
print(f['name'], f['mimeType'], f['id'])
def main():
store = file.Storage('storage.json')
creds = store.get()
if not creds or creds.invalid:
# You want to be sure to parse the args to take in consideration the new parameters
args = argparse.ArgumentParser(parents=[tools.argparser]).parse_args()
flow = client.OAuth2WebServerFlow(args.client_id, args.client_secret, SCOPES)
creds = tools.run_flow(flow, store, tools.argparser.parse_args())
drive_sdk = discovery.build('drive', 'v3', http=creds.authorize(Http()))
list_files(drive_sdk)
if __name__ == "__main__":
main()

GMAIL API: <HttpError 400 when requesting https://www.googleapis.com/gmail/v1/users/myemail%40gmail.com/labels?alt=json returned "Bad Request">

I am trying to write a python program that will access my gmail account through the gmail API using account services. I followed all the steps in the Using OAuth 2.0 for Server to Server Applications
here is my code:
from __future__ import print_function
import httplib2
import os
from apiclient.discovery import build
from httplib2 import Http
from oauth2client import client
from oauth2client import tools
from oauth2client.file import Storage
from oauth2client.service_account import ServiceAccountCredentials
try:
import argparse
flags = argparse.ArgumentParser(parents=[ tools.argparser]).parse_args()
except ImportError:
flags = None
def main():
SCOPES = ['https://www.googleapis.com/auth/gmail.modify']
APPLICATION_NAME = 'Gmail Application'
credentials = ServiceAccountCredentials.from_json_keyfile_name(
'MyKeyFile.json', scopes=SCOPES)
http_auth = credentials.authorize(Http())
service = build('gmail', 'v1', http=http_auth)
results = service.users().labels().list(userId='myemail#gmail.com').execute()
labels = results.get('labels', [])
if not labels:
print('No labels found.')
else:
print('Labels:')
for label in labels:
print(label['name'])
if __name__ == '__main__':
main()
When I run this i get a an error 400 Bad Request.
Anyone encountered this issue. I have been changing things around foro a couple of days now and nothing has worked: I used the service ID instead of my email, I used p12 credentials api, created new projects and new account services with new keys... you name it. I'm hitting a wall. Can anyone help?
Thanks

You should to Enable your API on Dashboard in your API Manager menu here
Then, choose you interested in API
Finally, you'll see something like this

Creating active users via Google Directory API

I am the admin for my Google domain and I am trying to automate provisioning of new users. I first attempted to do so using a command line tool called GAM.
https://github.com/jay0lee/GAM
While this tool works, at least half of the users I create are flagged for unusual activity and are created as suspended. I have tried setting the flags in a variety of combinations but am getting the same result.
I also tried to use the Directory API directly by writing a python program but all new users are still being created as suspended users. Here is my program. Can anyone shed some light on this? Thank you.
from __future__ import print_function
import httplib2
import os
from apiclient import discovery
import oauth2client
from oauth2client import client
from oauth2client import tools
try:
import argparse
flags = argparse.ArgumentParser(parents=[tools.argparser]).parse_args()
except ImportError:
flags = None
SCOPES = 'https://www.googleapis.com/auth/admin.directory.user'
CLIENT_SECRET_FILE = 'client_secret.json'
APPLICATION_NAME = 'Directory API Python Quickstart'
def get_credentials():
home_dir = os.path.expanduser('~')
credential_dir = os.path.join(home_dir, '.credentials')
if not os.path.exists(credential_dir):
os.makedirs(credential_dir)
credential_path = os.path.join(credential_dir,
'admin-directory_v1-python-quickstart.json')
store = oauth2client.file.Storage(credential_path)
credentials = store.get()
if not credentials or credentials.invalid:
flow = client.flow_from_clientsecrets(CLIENT_SECRET_FILE, SCOPES)
flow.user_agent = APPLICATION_NAME
if flags:
credentials = tools.run_flow(flow, store, flags)
else: # Needed only for compatibility with Python 2.6
credentials = tools.run(flow, store)
print('Storing credentials to ' + credential_path)
return credentials
def create_user():
credentials = get_credentials()
http = credentials.authorize(httplib2.Http())
service = discovery.build('admin', 'directory_v1', http=http)
userinfo = {'primaryEmail': 'jane#mydomain.com',
'name': { 'givenName': 'Jane', 'familyName': 'Smith' },
'password': '34gjklre304iojlo24j2kl3kdlj',}
service.users().insert(body=userinfo).execute()
if __name__ == '__main__':
create_user()
main()

Take note of the following statement from Directory API about Creating accounts:
If the Google account has purchased mail licenses, the new user
account is automatically assigned a mailbox. This assignment may take
a few minutes to be completed and activated.
So it may take sometime. Also note from the user resources that:
A new account is automatically suspended by Google until the initial
administrator login which activates the account. After the account is
activated, the account is no longer suspended.
So after creating the new account, try logging it in to officially activate it.

This issue ended up being attributed to testing on a free trial domain. Using a trial domain, both GAM and the python program above created some suspended users and some active users. However, when I tried both methods on my paid full version google domain I was able to consistently create all active users.

Google Analytics API forces browser open when authenticating in Python

I'm attempting to query data from the Google Analytics API. I've setup my credentials, and I am able to get the following code to run on my Windows local machine. However, when I run from the Ubuntu command line, it forces an SSL browser window to open at the following step:
service = initialize_service()
All code is below:
import httplib2
from apiclient.discovery import build
from oauth2client.client import flow_from_clientsecrets
from oauth2client.file import Storage
from oauth2client.tools import run
import sys
from apiclient.errors import HttpError
from oauth2client.client import AccessTokenRefreshError
CLIENT_SECRETS = 'C:\Users\mds78\Documents\code\google\client_secrets.json'
MISSING_CLIENT_SECRETS_MESSAGE = '%s is missing' % CLIENT_SECRETS
FLOW = flow_from_clientsecrets(CLIENT_SECRETS, scope='https://www.googleapis.com/auth/analytics.readonly', message=MISSING_CLIENT_SECRETS_MESSAGE)
TOKEN_FILE_NAME = 'analytics.dat'
def prepare_credentials():
storage = Storage(TOKEN_FILE_NAME)
credentials = storage.get()
if credentials is None or credentials.invalid:
credentials = run(FLOW, storage)
return credentials
def initialize_service():
http = httplib2.Http()
credentials = prepare_credentials()
http = credentials.authorize(http)
return build('analytics', 'v3', http=http)
service = initialize_service()