We Keep Coding

requests library with googleapiclient

Following is the code to access a google storage bucket using the httplib2 library
import json
from httplib2 import Http
from oauth2client.client import SignedJwtAssertionCredentials
from googleapiclient.discovery import build
from pprint import pprint
client_email = 'my.iam.gserviceaccount.com'
json_file = 'services.json'
cloud_storage_bucket = 'my_bucket'
files = 'reviews/reviews_myapp_201603.csv'
private_key = json.loads(open(json_file).read())['private_key']
credentials = SignedJwtAssertionCredentials(client_email,
private_key,'https://www.googleapis.com/auth/devstorage.read_only')
storage = build('storage', 'v1', http=credentials.authorize(Http()))
pprint(storage.objects().get(bucket=cloud_storage_bucket, object=files).execute())
Can someone tell me if I can make the http request using the Python Requests library here?
If yes, how?

Yes, you can use the HTTP header Authorization: Bearer <access_token> with requests or any library you want.
Service account
from google.oauth2 import service_account
credentials = service_account.Credentials.from_service_account_file(
'services.json',
scopes=['https://www.googleapis.com/auth/devstorage.read_only'],
)
# Copy access token
bearer_token = credentials.token
User account credentials
import json
from google.oauth2.credentials import Credentials
from google_auth_oauthlib.flow import InstalledAppFlow
flow = InstalledAppFlow.from_client_secrets_file(
'test.json',
'https://www.googleapis.com/auth/devstorage.read_only'
)
# Construct cache path for oauth2 token
oauth2_cache_path = 'test-oauth2.json'
credentials = None
try:
# Try to load existing oauth2 token
with open(oauth2_cache_path, 'r') as f:
credentials = Credentials(**json.load(f))
except (OSError, IOError) as e:
pass
if not credentials or not credentials.valid:
credentials = flow.run_console()
with open(oauth2_cache_path, 'w+') as f:
f.write(json.dumps({
'token': credentials.token,
'refresh_token': credentials.refresh_token,
'token_uri': credentials.token_uri,
'client_id': credentials.client_id,
'client_secret': credentials.client_secret,
'scopes': credentials.scopes,
}))
# Copy access token
bearer_token = credentials.token
Use requests lib
import requests
# Send request
response = requests.get(
'https://www.googleapis.com/storage/v1/<endpoint>?access_token=%s'
% bearer_token)
# OR
response = requests.get(
'https://www.googleapis.com/storage/v1/<endpoint>',
headers={'Authorization': 'Bearer %s' % bearer_token})
Use googleapiclient lib
I recommend you to use build() method and not requests directly because the google library do some checks before sending your API call (like checking params, endpoint, auth and the method you use). This library also raise exceptions when error is detected.
from googleapiclient.discovery import build
storage = build('storage', 'v1', credentials=credentials)
print(storage.objects().get(bucket='bucket', object='file_path').execute())
More informations here : https://developers.google.com/identity/protocols/OAuth2WebServer#callinganapi (click on "HTTP/REST" tab)

I suggest to use the official Google Auth library which is already implementing Requests Library. See this link for more information.
Here is a code to try (given that you have a service account file with required permissions):
from google.oauth2 import service_account
from google.auth.transport.requests import AuthorizedSession
service_account_file = 'service_account.json'
scopes = ['https://www.googleapis.com/auth/devstorage.full_control']
credentials = service_account.Credentials.from_service_account_file(
service_account_file, scopes=scopes)
session = AuthorizedSession(credentials)
bucket_name = 'YOUR-BUCKET-NAME'
response = session.get(f'https://storage.googleapis.com/storage/v1/b/{bucket_name}')
print(response.json())

How do I update my Python Google Sheet API credentials code to avoid package depreciation?

The Google Sheets API Python Quickstart currently uses deprecated packages in its example code, where the use of oauth2client and httplib2 rather than google-auth and google-auth-oauthlib (or maybe google-auth?) is outdated or soon to be.
How do I rewrite this code to use these new libraries in a way that best anticipates their current state and the eventual move to google-auth, etc.? In particular, is there a simple reimplementation of retrieval, storage, and use of credentials in the quickstart (below) that brings it up to date?
from __future__ import print_function
import httplib2
import os
from apiclient import discovery
from oauth2client import client
from oauth2client import tools
from oauth2client.file import Storage
try:
import argparse
flags = argparse.ArgumentParser(parents=[tools.argparser]).parse_args()
except ImportError:
flags = None
SCOPES = 'https://www.googleapis.com/auth/spreadsheets.readonly'
CLIENT_SECRET_FILE = 'client_secret.json'
APPLICATION_NAME = 'Google Sheets API Python Quickstart'
def get_credentials():
home_dir = os.path.expanduser('~')
credential_dir = os.path.join(home_dir, '.credentials')
if not os.path.exists(credential_dir):
os.makedirs(credential_dir)
credential_path = os.path.join(credential_dir, 'sheets.googleapis.com-python-quickstart.json')
store = Storage(credential_path)
credentials = store.get()
if not credentials or credentials.invalid:
flow = client.flow_from_clientsecrets(CLIENT_SECRET_FILE, SCOPES)
flow.user_agent = APPLICATION_NAME
if flags:
credentials = tools.run_flow(flow, store, flags)
else: # Needed only for compatibility with Python 2.6
credentials = tools.run(flow, store)
print('Storing credentials to ' + credential_path)
return credentials
def main():
credentials = get_credentials()
http = credentials.authorize(httplib2.Http())
discoveryUrl = 'https://sheets.googleapis.com/$discovery/rest?version=v4'
service = discovery.build('sheets', 'v4', http=http, discoveryServiceUrl=discoveryUrl)
# etc. ...

How about this answer? I prepared the sample script for the Quickstart of Sheets API.
In this sample, it supposes below.
Sheet API is enabled
You have a client_secret.json.
In this sample, at first, refresh token is retrieved using authorization code. The refresh token is saved to sheets.googleapis.com-python-quickstart.json. After 1st run, the access token is retrieved by the refresh token.
In this sample, cell values are retrieved from spreadsheet.
Sample script :
import copy
import json
import os
import pprint
import google.oauth2.credentials
from googleapiclient.discovery import build
from google_auth_oauthlib.flow import InstalledAppFlow
pp = pprint.PrettyPrinter(indent=2)
CLIENT_SECRETS_FILE = "client_secret.json"
SCOPES = ['https://www.googleapis.com/auth/spreadsheets.readonly']
API_SERVICE_NAME = 'sheets'
API_VERSION = 'v4'
def get_authenticated_service():
credential_path = os.path.join("./", 'sheets.googleapis.com-python-quickstart.json')
if os.path.exists(credential_path):
with open(credential_path, 'r') as f:
credential_params = json.load(f)
credentials = google.oauth2.credentials.Credentials(
credential_params["access_token"],
refresh_token=credential_params["refresh_token"],
token_uri=credential_params["token_uri"],
client_id=credential_params["client_id"],
client_secret=credential_params["client_secret"]
)
else:
flow = InstalledAppFlow.from_client_secrets_file(CLIENT_SECRETS_FILE, SCOPES)
credentials = flow.run_console()
with open(credential_path, 'w') as f:
p = copy.deepcopy(vars(credentials))
del p["expiry"]
json.dump(p, f, indent=4)
return build(API_SERVICE_NAME, API_VERSION, credentials=credentials)
def spreadsheets_get(service):
spreadsheetId = "### spreadsheet ID ###"
rangeName = "Sheet1!a1:a10"
results = service.spreadsheets().get(
spreadsheetId=spreadsheetId,
ranges=rangeName
).execute()
pp.pprint(results)
if __name__ == '__main__':
os.environ['OAUTHLIB_INSECURE_TRANSPORT'] = '1'
service = get_authenticated_service()
spreadsheets_get(service)
Note :
In order to save credentials, it was required to use oauth2client.file. So I prepared the sample script without using oauth2client.file. If you want to use oauth2client.file, please modify it.
I think that Quickstart for Sheets API may be also updated in the near future.
References :
References I refered to prepare this sample script are as follows.
Using OAuth 2.0 for Installed Applications
There is a sample script for python.
User Guide
google.oauth2.credentials module
Using OAuth 2.0 for Web Server Applications
If I misunderstand your question, I'm sorry.

'analytics' is not defined error with Upload data script with GA Management API

I'm trying to upload some custom data into GA with Python. It's the first I'm doing this so I'm not sure about nothing.
I've build the following script based on the example from the doc. When running it I have the following error :
File "import.py", line 50, in <module>
daily_upload = analytics.management().uploads().uploadData(
NameError: name 'analytics' is not defined
Here is my code :
import argparse
from apiclient.discovery import build
from oauth2client.service_account import ServiceAccountCredentials
import httplib2
import urllib2
from oauth2client import client
from oauth2client import file
from oauth2client import tools
def get_service(api_name, api_version, scope, key_file_location,
service_account_email):
"""Get a service that communicates to a Google API.
Args:
api_name: The name of the api to connect to.
api_version: The api version to connect to.
scope: A list auth scopes to authorize for the application.
key_file_location: The path to a valid service account p12 key file.
service_account_email: The service account email address.
Returns:
A service that is connected to the specified API.
"""
credentials = ServiceAccountCredentials.from_p12_keyfile(
service_account_email, key_file_location, scopes=scope)
http = credentials.authorize(httplib2.Http())
# Build the service object.
service = build(api_name, api_version, http=http)
return service
from apiclient.http import MediaFileUpload
try:
media = MediaFileUpload('mycsv.csv',
mimetype='application/octet-stream',
resumable=False)
daily_upload = analytics.management().uploads().uploadData(
accountId='XXXXXX',
webPropertyId='XXXXXXX',
customDataSourceId='XXXXXXXXXX',
media_body=media).execute()
except TypeError, error:
# Handle errors in constructing a query.
print 'There was an error in constructing your query : %s' % error
def main():
# Define the auth scopes to request.
scope = ['https://www.googleapis.com/auth/analytics']
# Use the developer console and replace the values with your
# service account email and relative location of your key file.
service_account_email = 'XXXXXX#XXXXXX'
key_file_location = 'XXXXXXXXXX.p12'
# Authenticate and construct service.
service = get_service('analytics', 'v3', scope, key_file_location,
service_account_email)
profile = get_first_profile_id(service)
print_results(get_results(service, profile))
if __name__ == '__main__':
main()
If my code isn't clear or show some other obvious mistake different from the one I'm questioning please be comprehensive I'm learning !
Edit: I've checked in my API Manager the Analytics API is well enable

Ok. It is a simple block alignement issue. I needed to align this part :
from apiclient.http import MediaFileUpload
try:
media = MediaFileUpload('mycsv.csv',
mimetype='application/octet-stream',
resumable=False)
daily_upload = analytics.management().uploads().uploadData(
accountId='XXXXXX',
webPropertyId='XXXXXXX',
customDataSourceId='XXXXXXXXXX',
media_body=media).execute()
except TypeError, error:
# Handle errors in constructing a query.
print 'There was an error in constructing your query : %s' % error
with the first part !

People API returns no connections when authenticating via Service Account [duplicate]

I'm trying to programmatically access the list of contacts on my own personal Google Account using the Python Client Library
This is a script that will run on a server without user input, so I have it set up to use credentials from a Service Account I set up. My Google API console setup looks like this.
I'm using the following basic script, pulled from the examples provided in the API docs -
import json
from httplib2 import Http
from oauth2client.service_account import ServiceAccountCredentials
from apiclient.discovery import build
# Only need read-only access
scopes = ['https://www.googleapis.com/auth/contacts.readonly']
# JSON file downloaded from Google API Console when creating the service account
credentials = ServiceAccountCredentials.from_json_keyfile_name(
'keep-in-touch-5d3ebc885d4c.json', scopes)
# Build the API Service
service = build('people', 'v1', credentials=credentials)
# Query for the results
results = service.people().connections().list(resourceName='people/me').execute()
# The result set is a dictionary and should contain the key 'connections'
connections = results.get('connections', [])
print connections #=> [] - empty!
When I hit the API it returns a result set without any 'connections' key. Specifically it returns -
>>> results
{u'nextSyncToken': u'CNP66PXjKhIBMRj-EioECAAQAQ'}
Is there something pertaining to my setup or code that's incorrect? Is there a way to see the response HTTP status code or get any further detail about what it's trying to do?
Thanks!
Side note: When I try it using the "Try it!" feature in the API docs, it correctly returns my contacts. Although I doubt that uses the client library and instead relies on user authorization via OAuth

The personFields mask is required. Specify one or more valid paths. Valid paths are documented at https://developers.google.com/people/api/rest/v1/people.connections/list/.
Additionally, use fields mask to specify which fields are included in a partial response.
Instead of:
results = service.people().connections().list(resourceName='people/me').execute()
... try:
results = service.people().connections().list(resourceName='people/me',personFields='names,emailAddresses',fields='connections,totalItems,nextSyncToken').execute()

Here is a working demo. I just tested it right now. Python 3.5.2
google-api-python-client==1.6.4
httplib2==0.10.3
oauth2client==4.1.2
You can save it to demo.py and then just run it. I left the create_contact function in case you might want to use it and have one more example on the API usage.
CLIENT_ID and CLIENT_SECRET are environment variables so I don't accidentally share that in code.
"""Google API stuff."""
import httplib2
import json
import os
from apiclient.discovery import build
from oauth2client.file import Storage
from oauth2client.client import OAuth2WebServerFlow
from oauth2client.tools import run_flow
CLIENT_ID = os.environ['CLIENT_ID']
CLIENT_SECRET = os.environ['CLIENT_SECRET']
SCOPE = 'https://www.googleapis.com/auth/contacts'
USER_AGENT = 'JugDemoStackOverflow/v0.1'
def make_flow():
"""Make flow."""
flow = OAuth2WebServerFlow(
client_id=CLIENT_ID,
client_secret=CLIENT_SECRET,
scope=SCOPE,
user_agent=USER_AGENT,
)
return flow
def get_people():
"""Return a people_service."""
flow = make_flow()
storage = Storage('info.dat')
credentials = storage.get()
if credentials is None or credentials.invalid:
credentials = run_flow(flow, storage)
http = httplib2.Http()
http = credentials.authorize(http)
people_service = build(serviceName='people', version='v1', http=http)
return people_service
def create_contact(people, user):
"""Create a Google Contact."""
request = people.createContact(
body={
'names': [{'givenName': user.name}],
'phoneNumbers': [
{'canonicalForm': user.phone, 'value': user.phone}],
}
)
return request.execute()
def demo():
"""Demonstrate getting contacts from Google People."""
people_service = get_people()
people = people_service.people()
connections = people.connections().list(
resourceName='people/me',
personFields='names,emailAddresses,phoneNumbers',
pageSize=2000,
)
result = connections.execute()
s = json.dumps(result)
# with open('contacts.json', 'w') as f:
# f.write(s)
return s
if __name__ == '__main__':
print(demo())

With service account, in DwD - G Suite Domain-wide Delegation, is necessary impersonate or delegate user in this way
delegate = credentials.create_delegated('user#xxxx.xxx')

For fellow googlers: I have the same problem using the JS API.
I succeded on my personal gmail address, but not on my work one (g-suite) neither on my secondary gmail address.
Can't see the pattern. It's possible that the work one has contact listing deactivated.

How to create Google API OAuth Credentials object from alternate source

I am working with this simple Google API example:
import httplib2
from apiclient.discovery import build
from oauth2client.client import flow_from_clientsecrets
from oauth2client.file import Storage
from oauth2client.tools import run
# Path to the client_secret.json file downloaded from the Developer Console
CLIENT_SECRET_FILE = 'client_secret.json'
# Check https://developers.google.com/gmail/api/auth/scopes for all available scopes
OAUTH_SCOPE = 'https://www.googleapis.com/auth/gmail.readonly'
# Location of the credentials storage file
STORAGE = Storage('gmail.storage')
# Start the OAuth flow to retrieve credentials
flow = flow_from_clientsecrets(CLIENT_SECRET_FILE, scope=OAUTH_SCOPE)
http = httplib2.Http()
# Try to retrieve credentials from storage or run the flow to generate them
credentials = STORAGE.get()
if credentials is None or credentials.invalid:
credentials = run(flow, STORAGE, http=http)
# Authorize the httplib2.Http object with our credentials
http = credentials.authorize(http)
# Build the Gmail service from discovery
gmail_service = build('gmail', 'v1', http=http)
And seeing as I have already gone through the OAuth flow previously (in a different non-Python app) and have my refresh tokens, etc. I would like to skip the first portion of this example and either manually create the expected storage file gmail.storage or create the credentials object some other way.
The problem is I can't find any documentation about the expected format of this storage file, or what should be in it, or how to instantiate the credentials object in any other way. Sorry that I cannot show any work here, but I'm at a loss. Any point in the right direction would be greatly appreciated.

Very simple, apparently this works:
from oauth2client.client import GoogleCredentials
from oauth2client import GOOGLE_TOKEN_URI
access_token = None
token_expiry = None
token_uri = GOOGLE_TOKEN_URI
user_agent = 'Python client library'
revoke_uri = None
gCreds = GoogleCredentials(
access_token,
client_id,
client_secret,
refresh_token,
token_expiry,
token_uri,
user_agent,
revoke_uri=revoke_uri
)

As explained here: in Google Cloud Platform's github
you can also use a string to setup this. Specially a json string
import json
import os
from google.oauth2 import service_account
from google.cloud import translate
info = json.loads(os.environ['GOOGLE_APPLICATION_CREDENTIALS_JSON_STRING'])
creds = service_account.Credentials.from_service_account_info(info)
# Instantiates a client
translate_client = translate.Client(credentials=creds)
Please note that I used Google Translate's API for this example but it's the same logic.
There is a bit more explanation in this git issue too: https://github.com/GoogleCloudPlatform/google-cloud-python/issues/4477

The oauth2client.file.Storage lib might be of interest to you:
from oauth2client.file import Storage
storage = Storage('gmail.storage')
credentials = storage.get()
storage.put(credentials)