We Keep Coding

How do I update my Python Google Sheet API credentials code to avoid package depreciation?

The Google Sheets API Python Quickstart currently uses deprecated packages in its example code, where the use of oauth2client and httplib2 rather than google-auth and google-auth-oauthlib (or maybe google-auth?) is outdated or soon to be.
How do I rewrite this code to use these new libraries in a way that best anticipates their current state and the eventual move to google-auth, etc.? In particular, is there a simple reimplementation of retrieval, storage, and use of credentials in the quickstart (below) that brings it up to date?
from __future__ import print_function
import httplib2
import os
from apiclient import discovery
from oauth2client import client
from oauth2client import tools
from oauth2client.file import Storage
try:
import argparse
flags = argparse.ArgumentParser(parents=[tools.argparser]).parse_args()
except ImportError:
flags = None
SCOPES = 'https://www.googleapis.com/auth/spreadsheets.readonly'
CLIENT_SECRET_FILE = 'client_secret.json'
APPLICATION_NAME = 'Google Sheets API Python Quickstart'
def get_credentials():
home_dir = os.path.expanduser('~')
credential_dir = os.path.join(home_dir, '.credentials')
if not os.path.exists(credential_dir):
os.makedirs(credential_dir)
credential_path = os.path.join(credential_dir, 'sheets.googleapis.com-python-quickstart.json')
store = Storage(credential_path)
credentials = store.get()
if not credentials or credentials.invalid:
flow = client.flow_from_clientsecrets(CLIENT_SECRET_FILE, SCOPES)
flow.user_agent = APPLICATION_NAME
if flags:
credentials = tools.run_flow(flow, store, flags)
else: # Needed only for compatibility with Python 2.6
credentials = tools.run(flow, store)
print('Storing credentials to ' + credential_path)
return credentials
def main():
credentials = get_credentials()
http = credentials.authorize(httplib2.Http())
discoveryUrl = 'https://sheets.googleapis.com/$discovery/rest?version=v4'
service = discovery.build('sheets', 'v4', http=http, discoveryServiceUrl=discoveryUrl)
# etc. ...

How about this answer? I prepared the sample script for the Quickstart of Sheets API.
In this sample, it supposes below.
Sheet API is enabled
You have a client_secret.json.
In this sample, at first, refresh token is retrieved using authorization code. The refresh token is saved to sheets.googleapis.com-python-quickstart.json. After 1st run, the access token is retrieved by the refresh token.
In this sample, cell values are retrieved from spreadsheet.
Sample script :
import copy
import json
import os
import pprint
import google.oauth2.credentials
from googleapiclient.discovery import build
from google_auth_oauthlib.flow import InstalledAppFlow
pp = pprint.PrettyPrinter(indent=2)
CLIENT_SECRETS_FILE = "client_secret.json"
SCOPES = ['https://www.googleapis.com/auth/spreadsheets.readonly']
API_SERVICE_NAME = 'sheets'
API_VERSION = 'v4'
def get_authenticated_service():
credential_path = os.path.join("./", 'sheets.googleapis.com-python-quickstart.json')
if os.path.exists(credential_path):
with open(credential_path, 'r') as f:
credential_params = json.load(f)
credentials = google.oauth2.credentials.Credentials(
credential_params["access_token"],
refresh_token=credential_params["refresh_token"],
token_uri=credential_params["token_uri"],
client_id=credential_params["client_id"],
client_secret=credential_params["client_secret"]
)
else:
flow = InstalledAppFlow.from_client_secrets_file(CLIENT_SECRETS_FILE, SCOPES)
credentials = flow.run_console()
with open(credential_path, 'w') as f:
p = copy.deepcopy(vars(credentials))
del p["expiry"]
json.dump(p, f, indent=4)
return build(API_SERVICE_NAME, API_VERSION, credentials=credentials)
def spreadsheets_get(service):
spreadsheetId = "### spreadsheet ID ###"
rangeName = "Sheet1!a1:a10"
results = service.spreadsheets().get(
spreadsheetId=spreadsheetId,
ranges=rangeName
).execute()
pp.pprint(results)
if __name__ == '__main__':
os.environ['OAUTHLIB_INSECURE_TRANSPORT'] = '1'
service = get_authenticated_service()
spreadsheets_get(service)
Note :
In order to save credentials, it was required to use oauth2client.file. So I prepared the sample script without using oauth2client.file. If you want to use oauth2client.file, please modify it.
I think that Quickstart for Sheets API may be also updated in the near future.
References :
References I refered to prepare this sample script are as follows.
Using OAuth 2.0 for Installed Applications
There is a sample script for python.
User Guide
google.oauth2.credentials module
Using OAuth 2.0 for Web Server Applications
If I misunderstand your question, I'm sorry.

People API returns no connections when authenticating via Service Account [duplicate]

I'm trying to programmatically access the list of contacts on my own personal Google Account using the Python Client Library
This is a script that will run on a server without user input, so I have it set up to use credentials from a Service Account I set up. My Google API console setup looks like this.
I'm using the following basic script, pulled from the examples provided in the API docs -
import json
from httplib2 import Http
from oauth2client.service_account import ServiceAccountCredentials
from apiclient.discovery import build
# Only need read-only access
scopes = ['https://www.googleapis.com/auth/contacts.readonly']
# JSON file downloaded from Google API Console when creating the service account
credentials = ServiceAccountCredentials.from_json_keyfile_name(
'keep-in-touch-5d3ebc885d4c.json', scopes)
# Build the API Service
service = build('people', 'v1', credentials=credentials)
# Query for the results
results = service.people().connections().list(resourceName='people/me').execute()
# The result set is a dictionary and should contain the key 'connections'
connections = results.get('connections', [])
print connections #=> [] - empty!
When I hit the API it returns a result set without any 'connections' key. Specifically it returns -
>>> results
{u'nextSyncToken': u'CNP66PXjKhIBMRj-EioECAAQAQ'}
Is there something pertaining to my setup or code that's incorrect? Is there a way to see the response HTTP status code or get any further detail about what it's trying to do?
Thanks!
Side note: When I try it using the "Try it!" feature in the API docs, it correctly returns my contacts. Although I doubt that uses the client library and instead relies on user authorization via OAuth

The personFields mask is required. Specify one or more valid paths. Valid paths are documented at https://developers.google.com/people/api/rest/v1/people.connections/list/.
Additionally, use fields mask to specify which fields are included in a partial response.
Instead of:
results = service.people().connections().list(resourceName='people/me').execute()
... try:
results = service.people().connections().list(resourceName='people/me',personFields='names,emailAddresses',fields='connections,totalItems,nextSyncToken').execute()

Here is a working demo. I just tested it right now. Python 3.5.2
google-api-python-client==1.6.4
httplib2==0.10.3
oauth2client==4.1.2
You can save it to demo.py and then just run it. I left the create_contact function in case you might want to use it and have one more example on the API usage.
CLIENT_ID and CLIENT_SECRET are environment variables so I don't accidentally share that in code.
"""Google API stuff."""
import httplib2
import json
import os
from apiclient.discovery import build
from oauth2client.file import Storage
from oauth2client.client import OAuth2WebServerFlow
from oauth2client.tools import run_flow
CLIENT_ID = os.environ['CLIENT_ID']
CLIENT_SECRET = os.environ['CLIENT_SECRET']
SCOPE = 'https://www.googleapis.com/auth/contacts'
USER_AGENT = 'JugDemoStackOverflow/v0.1'
def make_flow():
"""Make flow."""
flow = OAuth2WebServerFlow(
client_id=CLIENT_ID,
client_secret=CLIENT_SECRET,
scope=SCOPE,
user_agent=USER_AGENT,
)
return flow
def get_people():
"""Return a people_service."""
flow = make_flow()
storage = Storage('info.dat')
credentials = storage.get()
if credentials is None or credentials.invalid:
credentials = run_flow(flow, storage)
http = httplib2.Http()
http = credentials.authorize(http)
people_service = build(serviceName='people', version='v1', http=http)
return people_service
def create_contact(people, user):
"""Create a Google Contact."""
request = people.createContact(
body={
'names': [{'givenName': user.name}],
'phoneNumbers': [
{'canonicalForm': user.phone, 'value': user.phone}],
}
)
return request.execute()
def demo():
"""Demonstrate getting contacts from Google People."""
people_service = get_people()
people = people_service.people()
connections = people.connections().list(
resourceName='people/me',
personFields='names,emailAddresses,phoneNumbers',
pageSize=2000,
)
result = connections.execute()
s = json.dumps(result)
# with open('contacts.json', 'w') as f:
# f.write(s)
return s
if __name__ == '__main__':
print(demo())

With service account, in DwD - G Suite Domain-wide Delegation, is necessary impersonate or delegate user in this way
delegate = credentials.create_delegated('user#xxxx.xxx')

For fellow googlers: I have the same problem using the JS API.
I succeded on my personal gmail address, but not on my work one (g-suite) neither on my secondary gmail address.
Can't see the pattern. It's possible that the work one has contact listing deactivated.

Accessing Google spreadsheet with Python for server-to-server use

I am trying to update a Google spreadsheet from a server that some target users can see on a daily basis. Here is what I tried:
Created a project in "console.developers.google.com" then selected "drive API" -> "credentials" -> "add credentials" -> "service accounts" -> "create Json file"
Now with this JSON file (project name-e4sdfsdsdf0c.json) I tried to access Spreadsheets.
import gdata.spreadsheet.service
import gdata.service
import urlparse
import httplib2
from oauth2client.file import Storage
from oauth2client.client import flow_from_clientsecrets
from oauth2client import tools
spreadsheet_key = '13jQtgSUXKBExMvZjECf6sdfsfgLfmRFVmZw6t7hYyX3g0'
storage = Storage("creds.dat")
credentials = storage.get()
if credentials is None or credentials.invalid:
credentials = tools.run_flow(flow_from_clientsecrets("project name-e4sdfsdsdf0c.json", scope=["https://spreadsheets.google.com/feeds"]), storage)
if credentials.access_token_expired:
credentials.refresh(httplib2.Http())
spr_client = gdata.spreadsheet.service.SpreadsheetsService(
additional_headers={'Authorization' : 'Bearer %s' % credentials.access_token})
worksheets = spr_client.GetSpreadsheetsFeed(spreadsheet_key)
print worksheets.title
But I am getting this error:
Invalid file format. See https://developers.google.com/api-client-library/python/guide/aaa_client_secrets Expected a JSON object with a single property for a "web" or "installed" application"

You created a service account but it looks like you're trying to access it using a client flow.
Have a look at the service account documentation here:
https://developers.google.com/identity/protocols/OAuth2ServiceAccount
The first step would be to go back to the developer console and generate a p12 key. Then the basic flow for Python looks like this:
from oauth2client.client import SignedJwtAssertionCredentials
scope = 'https://www.googleapis.com/auth/drive.readonly https://spreadsheets.google.com/feeds'
client_email = '<your service account email address>'
with open("MyProject.p12") as f:
private_key = f.read()
credentials = SignedJwtAssertionCredentials(client_email, private_key, scope)
http_auth = credentials.authorize(Http())

Google Sheets API v4 appeared pretty straightforward here. After generating json file you may access spreadsheet with this code.
from oauth2client.service_account import ServiceAccountCredentials
from httplib2 import Http
from apiclient import discovery
scopes = ['https://www.googleapis.com/auth/spreadsheets']
credentials = ServiceAccountCredentials.from_json_keyfile_name(<path_to_your_client_secret.json>, scopes)
http_auth = credentials.authorize(Http())
discoveryUrl = ('https://sheets.googleapis.com/$discovery/rest?version=v4')
service = discovery.build('sheets', 'v4', http=http_auth, discoveryServiceUrl=discoveryUrl)
result = service.spreadsheets().values().update(...).execute()

Using refresh token with Google directory service API

Is there any way to use a refresh token with Google directory service API?
I couldn't find any examples how to do that (I'm using Python).
I'm looking for something similar to the following code (this works for Google Adwords API), with previosly set credentials:
oauth2_client = oauth2.GoogleRefreshTokenClient(
CLIENT_ID, CLIENT_SECRET, REFRESH_TOKEN)
adwords_client = adwords.AdWordsClient(
DEVELOPER_TOKEN, oauth2_client, USER_AGENT, CLIENT_CUSTOMER_ID)
self.managed_customer_service = adwords_client.GetService(
'ManagedCustomerService', version='v201402')
For Directory API I found just the following code snippet, but I have no idea how I could use a refresh token with it:
flow = OAuth2WebServerFlow(CLIENT_ID, CLIENT_SECRET, OAUTH_SCOPE, REDIRECT_URI)
authorize_url = flow.step1_get_authorize_url()
print 'Go to the following link in your browser: ' + authorize_url
code = raw_input('Enter verification code: ').strip()
credentials = flow.step2_exchange(code)
# Create an httplib2.Http object and authorize it with our credentials
http = httplib2.Http()
http = credentials.authorize(http)
self.directory_service = build('admin', 'directory_v1', http=http)
My final goal is to authorize my application using just the refresh token and without having to open the browser, login and get a new token each time.

The Python client library can automatically store, load, and refresh credentials if you use a Storage object. The Gmail API Python Quickstart sample shows how to use a file storage, which saves the credentials to disk. Here are the relevant lines of code:
from oauth2client.file import Storage
from oauth2client.tools import run
...
# Location of the credentials storage file
STORAGE = Storage('gmail.storage')
...
# Try to retrieve credentials from storage or run the flow to generate them
credentials = STORAGE.get()
if credentials is None or credentials.invalid:
credentials = run(flow, STORAGE, http=http)
There are additional storage classes built into the client library, or you can extend the Storage base class to implement you own.

how to set up Drive SDK so you dont need a success token each time you run .py

At the moment i am needing to get a "success code" each time i want to run my .py app. to access my googdrive files which is a pain.
I saw Ali Afshars great vid on https://developers.google.com/drive/search-parameters where he uses:
from auth import http to streamline this process.
Not sure what this function should contain... Can you point me in the right direction, so that i can set up my .py to do this automatically ...or at least only once.
many thanks
Dav-o
EDIT relevant current snip of code follows:
import logging
logging.basicConfig()
import httplib2
import pprint
from apiclient.discovery import build
from apiclient.http import MediaFileUpload
from oauth2client.client import *
from apiclient import errors
CLIENT_ID = "864350......ps.googleusercontent.com"
CLIENT_SECRET = "sw0yb.....-zR6XWzEgM"
OAUTH_SCOPE = 'https://www.googleapis.com/auth/drive'
REDIRECT_URI = 'urn:ietf:wg:oauth:2.0:oo...ooo' # Redirect URI for installed apps
flow = OAuth2WebServerFlow(CLIENT_ID, CLIENT_SECRET, OAUTH_SCOPE, REDIRECT_URI)
authorize_url = flow.step1_get_authorize_url()
print 'Go to the following link in your browser: ' + authorize_url
code = raw_input('Enter verification code: ').strip()
credentials = flow.step2_exchange(code)
http = httplib2.Http()
http = credentials.authorize(http)
drive_service = build('drive', 'v2', http=http)

What he probably does in auth is:
import httplib2
from oauth2client.client import OAuth2Credentials
http = httplib2.Http()
credentials = OAuth2Credentials(access_token, client_id, client_secret, refresh_token, None, '', '')
credentials.authorize(http)
You can wrap this code in a module/method and use the authenticate your requests. For multiple user scenarios, preserve access and refresh tokens (most likely in a db) and init credentials object with the stored tokens.