We Keep Coding

Unable to generate refresh token for Google Ads API using generate_refresh_token.py

I am very new to Python and Stackoverflow. I am working on connecting my Google Ads account with Python to automate few standard charts creation and email them to my team members. Please help me resolve this as I had not been able to find an answer to it upon Googling either.
Let me know if I have missed out on any info which might provide more context to the question here.
I have been using the steps as mentioned by #msaniscalchi. Created client ID and client Secret from https://console.developers.google.com and created googleads.yaml file in the same directory as the generate_refresh_token.py. When I run the script with respective client ID and client Secret values, I am getting "invalid syntax" error. I have verified my multiple times my client secret and ID values multiple times.
"""Generates refresh token for AdWords using the Installed Application flow."""
import argparse
import sys
from google_auth_oauthlib.flow import InstalledAppFlow
from oauthlib.oauth2.rfc6749.errors import InvalidGrantError
# Your OAuth2 Client ID and Secret. If you do not have an ID and Secret yet,
# please go to https://console.developers.google.com and create a set.
DEFAULT_CLIENT_ID = 609XXXXXXX22-58mbhXXXXXXXXXXXXXXXXXX6ri.apps.googleusercontent.com
DEFAULT_CLIENT_SECRET = 7uO7XXXXXXXXXXXXXX7dKBAP
# The AdWords API OAuth2 scope.
SCOPE = u'https://www.googleapis.com/auth/adwords'
# The redirect URI set for the given Client ID. The redirect URI for Client ID
# generated for an installed application will always have this value.
_REDIRECT_URI = 'urn:ietf:wg:oauth:2.0:oob'
parser = argparse.ArgumentParser(description='Generates a refresh token with '
'the provided credentials.')
parser.add_argument('--client_id', default=DEFAULT_CLIENT_ID,
help='Client Id retrieved from the Developer\'s Console.')
parser.add_argument('--client_secret', default=DEFAULT_CLIENT_SECRET,
help='Client Secret retrieved from the Developer\'s '
'Console.')
parser.add_argument('--additional_scopes', default=None,
help='Additional scopes to apply when generating the '
'refresh token. Each scope should be separated by a comma.')
class ClientConfigBuilder(object):
"""Helper class used to build a client config dict used in the OAuth 2.0 flow.
"""
_DEFAULT_AUTH_URI = 'https://accounts.google.com/o/oauth2/auth'
_DEFAULT_TOKEN_URI = 'https://accounts.google.com/o/oauth2/token'
CLIENT_TYPE_WEB = 'web'
CLIENT_TYPE_INSTALLED_APP = 'installed'
def __init__(self, client_type=None, client_id=None, client_secret=None,
auth_uri=_DEFAULT_AUTH_URI, token_uri=_DEFAULT_TOKEN_URI):
self.client_type = client_type
self.client_id = client_id
self.client_secret = client_secret
self.auth_uri = auth_uri
self.token_uri = token_uri
def Build(self):
"""Builds a client config dictionary used in the OAuth 2.0 flow."""
if all((self.client_type, self.client_id, self.client_secret,
self.auth_uri, self.token_uri)):
client_config = {
self.client_type: {
'client_id': self.client_id,
'client_secret': self.client_secret,
'auth_uri': self.auth_uri,
'token_uri': self.token_uri
}
}
else:
raise ValueError('Required field is missing.')
return client_config
def main(client_id, client_secret, scopes):
"""Retrieve and display the access and refresh token."""
client_config = ClientConfigBuilder(
client_type=ClientConfigBuilder.CLIENT_TYPE_WEB, client_id=client_id,
client_secret=client_secret)
flow = InstalledAppFlow.from_client_config(
client_config.Build(), scopes=scopes)
# Note that from_client_config will not produce a flow with the
# redirect_uris (if any) set in the client_config. This must be set
# separately.
flow.redirect_uri = _REDIRECT_URI
auth_url, _ = flow.authorization_url(prompt='consent')
print('Log into the Google Account you use to access your AdWords account '
'and go to the following URL: \n%s\n' % auth_url)
print('After approving the token enter the verification code (if specified).')
code = input('Code: ').strip()
try:
flow.fetch_token(code=code)
except InvalidGrantError as ex:
print('Authentication has failed: %s' % ex)
sys.exit(1)
print('Access token: %s' % flow.credentials.token)
print('Refresh token: %s' % flow.credentials.refresh_token)
if __name__ == '__main__':
args = parser.parse_args()
configured_scopes = [SCOPE]
if not (any([args.client_id, DEFAULT_CLIENT_ID]) and
any([args.client_secret, DEFAULT_CLIENT_SECRET])):
raise AttributeError('No client_id or client_secret specified.')
if args.additional_scopes:
configured_scopes.extend(args.additional_scopes.replace(' ', '').split(','))
main(args.client_id, args.client_secret, configured_scopes)
When I run the above code, I am getting the "Invalid Syntax" error highlighting at the numeric part of the Client ID and Secret.
Syntax error screenshot attached here
Editor Highlighter screenshot attached here

How would I unit test code like this?

I have the following class and am currently testing it via running a file which calls the methods in this file. I then use a mixture of print statements and checking the blog to make sure the code has worked.
I'd really like to write some pytest unit tests for this and automate it all, but how can I do this? Also, if authentication doesn't exist or had become invalid it opens the browser and prompts the user to input the auth code. Later on this will be handled by a gui form. Pytest doesn't take user input, and rightly so; it wouldn't be automated.
class BloggerInterface(object):
"""Connects to blogger api and authorises client."""
def get_credentials(self):
"""Gets google api credentials, or generates new credentials
if they don't exist or are invalid."""
scope = 'https://www.googleapis.com/auth/blogger'
flow = oauth2client.client.flow_from_clientsecrets(
'client_secret.json', scope,
redirect_uri='urn:ietf:wg:oauth:2.0:oob')
storage = oauth2client.file.Storage('credentials.dat')
credentials = storage.get()
if not credentials or credentials.invalid:
auth_uri = flow.step1_get_authorize_url()
webbrowser.open(auth_uri)
auth_code = input('Enter the auth code: ')
credentials = flow.step2_exchange(auth_code)
storage.put(credentials)
return credentials
def get_service(self):
"""Returns an authorised blogger api service."""
credentials = self.get_credentials()
http = httplib2.Http()
http = credentials.authorize(http)
service = apiclient.discovery.build('blogger', 'v3', http=http)
return service
def get_blog(self, blog_id):
"""Gets the details ofthe blog withthe id blog_id"""
BlogDetails = collections.namedtuple('BlogDetails', 'blog_id, name, desc, url')
conn = self.get_service()
request = conn.blogs().get(blogId=blog_id, view='ADMIN')
response = request.execute()
name = response.get('name')
desc = response.get('description')
url = response.get('url')
blog = BlogDetails(blog_id=blog_id, name=name, desc=desc, url=url)
return blog
def get_posts(self, blog_id, status='live'):
"""Gets all posts from the blog with the id blog_id"""
posts = []
conn = self.get_service()
request = conn.posts().list(blogId=blog_id, view='ADMIN',
status=status)
#Responses are paginated, so a paging loop is required.
while request:
response = request.execute()
for post in response.get('items', []):
post_id = post.get('id')
title = post.get('title')
url = post.get('url')
status = post.get('status')
content = post.get('content')
posts.append({'post_id':post_id, 'title':title, 'url':url,
'status':status, 'content':content})
request = conn.posts().list_next(request, response)
return posts
def add_post(self, blog_id, post, is_draft=True):
"""Adds a new post to the blog with the id blog_id"""
conn = self.get_service()
#post is in the form {title, content, (labels), author_name, author_id.
title, content, author_name, author_id, labels = post
data = {
'kind': 'blogger#post',
'title': title,
'content': content,
'labels': labels,
'author': {'displayName':author_name, 'id':author_id}
}
request = conn.posts().insert(blogId=blog_id, body=data,
isDraft=is_draft)
response = request.execute()
post_id = response.get('id')
return post_id

Don't test the oauth2client or webbrowser projects. Test how your code reacts to input and output from other parts. Those are black boxes, which you replace with your own mocks, so you can see how your code responds to different return values.
Use the unittest.mock module to produce the mocks. If you are using Python < 3.3, install the backport mock project to do so.
For example, for BloggerInterface.get_credentials(), you mock out oauth2client.client.flow_from_clientsecrets(), oauth2client.file.Storage(), webbrowser.open() and input. You can then play with the response from storage.get() to force your code to use webbrowser.open(), and test if your code correctly tried to open a webbrowser, and then called storage.put() to store the credentials:
with mock.patch('oauth2client.client.flow_from_clientsecrets') as mockflow, \
mock.patch('oauth2client.file.Storage') as MockStorage, \
mock.patch('webbrowser.open') as mockwbopen, \
mock.patch('yourmodule.input') as mockinput:
# set the credentials to invalid
storage = MockStorage.return_value
credentials = storage.get.return_value
credentials.invalid = True
# run the method and see if we get what we want
result = BloggerInterface().get_credentials()
# check that the flow was initialised correctly
mockflow.assert_called_with(
'client_secret.json', 'https://www.googleapis.com/auth/blogger',
redirect_uri='urn:ietf:wg:oauth:2.0:oob')
MockStorage.assert_called_with('credentials.dat')
# With invalid credentials, the code should obtain a auth url from the
# flow, pass it to the browser. Then the authentication code should be taken
# from input and passed back to the flow for exchange. Test these
# interactions took place:
flow.step1_get_authorize_url.assert_called_once_with()
mockwbopen.assert_called_once_with(flow.step1_get_authorize_url.return_value)
flow.step2_exchange.assert_called_once_with(mockinput.return_value)
storage.put(flow.step2_exchange.return_value)
assert result == flow.step2_exchange.return_value

Using Google Picasa API with Python

I have been using Google's Picasa API successfully from last 6 months or so.
Today I have started getting an error
raise GooglePhotosException(e.args[0])
GooglePhotosException: (403, 'Forbidden', 'Authorization required')
I checked my credentials.
self.gd_client = gdata.photos.service.PhotosService()
self.gd_client.email = EmailOfTheUploadingPictureAccount
self.gd_client.password = PasswordOfTheAccount
self.gd_client.source = 'destipak' #Not sure about that
self.feed_url = "/data/feed/api/user/"
self.entry_url = "/data/entry/api/user/"
self.gd_client.ProgrammaticLogin()
Everything was working well since yesterday. Anyone has any clues?
EDIT
Example given on Picasa for python is also not working.
URL
#!/usr/bin/python2.5
import gdata.photos.service
import gdata.media
import gdata.geo
gd_client = gdata.photos.service.PhotosService()
gd_client.email = '=change=' # Set your Picasaweb e-mail address...
gd_client.password = '=change=' # ... and password
gd_client.source = 'api-sample-google-com'
gd_client.ProgrammaticLogin()
albums = gd_client.GetUserFeed()
for album in albums.entry:
print 'Album: %s (%s)' % (album.title.text, album.numphotos.text)
photos = gd_client.GetFeed('/data/feed/api/user/default/albumid/%s?kind=photo' % (album.gphoto_id.text))
for photo in photos.entry:
print ' Photo:', photo.title.text
tags = gd_client.GetFeed('/data/feed/api/user/default/albumid/%s/photoid/%s?kind=tag' % (album.gphoto_id.text, photo.gphoto_id.text))
for tag in tags.entry:
print ' Tag:', tag.title.text
comments = gd_client.GetFeed('/data/feed/api/user/default/albumid/%s/photoid/%s?kind=comment' % (album.gphoto_id.text, photo.gphoto_id.text))
for comment in comments.entry:
print ' Comment:', comment.content.text
EDIT 2
Full traceback
Traceback (most recent call last):
File "/Users/mac/Picasa_API.py", line 158, in <module>
check_api()
File "/Users/mac/Picasa_API.py", line 140, in check_api
albums = gd_client.GetUserFeed()
File "/Users/mac/destipak/env/lib/python2.7/site-packages/gdata/photos/service.py", line 235, in GetUserFeed
return self.GetFeed(uri, limit=limit)
File "/Users/mac/destipak/env/lib/python2.7/site-packages/gdata/photos/service.py", line 180, in GetFeed
raise GooglePhotosException(e.args[0])
gdata.photos.service.GooglePhotosException: (403, 'Forbidden', 'Authorization required')

Here is the code I use to get OAuth2 authentication working with Picasa. First you need to create a client ID through the Google Developer Console: at https://console.developers.google.com/ and then you must download the client secrets as JSON and pass the filename to OAuth2Login.
The first time you run this code, you will have to authorize the client through your web browser, and paste the code you get there into the application. The credentials are then stored in the file specified by credential_store.
def OAuth2Login(client_secrets, credential_store, email):
scope='https://picasaweb.google.com/data/'
user_agent='myapp'
storage = Storage(credential_store)
credentials = storage.get()
if credentials is None or credentials.invalid:
flow = flow_from_clientsecrets(client_secrets, scope=scope, redirect_uri='urn:ietf:wg:oauth:2.0:oob')
uri = flow.step1_get_authorize_url()
webbrowser.open(uri)
code = raw_input('Enter the authentication code: ').strip()
credentials = flow.step2_exchange(code)
storage.put(credentials)
if (credentials.token_expiry - datetime.utcnow()) < timedelta(minutes=5):
http = httplib2.Http()
http = credentials.authorize(http)
credentials.refresh(http)
gd_client = gdata.photos.service.PhotosService(source=user_agent,
email=email,
additional_headers={'Authorization' : 'Bearer %s' % credentials.access_token})
return gd_client

I ran into this as well. Seems like email/password authentication has been turned off, and we need to switch to OAuth2. See
https://groups.google.com/forum/#!topic/Google-Picasa-Data-API/4meiAJ40l3E

How can I refresh a stored Google oAuth credential

I'm using the Python API that google provides. What I want to do is just make sure that the access token doesn't expire. I have the refresh_token stored in the credentials file. I'm just not sure how to 'check' that the token is still good before making the call to the API and if need be refreshing it and re-storing it in the credentials file.
I did a test that even if I delete the access tokens from the credentials file that it rewrites them into it using the refresh token. I'm hoping that will work for expired access tokens as well.
Thanks
storage = Storage('cred_storage.txt')
credentials = storage.get()
if not credentials:
flow = OAuth2WebServerFlow(CLIENT_ID, CLIENT_SECRET, OAUTH_SCOPE, REDIRECT_URI)
authorize_url = flow.step1_get_authorize_url()
print 'Go to the following link in your browser: ' + authorize_url
code = raw_input('Enter verification code: ').strip()
credentials = flow.step2_exchange(code)
storage.put(credentials)
http = httplib2.Http()
http = credentials.authorize(http)
print http
service = build('admin', 'reports_v1', http=http)
print service
data_query = service.customerUsageReports().get(**{'date':'2015-01-07'})
feed = data_query.execute()
print feed

Simply check the case of expired access token and refresh your expired access token like this:
if credentials.access_token_expired:
credentials.refresh(httplib2.Http())
Tip: While developing this, you can test by editing the access token expiry date in the credentials text file and forcing it to be older than an hour
Also, in your code on the line where you are checking if not credentials:, you can better handle that case with:
if credentials is None or credentials.invalid:

I came across this question while trying to find a way to refresh an access token when construction a credentials object when using from_authorized_user_info. Unfortunately, the following code did not work for me:
credentials.refresh(httplib2.Http())
But I found this documentation from the Oauth library that works wonder. Shared below:
import google.auth.transport.requests
import requests
request = google.auth.transport.requests.Request()
credentials.refresh(request)

Using Google API Python client with SignedJwtAssertionCredentials

I am trying to get a list of users from Google using google_api_python_client-1.4.0. I getting the access_denied error even through I have domain wide delegation authority.
Interesting thing is that.. when I use the same certificate/credentials with .net client library, it works.
The error I am getting is
File "/Library/Python/2.7/site-packages/oauth2client-1.4.6-py2.7.egg/oauth2client/client.py", line 807, in _do_refresh_request
oauth2client.client.AccessTokenRefreshError: access_denied: Requested client not authorized.
# Load the key in PKCS 12 format that you downloaded from the Google API
# Console when you created your Service account.
f = file('Gkeys/87ty8g87-privatekey.p12', 'rb')
key = f.read()
f.close()
# Create an httplib2.Http object to handle our HTTP requests and authorize it
# with the Credentials. Note that the first parameter, service_account_name,
# is the Email address created for the Service account. It must be the email
# address associated with the key that was created.
credentials = SignedJwtAssertionCredentials(
'889h98h98h98h98h9lurk#developer.gserviceaccount.com',
key,
scope=['https://www.googleapis.com/auth/admin.directory.group.readonly','https://www.googleapis.com/auth/admin.directory.user.readonly'],
private_key_password='notasecret',
sub='admin.user#domain.com'
)
http = httplib2.Http()
http = credentials.authorize(http)
directory_service = build('admin', 'directory_v1', http=http)
all_users = []
page_token = None
params = {'groupKey': 'groupname#domain.com'}
while True:
try:
if page_token:
params['pageToken'] = page_token
#current_page = directory_service.users().list(**params).execute()
#current_page = directory_service.members().list(**params).execute()
current_page = directory_service.members().list(groupKey='groupname#domain.com').execute()
all_users.extend(current_page['users'])
page_token = current_page.get('nextPageToken')
if not page_token:
break
except errors.HttpError as error:
print 'An error occurred: %s' % error
break
for user in all_users:
print user['primaryEmail']

Are you sure the scopes you authorized in the control panel exactly match those you're requesting here? If you authorized the read/write scope and are using the readonly scope here I believe that would cause your error.