i am trying to run the gmail API to get email from a specif user.
However i always receive the following error:
googleapiclient.errors.HttpError: https://www.googleapis.com/gmail/v1/users/me/settings/filters?alt=json returned "Insufficient Permission">
I have tried all the different options suggested from changing the scope to enabling the less secure applications in the gmail setting.
The last thing is that my code could be wrong:
from __future__ import print_function
from googleapiclient import discovery
from googleapiclient.discovery import build
from httplib2 import Http
from oauth2client import file, client, tools
# Setup the Gmail API
SCOPES = 'https://mail.google.com/'
store = file.Storage('credentials.json')
creds = store.get()
if not creds or creds.invalid:
flow = client.flow_from_clientsecrets('client_secret.json', SCOPES)
creds = tools.run_flow(flow, store)
service = build('gmail', 'v1', http=creds.authorize(Http()))
# Call the Gmail API
# results = service.users().labels().list(userId='me').execute()
# labels = results.get('labels', [])
# if not labels:
# print('No labels found.')
# else:
# print('Labels:')
# for label in labels:
# print(label['name'])
label_id = 'mine' # ID of user label to add
filter = {
'criteria': {
'from': 'string#test.com'
}#,
#'action': {
#'addLabelIds': [label_id],
#'removeLabelIds': ['INBOX']
#}
}
result = service.users().settings().filters().create(userId='me', body=filter).execute()
print ('Created filter: %s' % result.get('id'))
Could please help me?
Thanks
Just remove credentials.json file from the folder where you run this script from and rerun your script with the final desired list of scopes such as SCOPES = ['https://www.googleapis.com/auth/gmail.send', 'https://www.googleapis.com/auth/gmail.labels']. Authorize, at the resulting pop up browser window, the list of scopes for the desired gmail account. Then you should be all set.
Background story: It is likely you had authorized less permissive scopes (such as readonly) via the browser screen that popped up when your run_flow() ran first time. This would have created a file called credentials.json in the folder where you ran the script. This credentials.json is not again regenerated per your script above. It is just frozen with the first time scope. Just for curiosity, you can open and read the credentials.json and look for the key 'scopes'. Of course, you cannot add more scopes in this file manually and expect newer scopes to work. This has to be regenerated for different set of scopes if any.
Unrelated cautionary note: Avoid the too permissive scope https://mail.google.com/ if at all possible:)
None of the above fixed my problem with moving emails from the inbox into another "box/label". I am not sure if things changed with the gmail api, but what I found I had to do was to change the SCOPES line to the desired level i.e. SCOPES= ['https://www.googleapis.com/auth/gmail.modify'], then delete the token.pickle file (in that order). Following those two changes I reran the python script. Those two steps caused the authorization web page to appear reflecting the new permissions being granted and thus after me giving the OK, recreating a new token.pickle file. Everything ran as desired after that. No more 403 errors and the emails moved to the target box/label. I found the credentials.json file just allowed the script to access the emails but did not have any impact on the actions taken on the emails themselves. I found the article on Medium titled "A Beginner’s Guide to the Gmail API and Its Documentation" by Anton Odman to be the best overall guide to python and the gmail api. BTW, The other "gotcha" is the label created is not necessarily the label id. I recommend running code as mentioned at https://stackoverflow.com/a/55713119/4495318 to make sure you get the right label ID, it seems the label IDs and the label names are not the same except for the system generated labels.
Simply deleting the JSON file created using file.Storage('credentials.json') after you change or add SCOPES will help you and run the program once again.
Related
I am trying to build a script on my local machine using Python that will download a Google sheet from my account using a short lived federated identity token. I have seen documentation that speaks about differences between doing this depends on context, so starting on a MacOS running Monterey and VScode, and later to HA (Minikube, k9s, etc. and then cloud).
I have enabled the Sheets API via the browser for the current project, obtained a client ID and secret, and have the cli installed. I ran a gcloud auth application-default login. I am currently seeing a 403 and potentially insufficient scopes. Older posts here speaks of clobbering/modifying the .credentials and starting with a proper scope. I know a lot has changed with variable length tokens, scope and all since 2019...
Also I looked here on recommendation from Workspaces support, and seem to be in a pickle, pardon the pun;) Is it still a pickle I'm going to deal with here?
Here is my current code:
"""
BEFORE RUNNING:
---------------
1. If not already done, enable the Google Sheets API
and check the quota for your project at
https://console.developers.google.com/apis/api/sheets
2. Install the Python client library for Google APIs by running
`pip install --upgrade google-api-python-client`
"""
from pprint import pprint
from googleapiclient import discovery
# TODO: Change placeholder below to generate authentication credentials. See
# https://developers.google.com/sheets/quickstart/python#step_3_set_up_the_sample
#
# Authorize using one of the following scopes:
# 'https://www.googleapis.com/auth/drive'
# 'https://www.googleapis.com/auth/drive.file'
# 'https://www.googleapis.com/auth/drive.readonly'
# 'https://www.googleapis.com/auth/spreadsheets'
# 'https://www.googleapis.com/auth/spreadsheets.readonly'
credentials = None
service = discovery.build('sheets', 'v4', credentials=credentials)
# The ID of the spreadsheet to retrieve data from.
spreadsheet_id = '' # TODO: Update placeholder value.
# The A1 notation of the values to retrieve.
range_ = 'A1:Y1' # TODO: Update placeholder value.
# How values should be represented in the output.
# The default render option is ValueRenderOption.FORMATTED_VALUE.
value_render_option = '' # TODO: Update placeholder value.
# How dates, times, and durations should be represented in the output.
# This is ignored if value_render_option is
# FORMATTED_VALUE.
# The default dateTime render option is [DateTimeRenderOption.SERIAL_NUMBER].
date_time_render_option = '' # TODO: Update placeholder value.
# request = service.spreadsheets().values().get(spreadsheetId=spreadsheet_id, range=range_, valueRenderOption=value_render_option, dateTimeRenderOption=date_time_render_option)
# response = request.execute()
# errors here from above
request = service.spreadsheets().values().get(spreadsheetId=spreadsheet_id, range=range_)
response = request.execute()
# TODO: Change code below to process the `response` dict:
pprint(response)
and the latest error:
raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 403 when requesting https://sheets.googleapis.com/v4/spreadsheets/1234567long_spreadsheet_id/values/A1%3AY1?alt=json returned "Request had insufficient authentication scopes.". Details: "[{'#type': 'type.googleapis.com/google.rpc.ErrorInfo', 'reason': 'ACCESS_TOKEN_SCOPE_INSUFFICIENT', 'domain': 'googleapis.com', 'metadata': {'service': 'sheets.googleapis.com', 'method': 'google.apps.sheets.v4.SpreadsheetsService.GetValues'}}]"
If someone could point me in a good direction- either detailed current documentation as it relates to python (I may have to back up and punt with Apps Script or something for the time being and POC), or nudge me with some pseudocode in the way I need to properly authenticate to get my credential file working with a short lived token (IMHO better for the time being than having a small business trying to manage keys on GCP) and possibly for a refresh after it's stale, it would be greatly appreciated. Thanks in advance.
I was writing a Python script to automate uploading some files to Google Drive. Since I'm still a newbie Python programmer and this is an exercise as much as anything else, I started following the Google Quickstart and decided to use their quickstart.py as a basis on which to base my own script. In the part where it talks about how to create credentials for your Python script, it refers to the "Create credentials" link, at https://developers.google.com/workspace/guides/create-credentials
I follow the link, get into one of my Google Cloud projects, and try to set up the OAuth consent screen, using an "Internal" project, as they tell you... but I can't. Google says:
“Because you’re not a Google Workspace user, you can only make your
app available to external (general audience) users. ”
So I try to create an "External" project, and then proceed to create a new client ID, using a Desktop application. Then I download the JSON credentials and put them in the same folder as my Python script, as "credentials.json". I then execute the Python script in order to authenticate it: the browser opens, I log into my Google account, give it my permissions... and then the browser hangs, because it's redirecting to a localhost URL and obviously my little Python script isn't listening in my computer at all.
I believe they must have changed this recently, because a year ago I started following the same Python tutorial and could create credentials without problems, but the Google Drive API docs haven't been updated yet. So... how do I create credentials for a Python script now?
EDIT: adding here the source code for my script. As I said, it's very similar to Google's "quickstart.py":
from __future__ import print_function
import pickle
import os.path
from googleapiclient.discovery import build
from google_auth_oauthlib.flow import InstalledAppFlow
from google.auth.transport.requests import Request
from googleapiclient.errors import HttpError
# If modifying these scopes, delete the file token.pickle.
SCOPES = ['https://www.googleapis.com/auth/drive.metadata', 'https://www.googleapis.com/auth/drive']
def main():
"""Shows basic usage of the Drive v3 API.
Prints the names and ids of the first 10 files the user has access to.
"""
creds = None
# The file token.pickle stores the user's access and refresh tokens, and is
# created automatically when the authorization flow completes for the first
# time.
if os.path.exists('token_myappname.pickle'):
with open('token_myappname.pickle', 'rb') as token:
creds = pickle.load(token)
# If there are no (valid) credentials available, let the user log in.
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
flow = InstalledAppFlow.from_client_secrets_file(
'credentials.json', SCOPES)
creds = flow.run_local_server(port=0)
# Save the credentials for the next run
with open('token_myappname.pickle', 'wb') as token:
pickle.dump(creds, token)
service = build('drive', 'v3', credentials=creds)
# Call the Drive v3 API
results = service.files().list(
pageSize=10, fields="nextPageToken, files(id, name)").execute()
items = results.get('files', [])
if not items:
print('No files found.')
else:
#print(items[0])
print('Files:')
for item in items:
#print (item)
print(u'{0} {1} {2}'.format(item['name'], item['owners'], item['parents']))
I propose you to use a service account to access to the Drive.
For that, you need to share the drive (or the folder) with the service account email. And then use this code
from googleapiclient.discovery import build
import google.auth
SCOPES = ['https://www.googleapis.com/auth/drive.metadata', 'https://www.googleapis.com/auth/drive']
def main():
credentials, project_id = google.auth.default(scopes=SCOPES)
service = build('drive', 'v3', credentials=credentials)
# Call the Drive v3 API
results = service.files().list(
q=f"'1YJ6gMgACOqVVbcgKviJKtVa5ITgsI1yP' in parents",
pageSize=10, fields="nextPageToken, files(id, name, owners, parents)").execute()
items = results.get('files', [])
if not items:
print('No files found.')
else:
#print(items[0])
print('Files:')
for item in items:
#print (item)
print(u'{0} {1} {2}'.format(item['name'], item['owners'], item['parents']))
If you run your code on Google Cloud, in a compute engine instance for example, you need to customize the VM with the service account that you authorized in your drive. (Don't use the compute engine default service account, else you will need extra configuration on your VM)
If you run your script outside GCP, you need to generate a service account key file and to store it on your local server. Then, create an environment variable GOOGLE_APPLICATION_CREDENTIALS that reference the full path of the stored key file.
Aside from the other solution posted by Guillaume Blaquiere, I also found another one on my own, which I wanted to post here in case it's helpful. All I had to do is to... erm, actually read the code I was copying and pasting, in particular this line:
creds = flow.run_local_server(port=0)
I checked Google's documentation outside of the Quickstart and found in the following: https://google-auth-oauthlib.readthedocs.io/en/latest/reference/google_auth_oauthlib.flow.html
It turns out, the example code was opening a local port in my computer to listen to the request, and it wasn't working probably due to the "port 0" part, or some other network problem.
So the workaround I found was to use a different auth method found in the docs:
creds = flow.run_console()
In this case, you paste manually in the command line the auth code given to you by Google. I just tried it, and have my credentials happily stored in my local pickle file.
I'm using the default code provided from google here, and I don't quite understand why its not working. The code outputs the prompt Please visit this URL to authorize this application: [google login URL]. When attempting to log in with the account designated as owner of the script under the google developers console I get a Error 403: access_denied error with the message The developer hasn’t given you access to this app. It’s currently being tested and it hasn’t been verified by Google. If you think you should have access, contact the developer [the email I just tried to log in with].
from __future__ import print_function
import pickle
import os.path
from googleapiclient.discovery import build
from google_auth_oauthlib.flow import InstalledAppFlow
from google.auth.transport.requests import Request
# If modifying these scopes, delete the file token.pickle.
SCOPES = ['https://www.googleapis.com/auth/spreadsheets.readonly']
# The ID and range of a sample spreadsheet.
SAMPLE_SPREADSHEET_ID = '1vrZpCGW58qCCEfVXoJYlwlulraIlfWI2SmFXa1iPtuU'
SAMPLE_RANGE_NAME = 'Class Data!A2:E'
def main():
"""Shows basic usage of the Sheets API.
Prints values from a sample spreadsheet.
"""
creds = None
# The file token.pickle stores the user's access and refresh tokens, and is
# created automatically when the authorization flow completes for the first
# time.
if os.path.exists('token.pickle'):
with open('token.pickle', 'rb') as token:
creds = pickle.load(token)
# If there are no (valid) credentials available, let the user log in.
if not creds or not creds.valid:
if creds and creds.expired and creds.refresh_token:
creds.refresh(Request())
else:
flow = InstalledAppFlow.from_client_secrets_file(
'Google_API_Key.json', SCOPES)
creds = flow.run_local_server(port=0)
# Save the credentials for the next run
with open('token.pickle', 'wb') as token:
pickle.dump(creds, token)
service = build('sheets', 'v4', credentials=creds)
# Call the Sheets API
sheet = service.spreadsheets()
result = sheet.values().get(spreadsheetId=SAMPLE_SPREADSHEET_ID,
range=SAMPLE_RANGE_NAME).execute()
values = result.get('values', [])
if not values:
print('No data found.')
else:
print('Name, Major:')
for row in values:
# Print columns A and E, which correspond to indices 0 and 4.
print('%s, %s' % (row[0], row[4]))
def PrintToSheets():
main()
To fix this issue for me was this simple:
Go to https://console.developers.google.com/
On the top left beside the words "Google APIs" click the project dropdown on the right
Ensure that your correct project is selected
Click "OAuth consent screen" on the left side of the screen (below "Credentials")
If you have not created a consent screen, do that first
Under "Test users" there is a button called "+ ADD USERS"
Type the email of the account you will be testing with, press enter, then click save.
It should work now!
It seems like they updated this recently because last year I did not have to do this.
The error message you are getting is related to the fact that your application has not been verified yet.
As mentioned in that link all applications that access google APIs using sensitive scopes need to go though googles verification process. Normal you are given a grace period of 100 users accessing your application before your application will be locked down and you wont be able to authorize anymore users until you verify your application it sounds like you may have hit that point.
Your only options would be to go though the verification process or to create a whole new project on Google developer console and use that client id instead as the one you are using is currently locked for additional users.
Update Change
A change has been implemented on Google Developer console. You must now authorize users / testers to access your application before it has gone through the verification process. If you go to the Google developer console for your project, under consent screen you will find the following section
You can add tests users here, however you can not remove them and you can only add 100 of them so use it wisely.
My Python (3.6.7) code uses oauth2client to access Google Photos APIs. It successfully authenticates, but when it tries to access the Google Photos albums, it seems to be using the username as the project_id.
from __future__ import print_function
from apiclient.discovery import build
from httplib2 import Http
from oauth2client import file, client, tools
# Setup the Photo v1 API
SCOPES = 'https://www.googleapis.com/auth/photoslibrary.readonly'
store = file.Storage('credentials.json')
creds = store.get()
if not creds or creds.invalid:
flow = client.flow_from_clientsecrets('scripts/client_id.json', SCOPES)
creds = tools.run_flow(flow, store)
service = build('photoslibrary', 'v1', http=creds.authorize(Http()))
# Call the Photo v1 API
results = service.albums().list(
pageSize=10, fields="nextPageToken,albums(id,title)").execute()
items = results.get('albums', [])
if not items:
print('No albums found.')
else:
print('Albums:')
for item in items:
print('{0} ({1})'.format(item['title'].encode('utf8'), item['id']))
When executing the above code, it prompts me the auth page. When I successfully authenticate, it shows me the following error:
HttpError 403 when requesting {URL} returned "Photos Library API has not been used in project 123456 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/photoslibrary.googleapis.com/overview?project=123456 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.">
Interestingly, the number in bold 123456 (obviously changed) is actually the first part of the client_id found in the client_id.json
But the project_id looks something like this: test1-235515
So what I got from this error is that the oauth2client client is passing the client_id instead of the project_id. So even though I have enabled the Photos API, it will never access it correctly.
Please help with this error. How can I manually change the project_id?
The project ID is different from the project number. You will be able to see both in your Google Cloud Console configuration. See this documentation for more on how to identify your projects [1].
A single Google Cloud project can have many different OAuth client IDs configured. See this documentation for information about creating OAuth client credentials [2]. You should be only have to make sure that the client you created belongs to the project for which you have enabled APIs. Going to the URL provided in the error message should take you to the right configuration page.
[1] https://cloud.google.com/resource-manager/docs/creating-managing-projects#identifying_projects
[2] https://support.google.com/cloud/answer/6158849?hl=en
Anyone know if this is possible?
I just want to automate dropping some documents into my onedrive for business account.
I tried
import onedrivesdk
from onedrivesdk.helpers import GetAuthCodeServer
from onedrivesdk.helpers.resource_discovery import ResourceDiscoveryRequest
redirect_uri = 'http://localhost:8080'
client_id = 'appid'
client_secret = 'mysecret'
discovery_uri = 'https://api.office.com/discovery/'
auth_server_url='https://login.live.com/oauth20_authorize.srf?scope=wl.skydrive_update'
#auth_server_url='https://login.microsoftonline.com/common/oauth2/authorize',
auth_token_url='https://login.microsoftonline.com/common/oauth2/token'
http = onedrivesdk.HttpProvider()
auth = onedrivesdk.AuthProvider(http,
client_id,
auth_server_url=auth_server_url,
auth_token_url=auth_token_url)
auth_url = auth.get_auth_url(redirect_uri)
code = GetAuthCodeServer.get_auth_code(auth_url, redirect_uri)
auth.authenticate(code, redirect_uri, client_secret, resource=resource)
# If you have access to more than one service, you'll need to decide
# which ServiceInfo to use instead of just using the first one, as below.
service_info = ResourceDiscoveryRequest().get_service_info(auth.access_token)[0]
auth.redeem_refresh_token(service_info.service_resource_id)
client = onedrivesdk.OneDriveClient(service_info.service_resource_id + '/_api/v2.0/', auth, http)
I registered an APP and got a secret and id. But when I ran this I got scope is invalid errors. Plus it tries to launch a webpage which isn't great for a command line kinda environment. I think this SDK might be outdated as well because originally this script had login.microsoftonline, but that wasn't reachable so I changed it to login.live.com.
I wrote this sample code you posted. You replaced the auth_server_URLwith the authentication URL for Microsoft Account authentication, which can only be used to access OneDrive (the consumer product). You need to continue using the login.microsoftonline.com URL to log into your OneDrive for Business account.
You are correct that this pops up a dialog. However, you can write a little supporting code so that only happens the first time you log into a particular app. Follow these steps (assuming you are using the default implementation of AuthProvider:
Use the sample code above up through the line auth.redeem_refresh_token()
The AuthProvider will now have a Session object, which caches the credentials of the current user and session. Use AuthProvider.save_session() to save the credentials for later.
Next time you start your app, use AuthProvider.load_session() and AuthProvider.refresh_token() to retrieve the previous session and refresh the auth token. This will all be headless.
Take note that the default implementation of SessionBase (found here) uses Pickle and is not safe for product use. Make sure to create a new implementation of Session if you intend to deploy this app to other users.
Onerive's website shows "Not Yet" on "OneDrive SDK for Python" to "OneDrive for Business"
https://dev.onedrive.com/SDKs.htm
The github sample codes did not work for me either, it tried to popup a window of authentication, but IE can not find the address:
http://('https//login.microsoftonline.com/common/oauth2/authorize',)?redirect_uri=http%3A%2F%2Flocalhost%3A8080&client_id=034xxxx9-9xx8-4xxf-bexx-1bc5xxxxbd0c&response_type=code
or removed all the "-" in client id
http://('https//login.microsoftonline.com/common/oauth2/authorize',)?redirect_uri=http%3A%2F%2Flocalhost%3A8080&client_id=034xxxx99xx84xxfbexx1bc5xxxxbd0c&response_type=code
Either way, I got the same result, IE did not show the popup with a line "This page can’t be displayed"