Using Gmail API. My client secret file is downloaded and working for Ruby. When I try the quickstart.py (python) version I get this error
File "quickstart.py", line 70, in <module>
main()
File "quickstart.py", line 55, in main
credentials = get_credentials()
File "quickstart.py", line 38, in get_credentials
credentials = store.get()
File "/Library/Python/2.7/site-packages/oauth2client/client.py", line 374, in get
return self.locked_get()
File "/Library/Python/2.7/site-packages/oauth2client/file.py", line 79, in locked_get
credentials = Credentials.new_from_json(content)
File "/Library/Python/2.7/site-packages/oauth2client/client.py", line 281, in new_from_json
module = data['_module']
KeyError: '_module'
I have not changed the file at all, just added the client_secret.json to that working directory and also install the google-api-python-client. My python code came from here: https://developers.google.com/gmail/api/quickstart/python
Try replacing creds = store.get() with creds = None temporarily. If this works, you can refactor your code to always start with flow-based credentials instantiation. This worked for me. It seems Google samples are out of sync with their oauth2client.
oauth2client is trying to load credentials from a json file with an incorrect structure.
Maybe the Ruby client uses a different file format but I'd be surprised. Are you sure you didn't save client_secret.json as ~/.credentials/gmail-quickstart.json accidentally?
Regardless, removing ~/.credentials/gmail-quickstart.json and re-authenticating will generate a new credentials file with the correct structure.
I am learning Python myself and had a similar problem, but with the Calendar API example. It turned out that it was a typo with regards to the SCOPE.
## Typo - Invalid definition
SCOPES = 'https://ww.googleapies.com/auth/calendar.readonly'
## Correct Value for SCOPE
SCOPES = 'https://www.googleapis.com/auth/calendar'
Also, Matt's answer help point me in the right direction. The gmail-quickstart.json is not the same thing as the client_secret.json. The client_secret.json allows you to make a request for an OAuth2 token. While the gmail-quickstart.json contains the issued token and meta-data associated with it. The gmail-quickstart.json isn't created until you successfully login.
One last thought, in order to log in successfully, the quickstart.py app launched an instance of my web-browser (Firefox) and went to the Google login screen. In order for Firefox to run properly, I had to set my DISPLAY variable properly first.
$ export DISPLAY=:0
$ xhost +
access control disabled, clients can connect from any host
In this GitHub issue: error "KeyError: '_module'" when running gdrive_upload.py
sputnik-dev answared on 10 Jan 2016:
If someone have the same issue : auth_token.txt and client_secret.json are not the same! Don't link the client_secret.json from google API console. The file will be automatically created by the script.
Wrong way: gauth.SaveCredentialsFile("client_secret.json")
Right way: gauth.SaveCredentialsFile("<any random name>.json")
Extra:
PyDrive code that automate google drive api authetication. Use the browser just one time to authenticate and never more. It saves your credential data on mycreds.json :)
from pydrive.auth import GoogleAuth
from pydrive.drive import GoogleDrive
gauth = GoogleAuth()
gauth.LoadCredentialsFile("mycreds.json")
if gauth.credentials is None:
gauth.LocalWebserverAuth()
elif gauth.access_token_expired:
gauth.Refresh()
else:
gauth.Authorize()
gauth.SaveCredentialsFile("mycreds.json")
I solved this by moving the client_secret.json to the same directory as the py file that is trying to read it (quickstart.py), mine was on the Desktop while i had saved the json to Documents. I saved the json to the Desktop too and boy, It flew!.
I dont know why it doesnt work when they are in different directories, defining a custom credential_path doesn't help.
I followed several recommendations listed here:
moving the client_secret.json to the same directory as the py file that is trying to read it, as recommended by simic0de.
Working with skirill idea, instead of completely eliminating the 'credentials=store.get()', I decided to handle the exception, so it would work with flow-based credentials instantiation.
The final code is:
try:
credentials = store.get()
except:
print('Working with flow-based credentials instantiation')
You can write your own code in the except line. This approach will allow the store.get() command to work when conditions are met.
Related
I want to preface this by saying that I do know there's a question on here with the exact same title and similar problem but the solution given there didn't work for me which I'll explain as well.
I'm creating a Python tool to read and edit a google sheet using Google API. To get credentials, I use this code:
SCOPES = ['https://www.googleapis.com/auth/spreadsheets']
creds = Credentials.from_authorized_user_file('token.json', SCOPES)
##This exception is for pyinstaller
try:
service = build('sheets', 'v4', credentials=creds)
except:
DISCOVERY_SERVICE_URL = 'https://sheets.googleapis.com/$discovery/rest?version=v4'
service = build('sheets', 'v4', credentials=creds,discoveryServiceUrl=DISCOVERY_SERVICE_URL)
I'm trying to package this as an .exe with Pyinstaller. It seems to work fine when I'm not specifying --onefile but when I do, it can't seem to locate that JSON file anymore. When I run pyinstaller, I'm including --add-data "token.json;." but it still isn't working.
As I mentioned, this question was asked previously and the answer given was to use this code:
app_creds_dictionary = <paste contents of json file including {}>
creds = ServiceAccountCredentials.from_json_keyfile_dict(app_creds_dictionary, scope)
However, I'm not using service account credentials, I'm using OAuth 2.0 credentials so this doesn't work for me. So I either need an alternative way to read the credentials from a dictionary OR to just figure out how to get pyinstaller to bundle the file correctly. Any help would be greatly appreciated. Thank you!
blob.upload_from_filename(source) gives the error
raise exceptions.from_http_status(response.status_code, message, >response=response)
google.api_core.exceptions.Forbidden: 403 POST >https://www.googleapis.com/upload/storage/v1/b/bucket1-newsdata->bluetechsoft/o?uploadType=multipart: ('Request failed with status >code', 403, 'Expected one of', )
I am following the example of google cloud written in python here!
from google.cloud import storage
def upload_blob(bucket, source, des):
client = storage.Client.from_service_account_json('/path')
storage_client = storage.Client()
bucket = storage_client.get_bucket(bucket)
blob = bucket.blob(des)
blob.upload_from_filename(source)
I used gsutil to upload files, which is working fine.
Tried to list the bucket names using the python script which is also working fine.
I have necessary permissions and GOOGLE_APPLICATION_CREDENTIALS set.
This whole things wasn't working because I didn't have permission storage admin in the service account that I am using in GCP.
Allowing storage admin to my service account solved my problem.
As other answers have indicated that this is related to the issue of permission, I have found one following command as useful way to create default application credential for currently logged in user.
Assuming, you got this error, while running this code in some machine. Just following steps would be sufficient:
SSH to vm where code is running or will be running. Make sure you are user, who has permission to upload things in google storage.
Run following command:
gcloud auth application-default login
This above command will ask to create token by clicking on url. Generate token and paste in ssh console.
That's it. All your python application started as that user, will use this as default credential for storage buckets interaction.
Happy GCP'ing :)
This question is more appropriate for a support case.
As you are getting a 403, most likely you are missing a permission on IAM, the Google Cloud Platform support team will be able to inspect your resources and configurations.
This is what worked for me when the google documentation didn't work. I was getting the same error with the appropriate permissions.
import pathlib
import google.cloud.storage as gcs
client = gcs.Client()
#set target file to write to
target = pathlib.Path("local_file.txt")
#set file to download
FULL_FILE_PATH = "gs://bucket_name/folder_name/file_name.txt"
#open filestream with write permissions
with target.open(mode="wb") as downloaded_file:
#download and write file locally
client.download_blob_to_file(FULL_FILE_PATH, downloaded_file)
We are currently trying to access a folder of an Office 365 ProPlus tenant using the official OneDrive SDK for Python (https://github.com/OneDrive/onedrive-sdk-python). One of our clients would like to use a OneDrive folder as a way of storing and sharing programmatically generated files, therefore, we would like to provide basic file operations.
We have a working solution for a personal OneDrive account, however, when we try to apply the same approach for their OneDrive, we face an issue during the authentication process.
We asked them to register the application in the Azure AD following the steps in the official documentation. Next, they sent us the redirect URI, client ID and client secret that we included in our script. We are trying to use the following code:
redirect_uri = 'REDIRECT_URI'
client_secret = 'CLIENT_SECRET'
client_id='CLIENT_ID'
discovery_uri = 'https://api.office.com/discovery/'
auth_server_url='https://login.microsoftonline.com/common/oauth2/authorize'
auth_token_url='https://login.microsoftonline.com/common/oauth2/token'
http_provider = onedrivesdk.HttpProvider()
auth_provider = onedrivesdk.AuthProvider(http_provider,
client_id,
auth_server_url=auth_server_url,
auth_token_url=auth_token_url)
auth_url = auth_provider.get_auth_url(redirect_uri)
code = GetAuthCodeServer.get_auth_code(auth_url, redirect_uri)
However, we get the following error message when executing the last line:
Traceback (most recent call last):
File "onedrive-test.py", line 25, in
code = GetAuthCodeServer.get_auth_code(auth_url, redirect_uri)
File "/home/username/.local/lib/python3.6/site-packages/onedrivesdk/helpers/GetAuthCodeServer.py",
line 60, in get_auth_code
s = GetAuthCodeServer((host_address, port), code_acquired, GetAuthCodeRequestHandler)
File "/home/username/.local/lib/python3.6/site-packages/onedrivesdk/helpers/GetAuthCodeServer.py",
line 76, in init
HTTPServer.init(self, server_address, RequestHandlerClass)
File "/usr/lib/python3.6/socketserver.py", line 453, in init
self.server_bind()
File "/usr/lib/python3.6/http/server.py", line 136, in server_bind
socketserver.TCPServer.server_bind(self)
File "/usr/lib/python3.6/socketserver.py", line 467, in server_bind
self.socket.bind(self.server_address)
socket.gaierror: [Errno -2] Name or service not known
We also tried opening the auth_url manually, which took us one step further, but still could not authenticate the application with the following error:
AADSTS50020: User account 'USER ACCOUNT' from identity provider
'live.com' does not exist in tenant 'TENANT NAME' and cannot access
the application 'CLIENT ID' in that tenant. The account needs to be
added as an external user in the tenant first. Sign out and sign in
again with a different Azure Active Directory user account.
We have two questions:
What might casue the first error? This is the comment (see below) that can be found in the readme of the SDK about using the GetAuthCodeServer class. It seems to us that the server cannot be run. Are there any not explicitly defined dependencies that we should be aware of before trying to run the webserver? (We are running the script on Ubuntu 18.10)
If you want to remove some of that manual work, you can
use the helper class GetAuthCodeServer. That helper class spins up a
webserver, so this method cannot be used on all environments.
With respect to the second issue, can you recommend proper material for configuring OneDrive for Business for our use-case? We went through a lot of documentation, but after long hours of research, we still could not find the correct way to fix that issue, especially since we do not have direct acces to the tenant and we cannot easily experiment with things. We would need to give a step-by-step cookbook to our client to set up everything on their side.
Any help would be much appreciated! :)
I was playing around with google admin api and put the example code together with a little modification. I set things up correctly on the google end, but when I run the script it kept giving me errors. Below is my code in python:
from __future__ import print_function
from apiclient.discovery import build
from httplib2 import Http
from oauth2client import file, client, tools
SCOPES = 'https://www.googleapis.com/auth/admin.directory.user'
store = file.Storage('storage.json')
creds = store.get()
if not creds or creds.invalid:
flow = client.flow_from_clientsecrets('client_secret.json', SCOPES)
creds = tools.run_flow(flow, store)
SERVICE = build('admin', 'directory_v1', http=creds.authorize(Http()))
results = SERVICE.users().list(customer='my_customer', maxResults=10,
orderBy='email').execute()
users = results.get('users', [])
Error I am getting is this:
Traceback (most recent call last):
File "quickstart.py", line 11, in <module>
flow = client.flow_from_clientsecrets('client_secret.json', SCOPES)
File "build/bdist.macosx-10.12-intel/egg/oauth2client/_helpers.py", line 133, in positional_wrapper
File "build/bdist.macosx-10.12-intel/egg/oauth2client/client.py", line 2125, in flow_from_clientsecrets
File "build/bdist.macosx-10.12-intel/egg/oauth2client/clientsecrets.py", line 165, in loadfile
File "build/bdist.macosx-10.12-intel/egg/oauth2client/clientsecrets.py", line 126, in _loadfile
File "build/bdist.macosx-10.12-intel/egg/oauth2client/clientsecrets.py", line 101, in _validate_clientsecrets
oauth2client.clientsecrets.InvalidClientSecretsError: Missing property "client_secret" in a client type of "installed".
I am new to google api and would very much appreciate for any helps.
Thanks
The issue I've found is with the name of Google's OAuth 2.0 client IDs. For your client ID, the name must EXACTLY be Google Sheets API Quickstart, or else the "client_secret" won't be populated in the client_secret.json when you download it from Google.
Alternatively, you could manually add a property under the installed JSON object, such as "client_secret":"mysupercoolsecret", using the secret generated for the credential when you made the credential.
Either way, I have no idea why Google would only populate the "client_secret" field when the name of the credential doesn't match a certain value.
Hope this helps. I've encounter this issue with using the Google API for other languages as well.
You need to click RESET SECRET button in the JSON Download page.
After that, you will have a brand new Client Secret and auto generated in your Json file.
I just encountered this same exact problem, and solved it this way. Hope it helps.
Download the client_secret.json again. There is a problem with that file.
You can use this link for a small tutorial to get your client_secret.
I'm using PyGithub v1.25 to create a little webapp where members of my organization can create private repositories in our github organization. Right now, I'm getting a BadCredentialsException when trying to call the get_organization() method of the Github base class.
Here is the relevant portion of my code:
from github import Github
import settings
GIT_OBJECT = Github(login_or_token=settings.AUTH_TOKEN)
ORG_OBJECT = GIT_OBJECT.get_organization('My-Organization-Name')
The auth token I am using was generated from my github user account, which has sufficient privileges to create private repositories in this organization when using the github web interface. I created the token with "user", "repo", and "admin:org" scopes selected. I am getting an error at the creation of ORG_OBJECT.
The stack trace:
File "/local/path/to/my/code/github_console/console/org_manage.py", line 10, in <module>
ORG_OBJECT = GIT_OBJECT.get_organization(‘My-Organization-Name’)
File "/local/path/to/my/code/github_console/lib/github/MainClass.py", line 187, in get_organization
"/orgs/" + login
File "/local/path/to/my/code/github_console/lib/github/Requester.py", line 169, in requestJsonAndCheck
return self.__check(*self.requestJson(verb, url, parameters, headers, input, cnx))
File "/local/path/to/my/code/github_console/lib/github/Requester.py", line 177, in __check
raise self.__createException(status, responseHeaders, output)
BadCredentialsException: 401 {u'documentation_url': u'https://developer.github.com/v3', u'message': u'Bad credentials'}
If anyone who has used either PyGithub or the github API before (or someone who is better than me at reading docs) has any insights, I appreciate the help!
Here's the PyGithub source code, in case anyone wants a look at that.
Doh!
Apparently, the above displayed code works great, and I just effed up importing local settings into my settings module, so a dummy AUTH_TOKEN was being used, and of course, resulting in a BadCredentialsException.
On the plus side, I guess the above is a demonstration of correct PyGithub usage.