I'm currently making a GUI YouTube video uploader for my community, but since I don't want all of my users to get my client_id and client_secret, I encoded them. Problem is that whenever program runs (it's not being run from command line using parameters, it gets those informations from Tkinter GUI) it start to authenticate users via web link, which contains real client_id and client_secret. I tried to use --noauth_local_webserver parameter but without success, since nothing is being run from command-line (I haven't found way to run this parameter without command line). As I saw on official docs, this parameter is set to "False" by default, is there any way to change that, or is there any way to disable web authentication? This is my code which I use to authenticate and start uploading a video (it's pretty much default one from official docs, with few changes so it fits my needs):
def get_authenticated_service():
makeitreal() #this is function which decodes encoded client_id and client_secret
flow = flow_from_clientsecrets(os.path.abspath(os.path.join(os.path.dirname(__file__), "client_secrets.json")), scope=YOUTUBE_UPLOAD_SCOPE,
message=MISSING_CLIENT_SECRETS_MESSAGE)
storage = Storage("%s-oauth2.json" % sys.argv[0])
credentials = storage.get()
if credentials is None or credentials.invalid:
credentials = run(flow, storage)
return build(YOUTUBE_API_SERVICE_NAME, YOUTUBE_API_VERSION,
http=credentials.authorize(httplib2.Http()))
def initialize_upload():
makeitreal() #this is function which decodes encoded client_id and client_secret
youtube = get_authenticated_service()
os.remove(os.path.join(os.path.dirname(__file__), "upload_video.py-oauth2.json")) #I use this to remove this json since it's not being used anymore and it contains client_id and client_secret
tags = None
insert_request = youtube.videos().insert(
part="snippet,status",
body=dict(
snippet=dict(
title=video_title, #####
description=video_desc, # These 3 parameters are not being gathered through command line as it was in default code, I changed it so it gets these from Tkinter GUI
tags=video_keywords, ####
categoryId="22"
),
status=dict(
privacyStatus=VALID_PRIVACY_STATUSES[0]
)
),
# chunksize=-1 means that the entire file will be uploaded in a single
# HTTP request. (If the upload fails, it will still be retried where it
# left off.) This is usually a best practice, but if you're using Python
# older than 2.6 or if you're running on App Engine, you should set the
# chunksize to something like 1024 * 1024 (1 megabyte).
media_body=MediaFileUpload(filename, chunksize=-1, resumable=True)
)
makeitfake() #this is function which encodes previously decoded client_id and client_secret
resumable_upload(insert_request) #this function uploads video
Thanks in advance, Amar!
You're missing some code. Update to the latest API and examples and it's as simple as: args.noauth_local_webserver = True
Anyway, here's some of the code if you want to try adding support for argparser yourself. There's no longer a run but a run_flow. But you can pass args as the third parameter to your existing run function.
from oauth2client.tools import argparser, run_flow
args = argparser.parse_args()
args.noauth_local_webserver = True
credentials = run_flow(flow, storage, args)
Alternatively, if you must make it work, you can modify oauth2client/tools.py and search for if not flags.noauth_local_webserver and right above that just add flags.noauth_local_webserver = True However, I must point out that modifying core packages is not recommended as your changes will be clobbered the next time you update your packages. The cleanest solution is to update to the latest versions of everything which makes it easier to do what you want to do.
Related
Solution
So I don't think its a surprise to anyone but Google's documentation is god awful. It's so scattered and the Python docs still reference their old depreciated library. Anyways.
So what I really needed to look at was this link "Enabling Server Side Access for your App". This is not linked to anywhere. Keep in mind this is entirely different than "Authenticating with a Backend Server"
This was a start. On the iOS side of things, we need to specify the server or backend's client_id.
...
GIDSignIn.sharedInstance().clientID = SBConstants.Google.IOS_CLIENT_ID
GIDSignIn.sharedInstance().serverClientID = SBConstants.Google.SERVER_CLIENT_ID
...
And capture serverAuthCode from the sign method inside your sign-in delegate.
...
self.googleUser.userID = user.userID
self.googleUser.token = user.authentication.idToken
self.googleUser.serverAuthCode = user.serverAuthCode
...
Now when you want to perform some action in the backend on behalf of the frontend, we pass the captured serverAuthCode and send it as a parameter.
That was the easy part. In the backend, Google seems to have 13 different OAuth2 libraries for Python documented. Their example uses oauth2client which of course is deprecated.
What we want to use is their 'new' library google-api-python-client.
When the auth_token is passed to the backend we need to check if the user already has an access token in our database. If it does, we need to refresh. Otherwise, we need to request a new access token based on the auth_code. After much trial and error, here is the code to do so:
# we have record of this user
# we have record of this user
if user.exists:
# create new credentials, and refresh
credentials = Credentials(
token=user.token,
refresh_token=user.refresh_token,
client_id=CLIENT_ID,
client_secret=CLIENT_SECRET,
token_uri='https://oauth2.googleapis.com/token')
# now we have an access token
credentials.refresh(requests.Request())
else:
# get the auth_token
token_obj = json.loads(request.body)
code = token_obj.get('auth_code')
# request access token given auth_token
auth_flow = flow.Flow.from_client_secrets_file(creds, scopes=scopes)
auth_flow.fetch_token(code=code)
# now have access token
credentials = auth_flow.credentials
A warning: Pass or fail, the auth_token is only good for one request. This totally burned me. This also means once you have a successful backend interaction, you must store the user's token information to then request a refresh not a new access token.
Hope this helps someone.
Original Post
Following the documentation here, I am trying to authenticate a user in my iOS app and pass their ID token to my backend. The backend handles the Google API interactions for the iOS app.
I am missing how to actually authenticate that user in the backend. I read over the docs here regarding ID tokens but I am confused on where the service account comes into play.
Current endpoint:
#api_view(['POST'])
#authentication_classes([TokenAuthentication])
#permission_classes([IsAuthenticated])
def google_token_info(request):
try:
token_obj = json.loads(request.body)
token = token_obj['id_token']
id_info = id_token.verify_oauth2_token(token, requests.Request(), settings.IOS_CLIENT_ID)
# create session here - how?
This is all working fine. The ID info returns the expected decrypted JWT contents, and I have the user's unique Google ID at this point.
While testing I had authentication set up via my backend. I had code like this:
def google_auth(request):
web_flow = flow.Flow.from_client_secrets_file(creds, scopes=scopes)
web_flow.redirect_uri = request.build_absolute_uri(reverse('api.auth:oauth_callback'))
auth_url, state = web_flow.authorization_url(access_type='offline', include_granted_scopes='true', prompt='consent')
request.session['state'] = state
return redirect(auth_url)
def oauth_callback(request):
success_flow = flow.Flow.from_client_secrets_file(creds, scopes=scopes, state=request.session.get('state'))
success_flow.redirect_uri = request.build_absolute_uri(reverse('api.auth:oauth_callback'))
auth_response = request.build_absolute_uri()
success_flow.fetch_token(authorization_response=auth_response)
credentials = success_flow.credentials
if not request.session.get('google_credentials'):
request.session['google_credentials'] = _credentials_to_dict(credentials)
return redirect(reverse('api.auth:success'))
Which setup session credentials for the user. I'm assuming I need something similar, but I am unsure how to create a session without actual credentials.
Solution
So I don't think its a surprise to anyone but Google's documentation is god awful. It's so scattered and the Python docs still reference their old depreciated library. Anyways.
So what I really needed to look at was this link "Enabling Server Side Access for your App". This is not linked to anywhere. Keep in mind this is entirely different than "Authenticating with a Backend Server"
This was a start. On the iOS side of things, we need to specify the server or backend's client_id.
...
GIDSignIn.sharedInstance().clientID = SBConstants.Google.IOS_CLIENT_ID
GIDSignIn.sharedInstance().serverClientID = SBConstants.Google.SERVER_CLIENT_ID
...
And capture serverAuthCode from the sign method inside your sign-in delegate.
...
self.googleUser.userID = user.userID
self.googleUser.token = user.authentication.idToken
self.googleUser.serverAuthCode = user.serverAuthCode
...
Now when you want to perform some action in the backend on behalf of the frontend, we pass the captured serverAuthCode and send it as a parameter.
That was the easy part. In the backend, Google seems to have 13 different OAuth2 libraries for Python documented. Their example uses oauth2client which of course is deprecated.
What we want to use is their 'new' library google-api-python-client.
When the auth_token is passed to the backend we need to check if the user already has an access token in our database. If it does, we need to refresh. Otherwise, we need to request a new access token based on the auth_code. After much trial and error, here is the code to do so:
# we have record of this user
# we have record of this user
if user.exists:
# create new credentials, and refresh
credentials = Credentials(
token=user.token,
refresh_token=user.refresh_token,
client_id=CLIENT_ID,
client_secret=CLIENT_SECRET,
token_uri='https://oauth2.googleapis.com/token')
# now we have an access token
credentials.refresh(requests.Request())
else:
# get the auth_token
token_obj = json.loads(request.body)
code = token_obj.get('auth_code')
# request access token given auth_token
auth_flow = flow.Flow.from_client_secrets_file(creds, scopes=scopes)
auth_flow.fetch_token(code=code)
# now have access token
credentials = auth_flow.credentials
A warning: Pass or fail, the auth_token is only good for one request. This totally burned me. This also means once you have a successful backend interaction, you must store the user's token information to then request a refresh not a new access token.
I am using mendeley a lot for my research. I want to edit the "user-tags" for the documents in the catalog of my mendeley client. Specifically I want to add the reputation of the journal to this field (in form of h-factor or impact factor). I've successfully set up the OAuth2 Authentification and am able to retrieve all the documents in my catalog. However I am not able to change the details / sync them back.
Do you know if this would be possible with the Mendeley API?
I didn't find a method like set or sync in the documentation of the API.
from mendeley import Mendeley
# I've changed the authentication details of my script (of course)
client_id = ****
client_secret = "abcdefghijklmnop"
redirect_uri = "http://localhost:8080/someTest"
# These values should match the ones supplied when registering your application
mendeley = Mendeley(client_id, redirect_uri=redirect_uri)
auth = mendeley.start_implicit_grant_flow()
# The user needs to visit this URL, and log in to Mendeley.
login_url = auth.get_login_url()
res = requests.post(login_url, allow_redirects=False, data={
'username': 'mymail#myprovider.net',
'password': 'somePsasword?!'
})
auth_response = res.headers['Location']
# After logging in, the user will be redirected to a URL, auth_response.
session = auth.authenticate(auth_response)
# print(session.files.list().items)
for document in session.documents.iter(view='tags'):
print(document.title)
a = session.documents.get("5982d0ce-0425-3548-a063-519620c17886", view='tags')
a.tags = "TESTETETSETEST"
Another option would be to just change my catalog locally on my PC, however I wasn't able to find a file / database for this in my mendeley directory
Edit:
I've tried the update() method mentioned in the API by changing the loop to the following. Which didn't solve my problem for now
for document in session.documents.iter(view='tags'):
print(document.tags)
document.tags = ["Test"]
document.update()
I'm not familiar with the Mendeley Python SDK but it did seem odd to me that you didn't need to make some other calls after a.tags = "TESTETETSETEST".
Do you not need to use the update method?
So I'm trying to produce temporary globally readable URLs for my Google Cloud Storage objects using the google-cloud-storage Python library (https://googlecloudplatform.github.io/google-cloud-python/latest/storage/blobs.html) - more specifically the Blob.generate_signed_url() method. I doing this from within a Compute Engine instance in a command line Python script. And I keep getting the following error:
AttributeError: you need a private key to sign credentials.the credentials you are currently using <class 'oauth2cl
ient.service_account.ServiceAccountCredentials'> just contains a token. see https://google-cloud-python.readthedocs
.io/en/latest/core/auth.html?highlight=authentication#setting-up-a-service-account for more details.
I am aware that there are issues with doing this from within GCE (https://github.com/GoogleCloudPlatform/google-auth-library-python/issues/50) but I have created a new Service Account credentials following the instructions here: https://cloud.google.com/storage/docs/access-control/create-signed-urls-program and my key.json file most certainly includes a private key. Still I am seeing that error.
This is my code:
keyfile = "/path/to/my/key.json"
credentials = ServiceAccountCredentials.from_json_keyfile_name(keyfile)
expiration = timedelta(3) # valid for 3 days
url = blob.generate_signed_url(expiration, method="GET",
credentials=credentials)
I've read through the issue tracker here https://github.com/GoogleCloudPlatform/google-cloud-python/issues?page=2&q=is%3Aissue+is%3Aopen and nothing related jumps out so I am assuming this should work. Cannot see what's going wrong here.
I was having the same issue. Ended up fixing it by starting the storage client directly from the service account json.
storage_client = storage.Client.from_service_account_json('path_to_service_account_key.json')
I know I'm late to the party but hopefully this helps!
Currently, it's not possible to use blob.generate_signed_url without explicitly referencing credentials. (Source: Google-Cloud-Python documentation) However, you can do a workaround, as seen here, which consists of:
signing_credentials = compute_engine.IDTokenCredentials(
auth_request,
"",
service_account_email=credentials.service_account_email
)
signed_url = signed_blob_path.generate_signed_url(
expires_at_ms,
credentials=signing_credentials,
version="v4"
)
A much complete snippet for those asking where other elements come from. cc #AlbertoVitoriano
from google.auth.transport import requests
from google.auth import default, compute_engine
credentials, _ = default()
# then within your abstraction
auth_request = requests.Request()
credentials.refresh(auth_request)
signing_credentials = compute_engine.IDTokenCredentials(
auth_request,
"",
service_account_email=credentials.service_account_email
)
signed_url = signed_blob_path.generate_signed_url(
expires_at_ms,
credentials=signing_credentials,
version="v4"
)
Anyone know if this is possible?
I just want to automate dropping some documents into my onedrive for business account.
I tried
import onedrivesdk
from onedrivesdk.helpers import GetAuthCodeServer
from onedrivesdk.helpers.resource_discovery import ResourceDiscoveryRequest
redirect_uri = 'http://localhost:8080'
client_id = 'appid'
client_secret = 'mysecret'
discovery_uri = 'https://api.office.com/discovery/'
auth_server_url='https://login.live.com/oauth20_authorize.srf?scope=wl.skydrive_update'
#auth_server_url='https://login.microsoftonline.com/common/oauth2/authorize',
auth_token_url='https://login.microsoftonline.com/common/oauth2/token'
http = onedrivesdk.HttpProvider()
auth = onedrivesdk.AuthProvider(http,
client_id,
auth_server_url=auth_server_url,
auth_token_url=auth_token_url)
auth_url = auth.get_auth_url(redirect_uri)
code = GetAuthCodeServer.get_auth_code(auth_url, redirect_uri)
auth.authenticate(code, redirect_uri, client_secret, resource=resource)
# If you have access to more than one service, you'll need to decide
# which ServiceInfo to use instead of just using the first one, as below.
service_info = ResourceDiscoveryRequest().get_service_info(auth.access_token)[0]
auth.redeem_refresh_token(service_info.service_resource_id)
client = onedrivesdk.OneDriveClient(service_info.service_resource_id + '/_api/v2.0/', auth, http)
I registered an APP and got a secret and id. But when I ran this I got scope is invalid errors. Plus it tries to launch a webpage which isn't great for a command line kinda environment. I think this SDK might be outdated as well because originally this script had login.microsoftonline, but that wasn't reachable so I changed it to login.live.com.
I wrote this sample code you posted. You replaced the auth_server_URLwith the authentication URL for Microsoft Account authentication, which can only be used to access OneDrive (the consumer product). You need to continue using the login.microsoftonline.com URL to log into your OneDrive for Business account.
You are correct that this pops up a dialog. However, you can write a little supporting code so that only happens the first time you log into a particular app. Follow these steps (assuming you are using the default implementation of AuthProvider:
Use the sample code above up through the line auth.redeem_refresh_token()
The AuthProvider will now have a Session object, which caches the credentials of the current user and session. Use AuthProvider.save_session() to save the credentials for later.
Next time you start your app, use AuthProvider.load_session() and AuthProvider.refresh_token() to retrieve the previous session and refresh the auth token. This will all be headless.
Take note that the default implementation of SessionBase (found here) uses Pickle and is not safe for product use. Make sure to create a new implementation of Session if you intend to deploy this app to other users.
Onerive's website shows "Not Yet" on "OneDrive SDK for Python" to "OneDrive for Business"
https://dev.onedrive.com/SDKs.htm
The github sample codes did not work for me either, it tried to popup a window of authentication, but IE can not find the address:
http://('https//login.microsoftonline.com/common/oauth2/authorize',)?redirect_uri=http%3A%2F%2Flocalhost%3A8080&client_id=034xxxx9-9xx8-4xxf-bexx-1bc5xxxxbd0c&response_type=code
or removed all the "-" in client id
http://('https//login.microsoftonline.com/common/oauth2/authorize',)?redirect_uri=http%3A%2F%2Flocalhost%3A8080&client_id=034xxxx99xx84xxfbexx1bc5xxxxbd0c&response_type=code
Either way, I got the same result, IE did not show the popup with a line "This page can’t be displayed"
I am using tkinter to create gui application that returns the security groups. Currently if you want to change your credentials (e.g. if you accidentally entered the wrong ones) you would have to restart the application otherwise boto3 would carry on using the old credentials.
I'm not sure why it keeps using the old credentials because I am running everything again using the currently entered credentials.
This is a snippet of the code that sets the environment variables and launches boto3. It works perfectly fine if you enter the right credentials the first time.
os.environ['AWS_ACCESS_KEY_ID'] = self.accessKey
os.environ['AWS_SECRET_ACCESS_KEY'] = self.secretKey
self.sts_client = boto3.client('sts')
self.assumedRoleObject = self.sts_client.assume_role(
RoleArn=self.role,
RoleSessionName="AssumeRoleSession1"
)
self.credentials = self.assumedRoleObject['Credentials']
self.ec2 = boto3.resource(
'ec2',
region_name=self.region,
aws_access_key_id=credentials['AccessKeyId'],
aws_secret_access_key=credentials['SecretAccessKey'],
aws_session_token=credentials['SessionToken'],
)
The credentials variables are set using:
self.accessKey = str(self.AWS_ACCESS_KEY_ID_Form.get())
self.secretKey = str(self.AWS_SECRET_ACCESS_KEY_Form.get())
self.role = str(self.AWS_ROLE_ARN_Form.get())
self.region = str(self.AWS_REGION_Form.get())
self.instanceID = str(self.AWS_INSTANCE_ID_Form.get())
Is there a way to use different credentials in boto3 without restarting the program?
You need boto3.session.Session to overwrite the access credentials.
Just do this
reference http://boto3.readthedocs.io/en/latest/reference/core/session.html
import boto3
# Assign you own access
mysession = boto3.session.Session(aws_access_key_id='foo1', aws_secret_access_key='bar1')
# If you want to use different profile call foobar inside .aws/credentials
mysession = boto3.session.Session(profile_name="fooboar")
# Afterwards, just declare your AWS client/resource services
sqs_resource=mysession.resource("sqs")
# or client
s3_client=mysession.client("s3")
Basically, little change to your code. you just pass in the session instead of direct boto3.client/boto3.resource
self.sts_client = mysession.client('sts')
Sure, just create different sessions from botocore.session.Session object for each set of credentials:
import boto3
s1 = boto3.session.Session(aws_access_key_id='foo1', aws_secret_access_key='bar1')
s2 = boto3.session.Session(aws_access_key_id='foo2', aws_secret_access_key='bar2')
Also you can leverage set_credentials method to keep 1 session an change creds on the fly:
import botocore
session - botocore.session.Session()
session.set_credentials('foo', 'bar')
client = session.create_client('s3')
client._request_signer._credentials.access_key
u'foo'
session.set_credentials('foo1', 'bar')
client = session.create_client('s3')
client._request_signer._credentials.access_key
u'foo1'
The answers given by #mootmoot and #Vor clearly state the way of dealing with multiple credentials using a session.
#Vor's answer
import boto3
s1 = boto3.session.Session(aws_access_key_id='foo1', aws_secret_access_key='bar1')
s2 = boto3.session.Session(aws_access_key_id='foo2', aws_secret_access_key='bar2')
But some of you would be curious about
why does the boto3 client or resource behave in that manner in the first place?
Let's clear out a few points about Session and Client as they'll actually lead us to the answer to the aforementioned question.
Session
A 'Session' stores configuration state and allows you to create service clients and resources
Client
if the credentials are not passed explicitly as arguments to the boto3.client method, then the credentials configured for the session will automatically be used. You only need to provide credentials as arguments if you want to override the credentials used for this specific client
Now let's get to the code and see what actually happens when you call boto3.client()
def client(*args, **kwargs):
return _get_default_session().client(*args, **kwargs)
def _get_default_session():
if DEFAULT_SESSION is None:
setup_default_session()
return DEFAULT_SESSION
def setup_default_session(**kwargs):
DEFAULT_SESSION = Session(**kwargs)
Learnings from the above
The function boto3.client() is really just a proxy for the boto3.Session.client() method
If you once use the client, the DEFAULT_SESSION is set up and for the next consecutive creation of clients it'll keep using the DEFAULT_SESSION
The credentials configured for the DEFAULT_SESSION are used if the credentials are not explicitly passed as arguments while creating the boto3 client.
Answer
The first call to boto3.client() sets up the DEFAULT_SESSION and configures the session with the oldCredsAccessKey, oldCredsSecretKey, the already set values for env variables AWS_ACCESS_KEY_ID and AWS_SECRET_ACESS_KEY respectively.
So even if you set new values of credentials in the environment, i.e do this
os.environ['AWS_ACCESS_KEY_ID'] = newCredsAccessKey
os.environ['AWS_SECRET_ACCESS_KEY'] = newCredsSecretKey
The upcoming boto3.client() calls still pick up the old credentials configured for the DEFAULT_SESSION
NOTE
boto3.client() call in this whole answer means that no arguments passed to the client method.
References
https://boto3.amazonaws.com/v1/documentation/api/latest/_modules/boto3.html#client
https://boto3.amazonaws.com/v1/documentation/api/latest/_modules/boto3/session.html#Session
https://ben11kehoe.medium.com/boto3-sessions-and-why-you-should-use-them-9b094eb5ca8e