I currently have a monolithic Python script which performs an OAuth authentication, returning an OAuth1Session, and then proceeds to perform some business logic using that OAuth1Session to gain authorization to a third-party service.
I need to split this up into two separate scripts, one which performs the OAuth authentication and will run on one machine, and the other which will run on a remote machine to perform the business logic authorized against the third-party service.
How can I serialize the OAuth1Session object so that the authenticated tokens can be handed off seamlessly from the authentication script on machine A to the processing script on machine B?
I tried the obvious:
print(json.dumps(session))
But I got this error:
TypeError: Object of type OAuth1Session is not JSON serializable
Is there a canonical solution for this simple requirement?
UPDATE
Here's the entire source code. Please note this is not my code, I downloaded it from the author and now I'm trying to modify it to work a bit differently.
"""This Python script provides examples on using the E*TRADE API endpoints"""
from __future__ import print_function
import webbrowser
import json
import logging
import configparser
import sys
import requests
from rauth import OAuth1Service
def oauth():
"""Allows user authorization for the sample application with OAuth 1"""
etrade = OAuth1Service(
name="etrade",
consumer_key=config["DEFAULT"]["CONSUMER_KEY"],
consumer_secret=config["DEFAULT"]["CONSUMER_SECRET"],
request_token_url="https://api.etrade.com/oauth/request_token",
access_token_url="https://api.etrade.com/oauth/access_token",
authorize_url="https://us.etrade.com/e/t/etws/authorize?key={}&token={}",
base_url="https://api.etrade.com")
base_url = config["DEFAULT"]["PROD_BASE_URL"]
# Step 1: Get OAuth 1 request token and secret
request_token, request_token_secret = etrade.get_request_token(
params={"oauth_callback": "oob", "format": "json"})
# Step 2: Go through the authentication flow. Login to E*TRADE.
# After you login, the page will provide a text code to enter.
authorize_url = etrade.authorize_url.format(etrade.consumer_key, request_token)
webbrowser.open(authorize_url)
text_code = input("Please accept agreement and enter text code from browser: ")
# Step 3: Exchange the authorized request token for an authenticated OAuth 1 session
session = etrade.get_auth_session(request_token,
request_token_secret,
params={"oauth_verifier": text_code})
return(session, base_url)
# loading configuration file
config = configparser.ConfigParser()
config.read(sys.argv[1])
(session, base_url) = oauth()
print(base_url)
print(json.dumps(session))
#original code
#market = Market(session, base_url)
#quotes = market.quotes(sys.argv[2])
Please note the last two commented-out lines. That is the original code: Immediate after the oauth is performed, the code invokes some business functionality. I want to break this up into two separate scripts running as isolated processes: Script 1 performs the oauth and persists the session, Script 2 reads the session from a file and performs the business functionality.
Unfortunately it fails at the last line, print(json.dumps(session)).
"XY Problem" Alert
My goal is to split up the script into two so that the business logic can run in a separate machine from the authentication code. I believe that the way to do this is to serialize the session object and then parse it back in the second script. Printing out the session using json.dumps() is an intermediate step, 'Y', in my journey to solving problem 'X'. If you can think of a better way to achieve the goal, that could be a valid answer.
From the comments available in the source code here: https://github.com/litl/rauth/blob/a6d887d7737cf21ec896a8104f25c2754c694011/rauth/session.py
You only need to serialize some attributes of your object to reinstantiate it:
Line 103
def __init__(self,
consumer_key,
consumer_secret,
access_token=None,
access_token_secret=None,
signature=None,
service=None):
...
Thus I would suggest serializing the following dict on the first machine:
info_to_serialize = {
'consumer_key': session.consumer_key,
'consumer_secret': session.consumer_secret,
'access_token': session.access_token,
'access_token_secret': session.access_token_secret
}
serialized_data = json.dumps(info_to_serialize)
And on the second machine reinstantiate your session like that:
from rauth.session import OAuth1Session
info_deserialized = json.loads(serialized_data)
session = OAuth1Session(**info_deserialized)
Hope this helped
Related
First of all, I'm not a Python guru as you can probably tell... So here we go.
I'm trying to use Asana's API to pull data with Python requests (Projects, tasks, etc) and doing the authentication using Oauth 2.0... I've been trying to find a simple python script to have something to begin with but I haven't had any luck and I can't find a decent and simple example!
I already created the app and got my client_secret and client_secret. But I don't really know where or how to start... Could anybody help me please?
import sys, os, requests
sys.path.append(os.path.dirname(os.path.dirname(__file__)))
import asana
import json
from six import print_
import requests_oauthlib
from requests_oauthlib import OAuth2Session
client_id=os.environ['ASANA_CLIENT_ID'],
client_secret=os.environ['ASANA_CLIENT_SECRET'],
# this special redirect URI will prompt the user to copy/paste the code.
# useful for command line scripts and other non-web apps
redirect_uri='urn:ietf:wg:oauth:2.0:oob'
if 'ASANA_CLIENT_ID' in os.environ:
#Creates a client with previously obtained Oauth credentials#
client = asana.Client.oauth(
#Asana Client ID and Secret, set as a Windows environments to avoid hardcoding variables into the script#
client_id=os.environ['ASANA_CLIENT_ID'],
client_secret=os.environ['ASANA_CLIENT_SECRET'],
# this special redirect URI will prompt the user to copy/paste the code.
# useful for command line scripts and other non-web apps
redirect_uri='urn:ietf:wg:oauth:2.0:oob'
)
print ("authorized=", client.session.authorized)
# get an authorization URL:
(url, state) = client.session.authorization_url()
try:
# in a web app you'd redirect the user to this URL when they take action to
# login with Asana or connect their account to Asana
import webbrowser
webbrowser.open(url)
except Exception as e:
print_("Open the following URL in a browser to authorize:")
print_(url)
print_("Copy and paste the returned code from the browser and press enter:")
code = sys.stdin.readline().strip()
# exchange the code for a bearer token
token = client.session.fetch_token(code=code)
#print_("token=", json.dumps(token))
print_("authorized=", client.session.authorized)
me = client.users.me()
print "Hello " + me['name'] + "\n"
params = {'client_id' : client_id, 'redirect_uri' : redirect_uri, 'response_type' : token,}
print_("*************** Request begings *******************"+"\n")
print_("r = requests.get('https://app.asana.com/api/1.0/users/me)" + "\n")
r = requests.get('https://app.asana.com/api/1.0/users/me', params)
print_(r)
print_(r.json)
print_(r.encoding)
workspace_id = me['workspaces'][0]['id']
print_("My workspace ID is" + "\n")
print_(workspace_id)
print_(client.options)
I'm not sure how to use the requests lib with Asana. Their python doc did not help me. I'm trying to pull the available projects and their code colours so I can later plot them into a web browser (For a high-level view of the different projects and their respective colours - Green, yellow or red)
When I introduce the url (https://app.asana.com/api/1.0/users/me) into a browser, it gives me back a json response with the data, but when I try to do the same with the script, it gives me back a 401 (not authorized) response.
Does anybody know what I'm missing / doing wrong?
Thank you!!!
I believe the issue is that the Requests library is a lower level library. You would need to pass all of the parameters to your requests.
Is there a reason you are not exclusively using the Asana Python client library to make requests? All of the data you are looking to fetch from Asana (projects, tasks, etc.) are accessible using the Asana Python library. You will want to look in the library to find the methods you need. For example, the methods for the tasks resource can be found here. I think this approach will be easier (and less error-prone) than switching between the Asana lib and the Requests lib. The Asana lib is actually built on top of Requests (as seen here).
Anyone know if this is possible?
I just want to automate dropping some documents into my onedrive for business account.
I tried
import onedrivesdk
from onedrivesdk.helpers import GetAuthCodeServer
from onedrivesdk.helpers.resource_discovery import ResourceDiscoveryRequest
redirect_uri = 'http://localhost:8080'
client_id = 'appid'
client_secret = 'mysecret'
discovery_uri = 'https://api.office.com/discovery/'
auth_server_url='https://login.live.com/oauth20_authorize.srf?scope=wl.skydrive_update'
#auth_server_url='https://login.microsoftonline.com/common/oauth2/authorize',
auth_token_url='https://login.microsoftonline.com/common/oauth2/token'
http = onedrivesdk.HttpProvider()
auth = onedrivesdk.AuthProvider(http,
client_id,
auth_server_url=auth_server_url,
auth_token_url=auth_token_url)
auth_url = auth.get_auth_url(redirect_uri)
code = GetAuthCodeServer.get_auth_code(auth_url, redirect_uri)
auth.authenticate(code, redirect_uri, client_secret, resource=resource)
# If you have access to more than one service, you'll need to decide
# which ServiceInfo to use instead of just using the first one, as below.
service_info = ResourceDiscoveryRequest().get_service_info(auth.access_token)[0]
auth.redeem_refresh_token(service_info.service_resource_id)
client = onedrivesdk.OneDriveClient(service_info.service_resource_id + '/_api/v2.0/', auth, http)
I registered an APP and got a secret and id. But when I ran this I got scope is invalid errors. Plus it tries to launch a webpage which isn't great for a command line kinda environment. I think this SDK might be outdated as well because originally this script had login.microsoftonline, but that wasn't reachable so I changed it to login.live.com.
I wrote this sample code you posted. You replaced the auth_server_URLwith the authentication URL for Microsoft Account authentication, which can only be used to access OneDrive (the consumer product). You need to continue using the login.microsoftonline.com URL to log into your OneDrive for Business account.
You are correct that this pops up a dialog. However, you can write a little supporting code so that only happens the first time you log into a particular app. Follow these steps (assuming you are using the default implementation of AuthProvider:
Use the sample code above up through the line auth.redeem_refresh_token()
The AuthProvider will now have a Session object, which caches the credentials of the current user and session. Use AuthProvider.save_session() to save the credentials for later.
Next time you start your app, use AuthProvider.load_session() and AuthProvider.refresh_token() to retrieve the previous session and refresh the auth token. This will all be headless.
Take note that the default implementation of SessionBase (found here) uses Pickle and is not safe for product use. Make sure to create a new implementation of Session if you intend to deploy this app to other users.
Onerive's website shows "Not Yet" on "OneDrive SDK for Python" to "OneDrive for Business"
https://dev.onedrive.com/SDKs.htm
The github sample codes did not work for me either, it tried to popup a window of authentication, but IE can not find the address:
http://('https//login.microsoftonline.com/common/oauth2/authorize',)?redirect_uri=http%3A%2F%2Flocalhost%3A8080&client_id=034xxxx9-9xx8-4xxf-bexx-1bc5xxxxbd0c&response_type=code
or removed all the "-" in client id
http://('https//login.microsoftonline.com/common/oauth2/authorize',)?redirect_uri=http%3A%2F%2Flocalhost%3A8080&client_id=034xxxx99xx84xxfbexx1bc5xxxxbd0c&response_type=code
Either way, I got the same result, IE did not show the popup with a line "This page can’t be displayed"
I am using python social auth with django app. In email validation, the link received on the mail works fine in the same browser from which authentication was initiated but it show Missing needed parameter state in different browser.Did anybody fix this issue ?
The issue is discussed here Issue #577
This is because there's no partial pipeline data in other browser!
Christopher Keefer has worked on monkey-patch for this by fetching the session_key for Django session table. There's also a blog article here on this , refer Step 3 of this article.
# Monkey patching - an occasionally necessary evil.
from social import utils
from social.exceptions import InvalidEmail
from django.core import signing
from django.core.signing import BadSignature
from django.contrib.sessions.models import Session
from django.conf import settings
def partial_pipeline_data(backend, user=None, *args, **kwargs):
"""
Monkey-patch utils.partial_pipeline_data to enable us to retrieve session data by signature key in request.
This is necessary to allow users to follow a link in an email to validate their account from a different
browser than the one they were using to sign up for the account, or after they've closed/re-opened said
browser and potentially flushed their cookies. By adding the session key to a signed base64 encoded signature
on the email request, we can retrieve the necessary details from our Django session table.
We fetch only the needed details to complete the pipeline authorization process from the session, to prevent
nefarious use.
"""
data = backend.strategy.request_data()
if 'signature' in data:
try:
signed_details = signing.loads(data['signature'], key=settings.EMAIL_SECRET_KEY)
session = Session.objects.get(pk=signed_details['session_key'])
except BadSignature, Session.DoesNotExist:
raise InvalidEmail(backend)
session_details = session.get_decoded()
backend.strategy.session_set('email_validation_address', session_details['email_validation_address'])
backend.strategy.session_set('next', session_details.get('next'))
backend.strategy.session_set('partial_pipeline', session_details['partial_pipeline'])
backend.strategy.session_set(backend.name + '_state', session_details.get(backend.name + '_state'))
backend.strategy.session_set(backend.name + 'unauthorized_token_name',
session_details.get(backend.name + 'unauthorized_token_name'))
partial = backend.strategy.session_get('partial_pipeline', None)
if partial:
idx, backend_name, xargs, xkwargs = \
backend.strategy.partial_from_session(partial)
if backend_name == backend.name:
kwargs.setdefault('pipeline_index', idx)
if user: # don't update user if it's None
kwargs.setdefault('user', user)
kwargs.setdefault('request', backend.strategy.request_data())
xkwargs.update(kwargs)
return xargs, xkwargs
else:
backend.strategy.clean_partial_pipeline()
utils.partial_pipeline_data = partial_pipeline_data
This fixes the problem to much extent, still its not perfect. It will fail if session_key gets deleted/changed in the database. Django updates session_key each time the session data changes. So in case any other user logs in the same browser the session_key gets changed and user can't verify with the email link.
Omab has mentioned in the comment on issue,
I see the problem now, and even if I think that this could be solved with a re-write of the email validation pipeline, this affects all the pipeline functions that use the partial mechanism, so, I'm already working on a restructure of the pipeline serialization functionality that will improve this behavior. Basically the pipeline data will be dumped to a DB table and a hash code will be used to identify the processes which can be stopped and continue later, removing the dependency of the session.
Looking for update on this.
I'm new to Python, new to the jira-python library, and new to network programming, though I do have quite a bit of experience with application and integration programming and database queries (though it's been a while).
Using Python 2.7 and requests 1.0.3
I'm trying to use this library - http://jira-python.readthedocs.org/en/latest/ to query Jira 5.1 using Python. I successfully connected using an unauthenticated query, though I had to make a change to a line in client.py, changing
I changed
self._session = requests.session(verify=verify, hooks={'args': self._add_content_type})
to
self._session = requests.session()
I didn't know what I was doing exactly but before the change I got an error and after the change I got a successful list of project names returned.
Then I tried basic authentication so I can take advantage of my Jira permissions and do reporting. That failed initially too. And I made the same change to
def _create_http_basic_session
in client.py , but now I just get another error. So problem not solved. Now I get a different error:
HTTP Status 415 - Unsupported Media Type
type Status report
message Unsupported Media Type
description The server refused this request because the request entity is in
a format not` `supported by the requested resource for the requested method
(Unsupported Media Type).
So then I decided to do a super simple test just using the requests module, which I believe is being used by the jira-python module and this code seemed to log me in. I got a good response:
import requests
r = requests.get(the_url, auth=(my username , password))
print r.text
Any suggestions?
Here's how I use the jira module with authentication in a Python script:
from jira.client import JIRA
import logging
# Defines a function for connecting to Jira
def connect_jira(log, jira_server, jira_user, jira_password):
'''
Connect to JIRA. Return None on error
'''
try:
log.info("Connecting to JIRA: %s" % jira_server)
jira_options = {'server': jira_server}
jira = JIRA(options=jira_options, basic_auth=(jira_user, jira_password))
# ^--- Note the tuple
return jira
except Exception,e:
log.error("Failed to connect to JIRA: %s" % e)
return None
# create logger
log = logging.getLogger(__name__)
# NOTE: You put your login details in the function call connect_jira(..) below!
# create a connection object, jc
jc = connect_jira(log, "https://myjira.mydom.com", "myusername", "mypassword")
# print names of all projects
projects = jc.projects()
for v in projects:
print v
Below Python script connects to Jira and does basic authentication and lists all projects.
from jira.client import JIRA
options = {'server': 'Jira-URL'}
jira = JIRA(options, basic_auth=('username', 'password'))
projects = jira.projects()
for v in projects:
print v
It prints a list of all the project's available within your instance of Jira.
Problem:
As of June 2019, Atlassian Cloud users who are using a REST endpoint in Jira or Confluence Cloud with basic or cookie-based authentication will need to update their app or integration processes to use an API token, OAuth, or Atlassian Connect.
After June 5th, 2019 attempts to authenticate via basic auth with an Atlassian account password will return an invalid credentials error.
Reference: Deprecation of basic authentication with passwords for Jira and Confluence APIs
Solution to the Above-mentioned Problem:
You can use an API token to authenticate a script or other process with an Atlassian cloud product. You generate the token from your Atlassian account, then copy and paste it to the script.
If you use two-step verification to authenticate, your script will need to use a REST API token to authenticate.
Steps to Create an API Token from your Atlassian Account:
Log in to https://id.atlassian.com/manage/api-tokens
Click Create API token.
From the dialog that appears, enter a memorable and concise Label for your token and click Create.
Click Copy to clipboard, then paste the token to your script.
Reference: API tokens
Python 3.8 Code Reference
from jira.client import JIRA
jira_client = JIRA(options={'server': JIRA_URL}, basic_auth=(JIRA_USERNAME, JIRA_TOKEN))
issue = jira_client.issue('PLAT-8742')
print(issue.fields.summary)
Don't change the library, instead put your credentials inside the ~/.netrc file.
If you put them there you will also be able to test your calls using curl or wget.
I am not sure anymore about compatibility with Jira 5.x, only 7.x and 6.4 are currently tested. If you setup an instance for testing I could modify the integration tests to run against it, too.
My lucky guess is that you broke it with that change.
As of 2019 Atlassian has deprecated authorizing with passwords.
You can easily replace the password with an API Token created here.
Here's a minimalistic example:
pip install jira
from jira import JIRA
jira = JIRA("YOUR-JIRA-URL", basic_auth=("YOUR-EMAIL", "YOUR-API-TOKEN"))
issue = jira.issue("YOUR-ISSUE-KEY (e.g. ABC-13)")
print(issue.fields.summary)
I recommend storing your API Token as an environment variable and accessing it with os.environ[key].
I am trying to put together a bash or python script to play with the facebook graph API. Using the API looks simple, but I'm having trouble setting up curl in my bash script to call authorize and access_token. Does anyone have a working example?
Update 2018-08-23
Since this still gets some views and upvotes I just want to mention that by now there seems to exist a maintained 3rd party SDK: https://github.com/mobolic/facebook-sdk
Better late than never, maybe others searching for that will find it. I got it working with Python 2.6 on a MacBook.
This requires you to have
the Python facebook module installed: https://github.com/pythonforfacebook/facebook-sdk,
an actual Facebook app set up
and the profile you want to post to must have granted proper permissions to allow all the different stuff like reading and writing.
You can read about the authentication stuff in the Facebook developer documentation. See https://developers.facebook.com/docs/authentication/ for details.
This blog post might also help with this: http://blog.theunical.com/facebook-integration/5-steps-to-publish-on-a-facebook-wall-using-php/
Here goes:
#!/usr/bin/python
# coding: utf-8
import facebook
import urllib
import urlparse
import subprocess
import warnings
# Hide deprecation warnings. The facebook module isn't that up-to-date (facebook.GraphAPIError).
warnings.filterwarnings('ignore', category=DeprecationWarning)
# Parameters of your app and the id of the profile you want to mess with.
FACEBOOK_APP_ID = 'XXXXXXXXXXXXXXX'
FACEBOOK_APP_SECRET = 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'
FACEBOOK_PROFILE_ID = 'XXXXXX'
# Trying to get an access token. Very awkward.
oauth_args = dict(client_id = FACEBOOK_APP_ID,
client_secret = FACEBOOK_APP_SECRET,
grant_type = 'client_credentials')
oauth_curl_cmd = ['curl',
'https://graph.facebook.com/oauth/access_token?' + urllib.urlencode(oauth_args)]
oauth_response = subprocess.Popen(oauth_curl_cmd,
stdout = subprocess.PIPE,
stderr = subprocess.PIPE).communicate()[0]
try:
oauth_access_token = urlparse.parse_qs(str(oauth_response))['access_token'][0]
except KeyError:
print('Unable to grab an access token!')
exit()
facebook_graph = facebook.GraphAPI(oauth_access_token)
# Try to post something on the wall.
try:
fb_response = facebook_graph.put_wall_post('Hello from Python', \
profile_id = FACEBOOK_PROFILE_ID)
print fb_response
except facebook.GraphAPIError as e:
print 'Something went wrong:', e.type, e.message
Error checking on getting the token might be better but you get the idea of what to do.
Here you go, as simple as it can get. Doesn’t require any 3rd-party SDK etc.
Make sure Python 'requests' module is installed
import requests
def get_fb_token(app_id, app_secret):
url = 'https://graph.facebook.com/oauth/access_token'
payload = {
'grant_type': 'client_credentials',
'client_id': app_id,
'client_secret': app_secret
}
response = requests.post(url, params=payload)
return response.json()['access_token']
Easy! Just use facebook-sdk.
import facebook
app_id = 'YOUR_APP_ID'
app_secret = 'YOUR_APP_SECRET'
graph = facebook.GraphAPI()
# exactly what you're after ;-)
access_token = graph.get_app_access_token(app_id, app_secret)
You first need to set up an application. The following will then spit out an access token given your application ID and secret:
> curl -F type=client_cred -F client_id=[...] -F client_secret=[...] https://graph.facebook.com/oauth/access_token
Since a web browser needs to be involved for the actual authorization, there is no such thing as a "standalone script" that does it all. If you're just playing with the API, or are writing a script to automate something yourself, and want a access_token for yourself that does not expire, you can grab one here: http://fbrell.com/auth/offline-access-token
There IS a way to do it, I've found it, but it's a lot of work and will require you to spoof a browser 100% (and you'll likely be breaking their terms of service)
Sorry I can't provide all the details, but the gist of it:
assuming you have a username/password for a facebook account, go curl for the oauth/authenticate... page. Extract any cookies returned in the "Set-Cookie" header and then follow any "Location" headers (compiling cookies along the way).
scrape the login form, preserving all fields, and submit it (setting the referer and content-type headers, and inserting your email/pass) same cookie collection from (1) required
same as (2) but now you're going to need to POST the approval form acquired after (2) was submitted, set the Referer header with thr URL where the form was acquired.
follow the redirects until it sends you back to your site, and get the "code" parameter out of that URL
Exchange the code for an access_token at the oauth endpoint
The main gotchas are cookie management and redirects. Basically, you MUST mimic a browser 100%. I think it's hackery but there is a way, it's just really hard!
s29 has the correct answer but leaves some steps to solve. The following script demonstrates a working script for acquiring an access token using the Facebook SDK:
__requires__ = ['facebook-sdk']
import os
import facebook
def get_app_access_token():
client = facebook.GraphAPI()
return client.get_app_access_token(
os.environ['FACEBOOK_APP_ID'],
os.environ['FACEBOOK_APP_SECRET'],
)
__name__ == '__main__' and print(get_app_access_token())
This script expects the FACEBOOK_APP_ID and FACEBOOK_APP_SECRET environment variables are set to the values for your app. Feel free to adapt that technique to load those values from a different source.
You must first install the Facebook SDK (pip install facebook-sdk; python get-token.py) or use another tool like rwt to invoke the script (rwt -- get-token.py).
Here is the Python Code. Try running some of these examples on command line, they work fine for me. See also — http://www.pythonforfacebook.com/