I'm just beginning to learn about python/django. I know PHP, but I wanted to get to know about this framework. I'm trying to work with yelp's API. I'm trying to figure out what to do when someone brings in a new file into the project.
In their business.py they have this:
import json
import oauth2
import optparse
import urllib
import urllib2
parser = optparse.OptionParser()
parser.add_option('-c', '--consumer_key', dest='consumer_key', help='OAuth consumer key (REQUIRED)')
parser.add_option('-s', '--consumer_secret', dest='consumer_secret', help='OAuth consumer secret (REQUIRED)')
parser.add_option('-t', '--token', dest='token', help='OAuth token (REQUIRED)')
parser.add_option('-e', '--token_secret', dest='token_secret', help='OAuth token secret (REQUIRED)')
parser.add_option('-a', '--host', dest='host', help='Host', default='api.yelp.com')
parser.add_option('-i', '--id', dest='id', help='Business')
parser.add_option('-u', '--cc', dest='cc', help='Country code')
parser.add_option('-n', '--lang', dest='lang', help='Language code')
options, args = parser.parse_args()
# Required options
if not options.consumer_key:
parser.error('--consumer_key required')
if not options.consumer_secret:
parser.error('--consumer_secret required')
if not options.token:
parser.error('--token required')
if not options.token_secret:
parser.error('--token_secret required')
if not options.id:
parser.error('--id required')
url_params = {}
if options.cc:
url_params['cc'] = options.cc
if options.lang:
url_params['lang'] = options.lang
path = '/v2/business/%s' % (options.id,)
def request(host, path, url_params, consumer_key, consumer_secret, token, token_secret):
"""Returns response for API request."""
# Unsigned URL
encoded_params = ''
if url_params:
encoded_params = urllib.urlencode(url_params)
url = 'http://%s%s?%s' % (host, path, encoded_params)
print 'URL: %s' % (url,)
# Sign the URL
consumer = oauth2.Consumer(consumer_key, consumer_secret)
oauth_request = oauth2.Request('GET', url, {})
oauth_request.update({'oauth_nonce': oauth2.generate_nonce(),
'oauth_timestamp': oauth2.generate_timestamp(),
'oauth_token': token,
'oauth_consumer_key': consumer_key})
token = oauth2.Token(token, token_secret)
oauth_request.sign_request(oauth2.SignatureMethod_HMAC_SHA1(), consumer, token)
signed_url = oauth_request.to_url()
print 'Signed URL: %s\n' % (signed_url,)
# Connect
try:
conn = urllib2.urlopen(signed_url, None)
try:
response = json.loads(conn.read())
finally:
conn.close()
except urllib2.HTTPError, error:
response = json.loads(error.read())
return response
response = request(options.host, path, url_params, options.consumer_key, options.consumer_secret, options.token, options.token_secret)
print json.dumps(response, sort_keys=True, indent=2)
Its very lengthy, I appologize for that. But my concern is what do I do with this? They've set up a def request() in here, and I'm assuming that I have to import this into my views?
I've been following the django documentation of creating a new app. In the documentation they've set up a bunch of def inside the views.py file. I'm just confused as to how am I supposed to make this work with my project? If I wanted to search for a business in the URL, how would it send the data out?
Thanks for your help.
This is a command line script that makes http requests to the yelp api. You probably don't want to make such an external request within the context of a main request handler. Well, you could call a request handler that makes this call to yelp. Let's see ...
You could import the request function and instead of invoking it with command line options, call it yourself.
from yelp.business import request as yelp_req
def my_request_handler(request):
json_from_yelp = yelp_req(...
# do stuff and return a response
Making this kind of external call inside a request handler is pretty meh though (that is, making an http request to an external service within a request handler). If the call is in ajax, it may be ok for the ux.
This business.py is just an example showing you how to create a signed request with oauth2. You may be able to just import the request function and use it. OTOH, you may prefer to write your own (perhaps using the requests library). You probably want to use celery or some other async means to make the calls outside of your request handlers and/or cache the responses to avoid costly external http io with every request.
Related
I'm trying to work with a third party API and I am having problems with sending the request when using the requests or even urllib.request.
Somehow when I use http.client I am successful sending and receiving the response I need.
To make life easier for me, I created an API class below:
class API:
def get_response_data(self, response: http.client.HTTPResponse) -> dict:
"""Get the response data."""
response_body = response.read()
response_data = json.loads(response_body.decode("utf-8"))
return response_data
The way I use it is like this:
api = API()
rest_api_host = "api.app.com"
connection = http.client.HTTPSConnection(rest_api_host)
token = "my_third_party_token"
data = {
"token":token
}
payload = json.loads(data)
headers = {
# some headers
}
connection.request("POST", "/some/endpoint/", payload, headers)
response = connection.getresponse()
response_data = api.get_response_data(response) # I get a dictionary response
This workflow works for me. Now I just want to write a test for the get_response_data method.
How do I instantiate a http.client.HTTPResponse with the desired output to be tested?
For example:
from . import API
from unittest import TestCase
class APITestCase(TestCase):
"""API test case."""
def setUp(self) -> None:
super().setUp()
api = API()
def test_get_response_data_returns_expected_response_data(self) -> None:
"""get_response_data() method returns expected response data in http.client.HTTPResponse"""
expected_response_data = {"token": "a_secret_token"}
# I want to do something like this
response = http.client.HTTPResponse(expected_response_data)
self.assertEqual(api.get_response_data(response), expected_response_data)
How can I do this?
From the http.client docs it says:
class http.client.HTTPResponse(sock, debuglevel=0, method=None, url=None)
Class whose instances are returned upon successful connection. Not instantiated directly by user.
I tried looking at socket for the sock argument in the instantiation but honestly, I don't understand it.
I tried reading the docs in
https://docs.python.org/3/library/http.client.html#http.client.HTTPResponse
https://docs.python.org/3/library/socket.html
Searched the internet on "how to test http.client.HTTPResponse" but I haven't found the answer I was looking for.
I'm using Python on a Jupyter notebook for data analysis, and want access to a third party API (Mendeley) that uses OAuth. There used to be a work-around with a server on Heroku that produced a token manually, but that's been discontinued recently.
This must be an insanely common problem, but I can't find a maintained library that supports it. Most Python OAuth libraries are server-only; there's a well-supported JupyterHub-OAuthenticator, but IFAICS that is using OAuth for a different purpose.
ipyauth looks the business, but it's not been updated much and it's not documented how to extend it for new services. That situation usually means there's something better-supported available.
What is the currently-maintained Jupyter-Python-ThirdPartyAPI library, please?
Well, one answer turns out to be just to use the requests package, and to copy and paste the redirected URL each time:
from requests_oauthlib import OAuth2Session
scope = 'all'
redirect_uri='http://localhost:8888/'
oauth = OAuth2Session('YourApplicationApiIdNumber', redirect_uri=redirect_uri, scope=scope)
authorization_url, state = oauth.authorization_url(
"https://api.mendeley.com/oauth/authorize" )
print( 'Please go to %s to authorize access, and copy the final localhost URL' % authorization_url )
assert(False) # Stop processing until this is done.
... into a variable:
authorization_response = 'http://localhost:8888/tree?TheStuffWereInterestedIn'
... And then take it from there:
token = oauth.fetch_token(
'https://api.mendeley.com/oauth/token',
authorization_response=authorization_response.replace('http', 'https').replace(',',''),
client_secret='YourClientSecret')
r = oauth.get('https://api.mendeley.com/documents?sort=last_modified&order=desc&limit=500', timeout=30)
You have to configure the Mendeley application interface with the callback URL. This is http://localhost:8888/ , because that's something that Jupyter can display without losing the additional OAuth2 parameters. But Requests OAuth implementation doesn't accept non-https links (nor a trailing comma that Jupyter adds occasionally), so we fudge it as shown.
I guess this approach will work with pretty much any OAuth2 API. Certainly Requests lists quite a few.
Not sure if this helps or not, but I had to do an Oauth using PKCE Authorization flow with client_id, a registered callback url and no secret. And it had to run in Jupyter. The user is required to login into the authorization provider's website as part of the redirect (I think. I get mixed up with the terms because for me the Auth provider and the resource were from the same source). To do this I used the python HTTPServer and request handler to intercept the callback url. The Httpserver then hands over the relevant authorization code to my OAuth client when fetching the access token.
from http.server import BaseHTTPRequestHandler, HTTPServer
from urllib import parse
import random
import string
import hashlib
import base64
from typing import Any
import webbrowser
from authlib.integrations.requests_client import OAuth2Session
config = {
'scopes': ['openid', 'profile'],
'port' : 8888,
'redirect_url': 'http://localhost:8888/auth/callback',
'access_token_url': 'https://some_oauth_proivder.com/oauth2/access',
'auth_code_url': 'https://some_oauth_proivder.com/oauth2/authz/',
'client_id': 'providedByOauthprovider'}
def generate_code() -> tuple[str, str]:
rand = random.SystemRandom()
code_verifier = ''.join(rand.choices(string.ascii_letters + string.digits, k=128))
code_sha_256 = hashlib.sha256(code_verifier.encode('utf-8')).digest()
b64 = base64.urlsafe_b64encode(code_sha_256)
code_challenge = b64.decode('utf-8').replace('=', '')
return (code_verifier, code_challenge)
def login(config: dict[str, Any]) -> str:
class OAuthHttpServer(HTTPServer):
def __init__(self, *args, **kwargs):
HTTPServer.__init__(self, *args, **kwargs)
self.authorization_code = ""
class OAuthHttpHandler(BaseHTTPRequestHandler):
def do_GET(self):
self.send_response(200)
self.send_header("Content-Type", "text/html")
self.end_headers()
self.wfile.write("Redirecting to the My Auth provider login\n".encode("UTF-8"))
parsed = parse.urlparse(self.path)
client.fetch_token(url=config['access_token_url'],
authorization_response=self.path,
state=state,
code_verifier=code_verifier,
grant_type = 'authorization_code')
self.wfile.write("""
<html>
<body>
<h2>Authorization request to My Auth provider has been completed.</h1>
<h3>You may close this tab or window now.</h3>
</body>
</html>
""".encode("UTF-8"))
self.wfile.write('<script> setTimeout("window.close()", 2500);</script>'.encode("UTF-8")) #Timeout only works if already logged for some reason
with OAuthHttpServer(('', config["port"]), OAuthHttpHandler) as httpd:
client = OAuth2Session(client_id=config['client_id'],
scope=config['scopes'],
redirect_uri=config['redirect_url'],
code_challenge_method='S256')
code_verifier, code_challenge = generate_code()
auth_uri, state = client.create_authorization_url(config['auth_code_url'], code_verifier=code_verifier)
webbrowser.open_new(auth_uri)
httpd.handle_request()
clear_output()
print("Logged in successfully")
return client.token['access_token']
I borrowed a lot of this from a sample from this guy, but I had to swap out the Oauth client because it didn't work for me. And I had to rearrange the http request handler a little.
https://github.com/CamiloTerevinto/Blog/tree/main/Samples
This simple Falcon API will take a HTTP POST with enctype=multipart/form-data and a file upload in the file parameter and print the file's content on the console:
# simple_api.py
import cgi
import falcon
class SomeTestApi(object):
def on_post(self, req, resp):
upload = cgi.FieldStorage(fp=req.stream, environ=req.env)
upload = upload['file'].file.read()
print(upload)
app = falcon.API()
app.add_route('/', SomeTestApi())
One might also use the falcon-multipart middleware to achieve the same goal.
To try it out, run it e.g. with gunicorn (pip install gunicorn),
gunicorn simple_api.py
then use cUrl (or any REST client of choice) to upload a text file:
# sample.txt
this is some sample text
curl -F "file=#sample.txt" localhost:8000
I would like to test this API now with Falcon's testing helpers by simulating a file upload. However, I do not understand yet how to do this (if it is possible at all?). The simulate_request method has a file_wrapper parameter which might be useful but from the documentation I do not understand how this is supposed to be filled.
Any suggestions?
This is what I came up with, which tries to simulate what my Chrome does.
Note that this simulates the case when you are uploading only one file, but you can simply modify this function to upload multiple files, each one separated by two new lines.
def create_multipart(data, fieldname, filename, content_type):
"""
Basic emulation of a browser's multipart file upload
"""
boundry = '----WebKitFormBoundary' + random_string(16)
buff = io.BytesIO()
buff.write(b'--')
buff.write(boundry.encode())
buff.write(b'\r\n')
buff.write(('Content-Disposition: form-data; name="%s"; filename="%s"' % \
(fieldname, filename)).encode())
buff.write(b'\r\n')
buff.write(('Content-Type: %s' % content_type).encode())
buff.write(b'\r\n')
buff.write(b'\r\n')
buff.write(data)
buff.write(b'\r\n')
buff.write(boundry.encode())
buff.write(b'--\r\n')
headers = {'Content-Type': 'multipart/form-data; boundary=%s' %boundry}
headers['Content-Length'] = str(buff.tell())
return buff.getvalue(), headers
You can then use this function like the following:
with open('test/resources/foo.pdf', 'rb') as f:
foodata = f.read()
# Create the multipart data
data, headers = create_multipart(foodata, fieldname='uploadFile',
filename='foo.pdf',
content_type='application/pdf')
# Post to endpoint
client.simulate_request(method='POST', path=url,
headers=headers, body=data)
You can craft a suitable request body and Content-Type using the encode_multipart_formdata function in urllib3, documented here. An example usage:
from falcon import testing
import pytest
import myapp
import urllib3
# Depending on your testing strategy and how your application
# manages state, you may be able to broaden the fixture scope
# beyond the default 'function' scope used in this example.
#pytest.fixture()
def client():
# Assume the hypothetical `myapp` package has a function called
# `create()` to initialize and return a `falcon.App` instance.
return testing.TestClient(myapp.create())
# a dictionary mapping the HTML form label to the file uploads
fields = {
'file_1_form_label': ( # label in HTML form object
'file1.txt', # filename
open('path/to/file1.txt').read(), # file contents
'text/plain' # MIME type
),
'file_2_form_label': (
'file2.json',
open('path/to/file2.json').read(),
'application/json'
)
}
# create the body and header
body, content_type_header = urllib3.encode_multipart_formdata(fields)
# NOTE: modify these headers to reflect those generated by your browser
# and/or required by the falcon application you're testing
headers = {
'Content-Type': content_type_header,
}
# craft the mock query using the falcon testing framework
response = client.simulate_request(
method="POST",
path='/app_path',
headers=headers,
body=body)
print(response.status_code)
Note the syntax of the fields object, which is used as input for the encode_multipart_formdata function.
See Tim Head's blog post for another example:
https://betatim.github.io/posts/python-create-multipart-formdata/
Falcon testing example copied from their docs:
https://falcon.readthedocs.io/en/stable/api/testing.html
I am trying to authenticate user using FIWARE.
It returns a 404. Thus fails at Step 1 itself. What is the access token url ? Any other pointers to check
I have tried variations with 'oauth/access_token', 'oauth/token' 'oauth2/token' 'oauth2/access_token' . All of them dont seem to work.
My Code is Below:
import oauth2 as oauth
# OAuth secret into your project's settings.
consumer = oauth2.Consumer(settings.FIWARE_CLIENT_ID,settings.FIWARE_CLIENT_SECRET)
client = oauth2.Client(consumer)
access_token_url = 'https://account.lab.fiware.org/oauth2/access_token'
# This is the slightly different URL used to authenticate/authorize.
authenticate_url = 'https://account.lab.fiware.org/oauth2/authorize'
def fiware_login(request):
# Step 1. Get a request token from FIWARE.
resp, content = client.request(access_token_url, "GET")
print resp
if resp['status'] != '200':
print content
raise Exception("Invalid response from FIWARE.")
# Step 2. Redirect the user to the authentication URL.
url = "%s?access_token=%s" % (authenticate_url,
resp['access_token'])
return HttpResponseRedirect(url)
Correct endpoint is "/oauth2/token".
Maybe you should use POST method instead of GET.
For more information see https://github.com/ging/fi-ware-idm/wiki/Using-the-FI-LAB-instance
I'm trying to create an event with facebook from an external application. I've read this con events, where it states that you can create an event via POST, so i have the following-
data = dict()
data['access_token'] = self.access_token
data['name'] = 'Fb event from Python!'
data['start_time'] = datetime.datetime.now().isoformat()
data = urllib.urlencode(data)
url = 'https://graph.facebook.com/me/events'
request = urllib2.Request(url=url, data=data)
response = urllib2.urlopen(request)
Where I already have my access token and my fb permissions set so my app can create events and so forth. But I get an error 400 = Bad Request, so if anyone could help I'd be more than happy thanks in advance
Well I would say you should think about using facebook-sdk. From your error i.e http 400 you know that what you are doing wrong is how you are sending the http request itself. The request requires certain parameters in a certain form that you are not giving. If you look at
the request function in here
It makes the request in this form where path is id( I am also not sure me is valid here, but perhaps we can use me for the resource whose access-token we are using)
I think args access token .
file = urllib2.urlopen("https://graph.facebook.com/" + path + "?" +
urllib.urlencode(args),
post_data, timeout=self.timeout)
Thus change the form in which you are making the request. You can go through facebook.py's code and docs to figure out the variables mean based on your needs