i want to make an tornado proxy page.
So when I visit proxypage, I will get the content from google.com.
I made a small simple proxy page in php only to find out tornado webserver dont support php.
Here is t he php code to explain better what i'm looking for:
<?php
$file = file_get_contents($_GET['requrl']);
echo $file;
?>
so its nothing to extreme.
I am making ajax requests to a site that gives json response, no images nothing special just json. but ajax dont work with cross-domain requests (same-origin-policy). and the site dont support jsonp or cors, so a proxy page is the only way.
add a handler that looks something like:
(r"/", proxyHandler ),
and the handler class
class proxyHandler(RequestHandler):
def get(self):
connection = httplib.HTTPConnection( "www.google.com" )
connection.request( 'GET', '[uri]', '[body]', '[header]' )
response = connection.getresponse()
if response.status !=200:
print response.status, response.reason
data = response.read()
connection.close()
try:
dataJSON = json.loads(data)
if DEBUG_HTTP:
printDoc( dataJSON, 'http response json')
return dataJSON
except Exception, e:
print 'data is of type, '+str(type(data))+', returning as is'
return data
in connection.request() call, uri param is not optional ( can pass '' ), body and header are.
This could also be done in async fashion.
Related
I have been using this function to handle http requests with no problems:
def do_request(self, method, url, **kwargs):
params = kwargs.get('params', None)
headers = kwargs.get('headers', None)
payload = kwargs.get('data', None)
request_method = {'GET':requests.get, 'POST': requests.post, 'PUT': requests.put, 'DELETE': requests.delete}
request_url = url
req = request_method[method]
try:
res = req(request_url, headers=headers, params=params, data=json.dumps(payload))
except (requests.exceptions.ConnectionError, requests.exceptions.RequestException) as e:
data = {'has_error':True, 'error_message':e.message}
return data
try:
data = res.json()
data.update({'has_error':False, 'error_message':''})
except ValueError as e:
msg = "Cannot read response, %s" %(e.message)
data = {'has_error':True, 'error_message':msg}
if not res.ok:
msg = "Response not ok"
data.update({'has_error':True, 'error_message':msg})
if res.status_code >= 400:
msg = 'Error code: ' + str(res.status_code) + '\n' + data['errorCode']
data.update({'has_error':True, 'error_message': msg})
return data
When I have to do a DELETE request without body entity I have no problems but when I try to add one (when required by the server) I get an error message from the server telling that the body cannot be null as if no body has been sent. Any ideas why this might be happening? I'm using requests module and python 2.7.12. As far as I know data can be send in a DELETE request. Thanks!
There are problems with some clients and some servers when DELETE includes entity body: Is an entity body allowed for an HTTP DELETE request? for example & lots of search results.
Some servers (apparently) convert the DELETE into a POST, others simply perform the DELETE but drop the body. In your case, you've investigated that indeed, the body of a DELETE is dropped by the server & it has been suggested that you change the DELETE to POST.
Mmm... I can send a DELETE with body with Postman and works OK. But I cant get the same result with Requests 2.17.3
This is a issue related to Requests
I'm just beginning to learn about python/django. I know PHP, but I wanted to get to know about this framework. I'm trying to work with yelp's API. I'm trying to figure out what to do when someone brings in a new file into the project.
In their business.py they have this:
import json
import oauth2
import optparse
import urllib
import urllib2
parser = optparse.OptionParser()
parser.add_option('-c', '--consumer_key', dest='consumer_key', help='OAuth consumer key (REQUIRED)')
parser.add_option('-s', '--consumer_secret', dest='consumer_secret', help='OAuth consumer secret (REQUIRED)')
parser.add_option('-t', '--token', dest='token', help='OAuth token (REQUIRED)')
parser.add_option('-e', '--token_secret', dest='token_secret', help='OAuth token secret (REQUIRED)')
parser.add_option('-a', '--host', dest='host', help='Host', default='api.yelp.com')
parser.add_option('-i', '--id', dest='id', help='Business')
parser.add_option('-u', '--cc', dest='cc', help='Country code')
parser.add_option('-n', '--lang', dest='lang', help='Language code')
options, args = parser.parse_args()
# Required options
if not options.consumer_key:
parser.error('--consumer_key required')
if not options.consumer_secret:
parser.error('--consumer_secret required')
if not options.token:
parser.error('--token required')
if not options.token_secret:
parser.error('--token_secret required')
if not options.id:
parser.error('--id required')
url_params = {}
if options.cc:
url_params['cc'] = options.cc
if options.lang:
url_params['lang'] = options.lang
path = '/v2/business/%s' % (options.id,)
def request(host, path, url_params, consumer_key, consumer_secret, token, token_secret):
"""Returns response for API request."""
# Unsigned URL
encoded_params = ''
if url_params:
encoded_params = urllib.urlencode(url_params)
url = 'http://%s%s?%s' % (host, path, encoded_params)
print 'URL: %s' % (url,)
# Sign the URL
consumer = oauth2.Consumer(consumer_key, consumer_secret)
oauth_request = oauth2.Request('GET', url, {})
oauth_request.update({'oauth_nonce': oauth2.generate_nonce(),
'oauth_timestamp': oauth2.generate_timestamp(),
'oauth_token': token,
'oauth_consumer_key': consumer_key})
token = oauth2.Token(token, token_secret)
oauth_request.sign_request(oauth2.SignatureMethod_HMAC_SHA1(), consumer, token)
signed_url = oauth_request.to_url()
print 'Signed URL: %s\n' % (signed_url,)
# Connect
try:
conn = urllib2.urlopen(signed_url, None)
try:
response = json.loads(conn.read())
finally:
conn.close()
except urllib2.HTTPError, error:
response = json.loads(error.read())
return response
response = request(options.host, path, url_params, options.consumer_key, options.consumer_secret, options.token, options.token_secret)
print json.dumps(response, sort_keys=True, indent=2)
Its very lengthy, I appologize for that. But my concern is what do I do with this? They've set up a def request() in here, and I'm assuming that I have to import this into my views?
I've been following the django documentation of creating a new app. In the documentation they've set up a bunch of def inside the views.py file. I'm just confused as to how am I supposed to make this work with my project? If I wanted to search for a business in the URL, how would it send the data out?
Thanks for your help.
This is a command line script that makes http requests to the yelp api. You probably don't want to make such an external request within the context of a main request handler. Well, you could call a request handler that makes this call to yelp. Let's see ...
You could import the request function and instead of invoking it with command line options, call it yourself.
from yelp.business import request as yelp_req
def my_request_handler(request):
json_from_yelp = yelp_req(...
# do stuff and return a response
Making this kind of external call inside a request handler is pretty meh though (that is, making an http request to an external service within a request handler). If the call is in ajax, it may be ok for the ux.
This business.py is just an example showing you how to create a signed request with oauth2. You may be able to just import the request function and use it. OTOH, you may prefer to write your own (perhaps using the requests library). You probably want to use celery or some other async means to make the calls outside of your request handlers and/or cache the responses to avoid costly external http io with every request.
Let me start with I have successfully connected to my service end point. When I run my program, I get a 200 OK and a text response of XML request null. These are both good things! It's exactly what I get if I paste the URL of the service end point into a browser.
My question is, in my program I thought I was properly attaching my XML request payload to my POST. However with the response of XML request null. indeed I am not.
xml = '''xml request here '''
host = "www.xmlendpoint.com"
url = "/service/endpoint"
def doRequest():
conn = httplib.HTTPSConnection(host)
headers = {"Content-type": "text/xml","Content-Length": "%d" % len(xml)}
conn.request("POST", url, "", headers)
conn.send(xml)
res = conn.getresponse()
print res.status, res.reason
print res.read()
conn.close()
if __name__ == "__main__":
doRequest()
Anything glaring wrong here?
I have VMware setup for testing. I create one user abc/abc123 to access the Org url "http://localhost/cloud/org/MyOrg". I want to access the RestAPI of the VCloud. I tried with RestClient plugin in firefox. Its working fine.
Now I tried with python code.
url = 'https://localhost/api/sessions/'
req = urllib2.Request(url)
base64string = base64.encodestring('%s:%s' % ('abc#MyOrg', 'abc123'))[:-1]
authheader = "Basic %s" % base64string
req.add_header("Authorization", authheader)
req.add_header("Accept", 'application/*+xml;version=1.5')
f = urllib2.urlopen(req)
data = f.read()
print(data)
This is the code i get from stackoverflow. But for my example its give "urllib2.HTTPError: HTTP Error 403: Forbidden" Error.
I also tried HTTP authentication for the same.
After doing some googling I found the solution from the post https://stackoverflow.com/a/6348729/243031. I change the code for my usability. I am posting the answer because if some one has same error then he will get the answer directly.
My change code is:
import urllib2
import base64
# make a string with the request type in it:
method = "POST"
# create a handler. you can specify different handlers here (file uploads etc)
# but we go for the default
handler = urllib2.HTTPSHandler()
# create an openerdirector instance
opener = urllib2.build_opener(handler)
# build a request
url = 'https://localhost/api/sessions'
request = urllib2.Request(url)
# add any other information you want
base64string = base64.encodestring('%s:%s' % ('abc#MyOrg', 'abc123'))[:-1]
authheader = "Basic %s" % base64string
request.add_header("Authorization", authheader)
request.add_header("Accept",'application/*+xml;version=1.5')
# overload the get method function with a small anonymous function...
request.get_method = lambda: method
# try it; don't forget to catch the result
try:
connection = opener.open(request)
except urllib2.HTTPError,e:
connection = e
# check. Substitute with appropriate HTTP code.
if connection.code == 200:
data = connection.read()
print "Data :", data
else:
print "ERRROR", connection.code
Hope this will help some one who want to send POST request without the data.
I'm trying to use Tornado to start a server and post a string to it. I've found lots of examples of how to write the post method in the handler class, but no examples of how to write the post request. My current code does cause the post method to execute, but get_argument isn't getting the data--it just prints the default "No data received" every time. What am I doing wrong?
My code looks like this:
class MainHandler(tornado.web.RequestHandler):
def post(self):
data = self.get_argument('body', 'No data received')
self.write(data)
application = tornado.web.Application([
(r"/", MainHandler),
])
if __name__ == "__main__":
def handle_request(response):
if response.error:
print "Error:", response.error
else:
print response.body
tornado.ioloop.IOLoop.instance().stop()
application.listen(8888)
test = "test data"
http_client = tornado.httpclient.AsyncHTTPClient()
http_client.fetch("http://0.0.0.0:8888", handle_request, method='POST', headers=None, body=test)
tornado.ioloop.IOLoop.instance().start()
Is putting the string I want to send in the "body" parameter the right thing to do? In some examples I've seen, like here, it seems people create their own parameters, but if I try to add a new parameter to the request, like
http_client.fetch("http://0.0.0.0:8888", handle_request, method='POST', headers=None, data=test)
I just get an error saying "TypeError: init() got an unexpected keyword argument 'data'"
Thanks!
it seems people create their own parameters
Not quite. From the docs:
fetch(request, **kwargs)
Executes a request, returning an
HTTPResponse.
The request may be either a string URL or an HTTPRequest object. If it
is a string, we construct an HTTPRequest using any additional kwargs:
HTTPRequest(request, **kwargs)
(Link)
So the kwargs are actually from this method.
Anyways, to the real meat of the problem: How do you send POST data? You were on the right track, but you need to url encode your POST data and use that as your body kwarg. Like this:
import urllib
post_data = { 'data': 'test data' } #A dictionary of your post data
body = urllib.urlencode(post_data) #Make it into a post request
http_client.fetch("http://0.0.0.0:8888", handle_request, method='POST', headers=None, body=body) #Send it off!
Then to get the data:
data = self.get_argument('data', 'No data received')