I apologize if this has been answered elsewhere; I've looked around and haven't found a definitive answer.
I'd like to use Tornado to accept an HTTP request with querystring parameters, use those params to call a NOAA web service to fetch weather data, process/parse the NOAA response, then return the final data to the user.
I'm looking at Tornado because I can't count on the latency or availability of the web service request and I need the calls to be non-blocking. (otherwise I'd just use Django)
I also want to make sure I can set an appropriate timeout on the NOAA request so I can give up as necessary.
Note: I'm also open to using Twisted, though it seems to have a much steeper learning curve and my needs feel pretty simple. (I would do this in Node.js, but I'm much more comfortable handling the parsing requirements in Python)
Thanks in advance for anyone who can help point me in the right direction.
I will open-source the server process once finished and credit anyone who contributes examples or RTFM links to the appropriate documentation.
I've extracted code sample from my project. It's not perfect, but it gives an idea how to use Tornadp's AsyncHTTPClient
#tornado.gen.engine
def async_request(self, callback, server_url, method=u'GET', body=None, **kwargs):
"""
Make async request to server
:param callback: callback to pass results
:param server_url: path to required API
:param method: HTTP method to use, default - GET
:param body: HTTP request body for POST request, default - None
:return: None
"""
args = {}
if kwargs:
args.update(kwargs)
url = '%s?%s' % (server_url, urlencode(args))
request = tornado.httpclient.HTTPRequest(url, method, body=body)
http = tornado.httpclient.AsyncHTTPClient()
response = yield tornado.gen.Task(http.fetch, request)
if response.error:
logging.warning("Error response %s fetching %s", response.error, response.request.url)
callback(None)
return
data = tornado.escape.json_decode(response.body) if response else None
callback(data)
Related
I'm new to Locust, and I am attempting to log statistics for a POST request, and I'm using the following code along with a generic call to locust.
import json
from locust import HttpUser, task, between
import cfg
class BasicUser(HttpUser):
wait_time = between(1, 3)
v1_data = json.load(open("v1_sample_data.json", "r"))
#task
def get_v1_prediction(self):
route = "/" + cfg.lookup("model.v1.route")
response = self.client.post(
route,
json=self.v1_data,
catch_response=True,
name="API Call"
)
print(response.text)
When I start an experiment, the host is called successfully, and response.text has the expected value and is printed to the console repeatedly. However, the statistics aren't logged.
When I use a GET request in place of the POST without passing data, statistics are logged (though it's only failures because the web app only allows POST requests). Any idea what's going on here?
The catch_response=True is the culprit.
From the documentation:
catch_response – (optional) Boolean argument that, if set, can be used to make a request return a context manager to work as argument to a with statement. This will allow the request to be marked as a fail based on the content of the response, even if the response code is ok (2xx). The opposite also works, one can use catch_response to catch a request and then mark it as successful even if the response code was not (i.e 500 or 404).
I have been working on some code that will grab emergency incident information from a service called PulsePoint. It works with software built into computer controlled dispatch centers.
This is an app that empowers citizen heroes that are CPR trained to help before a first resonder arrives on scene. I'm merely using it to get other emergency incidents.
I reversed-engineered there app as they have no documentation on how to make your own requests. Because of this i have knowingly left in the api key and auth info because its in plain text in the Android manifest file.
I will definitely make a python module eventually for interfacing with this service, for now its just messy.
Anyhow, sorry for that long boring intro.
My real question is, how can i simplify this function so that it looks and runs a bit cleaner in making a timed request and returning a json object that can be used through subscripts?
import requests, time, json
def getjsonobject(agency):
startsecond = time.strftime("%S")
url = REDACTED
body = []
currentagency = requests.get(url=url, verify=False, stream=True, auth=requests.auth.HTTPBasicAuth(REDACTED, REDCATED), timeout = 13)
for chunk in currentagency.iter_content(1024):
body.append(chunk)
if(int(startsecond) + 5 < int(time.strftime("%S"))): #Shitty internet proof, with timeout above
raise Exception("Server sent to much data")
jsonstringforagency = str(b''.join(body))[2:][:-1] #Removes charecters that define the response body so that the next line doesnt error
currentagencyjson = json.loads(jsonstringforagency) #Loads response as decodable JSON
return currentagencyjson
currentincidents = getjsonobject("lafdw")
for inci in currentincidents["incidents"]["active"]:
print(inci["FullDisplayAddress"])
Requests handles acquiring the body data, checking for json, and parsing the json for you automatically, and since you're giving the timeout arg I don't think you need separate timeout handling. Request also handles constructing the URL for get requests, so you can put your query information into a dictionary, which is much nicer. Combining those changes and removing unused imports gives you this:
import requests
params = dict(both=1,
minimal=1,
apikey=REDACTED)
url = REDACTED
def getjsonobject(agency):
myParams = dict(params, agency=agency)
return requests.get(url, verify=False, params=myParams, stream=True,
auth=requests.auth.HTTPBasicAuth(REDACTED, REDACTED),
timeout = 13).json()
Which gives the same output for me.
I am sending post request in the body of some json data, to process on server and I want the results back to client(c++ app on phone) in the form of json data and hence parse on mobile.
I have the following code inside handler:
class ServerHandler(tornado.web.RequestHandler):
def post(self):
data = tornado.escape.json_decode(self.request.body)
id = data.get('id',None)
#process data from db (take a while) and pack in result which is dictinary
result = process_data(id)# returns dictionary from db= takes time
print 'END OF HANDLER'
print json.dumps(result)
#before this code below I have tried also
#return result
#return self.write(result)
#return self.write(json.dumps(result))
#return json.dumps(result)
self.set_header('Content-Type', 'application/json')
json_ = tornado.escape.json_encode(result)
self.write(json_)
self.finish()
#return json.dumps(result)
I always get printed 'END OF HANDLER' and valid dictinary/json below on console but when I read at client mobile I always get
<html><title>405: Method Not Allowed</title><body>405: Method Not Allowed</body></html>
Does anyone have any idea what is the bug ?
(I am using CIwGameHttpRequest for sending request and it works when file is static =>name.json but now same content is giving error in post request. )
The error (HTTP 405 Method Not Allowed) means that you have made a request to a valid URL, but you are using an HTTP verb (e.g. GET, POST, PUT, DELETE) that cannot be used with that URL.
Your web service code appears to handle the POST verb, as evidenced by the post method name, and also by the fact that incoming requests appear to have a request body. You haven't shown us your C++ client code, so all I can do is to speculate that it is making a GET request. Does your C++ code call Request->setPOST();? (I haven't worked with CIwGameHttpRequest before, but Googling for it I found this page from which I took that line of code.)
I've not worked with Tornado before, but I imagine that there is some mechanism somewhere that allows you to connect a URL to a RequestHandler. Given that you have a 405 Method Not Allowed error rather than 404 Not Found, it seems that however this is done you've done it correctly. You issue a GET request to Tornado for the URL, it determines that it should call your handler, and only when it tries to use your handler it realises that it can't handle GET requests, concludes that your handler (and hence its URL) doesn't support GETs and returns a 405 error.
I am trying to send data from a javascript app running in GTK webkit to Python via a HTTP request with the data sent in POST.
I can capture the request using resource-request-starting and checking the uri of the request.
I know the request works because I can send data through the request headers and view it with
def on_resource_request_starting(view, frame, resource, request, response):
uri = urllib.unquote(request.props.uri)
if uri.startswith('http://appname.local/'):
print request.get_message().request_headers.get_one('foobar')
But when I use print request.get_message().request_body.data I don't get anything.
How do I view the POST data?
I haven't used this API, but I believe you need to call the binding's equivalent of soup_message_body_flatten() before reading the data field. See the documentation for SoupMessageBody in the C API.
So, at a guess:
print request.get_message().request_body.flatten().data
Hooking to SoupSession "request-queued" signal and getting
buffer(s) using soup_message_body_get_chunk(soupmsgbody, num);
seems to work (in webkitgtk1 today, Jun 2015).
webkit_get_default_session() returns the SoupSession in question.
I am a newcomer to the Python and Twisted game so excuse the ignorance I will likely be asking this question with. As a sort of first program, I am trying to write a basic HTTP server using twisted.web.sever which would simply print to screen the HTTP request, and then print to screen the HTTP response. I am trying to print the entire message. Here is what I have so far:
from twisted.internet import reactor
from twisted.web.server import Site
from twisted.web.resource import Resource
import time
class TestPage(Resource):
isLeaf = True
def render_GET(self, request):
response = "Success"
print "You're request was %s" % request
print "The sever's response was %s" % response
return response
resource = TestPage()
factory = Site(resource)
reactor.listenTCP(8000, factory)
reactor.run()
So far, I am having success printing the request. What I want to know is where I can access the raw response data, not just the textual message. Also, if I wanted to start parsing the request/response for information, what would be the best way to go about doing that?
Edit: I'm also new to stackoverflow, how do I get this code to display properly?
Take a look at the Request and IRequest API docs to get an idea of what that request parameter offers you. You should be able to find just about everything in the request there.
I'm not sure what you mean by raw response data though. The response is up to you to generate.