I have a flask server up and running on pythonanywhere and I am trying to write a python script which I can run locally which will trigger a particular response - lets say the server time, for the sake of this discussion.
There is tonnes and tonnes of documentation on how to write the Flask server side of this process, but non/very little on how to write something which can trigger the Flask app to run.
I have tried sending XML in the form of a simple curl command e.g.
curl -X POST -d '<From>Jack</From><Body>Hello, it worked!</Body>' URL
But this doesnt seem to work (errors about referral headers).
Could someone let me know the correct way to compose some XML which can be sent to a listening flask server.
Thanks,
Jack
First, i would add -H "Content-Type: text/xml" to the headers in the cURL call so the server knows what to expect. It would be helpful if you posted the server code (not necessarily everything, but at least what's failing).
To debug this i would use
#app.before_request
def before_request():
if True:
print "HEADERS", request.headers
print "REQ_path", request.path
print "ARGS",request.args
print "DATA",request.data
print "FORM",request.form
It's a bit rough, but helps to see what's going on at each request. Turn it on and off using the if statement as needed while debugging.
Running your request without the xml header in the cURL call sends the data to the request.form dictionary. Adding the xml header definition results in the data appearing in request.data. Without knowing where your server fails, the above should give you at least a hint on how to proceed.
EDIT referring to comment below:
I would use the excellent xmltodict library. Use this to test:
import xmltodict
#app.before_request
def before_request():
print xmltodict.parse(request.data)['xml']['From']
with this cURL call:
curl -X POST -d '<xml><From>Jack</From><Body>Hello, it worked!</Body></xml>' localhost:5000 -H "Content-Type: text/xml"
'Jack' prints out without issues.
Note that this call has been modified from your question- the 'xml' tag has been added since XML requires a root node (it's called an xml tree for a reason..). Without this tag you'll get a parsing error from xmltodict (or any other parser you choose).
Related
Here is how I want my program to work. Step 2 is what I am unsure of how to implement.
Client makes API call to /email endpoint
/email endpoint has a script run that gather emails from GMAIL API
Put contents into response object
Returns response object back to client
I understand how to make a static api response. But I can't get a python script to run when the api endpoint is hit.
I saw the flask tag in your post.
I only played around with flask for certain interviews, but know enough to say calling a python script outside your running server is somewhat of an antipattern.
I assume your backend is a flask app, so ideally, you'd want to wrap whatever script you have in your python script file in a function and simply call it from your flask method when the endpoint is hit.
Something like:
from flask import Flask
from custom_emails_module import gather_email
#api.route('/email', methods=["GET"])
def method_associated_with_your_endpoint():
# additional
gather_email()
where custom_emails_module should be the module you create for your gather_emails script.
Now, at the end of your gather_emails function, simple remember to return the correct type, usually done with:
return json.dumps("success": True, "data": python_object_with_several_emails)
Use something like PostMan for local debugging and remember to use application/json in header for Content-Type.
Good luck!
I'm looking to push json packets to xively using this command:
jstr = '''{ "version":"1.0.0", "datastreams":[{"id":"one","current_value":"100.00"},{"id":"two","current_value":"022.02"},{"id":"three","current_value":"033.03"},{"id":"four","current_value":"044.04"}] }'''
and running it by calling this:
cmd = "curl --request PUT %s --header %s --verbose %s" % (jstr,apiKey,feedId)
I'm doing it this way so I can manipulate the JSON between transmissions (I change it to a dict and back).
It's throwing an error saying that there is no data sent. I'm new to curl, xively and python so it's really confusing me. Any help would be greatly appreciated please.
The best way to achieve this will be using official Python module provided by Xively.
Here are few reasons for not doing it the way you just described:
the official library provides a nice and simple API
you don't need to care what the data format actually is
calling curl command to make an HTTP request every time is absolutely inefficient as it take time for the OSto spawn and new process
I'm writing flask application that accepts POST requests with json data. I noticed huge differences in response time based on data size being passed to application. After debugging I narrowed down issue to the line where I was retrieving json data from request object. It may be important to note that testing was done on flask development server.
start = time.time()
resp = json.dumps(request.json)
return str(time.time() - start)
I timed this line and for data of 1024 (probably not coincidence) and less characters this took 0.002s and for anything over 1024 over 1 second!
What is happening here? Is this the limitation of development server?
EDIT:
Same thing happens for getting POST data through request.form.get('somedata') with content lenght over 1024
EDIT:
I couldn't replicate issue with same example served by Apache
EDIT:
I started digging into Werkzeug module and found that slowness occurs when reading response message self._read(to_read) in wsgi.py module which is passed from BaseHTTPRequestHandler. Still don't know why so slow.
Here's environment details:
Ubuntu - 10.04
Python - 2.6.5
Flask - 0.9
Werkzeug - 0.8.3
The flask development server is expected to be slow. From http://flask.pocoo.org/docs/deploying/:
You can use the builtin server during development, but you should use a full deployment option for production applications. (Do not use the builtin development server in production.)
As Marcus mentioned in the comments, another WSGI server like gunicorn or tornado would be much faster and more reliable, so definitely use one of those for deployment and benchmarking.
If you're worried about working quickly during development, you can use gunicorn in development just like you would in deployment. If you're deploying to heroku, for example, you can run "foreman start" and the gunicorn server will start right up.
I had this problem on a line like this, it was taking about 1.0 second! It's in a flask post handler:
username=request.form.get('username')
I was testing it with curl -F:
curl -F username="x" http://127.0.0.1:5000/func
I just changed -F to -d and it got 0.0004 seconds!!!
curl -d username="x" http://127.0.0.1:5000/func
I think flask has a problem to retrieving "multipart/form-data" content-type.
If you use curl to send a request, Expect: 100-continue might cause the behavior. I met a similar behavior with uwsgi, flask and curl. What happens in my case are the following:
If the request body size is larger than 1024 byte, curl posts data with Expect: 100-continue header.
However, uwsgi can't deal with the header. So the uwsgi doesn't respond 100-continue.
curl waits for the100-continue response until about one second time-out.
When curl sends 100-continue | Georg's Log was useful for me to know the curl behavior.
I just checked my webspace and it's signature says: Apache/2.2.9 (Debian) mod_python/3.3.1 Python/2.5.2 mod_ssl/2.2.9 OpenSSL/0.9.8g
This give me hope that Python is somehow supported. Why is python listed twice? mod_python/3.3.1 AND Python/2.5.2 ???
There is a cgi-bin folder on my webspace.
What I want to do: I need to do a cross-site call to get some text-data from a server. The text-data is not JSON but I guess I should convert it to JSON (or is there an option to do cross-site without JSON?)
The python script gets the request for some JSONP. Depending on the request (I guess I should somehow parse the URL) the python script is to load the a requested text-data file from the webserver and wrap it in some JSON and return it.
Can somebody tell me how I do these three steps with python on my webspace?
First off, the signature isn't listing python twice. Its listing first the version of mod_python, which is an Apache web server plugin, then it is listing the version of the python interpreter on the system.
python cgi module - This is really an inefficient approach to writing python server code, but here it is. Ultimately you should consider one of the many amazing python web frameworks out there. But, using the cgi module, your response would always start with this:
print 'Content-Type: application/json\n\n'
Your python script would run on the server from an HTTP request. In that script you would check the request and determine the data you will want to serve from either the URL value or the query string.
At the very least you would just wrap your return value in a basic JSON data structure. The text data itself can just be a string:
import json
text_data = "FOO"
json_data = json.dumps({'text': text_data})
print json_data
# {"text": "FOO"}
For the JSONP aspect, you would usually check the query string to see if the request contains a specific name for the callback function the client wants, or just default to 'callback'
print "callback(%s);" % json_data
# callback({"text": "FOO"});
Returning that would be a JSONP type response, because when the client receives it, the callback is executed for the client.
And to conclude, let me add that you should be aware that python cgi scripts will need to start a brand new python interpreter process for every single request (even repeat requests from the same client). This can easily overwhelm a server under increased load. For this reason, people usually go with the wsgi route (mod_wsgi in apache). wsgi allows a persistant application to keep running, and handles ongoing requests.
I'm trying to get a PUT request from Python into a servlet in Tomcat. The parameters are missing when I get into Tomcat.
The same code is happily working for POST requests, but not for PUT.
Here's the client:
lConnection = httplib.HTTPConnection('localhost:8080')
lHeaders = {"Content-type": "application/x-www-form-urlencoded",
"Accept": "text/plain"}
lParams = {'Username':'usr', 'Password':'password', 'Forenames':'First','Surname':'Last'}
lConnection.request("PUT", "/my/url/", urllib.urlencode(lParams), lHeaders)
Once in the server, a request.getParameter("Username") is returning null.
Has anyone got any clues as to where I'm losing the parameters?
I tried your code and it seems that the parameters get to the server using that code. Tcpdump gives:
PUT /my/url/ HTTP/1.1
Host: localhost
Accept-Encoding: identity
Content-Length: 59
Content-type: application/x-www-form-urlencoded
Accept: text/plain
Username=usr&Password=password&Surname=Last&Forenames=First
So the request gets to the other side correctly, it must be something with either tomcat configuration or the code that is trying to read the parameters.
I don't know what the Tomcat side of your code looks like, or how Tomcat processes and provides access to request parameters, but my guess is that Tomcat is not "automagically" parsing the body of your PUT request into nice request parameters for you.
I ran into the exact same problem using the built-in webapp framework (in Python) on App Engine. It did not parse the body of my PUT requests into request parameters available via self.request.get('param'), even though they were coming in as application/x-www-form-urlencoded.
You'll have to check on the Tomcat side to confirm this, though. You may end up having to access the body of the PUT request and parse out the parameters yourself.
Whether or not your web framework should be expected to automagically parse out application/x-www-form-urlencoded parameters in PUT requests (like it does with POST requests) is debatable.
I'm guessing here, but I think the problem is that PUT isn't meant to be used that way. The intent of PUT is to store a single entity, contained in the request, into the resource named in the headers. What's all this stuff about user name and stuff?
Your Content Type is application/X-www-form-urlencoded, which is a bunch of field contents. What PUT wants is something like an encoded file. You know, a single bunch of data it can store somewhere.