We Keep Coding

Handle post request in flask and bottle

I've been trying to build an API for more than 4 hours now and I searched and asked everywhere I could but I can't find help. The problem is at the level of handling POST requests. I tried with NodeJS (testify and express (as well as middlewares)) and Python (Flask, bottle) and I still can't get why I get an empty object with express or None in python. I have the following code with bottle
1 from bottle import run, Bottle, post, request, redirect
2 import json
3
4 pika = Bottle()
5
6 #post("/shorturl")
7 def shorten():
8 data = request.json
9 #data = json.dumps(rdata)
10 print(data)
11 return f"You posted {data}"
12
13 run(host="localhost", port=3000, debug=True)
And I had the following code at the beginning (I deleted and restarted from scratch) - you can find tweet here.
I can't get None with flask and bottle when using request.get_json() and request.json() respectively which I've found are the way to do it from the docs.
Any help appreciated.
Thanks

Your python code seems to be correct, because I tried and it worked.
For troubleshooting you can insert:
print(request.headers['Content-Type']) #output should be: application/json'
print(request.body.read()) #output should be like: b'{"key":"value"}'
I used Postman and when I tried the first time, I made the mistake to select body form instead of raw. In Postman you have to write Content-Type:appliction/json in the header manually and insert the json as raw.
I assume in Restler it's similar (I never used Restler).
So if you have to configure it manually, make sure that your header contains 'Content-Type':'application/json'
And your body is set to raw and look like this.
If for example form-data was selected, the manually set header would not be used by postman and print(request.header['Content-Type']) would output something like this: multipart/form-data; boundary=--------------------------742283902180513961059188
Could imagine Restler has the same snare.

Here is a way to handle a dynamic routing of the api. Now you just have to add methods to the API class and they are automatically picked up by the bottle app. I am merging POST and GET into one method string to merge query parameters and forms into one payload which you can access via self.payload
import ujson as json
from login import User
def check(data):
try:
if isinstance(data, (str,bytes)):
return json.loads(data)
except:
return data
return data
def merge_dicts(*args):
result = {}
for dictionary in args:
result.update(dictionary or {})
return result
class API:
def __init__(self, payload, user):
self.payload = payload
self.option = ''
self.request = None
#classmethod
def bot(cls, request, option, user):
payload = merge_dicts(dict(request.forms), dict(request.query.decode())) # merge query and form inputs
slf = cls(payload, user)
slf.request = request
slf.option = str(option)
return slf
def test(self): # localhost/api/test
return self.payload
#get('/api/<command>')
#post('/api/<command>')
#get('/api/<command>/<option>')
#post('/api/<command>/<option>')
def routeapi(command='', option=''):
user = User()
wapi = API.bot(request, option, user)
func = getattr(wapi, f"{command}", None)
if callable(func):
result = func()
if result:
if request.method == 'GET':
response.headers['Content-Type'] = 'application/json'
response.headers['Cache-Control'] = 'no-cache'
return {command:json.check(result)}
else:
return {command:None}

How to return plain text from flask endpoint? Needed by Prometheus

I need to setup a /metrics endpoint so that Prometheus can consume statistics about an endpoint. How do I go about doing this?
I have this in my Flask app:
#app.route('/metrics')
def metrics():
def generateMetrics():
metrics = ""
... some string builder logic
return metrics
response = make_response(generateMetrics(), 200)
response.mimetype = "text/plain"
return response
Is this the best way? What is the difference between returning a String (just returning metrics here) and returning plain text? Why do I need the mimetype?

Is this the best way?
There are several ways to set the MIME type, better explained and discussed in this StackOverflow question. Your way works fine and gets the job done.
What is the difference between returning a String and returning plain text?
If you return a string Flask will automatically handle some of the Response logic for you. This means using some default values. If you set up two different endpoints you'll see that the difference turns out to be that your code returns the following header:
Content-Type:"text/plain; charset=utf-8"
While just returning a string (and default MIME type) would return the following header:
Content-Type:"text/html; charset=utf-8"
Why do I need the mimetype?
You might say that it is technically more correct, given that your response is simply plain text, and not HTML. However, a more forcing reason for needing it would be that a third party system you are using (Prometheus) relies on or cares about the contents of the "Content-Type" header. If they do, then you must set it for them to accept it.
Example code
For the Content-Type header demonstration I used the following example Python code:
from flask import Flask, make_response
app = Flask(__name__)
def generateMetrics():
return "hello world"
#app.route('/metrics')
def metrics():
response = make_response(generateMetrics(), 200)
response.mimetype = "text/plain"
return response
#app.route('/metrics2')
def metrics2():
return generateMetrics()
I then viewed the returned body and headers using Postman.

Flask request and application/json content type

I have a flask app with the following view:
#menus.route('/', methods=["PUT", "POST"])
def new():
return jsonify(request.json)
However, this only works if the request's content type is set to application/json, otherwise the dict request.json is None.
I know that request.data has the request body as a string, but I don't want to be parsing it to a dict everytime a client forgets to set the request's content-type.
Is there a way to assume that every incoming request's content-type is application/json? All I want is to always have access to a valid request.json dict, even if the client forgets to set the application content-type to json.

Use request.get_json() and set force to True:
#menus.route('/', methods=["PUT", "POST"])
def new():
return jsonify(request.get_json(force=True))
From the documentation:
By default this function will only load the json data if the mimetype is application/json but this can be overridden by the force parameter.
Parameters:
force – if set to True the mimetype is ignored.
For older Flask versions, < 0.10, if you want to be forgiving and allow for JSON, always, you can do the decode yourself, explicitly:
from flask import json
#menus.route('/', methods=["PUT", "POST"])
def new():
return jsonify(json.loads(request.data))

the request object already has a method get_json which can give you the json regardless of the content-type if you execute it with force=True so your code would be something like the following:
#menus.route('/', methods=["PUT", "POST"])
def new():
return jsonify(request.get_json(force=True))
in fact, the flask documentation says that request.get_json should be used instead of request.json: http://flask.pocoo.org/docs/api/?highlight=json#flask.Request.json

How do you access the query string in Flask routes?

How do you access query parameters or the query string in Flask routes? It's not obvious from the Flask documentation.
The example route /data below illustrates the context that I would like to access that data. If someone requests something like example.com/data?abc=123, I would like access to the string ?abc=123 or to be able to retrieve the value of parameters like abc.
#app.route("/data")
def data():
# query_string = ???
return render_template("data.html")

from flask import request
#app.route('/data')
def data():
# here we want to get the value of user (i.e. ?user=some-value)
user = request.args.get('user')

The full URL is available as request.url, and the query string is available as request.query_string.decode().
Here's an example:
from flask import request
#app.route('/adhoc_test/')
def adhoc_test():
return request.query_string
To access an individual known param passed in the query string, you can use request.args.get('param'). This is the "right" way to do it, as far as I know.
ETA: Before you go further, you should ask yourself why you want the query string. I've never had to pull in the raw string - Flask has mechanisms for accessing it in an abstracted way. You should use those unless you have a compelling reason not to.

I came here looking for the query string, not how to get values from the query string.
request.query_string returns the URL parameters as raw byte string (Ref 1).
Example of using request.query_string:
from flask import Flask, request
app = Flask(__name__)
#app.route('/data', methods=['GET'])
def get_query_string():
return request.query_string
if __name__ == '__main__':
app.run(debug=True)
Output:
References:
Official API documentation on query_string

We can do this by using request.query_string.
Example:
Lets consider view.py
from my_script import get_url_params
#app.route('/web_url/', methods=('get', 'post'))
def get_url_params_index():
return Response(get_url_params())
You also make it more modular by using Flask Blueprints - https://flask.palletsprojects.com/en/1.1.x/blueprints/
Lets consider first name is being passed as a part of query string
/web_url/?first_name=john
## here is my_script.py
## import required flask packages
from flask import request
def get_url_params():
## you might further need to format the URL params through escape.
firstName = request.args.get('first_name')
return firstName
As you see this is just a small example - you can fetch multiple values + formate those and use it or pass it onto the template file.

Werkzeug/Flask as already parsed everything for you. No need to do the same work again with urlparse:
from flask import request
#app.route('/')
#app.route('/data')
def data():
query_string = request.query_string ## There is it
return render_template("data.html")
The full documentation for the request and response objects is in Werkzeug: http://werkzeug.pocoo.org/docs/wrappers/

Try like this for query string:
from flask import Flask, request
app = Flask(__name__)
#app.route('/parameters', methods=['GET'])
def query_strings():
args1 = request.args['args1']
args2 = request.args['args2']
args3 = request.args['args3']
return '''<h1>The Query String are...{}:{}:{}</h1>''' .format(args1,args2,args3)
if __name__ == '__main__':
app.run(debug=True)
Output:

Every form of the query string retrievable from flask request object as described in O'Reilly Flask Web Devleopment:
From O'Reilly Flask Web Development, and as stated by Manan Gouhari earlier, first you need to import request:
from flask import request
request is an object exposed by Flask as a context variable named (you guessed it) request. As its name suggests, it contains all the information that the client included in the HTTP request. This object has many attributes and methods that you can retrieve and call, respectively.
You have quite a few request attributes which contain the query string from which to choose. Here I will list every attribute that contains in any way the query string, as well as a description from the O'Reilly book of that attribute.
First there is args which is "a dictionary with all the arguments passed in the query string of the URL." So if you want the query string parsed into a dictionary, you'd do something like this:
from flask import request
#app.route('/'):
queryStringDict = request.args
(As others have pointed out, you can also use .get('<arg_name>') to get a specific value from the dictionary)
Then, there is the form attribute, which does not contain the query string, but which is included in part of another attribute that does include the query string which I will list momentarily. First, though, form is "A dictionary with all the form fields submitted with the request." I say that to say this: there is another dictionary attribute available in the flask request object called values. values is "A dictionary that combines the values in form and args." Retrieving that would look something like this:
from flask import request
#app.route('/'):
formFieldsAndQueryStringDict = request.values
(Again, use .get('<arg_name>') to get a specific item out of the dictionary)
Another option is query_string which is "The query string portion of the URL, as a raw binary value." Example of that:
from flask import request
#app.route('/'):
queryStringRaw = request.query_string
Then as an added bonus there is full_path which is "The path and query string portions of the URL." Por ejemplo:
from flask import request
#app.route('/'):
pathWithQueryString = request.full_path
And finally, url, "The complete URL requested by the client" (which includes the query string):
from flask import request
#app.route('/'):
pathWithQueryString = request.url
Happy hacking :)

I prefer
user = request.args['user'] if 'user' in request.args else 'guest'
over
user = request.args.get('user')
this way, you can check the url actually contains the query string first

The implementation below worked for me.
from flask import request
def getVerificationStatus():
try:
requestId=int(request.args.get('requestId'))
print(requestId)
status= verificationStepRepository.getVerificationStatus(requestId)
return tb.responsify(200, "success", status)
except Exception as e:
return errorHandler.dispatchInternalServerError(str(e))

Often we just want to map the whole query string into an appropriate python data structure and take it from there. The appropriate structure is a multi-dictionary because keywords can repeat, for example we need to handle A=123&A=456&B=789. A multi-dictionary is a list of 2-tuples where each 2-tuple contains the key as its first item and the list of values as its second, so the above goes to [('A',['123','456']),('B',['789'])]. All of this is achieved by
qstr = request.args.lists() # A generator for the multi-dict
qstr = list(qstr) # To get the actual multi-dict
If all you want is a dictionary where the first occurrence of a duplicate keyword is used you can just go
qstr = request.args.to_dict()

This can be done using request.args.get().
For example if your query string has a field date, it can be accessed using
date = request.args.get('date')
Don't forget to add "request" to list of imports from flask,
i.e.
from flask import request

If the request if GET and we passed some query parameters then,
fro`enter code here`m flask import request
#app.route('/')
#app.route('/data')
def data():
if request.method == 'GET':
# Get the parameters by key
arg1 = request.args.get('arg1')
arg2 = request.args.get('arg2')
# Generate the query string
query_string="?arg1={0}&arg2={1}".format(arg1, arg2)
return render_template("data.html", query_string=query_string)

This Code worked for me:
from flask import Flask, request
app = Flask(__name__)
#app.route('/')
def search():
query = request.args
for key,value in query.items():
print(key,value)
return "Hello World"
if __name__ == '__main__':
app.run(debug=True)

Unit testing Django JSON View

I'm trying to write some unit tests for some Django json_view views and I'm having trouble passing the json_string to the view. I posted a related question yesterday about passing a json string to a Django view from the JS, the issue was that in my JS I was just passing the json string where I needed to be passing the string as the attribute of an object, because I was failing to do this the string was being taken as the key for the resulting query dict. I'm having a similar problem again except that this time it is form a Django unit test to the Django View. Here is a simplified version of my code which produces the same result.
class MyTestCase(TestCase):
def setUp(self):
self.u = User.objects.create_user('test','test','test')
self.u.is_active = True
self.u.save()
self.client.login(username='test',password='test')
def test_create_object_from_form(self):
"""Test the creation of the Instance from the form data."""
import json
json_string json.dumps({'resource':{'type':'book','author':'John Doe'}})
print(json_string)
response = self.client.post(reverse('ajax_view'),
{'form':json_string},'json')
self.assetNotContains(response,'error')
and the view looks like this
#json_view
def ajax_view(request):
"""Process the incoming form data."""
if request.method == 'POST':
print(request.POST)
form_data = json.loads(request.POST['form'])
resource_data = form_data['resource']
form = MyUserForm(resource_data)
if form.is_valid():
...
Here is what the two print statements produce when the test is run. The json_string is
{"resource": {"type": "book", "author": "John Doe"}}
and the query dict looks like
<QueryDict: {u'{\'form\': \'{"resource": {"type": "book", "author": "John Doe"}}\'}': [u'']}>
I'm total newbie with JS and ajax, so don't worry about hurting my pride, the answer is probably so close it could jump up and bite me.

Final edit
I originally stated that header HTTP_X_REQUESTED_WITH='XMLHttpRequest' was necessary in the post call but this is currently false while in tests. This header is necessary for the csrf middleware but csrf is disabled in tests. However, I still believe it is a good practice to put in test even if middleware disables csrf since most javascript library already pass this header by default when doing ajax. Also, if another piece of code that is not disabled ever use the is_ajax method, you won't need to debug your unittest for hours to figure out that the header was missing.
The problem is with the content-type because when django gets a value in there that is different than text/html, it doesn't use the default post data handling which is to format your data like in a query: type=book&author=JohnDoe for example.
Then the fixed code is:
response = self.client.post(reverse('ajax_view'),
{'form':json_string},
HTTP_X_REQUESTED_WITH='XMLHttpRequest')
Here's how I'm using it myself:
post_data = {
"jsonrpc" : "2.0", "method": method, "params" : params, "id" : id }
return client.post('/api/json/',
json.dumps(post_data), "text/json",
HTTP_X_REQUESTED_WITH='XMLHttpRequest')
to do some json-rpc. Notice that since I pass a different content-type than the default value, my data is passed as is in the post request.

Thank you to #Eric_Fortin for turning me on to the header, it does not however resolve my issue with the malformed query dictionary using 'client.post'. Once I made the change from POST to GET with the XMLHttpRequest header my query dictionary straitened itself out. Here is the current solution:
response = self.client.get(reverse('ajax_view'),
{'form':json_string},'json',
HTTP_X_REQUESTED_WITH='XMLHttpRequest')
this is only a partial answer since this request is going to change data on the server and should be POST not a GET.
Edit:
Here is the final code in my test that works for passing a JSON string via POST to my view:
response = self.client.post(reverse('ajax_view'),
{'form':json.dumps(json_dict)})
Now printing from the view shows that the query dictionary is well formed.
<QueryDict: {u'form': [u'{"resource": {"status": "reviewed", "name": "Resource Test", "description": "Unit Test"}}']}>
I found the answer while tinkering with one of my co-workers, removing the content_type 'json' fixed the malformed query dictionary. The view that is being tested does not make use of or call the 'HttpRequest.is_ajax()', sending the header XMLHttpRequest' has no impact on my issue, though including the header would constitute good-form since this post is an ajax request.