We Keep Coding

Return data on HTTP upload to sanic server

I'm experimenting with uploading data to a sanic web server. To do so I issue a POST request using curl. I try to return some data after the POST request. The rationale behind this is to return some ID now representing the upload on the server side. But this doesn't seem to work. Now I'm wondering: Is my program wrong? Does curl not write the output? Or is this a bug in sanic? Could anyone help me here? Thanks!
Here's the Python program:
import signal
import asyncio
import uvloop
import sanic
app = sanic.Sanic(__name__)
loop = asyncio.get_event_loop()
server = app.create_server(host="localhost", port=3002)
task = asyncio.ensure_future(server)
#app.post("/testUpload", stream=True)
async def api_testUpload(request):
async def doStream(response):
while True:
body = await request.stream.get()
if body is None:
break
sanic.response.json({
"result": "good!"
})
return sanic.response.stream(doStream)
signal.signal(signal.SIGINT, lambda s, f: loop.stop())
try:
loop.run_forever()
except:
loop.stop()
You can invoke curl like this:
curl -v --data-binary "#somefile.data" http://localhost:3002/testUpload
And here is what curl writes to STDOUT:
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 3002 (#0)
> POST /testUpload HTTP/1.1
> Host: localhost:3002
> User-Agent: curl/7.47.0
> Accept: */*
> Content-Length: 334504
> Content-Type: application/x-www-form-urlencoded
> Expect: 100-continue
>
< HTTP/1.1 200 OK
< Keep-Alive: 5
< Transfer-Encoding: chunked
< Content-Type: text/plain; charset=utf-8
<
* Done waiting for 100-continue
* We are completely uploaded and fine
* Connection #0 to host localhost left intact
As you can see, a text/plain response is generated. This should be an application/json with my data, shouldn't it?

This is not the right way to do it. The sanic API does not work that intuitively here. In order to write data after receiving the bytes sent you have to do it this way within the streaming function doStream():
await response.write(json.dumps({
"result": "good!"
}))

If I understand your question, then you do not need to stream at all. It seems to me like what you are attempting to do is:
upload a file
process the file and do something
return a response to say that something was done
Assuming this is the case, then you do not need to stream at all.
import sanic
app = sanic.Sanic(__name__)
def do_something(file):
print(f'doing something to {file.name}')
#app.post("/testUpload")
async def api_testUpload(request):
myfile = request.files.get('myfile')
do_something(myfile)
return sanic.response.json({
'body': myfile.body,
'name': myfile.name,
'type': myfile.type,
})
app.run(host='localhost', port=3002, auto_reload=True)
You can then hit your endpoint with a file ...
curl -i -F "myfile=#/tmp/somefile.txt" http://localhost:3002/testUpload
And you should see in your logs:
[2018-10-25 10:09:56 +0300] [16051] [INFO] Goin' Fast # http://localhost:3002
[2018-10-25 10:09:56 +0300] [16055] [INFO] Starting worker [16055]
doing something to somefile.txt
[2018-10-25 10:09:59 +0300] - (sanic.access)[INFO][127.0.0.1:39306]: POST http://localhost:3002/testUpload 200 62
And your return
HTTP/1.1 200 OK
Connection: keep-alive
Keep-Alive: 5
Content-Length: 62
Content-Type: application/json
{"body":"FOOBAR\n","name":"somefile.txt","type":"text\/plain"}
Now, I suppose if you are uploading a file, and you wanted to stream the response of that upload as it was happening to that request or another, that would be a different issue.
I hope this helps.

Python. Cannot make a request to simple site api. Flask and requests

I am trying to create simple API for my site. I created the route with flask:
#api.route('/api/rate&message_id=<message_id>&performer=<performer_login>', methods=['POST'])
def api_rate_msg(message_id, performer_login):
print("RATE API ", message_id, ' ', performer_id)
return 400
print(...) function don't execute...
I use flask-socketio to communicate between client and server.
I send json from client and process it with:
#socket.on('rate')
def handle_rate(data):
print(data)
payload = {'message_id':data['message_id'], 'performer':data['performer']}
r = requests.post('/api/rate', params=payload)
print (r.status_code)
Note, that data variable is sending from client and is correct(I've checked it).
print(r.status_code) don't exec too...
Where I'm wrong? Please, sorry for my bad english :(
This api function must increase rate of message, which stored in mongodb, if interesting.

Don't put &message_id=<message_id>&performer=<performer_login> in your route string. Instead, get these arguments from request.args.
Try it:
from flask import request
...
#api.route('/api/rate', methods=['POST'])
def api_rate_msg():
print(request.args)
return ''
I've tested it with httpie:
$ http -v POST :5000/api/rate message_id==123 performer_login==foo
POST /api/rate?message_id=123&performer_login=foo HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Length: 0
Host: localhost:5000
User-Agent: HTTPie/0.9.8
HTTP/1.0 200 OK
Content-Length: 0
Content-Type: text/html; charset=utf-8
Date: Sun, 02 Apr 2017 13:54:40 GMT
Server: Werkzeug/0.11.11 Python/2.7.13
And from flask's log:
ImmutableMultiDict([('message_id', u'123'), ('performer_login', u'foo')])
127.0.0.1 - - [02/Apr/2017 22:54:40] "POST /api/rate?message_id=123&performer_login=foo HTTP/1.1" 200 -

Remove the below part from your api route
&message_id=<message_id>&performer=<performer_login
This is not required in POST request. It helps in GET requests. API call in request is not matching the route definition and therefore you have the current problem

Uploading file to Flask app using curl not working [duplicate]

This question already has answers here:
POST XML file using cURL command line
(9 answers)
Closed 6 years ago.
I am trying to upload a file to a Flask app and use it as a pandas DataFrame, not store it.
I have a Flask app with the following endpoint:
```
#app.route('/upload_file/', methods=['POST'])
def upload_file():
afile = request.files['file']
ff = file.read(afile)
print ff
return str(ff)
```
I am trying to use curl to use POST to upload the file using:
```curl -v -F file='/Users/asharma/../cpr_all_platforms.csv' http://127.0.0.1:5000/upload_file/```
I get the following error:
```
* Trying 127.0.0.1...
* Connected to 127.0.0.1 (127.0.0.1) port 5000 (#0)
> POST /upload_file/ HTTP/1.1
> Host: 127.0.0.1:5000
> User-Agent: curl/7.49.0
> Accept: */*
> Content-Length: 207
> Expect: 100-continue
> Content-Type: multipart/form-data; boundary=------------------------b313bf20e62c9f7c
>
< HTTP/1.1 100 Continue
* HTTP 1.0, assume close after body
< HTTP/1.0 400 BAD REQUEST
< Content-Type: text/html
< Content-Length: 192
< Server: Werkzeug/0.11.11 Python/2.7.12
< Date: Fri, 17 Feb 2017 19:00:37 GMT
<
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">
<title>400 Bad Request</title>
<h1>Bad Request</h1>
<p>The browser (or proxy) sent a request that this server could not understand.</p>
* Closing connection 0
```

#app.route('/upload_file/', methods=['POST'])
def upload_file():
file = request.files['file']
ff = pd.read_csv(file)
return str(ff)
The endpoint is hit using curl and the filename:
curl -i -X POST -F file=#/Users/asharma/../cpr_all_platforms.csv http://127.0.0.1:5000/upload_file/

Bottle framework with a tunnel doesn't receive GET arguments

I'm trying to make a simple python script which returns an argument from GET request. The issue is that it does not receive any arguments and returns blank body. There is one peculiar thing, though. In order to test GET requests I use requestmaker.com, hurl.it and apikitchen.com. While requestmaker and apikitchen return an empty body, hurl.it actually returns the required parameter.
I have tried Bottle, Flask and Tornado with the same results. I'm using ngrok for tunneling but I've also tried forwardhq.com.
The code (with bottle framework):
import bottle
from bottle import route, run, request, response
bottle.debug(True)
#route('/')
def home():
return "Great Scott!"
#route('/valley')
def thevalley():
theflux = request.query.flux
return theflux
run(host='0.0.0.0', port=8515, reloader=True)
ngrok status:
Tunnel Status online
Version 1.6/1.5
Forwarding http://88mph.ngrok.com -> 127.0.0.1:8515
The results I get from GET requests:
requestmaker.com
Request Headers Sent:
GET /valley HTTP/1.1
Host: 88mph.ngrok.com
Accept: */*
Content-Length: 10
Content-Type: application/x-www-form-urlencoded
Response Headers:
HTTP/1.1 200 OK
Server: nginx/1.4.3
Date: Sun, 09 Feb 2014 13:51:20 GMT
Content-Type: text/html; charset=UTF-8
Content-Length: 0
Connection: keep-alive
bottle:
127.0.0.1 - - [09/Feb/2014 13:51:20] "GET /valley HTTP/1.1" 200 0
hurl.it
Request:
Accept: */*
Accept-Encoding: gzip, deflate, compress
User-Agent: runscope/0.1
Response:
Connection: keep-alive
Content-Length: 5
Content-Type: text/html; charset=UTF-8
Date: Sun, 09 Feb 2014 14:02:55 GMT
Server: nginx/1.4.3
Body:
121gw
bottle:
127.0.0.1 - - [09/Feb/2014 14:02:55] "GET /valley?flux=121gw HTTP/1.1" 200 5
https://88mph.ngrok.com/valley?flux=121gw
Finally, just entering the URL into the address bar works as well, I get "121gw".
bottle:
127.0.0.1 - - [09/Feb/2014 14:05:46] "GET /valley?flux=121gw HTTP/1.1" 200 5
End
Every request maker can connect to server (200 OK) and even return "Great Scott" when accessing root. However, only webbrowser and hurl return the argument. Any ideas what is at fault here?

No response with POST request and Content-Type "application/json" in flask

I'm having problems with a Flask view that should return a response with content-type "application/json" in response to a POST request.
Specifically, if I do:
curl -v -d 'foo=bar' http://example.org/jsonpost
to this view:
#app.route('/jsonpost', methods=['GET', 'POST'])
def json_post():
resp = make_response('{"test": "ok"}')
resp.headers['Content-Type'] = "application/json"
return resp
I get some sort of connection reset:
* About to connect() to example.org port 80 (#0)
* Trying xxx.xxx.xxx.xxx... connected
* Connected to example.org (xxx.xxx.xxx.xxx) port 80 (#0)
> POST /routing/jsonpost HTTP/1.1
> User-Agent: curl/7.19.7 (i486-pc-linux-gnu) libcurl/7.19.7 OpenSSL/0.9.8k zlib/1.2.3.3 libidn/1.15
> Host: example.org
> Accept: */*
> Content-Length: 7
> Content-Type: application/x-www-form-urlencoded
>
< HTTP/1.1 200 OK
< Server: nginx/1.2.4
< Date: Thu, 27 Dec 2012 14:07:59 GMT
< Content-Type: application/json
< Content-Length: 14
< Connection: keep-alive
< Set-Cookie: session="..."; Path=/; HttpOnly
< Cache-Control: public
<
* transfer closed with 14 bytes remaining to read
* Closing connection #0
curl: (18) transfer closed with 14 bytes remaining to read
If instead I do:
curl -d 'foo=bar' http://example.org/htmlpost
to:
#app.route('/htmlpost', methods=['GET', 'POST'])
def html_post():
resp = make_response('{"test": "ok"}')
resp.headers['Content-Type'] = "text/html"
return resp
I get the expected the full response (200-ok)
{"test": "ok"}
By the way, if I send a GET request to the same JSON route:
curl http://example.org/jsonpost
I also get the expected response..
Any ideas?

Thanks to Audrius's comments I tracked a possible source of the problem to the interaction between uWSGI and nginx: apparently, if you receive POST data in a request you must read it before returning a response.
This, for example, fixes my issue.
#app.route('/jsonpost', methods=['GET', 'POST'])
def json_post():
if request.method == 'POST':
dummy = request.form
resp = make_response('{"test": "ok"}')
resp.headers['Content-Type'] = "application/json"
return resp
A different solution involves passing --post-buffering 1 to uWSGI as described by uWSGI's author Roberto De Ioris.
I still don't understand why the problem does not present itself with Content-Type set to "text/html"