We Keep Coding

flask - empty form data when sending request from file with netcat

I have a remote server running a simple flask server on it. I use this server for debugging purposes of my MCU cellular interface.
from flask import Flask
from flask import request
app = Flask(__name__)
#app.route("/b61d39e9-94ca-49e2-96ba-bdc3325a8aeb", methods = ["POST"])
def root():
print(request.headers)
if(request.is_json):
content = request.get_json()
print (content)
print(request.form.get("uuid"))
return "OK"
if(__name__ == "__main__"):
app.run(host = "0.0.0.0", port = 65432, debug = True)
And I send this request using netcat and the command nc ... 65432 < Request.txt
POST /b61d39e9-94ca-49e2-96ba-bdc3325a8aeb HTTP/1.1
Host: ESP32-DevKit VE
Content-Type: multipart/form-data;boundary="data"
--data
Content-Disposition: form-data; name="uuid"
ff0dbd36-dc36-4df5-8196-324adf76ed3f
--data--
netcat is reporting error 200, so everything was fine, but the flask web server doesn´t output the value of the field uuid.
... - - [13/Aug/2022 09:21:21] "POST /b61d39e9-94ca-49e2-96ba-bdc3325a8aeb HTTP/1.1" 200 -
Host: ESP32-DevKit VE
Content-Type: multipart/form-data;boundary="data"
None
I can not figure out what kind of problem is here.

It appears you're missing Content-Length header. I was able to get uuid using your python code above and this request:
POST /b61d39e9-94ca-49e2-96ba-bdc3325a8aeb HTTP/1.1
Content-Type: multipart/form-data; boundary=--------------------------data
Content-Length: 135
----------------------------data
Content-Disposition: form-data; name="uuid"
ff0dbd36-dc36-4df5-8196-324adf76ed3f
----------------------------data--

Batch Geocode Request with HERE Maps

I am working with the batch geocoding service on HERE Maps: https://developer.here.com/documentation/batch-geocoder/dev_guide/topics/request-constructing.html
The documentation is a bit sparse on exactly how to get results from a post request for a batch geocode.
I have been trying to make this work with Python and/or cURL but don't really understand what I am missing.
To my understanding I need to hit the url formatted like:
curl --cacert ./my_ssl_cert.txt -X POST -H "Content-Type: text/plain" --data-binary #locations.txt
"https://batch.geocoder.ls.hereapi.com
/6.2/jobs?
&apiKey={YOUR_API_KEY}
&action=run
&header=true
&inDelim=|
&outDelim=,
&outCols=recId,latitude,longitude,locationLabel
&outputcombined=true
&language=en-US
I have a test locations.txt file like:
recId|searchText|country
0001|Invalidenstraße 116 10115 Berlin|DEU
0002|Am Kronberger Hang 8 65824 Schwalbach|DEU
0003|425 W Randolph St Chicago IL 60606|USA
0004|One Main Street Cambridge MA 02142|USA
0005|200 S Mathilda Ave Sunnyvale CA 94086|USA
However, I would prefer to work with this process in something like Python using the requests library but am unsure of how to go about this process?
To my understanding I make a POST request and get a request ID then follow-up with a GET request using that request ID to obtain my results? I'm not sure how I would do this in python...
General thought flow in Python:
import requests
batch_url = '''https://batch.geocoder.ls.hereapi.com/6.2/jobs?&apiKey={MY_API_KEY}
&action=run
&header=true
&inDelim=|
&outDelim=,
&outCols=recId,latitude,longitude,locationLabel
&outputcombined=true
&language=en-US'''.replace("\n",'')
resp = requests.post(url=batch_url, verify="./my_ssl_cert.txt")
but I don't get a good response. I have ran the cURL command in terminal and received some information like:
* Connected to batch.geocoder.ls.hereapi.com (IP ADDRESS) port PORT (#0)
* schannel: disabled automatic use of client certificate
* schannel: ALPN, offering http/1.1
* schannel: added 2 certificate(s) from CA file
* schannel: connection hostname (batch.geocoder.ls.hereapi.com) validated against certificate name (batch.geocoder.ls.hereapi.com)
* ALPN, server did not agree to a protocol
> POST /6.2/jobs?&apiKey=MY_API_KEY&action=run&header=true&inDelim=|&outDelim=,&outCols=recId,latitude,longitude,locationLabel&outputcombined=true&language=en-US HTTP/1.1
> Host: batch.geocoder.ls.hereapi.com
> User-Agent: curl/7.79.1
> Accept: */*
> Content-Type: text/plain
> Content-Length: 254
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 400
< Date: Mon, 14 Mar 2022 13:03:11 GMT
< Server: openresty
< X-NLP-IRT: XXX
< X-Request-Id: REQ-XXXXXXXXXX
< X-Served-By: XXXXXXXXXXXXXX
< Content-Length: 0
< Connection: keep-alive
* Connection #0 to host batch.geocoder.ls.hereapi.com left intact
NOTE: I masked some information from the above output.

Return data on HTTP upload to sanic server

I'm experimenting with uploading data to a sanic web server. To do so I issue a POST request using curl. I try to return some data after the POST request. The rationale behind this is to return some ID now representing the upload on the server side. But this doesn't seem to work. Now I'm wondering: Is my program wrong? Does curl not write the output? Or is this a bug in sanic? Could anyone help me here? Thanks!
Here's the Python program:
import signal
import asyncio
import uvloop
import sanic
app = sanic.Sanic(__name__)
loop = asyncio.get_event_loop()
server = app.create_server(host="localhost", port=3002)
task = asyncio.ensure_future(server)
#app.post("/testUpload", stream=True)
async def api_testUpload(request):
async def doStream(response):
while True:
body = await request.stream.get()
if body is None:
break
sanic.response.json({
"result": "good!"
})
return sanic.response.stream(doStream)
signal.signal(signal.SIGINT, lambda s, f: loop.stop())
try:
loop.run_forever()
except:
loop.stop()
You can invoke curl like this:
curl -v --data-binary "#somefile.data" http://localhost:3002/testUpload
And here is what curl writes to STDOUT:
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 3002 (#0)
> POST /testUpload HTTP/1.1
> Host: localhost:3002
> User-Agent: curl/7.47.0
> Accept: */*
> Content-Length: 334504
> Content-Type: application/x-www-form-urlencoded
> Expect: 100-continue
>
< HTTP/1.1 200 OK
< Keep-Alive: 5
< Transfer-Encoding: chunked
< Content-Type: text/plain; charset=utf-8
<
* Done waiting for 100-continue
* We are completely uploaded and fine
* Connection #0 to host localhost left intact
As you can see, a text/plain response is generated. This should be an application/json with my data, shouldn't it?

This is not the right way to do it. The sanic API does not work that intuitively here. In order to write data after receiving the bytes sent you have to do it this way within the streaming function doStream():
await response.write(json.dumps({
"result": "good!"
}))

If I understand your question, then you do not need to stream at all. It seems to me like what you are attempting to do is:
upload a file
process the file and do something
return a response to say that something was done
Assuming this is the case, then you do not need to stream at all.
import sanic
app = sanic.Sanic(__name__)
def do_something(file):
print(f'doing something to {file.name}')
#app.post("/testUpload")
async def api_testUpload(request):
myfile = request.files.get('myfile')
do_something(myfile)
return sanic.response.json({
'body': myfile.body,
'name': myfile.name,
'type': myfile.type,
})
app.run(host='localhost', port=3002, auto_reload=True)
You can then hit your endpoint with a file ...
curl -i -F "myfile=#/tmp/somefile.txt" http://localhost:3002/testUpload
And you should see in your logs:
[2018-10-25 10:09:56 +0300] [16051] [INFO] Goin' Fast # http://localhost:3002
[2018-10-25 10:09:56 +0300] [16055] [INFO] Starting worker [16055]
doing something to somefile.txt
[2018-10-25 10:09:59 +0300] - (sanic.access)[INFO][127.0.0.1:39306]: POST http://localhost:3002/testUpload 200 62
And your return
HTTP/1.1 200 OK
Connection: keep-alive
Keep-Alive: 5
Content-Length: 62
Content-Type: application/json
{"body":"FOOBAR\n","name":"somefile.txt","type":"text\/plain"}
Now, I suppose if you are uploading a file, and you wanted to stream the response of that upload as it was happening to that request or another, that would be a different issue.
I hope this helps.

Python. Cannot make a request to simple site api. Flask and requests

I am trying to create simple API for my site. I created the route with flask:
#api.route('/api/rate&message_id=<message_id>&performer=<performer_login>', methods=['POST'])
def api_rate_msg(message_id, performer_login):
print("RATE API ", message_id, ' ', performer_id)
return 400
print(...) function don't execute...
I use flask-socketio to communicate between client and server.
I send json from client and process it with:
#socket.on('rate')
def handle_rate(data):
print(data)
payload = {'message_id':data['message_id'], 'performer':data['performer']}
r = requests.post('/api/rate', params=payload)
print (r.status_code)
Note, that data variable is sending from client and is correct(I've checked it).
print(r.status_code) don't exec too...
Where I'm wrong? Please, sorry for my bad english :(
This api function must increase rate of message, which stored in mongodb, if interesting.

Don't put &message_id=<message_id>&performer=<performer_login> in your route string. Instead, get these arguments from request.args.
Try it:
from flask import request
...
#api.route('/api/rate', methods=['POST'])
def api_rate_msg():
print(request.args)
return ''
I've tested it with httpie:
$ http -v POST :5000/api/rate message_id==123 performer_login==foo
POST /api/rate?message_id=123&performer_login=foo HTTP/1.1
Accept: */*
Accept-Encoding: gzip, deflate
Connection: keep-alive
Content-Length: 0
Host: localhost:5000
User-Agent: HTTPie/0.9.8
HTTP/1.0 200 OK
Content-Length: 0
Content-Type: text/html; charset=utf-8
Date: Sun, 02 Apr 2017 13:54:40 GMT
Server: Werkzeug/0.11.11 Python/2.7.13
And from flask's log:
ImmutableMultiDict([('message_id', u'123'), ('performer_login', u'foo')])
127.0.0.1 - - [02/Apr/2017 22:54:40] "POST /api/rate?message_id=123&performer_login=foo HTTP/1.1" 200 -

Remove the below part from your api route
&message_id=<message_id>&performer=<performer_login
This is not required in POST request. It helps in GET requests. API call in request is not matching the route definition and therefore you have the current problem

No response with POST request and Content-Type "application/json" in flask

I'm having problems with a Flask view that should return a response with content-type "application/json" in response to a POST request.
Specifically, if I do:
curl -v -d 'foo=bar' http://example.org/jsonpost
to this view:
#app.route('/jsonpost', methods=['GET', 'POST'])
def json_post():
resp = make_response('{"test": "ok"}')
resp.headers['Content-Type'] = "application/json"
return resp
I get some sort of connection reset:
* About to connect() to example.org port 80 (#0)
* Trying xxx.xxx.xxx.xxx... connected
* Connected to example.org (xxx.xxx.xxx.xxx) port 80 (#0)
> POST /routing/jsonpost HTTP/1.1
> User-Agent: curl/7.19.7 (i486-pc-linux-gnu) libcurl/7.19.7 OpenSSL/0.9.8k zlib/1.2.3.3 libidn/1.15
> Host: example.org
> Accept: */*
> Content-Length: 7
> Content-Type: application/x-www-form-urlencoded
>
< HTTP/1.1 200 OK
< Server: nginx/1.2.4
< Date: Thu, 27 Dec 2012 14:07:59 GMT
< Content-Type: application/json
< Content-Length: 14
< Connection: keep-alive
< Set-Cookie: session="..."; Path=/; HttpOnly
< Cache-Control: public
<
* transfer closed with 14 bytes remaining to read
* Closing connection #0
curl: (18) transfer closed with 14 bytes remaining to read
If instead I do:
curl -d 'foo=bar' http://example.org/htmlpost
to:
#app.route('/htmlpost', methods=['GET', 'POST'])
def html_post():
resp = make_response('{"test": "ok"}')
resp.headers['Content-Type'] = "text/html"
return resp
I get the expected the full response (200-ok)
{"test": "ok"}
By the way, if I send a GET request to the same JSON route:
curl http://example.org/jsonpost
I also get the expected response..
Any ideas?

Thanks to Audrius's comments I tracked a possible source of the problem to the interaction between uWSGI and nginx: apparently, if you receive POST data in a request you must read it before returning a response.
This, for example, fixes my issue.
#app.route('/jsonpost', methods=['GET', 'POST'])
def json_post():
if request.method == 'POST':
dummy = request.form
resp = make_response('{"test": "ok"}')
resp.headers['Content-Type'] = "application/json"
return resp
A different solution involves passing --post-buffering 1 to uWSGI as described by uWSGI's author Roberto De Ioris.
I still don't understand why the problem does not present itself with Content-Type set to "text/html"