While implementing a rest API in python flask, I have used several options to return a file (any type) , read it and save it to local repository of request but encountered with multiple errors as below:
Case 1:
def download_file():
return send_file('any_file.pdf')
r = requests.get(url = 'http://localhost:5000/download').read()
has responded with a error Response object has no attribute read/text/content
Case 2:
def download_file():
file = open('any_file.pdf','r').read()
return file
r = requests.get(url = 'http://localhost:5000/download')
has responded with a error Return doesn't accept this
So How can I do this as flask is not allowing to return a file without response object and response object is not readable and doesn't support to save that file directly.
The Flask server code in Case 1 is correct. A more complete example:
#app.route('/download')
def download_file():
# Some logic here
send_file('any_file.pdf')
However the Response object returned by requests.get doesn't have a read method. The correct way is to use:
Response.content: Content of the response, in bytes.
So, the client code should be:
r = requests.get('http://localhost:5000/download')
bytes = r.content
# Now do something with bytes, for example save it:
with open('downloaded_file.ext', 'wb') as f:
f.write(bytes)
Related
I'm trying to write the output of an API 'get' request to a file. I know the function works as the output of the request is returned successfully and in full in shell.
The function creates the file, but the file only includes the status code for the get request - can anyone help me understand why? Still pretty new to Python and very new to calling APIs...
def user_info(token):
heads = {"Authorization": "bearer " + token, "Content-type": "application/json"}
url = "xxxxx"
request = req.get(url, headers=heads)
with open('userdata.txt', 'w') as f:
f.write(str(request))
if request.status_code == 200:
return request.json()
else:
return "User Info FAILED! Failure code: {}, error: {}".format(request.status_code,request.json()['error'])
The file just contains:
<Response [200]>
Thanks!
A Response object in the requests module when called as a string will return its status code as detailed in the Response python code
def __repr__(self):
return '<Response [%s]>' % (self.status_code)
If you want to write the json from the response as a string then you can use the built in json module and use its dumps method to dump it as a string not a dict.
import requests
import json
with open("myfile.txt", "w") as output:
resp = requests.get("https://jsonplaceholder.typicode.com/todos/1")
output.write(json.dumps(resp.json()))
I have an endpoint which is receiving a text file from a request, which the flask server processes and now need to send a response which should be a json file. I have read and performed the operations of received txt file, and now when I send the dict by jsonify I get this in terminal.
Response 444 bytes [200 OK]
How do I get the json file or is there any other way of sending the JSON file in response? Since I won't be in control of the server that will send the request I need a way to send JSON file.
Code Handling the response
#app.route('/automated_testing', methods=['GET','POST'])
def getfile():
if request.method == 'POST':
for file in request.files:
links = request.files[file].read()
#Since a byte sized object is returned
links = [links.decode('utf8').strip()]
links = links[0].split("\n")
res = dict()
for i in links:
f = FlarePredictor(i)
res[i] = f[0]
print(jsonify(res), file=sys.stderr)
return jsonify(res)
else:
return "GET REQ"
return "Hi"
Code used to send the request
import requests
with open('test.txt', 'rb') as f:
r = requests.post('http://127.0.0.1:5000/automated_testing', files={'test.txt': f})
requests supports JSON out-of-the-box:
json = r.json()
Now json will be a regular Python dictionary containing the data.
I try to use falcon package in my project. Problem is I didn't find a way to get body data from the HTTP post request.
I used code from example, but req.stream.read() doesn't return JSON as expected.
The code is:
raw_json = req.stream.read()
result.json(raw_json, encoding='utf-8')
resp.body = json.dumps(result_json, encoding='utf-8')
How to get the POST data?
Thanks for any help
in falcon 2, if you work with json type, use req.media
for example:
import falcon
from json import dumps
class Resource(object):
def on_post(self, req, resp, **kwargs):
result = req.media
# do your job
resp.body = dumps(result)
api = falcon.API()
api.add_route('/test', Resource())
Little digging into the problem led to the following linked issue on github. It states that falcon framework at least in its version 0.3 and working with Python 2 didn't parse data 'POSTed' as string if they are aptly escaped. We could use more information on what data you are trying to send over POST request and in what format is that being sent, as in if its being send as simple text, or with Header Information Content-Type:application/json, or if its coming through an HTML form.
While the exact issue is not clear from the question I could still suggest trying to use bounded_stream instead of stream as in:
raw_json = req.bounded_stream.read()
result.json(raw_json, encoding='utf-8')
resp.body = json.dumps(result_json, encoding='utf-8')
for the official documentation suggests use of bounded_stream where uncertain conditions such as Content-Length undefined or 0, or if header information is missing altogether.
bounded_stream is described as the following in the official falcon documentation.
File-like wrapper around stream to normalize certain differences between the native input objects employed by different WSGI servers. In particular, bounded_stream is aware of the expected Content-Length of the body, and will never block on out-of-bounds reads, assuming the client does not stall while transmitting the data to the server.
Falcon receives the HTTP requests data as buffer object as passed by WSGI wrapper which receives the data from client, and its possible it doesn't run proper parsing on top of the data to convert to a more usable data structure for performance reasons.
Big thanks to Ryan (and Prateek Jain) for the answer.
The solution is simply to put app.req_options.auto_parse_form_urlencoded=True. For example:
import falcon
class ThingsResource(object):
def on_post(self, req, resp):
value = req.get_param("value", required=True)
#do something with value
app = falcon.API()
app.req_options.auto_parse_form_urlencoded=True
things = ThingsResource()
app.add_route('/things', things)
The field you're looking for is somewhat confusingly named, but it's req.media:
Returns a deserialized form of the request stream. When called, it will attempt to deserialize the request stream using the Content-Type header as well as the media-type handlers configured via falcon.RequestOptions.
If the request is JSON, req.media already contains a python dict.
I have added changes in request.py in falcon framework to parse application/x-www-form-urlencoded and multipart/from-data.
I have raised pull request - https://github.com/falconry/falcon/pull/1236 but it is not yet merged in master.
Check this - https://github.com/branelmoro/falcon
I have added new code to parse POST, PUT and DELETE application/x-www-form-urlencoded and multipart/form-data.
Text fields will be available in req.form_data dictionary and upload file buffer stream will be available in req.files dictionary.
I hope this will help to access POST and GET parameters separately and we will be able to upload files as well.
Good thing about the change is that it will not load entire uploaded file in memory.
Below is sample code to show how to use POST, PUT and DELETE application/x-www-form-urlencoded and multipart/form-data:
import falcon
class Resource(object):
def on_post(self, req, resp):
# req.form_data will return dictionary of text field names and their values
print(req.form_data)
# req.form_data will return dictionary of file field names and
# their buffer class FileStream objects as values
print(req.files)
# support we are uploading a image.jpg in `pancard` file field then
# req.files["pancard"] will be FileStream buffer object
# We can use set_max_upload_size method to set maximum allowed
# file size let say 1Mb = 1*1024*1024 bytes for this file
req.files["pancard"].set_max_upload_size(1*1024*1024)
# We can use uploadto method to upload file on required path (Note: absolute filepath is required)
# This method returns boolean - `True` on successful upload
# and if upload is unsuccessful then it returns `False` and sets error on failure.
path = "/tmp/" + req.files["pancard"].name
response = req.files["pancard"].uploadto("/tmp/" + path)
print(response)
# Once file is uploaded sucessfully, we can check it's size
print(req.files["pancard"].size)
# If file is not uploaded sucessfully, we can check it's error
print(req.files["pancard"].error)
resp.body = "Done file upload"
resp.status = falcon.HTTP_200
# falcon.API instances are callable WSGI apps
app = falcon.API()
things = Resource()
# things will handle post requests to the '/post_path' URL path
app.add_route('/post_path', things)
Do let me know if you have any doubts.
So far... for me bounded_stream.read() and stream.read() both get the posted data as type str. I have only found one way around the issue so far:
def on_post(self, req, resp):
posted_data = json.loads(req.stream.read())
print(str(type(posted_data)))
print(posted_data)
Loading the string into a json dict once the posted data is received is my only solution that I can come up with
Here's something I used while designing an API.
import falcon
import json
class VerifierResource():
def on_post(self, req, resp):
credentials = json.loads(req.stream.read())
if credentials['username'] == USER \
and credentials['passwd'] == PASSWORD:
resp.body = json.dumps({"status": "verified"})
else:
resp.body = json.dumps({"status": "invalid"})
api = falcon.API()
api.add_route('/verify', VerifierResource())
This returns a serialized JSON with corresponding response body.
there is a sample way to get media from body. I use to get the body in the post method:
def on_post(req,resp)
arguments = {}
# get body media on post method
body = req.get_media()
if 'something' in body:
arguments['something'] = body['something']
send body content type Media-Type and print resp or use in code, but if want to send JSON body your code should cover give JSON parameters.
Do let me know if you have any doubts.
I need help with python requests. I want to post a file to a server. Here is my example code for posting the file
import requests
with open('try.txt', 'rb') as myfile:
url = <myurl>
files = {'value':myfile}
response = requests.post(url, files=files)
try.txt is not a blank file. It contains some sentences
On the server side, here is the snippet for receiving the file.
from flask_restful import Api, Resource, reqparse
class MyThing(Resource):
def get(self):
pass
def post(self):
parser = reqparse.RequestParser()
parser.add_argument('value')
args = parser.parse_args()
value = args['value']
if value.endswith(".txt") or value.endswith(".csv"):
## do something
else:
## do something
However when I run the server and try to post the file to the server, it return an error saying that
if value.endswith(".txt") or value.endswith(".csv"):
AttributeError: 'NoneType' object has no attribute 'endswith'
I check the value by print files and it return like this
{'value': <open file 'try.txt', mode 'rb' at 0x7f041db8e420>}
Does this mean value is null? or did I missed something on my server side code? Thank you for the help.
I'm trying to build a simple proxy using Flask and requests. The code is as follows:
#app.route('/es/<string:index>/<string:type>/<string:id>',
methods=['GET', 'POST', 'PUT']):
def es(index, type, id):
elasticsearch = find_out_where_elasticsearch_lives()
# also handle some authentication
url = '%s%s%s%s' % (elasticsearch, index, type, id)
esreq = requests.Request(method=request.method, url=url,
headers=request.headers, data=request.data)
resp = requests.Session().send(esreq.prepare())
return resp.text
This works, except that it loses the status code from Elasticsearch. I tried returning resp (a requests.models.Response) directly, but this fails with
TypeError: 'Response' object is not callable
Is there another, simple, way to return a requests.models.Response from Flask?
Ok, found it:
If a tuple is returned the items in the tuple can provide extra information. Such tuples have to be in the form (response, status, headers). The status value will override the status code and headers can be a list or dictionary of additional header values.
(Flask docs.)
So
return (resp.text, resp.status_code, resp.headers.items())
seems to do the trick.
Using text or content property of the Response object will not work if the server returns encoded data (such as content-encoding: gzip) and you return the headers unchanged. This happens because text and content have been decoded, so there will be a mismatch between the header-reported encoding and the actual encoding.
According to the documentation:
In the rare case that you’d like to get the raw socket response from the server, you can access r.raw. If you want to do this, make sure you set stream=True in your initial request.
and
Response.raw is a raw stream of bytes – it does not transform the response content.
So, the following works for gzipped data too:
esreq = requests.Request(method=request.method, url=url,
headers=request.headers, data=request.data)
resp = requests.Session().send(esreq.prepare(), stream=True)
return resp.raw.read(), resp.status_code, resp.headers.items()
If you use a shortcut method such as get, it's just:
resp = requests.get(url, stream=True)
return resp.raw.read(), resp.status_code, resp.headers.items()
Flask can return an object of type flask.wrappers.Response.
You can create one of these from your requests.models.Response object r like this:
from flask import Response
return Response(
response=r.reason,
status=r.status_code,
headers=dict(r.headers)
)
I ran into the same scenario, except that in my case my requests.models.Response contained an attachment. This is how I got it to work:
return send_file(BytesIO(result.content), mimetype=result.headers['Content-Type'], as_attachment=True)
My use case is to call another API in my own Flask API. I'm just propagating unsuccessful requests.get calls through my Flask response. Here's my successful approach:
headers = {
'Authorization': 'Bearer Muh Token'
}
try:
response = requests.get(
'{domain}/users/{id}'\
.format(domain=USERS_API_URL, id=hit['id']),
headers=headers)
response.raise_for_status()
except HTTPError as err:
logging.error(err)
flask.abort(flask.Response(response=response.content, status=response.status_code, headers=response.headers.items()))