I'm trying to build a simple proxy using Flask and requests. The code is as follows:
#app.route('/es/<string:index>/<string:type>/<string:id>',
methods=['GET', 'POST', 'PUT']):
def es(index, type, id):
elasticsearch = find_out_where_elasticsearch_lives()
# also handle some authentication
url = '%s%s%s%s' % (elasticsearch, index, type, id)
esreq = requests.Request(method=request.method, url=url,
headers=request.headers, data=request.data)
resp = requests.Session().send(esreq.prepare())
return resp.text
This works, except that it loses the status code from Elasticsearch. I tried returning resp (a requests.models.Response) directly, but this fails with
TypeError: 'Response' object is not callable
Is there another, simple, way to return a requests.models.Response from Flask?
Ok, found it:
If a tuple is returned the items in the tuple can provide extra information. Such tuples have to be in the form (response, status, headers). The status value will override the status code and headers can be a list or dictionary of additional header values.
(Flask docs.)
So
return (resp.text, resp.status_code, resp.headers.items())
seems to do the trick.
Using text or content property of the Response object will not work if the server returns encoded data (such as content-encoding: gzip) and you return the headers unchanged. This happens because text and content have been decoded, so there will be a mismatch between the header-reported encoding and the actual encoding.
According to the documentation:
In the rare case that you’d like to get the raw socket response from the server, you can access r.raw. If you want to do this, make sure you set stream=True in your initial request.
and
Response.raw is a raw stream of bytes – it does not transform the response content.
So, the following works for gzipped data too:
esreq = requests.Request(method=request.method, url=url,
headers=request.headers, data=request.data)
resp = requests.Session().send(esreq.prepare(), stream=True)
return resp.raw.read(), resp.status_code, resp.headers.items()
If you use a shortcut method such as get, it's just:
resp = requests.get(url, stream=True)
return resp.raw.read(), resp.status_code, resp.headers.items()
Flask can return an object of type flask.wrappers.Response.
You can create one of these from your requests.models.Response object r like this:
from flask import Response
return Response(
response=r.reason,
status=r.status_code,
headers=dict(r.headers)
)
I ran into the same scenario, except that in my case my requests.models.Response contained an attachment. This is how I got it to work:
return send_file(BytesIO(result.content), mimetype=result.headers['Content-Type'], as_attachment=True)
My use case is to call another API in my own Flask API. I'm just propagating unsuccessful requests.get calls through my Flask response. Here's my successful approach:
headers = {
'Authorization': 'Bearer Muh Token'
}
try:
response = requests.get(
'{domain}/users/{id}'\
.format(domain=USERS_API_URL, id=hit['id']),
headers=headers)
response.raise_for_status()
except HTTPError as err:
logging.error(err)
flask.abort(flask.Response(response=response.content, status=response.status_code, headers=response.headers.items()))
Related
I have a function like this:
def get_some_data(api_url, **kwargs)
# some logic on generating headers
# some more logic
response = requests.get(api_url, headers, params)
return response
I need to create a fake/mock "api_url", which, when made request to, would generate a valid response.
I understand how to mock the response:
def mock_response(data):
response = requests.Response()
response.status_code = 200
response._content = json.dumps(data)
return response
But i need to make the test call like this:
def test_get_some_data(api_url: some_magic_url_path_that_will_return_mock_response):
Any ideas on how to create an url path returning a response within the scope of the test (only standard Django, Python, pytest, unittest) would be very much appreciated
The documentation is very well written and more than clear on how to mock whatever you want. But, let say you have a service that makes the 3rd party API call:
def foo(url, params):
# some logic on generating headers
# some more logic
response = requests.get(url, headers, params)
return response
In your test you want to mock the return value of this service.
#patch("path_to_service.foo")
def test_api_call_response(self, mock_response):
mock_response.return_value = # Whatever the return value you want it to be
# Here you call the service as usual
response = foo(..., ...)
# Assert your response
This is my implementation,
full_url = url + '?' + params
req = urllib2.Request(full_url, params)
req.add_header('Content-Type', 'application/javascript')
req.add_header('Access-Control-Allow-Origin','*')
req.add_header('Accept', 'application/javascript')
req.add_header('x-lang', 'en')
req.add_header('x-app-version', '1.2.3')
req.add_header('x-consumer-key', 'abc')
req.add_header('x-source', 'web')
req.add_header('x-device-id', 'abc-2501015753736970469271537365900144030')
req.add_header('x-signature', signature)
req.add_header('X-Content-Type-Options', 'nosniff')
req.add_header('x-request-id', request)
req.add_header('x-timestamp', timeStamp)
response = urllib2.urlopen(req)
result = response.read()
result = result.decode('latin-1')
respjson = json.loads(result)
return respjson
Reading the output in ext.js
var script = document.createElement("script");
script.setAttribute("src", url);
script.setAttribute("type", "text/javascript");
script.setAttribute("id", trans.scriptId);
document.getElementsByTagName("head")[0].appendChild(script);
Thank you in advance.
Error shows in the browser "was loaded even though its MIME type (“application/json”) is not a valid JavaScript MIME"
[Error in the browser][1]
Short Answer:
Use JSONP instead of JSON in ExtJS.
Tons of additional information
better ways to retrieve a JSON from ExtJS
But this was not the question I assume.
Let's say you have your JSON and you are trying to populate a store.
Instead of injecting the JSON into the DOM, you could use:
Ext.StoreManager.get('storeId').load(JSON);
// or for the store of a grid
Ext.ComponentQuery.query('grid')[0].getStore().load(JSON);
// or for xtype main
Ext.first('main').getViewModel().set('initData', JSON);
All these methods would get the data into an ExtJS App
Retrieving Data
Getting Data into ExtJS typically would be done using
Ext.Ajax.request({
url: full_url,
header: ...
success: success_fn,
scope: this
})
Or directly in your model/store
I'm working with django rest_framework and consuming an API that brings a header value to validate the sender. I have the problem when I use APIClient to test the webhook that I made.
#pytest.mark.django_db
def test_change_status_webhook_ok(webhook_client, status_changes):
fixture_signature = (
"51b93156c361bfce14c527ddcb27cc3791e9ea6ede23bc5a56efa3be28e6a54d"
)
url = reverse("webhook_v1:notification")
response = webhook_client.post(
url,
json.dumps(status_changes),
content_type="application/json",
**{"X-Extserv-Signature": fixture_signature}
)
assert response.status_code == 200
The problem is when I tried to get the X-Extserv-Signature from headers.
I tried with:
ret = request.headers.get("X-Extserv-Signature")
this way only works when I receive a request from postman.. but the below form works when I do the request from APIClient and cannot get the value using the same above code.
ret = request.META["X-Extserv-Signature"]
You know how I can set the X-Extserv-Signature key value to headers in APIClient?
From the docs:
A case insensitive, dict-like object that provides access to all HTTP-prefixed headers
request.headers will only contain headers that are HTTP-prefixed, so:
response = webhook_client.post(
url,
json.dumps(status_changes),
content_type="application/json",
**{"HTTP_X_EXTSERV_SIGNATURE": fixture_signature} # <-- Use this
)
I wrote a certain API wrapper using Python's requests library.
When it gets a response using requests.get, it attempts to parse as json and takes the raw content if it doesn't work:
resp = requests.get(url, ...)
try:
resp_content = resp.json()
except ValueError:
resp_content = resp.content
return resp_content
This is correct for my purposes. The problem is how long it takes when the downloaded response is an image file, for example, if it is large, then it takes an extremely long time between entering the try, and failing the json parse and entering the except.
(I don't know if it takes super long for the .json() to error at all, or if once it errors it then takes a while to get into the except.)
Is there a way to see if resp is json-parsable without attempting to parse it with .json()? Something like resp.is_json, so I can instantly know which branch to take (resp.json() or resp.content), instead of waiting 30 seconds (large files can take minutes).
Depending on the consistency of the response, you could check if the returned headers include content-type application/json:
resp.headers.get('content-type') == 'application/json'
You can check if the content-type application/json is in the response headers:
'application/json' in response.headers.get('Content-Type', '')
(Addressing Daniel Kats comment in a previous response)
You could check if the returned headers include Content-Type application/json:
response.headers.get('Content-Type').startswith('application/json')
By using startswith, you're accounting for all allowed formats from https://www.w3.org/Protocols/rfc1341/4_Content-Type.html.
That doesn't guaranty that it will be a valid JSON, but at least that will catch responses which aren't being declared as JSON.
If using Session and not directly requests.(METHOD)
from requests import Session
from simplejson.errors import JSONDecodeError
class MySession(Session):
def __init__(self, *args, **kwargs) -> None:
super().__init__(*args, **kwargs)
def request(self, *args, **kwargs):
res = super().request(*args, **kwargs)
json = res.json
def wrapper():
try:
return json()
except JSONDecodeError:
return None
res.json = wrapper
return res
session = MySession()
res = session.get("https://api64.ipify.org")
if res.json():
print("ok")
Have you tried -
#resp_content = resp.json()
if resp_content.ok:
resp_content.json()
else:
resp_content = resp.content
I would check the first few 100 bytes and count the number of json characters like {":. Or you could check for image signatures (JFIF, PNG, GIF89A)..
Im trying to test a PUT request in my Flask app, using flasks test client.
Everything looks good to me but i keep getting 400 BAD request.
I tried the same request using POSTMAN and I get the response back.
Here is the code
from flask import Flask
app = Flask(__name__)
data = {"filename": "/Users/resources/rovi_source_mock.csv"}
headers = {'content-type': 'application/json'}
api = "http://localhost:5000/ingest"
with app.test_client() as client:
api_response = client.put(api, data=data, headers=headers)
print(api_response)
Output
Response streamed [400 BAD REQUEST]
You do need to actually encode the data to JSON:
import json
with app.test_client() as client:
api_response = client.put(api, data=json.dumps(data), headers=headers)
Setting data to a dictionary treats that as a regular form request, so each key-value pair would be encoded into application/x-www-form-urlencoded or multipart/form-data content, if you had used either content type. As it is, your data is entirely ignored instead.
I think it is simpler to just pass the data using the json parameter instead of the data parameter:
reponse = test_client.put(
api,
json=data,
)
Quoting from here:
Passing the json argument in the test client methods sets the request
data to the JSON-serialized object and sets the content type to
application/json.