In a Django project of mine, I've written simple webhook logic that pings a url for certain payload if some condition is met. Here's the code:
#csrf_exempt
def webhook_event(request,*args,**kwargs):
if request.method == 'POST':
data = json.loads(request.body)
event_type = data['event']
# use CDN URL from webhook payload
if event_type == 'project.datafile_updated':
url = data['data']['cdn_url']
config_manager.set_obj(url)
return json.dumps({'success':True}), 200, {'ContentType':'application/json'}
return json.dumps({'success':False}), 400, {'ContentType':'application/json'}
else:
return render(request,"404.html",{})
I'm using the following to test it:
import requests
import json
# Update project_id
project_id = "<some_id>"
data = {"timestamp": 1463602412, "project_id": project_id, "data": {"cdn_url": "https://cdn.example.com/json/{0}.json".format(project_id), "origin_url": "https://example.s3.amazonaws.com/json/{0}.json".format(project_id), "revision": 15}, "event": "project.datafile_updated"}
r = requests.post("http://127.0.0.1:8000/ohook/", data=json.dumps(data), headers={'Content-Type': 'application/json'})
print r
It all works perfectly, but the tuple returned by webhook_event is giving me the following error:
Internal Server Error: /ohook/
Traceback (most recent call last):
File "/home/hassan/.virtualenvs/myenv/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, in get_response
response = middleware_method(request, response)
File "/home/hassan/.virtualenvs/myenv/local/lib/python2.7/site-packages/newrelic-2.56.0.42/newrelic/hooks/framework_django.py", line 328, in wrapper
return wrapped(*args, **kwargs)
File "/home/hassan/.virtualenvs/myenv/local/lib/python2.7/site-packages/django/middleware/clickjacking.py", line 30, in process_response
if response.get('X-Frame-Options', None) is not None:
AttributeError: 'tuple' object has no attribute 'get'
Can anyone help me diagnose this?
As the error says, you're returning a tuple: the JSON, an integer (presumably the status code) and a dict (presumably the headers).
Even if you fixed this, you can't just return JSON from a view; you have to return an instance of HttpResponse or one of its subclasses.
Instead of doing return json.dumps(...) in your POST blocks, you should use JsonResponse. This accepts a Python datastructure and returns a response containing the serialized JSON; as a bonus, it also sets the content-type appropriately.
if event_type == 'project.datafile_updated':
...
return JsonResponse({'success':True})
return JsonResponse({'success':False}, status=400)
(Note that render is explicitly a shortcut which renders a template and returns it as an HttpResponse; this isn't the case with json.dumps().)
Related
I am trying to create a python api that works on the input data and run different scripts according to the provided input. I was using flask + gunicorn earlier with 8 workers. I got to know about asyncio library and I switched from flask to quart and gunicorn to hypercorn using the following [guide][1].
My functions look like :
#app.route("/pyscript", methods=["POST"])
##cross_origin(supports_credentials=True)
async def pyscript():
#check for pre defined token
try:
if(token == consts.authToken):
print("TOKEN VALIDATED")
else:
return jsonify({'message': 'token is invalid'})
except:
return jsonify({'message': 'token is invalid'})
json_all_data = await request.get_json(force=True)
print(json_all_data)
json_data = json_all_data['data']
json_data = json.dumps(json_data)
if json_data is None:
return "Bad request, invalid JSON Format inserted", 401
json_LOC = (json_all_data['LOC'])
json_LOI = (json_all_data['LOB'])
try:
if(json_LOC in keys and json_LOI in keys[json_LOC]):
chosen_operation_function = carriers.get(json_LOC).get(json_LOI)
return_UUID = uuid.uuid4()
app.add_background_task(run_task(json_data, json_LOC, json_LOI, chosen_operation_function, return_UUID))
out_int = {
"job_id" : return_UUID,
"status" : "Running",
"input" : json.loads(json_data)
}
return out_int, 202
else:
return jsonify({'message': 'LOC/LOB is invalid'})
except Exception as e:
print(e)
return e
run_task is a function in other file that is an async defined function.
Expected Output : I wanted to process the run_task in background while returning 202 response and a json data associated with it.
Whats happening : The value is returned in the postman request but the server shows the following error :
Traceback (most recent call last):
File "/home/user/.local/lib/python3.8/site-packages/quart/app.py", line 1396, in _wrapper
await copy_current_app_context(func)(*args, **kwargs)
File "/home/user/.local/lib/python3.8/site-packages/quart/ctx.py", line 333, in wrapper
return await app_context.app.ensure_async(func)(*args, **kwargs)
File "/home/user/.local/lib/python3.8/site-packages/quart/utils.py", line 55, in _wrapper
None, copy_context().run, partial(func, *args, **kwargs)
TypeError: the first argument must be callable
I am not able to identify the error and any help would be appreciated.
Thanks
[1]: https://pgjones.gitlab.io/quart/how_to_guides/flask_migration.html
I think you are calling add_background_task incorrectly, rather than,
app.add_background_task(run_task(json_data, json_LOC, json_LOI, chosen_operation_function, return_UUID))
It should be called as,
app.add_background_task(run_task, json_data, json_LOC, json_LOI, chosen_operation_function, return_UUID)
Note that run_task is not called in the second example in your code, instead it and the arguments to call it are passed to the add_background_task function.
In the Django Rest Framework I would like to post a file, received as an InMemoryUploadedFile, to a different server as soon as it is received.
It sounds simple, but the request.post() function does not seem to properly send over such a file :
def post(self, request, *args, **kwargs):
data = request.data
print(data)
# <QueryDict: {'file': [<InMemoryUploadedFile: myfile.pdf (application/pdf)>]}>
endpoint = OTHER_API_URL + "/endpoint"
r = requests.post(endpoint, files=data)
My other server receives the request (through flask) with the name of the file, but not the content:
#app.route("/endpoint", methods=["POST"])
def endpoint():
if flask.request.method == "POST":
# I removed the many checks to simplify the code
file = flask.request.files['file']
path = os.path.join(UPLOAD_FOLDER, file.filename)
file.save(path)
print(file) #<FileStorage: u'file.pdf' (None)>
print(os.path.getsize(path)) #0
return [{"response":"ok"}]
When posting a file directly to that api in form-data with postman, It works as expected:
print(file) # <FileStorage: u'file.pdf' ('application/pdf')>
print(os.path.getsize(path)) #8541
Any help on how to fix this, i.e. transform the InMemoryUploadedFile type in something a normal REST api can understand? Or maybe just adding the right headers?
I had to figure this issue out passing an uploaded file from a Django front end website to a Django backend API in Python 3. The InMemoryUploadedFile's actual file data can be accessed via the object's .file property's .getvalue() method.
path="path/to/api"
in_memory_uploaded_file = request.FILES['my_file']
io_file = in_memory_uploaded_file.file
file_value = io_file.getvalue()
files = {'my_file': file_value}
make_http_request(path, files=files)
and can be shortened
file = request.FILES['my_file'].file.getvalue()
files = {'my_file': file}
Before this, trying to send InMemoryUploadFile objects, the file property, or the result of the read() method all proved to send a blank/empty file by the time it got to the API.
I had the same problem and the same case.
My working solution
headers = {
"Host": API_HOST,
"cache-control": "no-cache",
}
try:
data = json_request = request.POST['json_request'].strip()
data = json.loads(data) # important!
except:
raise Http404
try:
in_memory_uploaded_file = request.FILES['file'].file.getvalue()
files = {'photo': in_memory_uploaded_file} # important!
except:
files = {}
if USE_HTTPS:
API_HOST = f'https://{API_HOST}'
else:
API_HOST = f'http://{API_HOST}'
if authorization_key and len(authorization_key) > 0:
response = requests.post(f'{API_HOST}/api/json/?authorization_key={authorization_key}', headers=headers, data=data, files=files)
else:
response = requests.post(f'{API_HOST}/api/json/', headers=headers, data=data)
json_response = json.dumps(response.json())
I'm writing an endpoint to receive and parse GitHub Webhook payloads using Django Rest Framework 3. In order to match the payload specification, I'm writing a payload request factory and testing that it's generating valid requests.
However, the problem comes when trying to test the request generated with DRF's Request class. Here's the smallest failing test I could come up with - the problem is that a request generated with DRF's APIRequestFactory seems to not be parsable by DRF's Request class. Is that expected behaviour?
from rest_framework.request import Request
from rest_framework.parsers import JSONParser
from rest_framework.test import APIRequestFactory, APITestCase
class TestRoundtrip(APITestCase):
def test_round_trip(self):
"""
A DRF Request can be loaded into a DRF Request object
"""
request_factory = APIRequestFactory()
request = request_factory.post(
'/',
data={'hello': 'world'},
format='json',
)
result = Request(request, parsers=(JSONParser,))
self.assertEqual(result.data['hello'], 'world')
And the stack trace is:
E
======================================================================
ERROR: A DRF Request can be loaded into a DRF Request object
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/james/active/prlint/venv/lib/python3.4/site-packages/rest_framework/request.py", line 380, in __getattribute__
return getattr(self._request, attr)
AttributeError: 'WSGIRequest' object has no attribute 'data'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/james/active/prlint/prlint/github/tests/test_payload_factories/test_roundtrip.py", line 22, in test_round_trip
self.assertEqual(result.data['hello'], 'world')
File "/home/james/active/prlint/venv/lib/python3.4/site-packages/rest_framework/request.py", line 382, in __getattribute__
six.reraise(info[0], info[1], info[2].tb_next)
File "/home/james/active/prlint/venv/lib/python3.4/site-packages/django/utils/six.py", line 685, in reraise
raise value.with_traceback(tb)
File "/home/james/active/prlint/venv/lib/python3.4/site-packages/rest_framework/request.py", line 186, in data
self._load_data_and_files()
File "/home/james/active/prlint/venv/lib/python3.4/site-packages/rest_framework/request.py", line 246, in _load_data_and_files
self._data, self._files = self._parse()
File "/home/james/active/prlint/venv/lib/python3.4/site-packages/rest_framework/request.py", line 312, in _parse
parsed = parser.parse(stream, media_type, self.parser_context)
File "/home/james/active/prlint/venv/lib/python3.4/site-packages/rest_framework/parsers.py", line 64, in parse
data = stream.read().decode(encoding)
AttributeError: 'str' object has no attribute 'read'
----------------------------------------------------------------------
I'm obviously doing something stupid - I've messed around with encodings... realised that I needed to pass the parsers list to the Request to avoid the UnsupportedMediaType error, and now I'm stuck here.
Should I do something different? Maybe avoid using APIRequestFactory? Or test my built GitHub requests a different way?
More info
GitHub sends a request out to registered webhooks that has a X-GitHub-Event header and therefore in order to test my webhook DRF code I need to be able to emulate this header at test time.
My path to succeeding with this has been to build a custom Request and load a payload using a factory into it. This is my factory code:
def PayloadRequestFactory():
"""
Build a Request, configure it to look like a webhook payload from GitHub.
"""
request_factory = APIRequestFactory()
request = request_factory.post(url, data=PingPayloadFactory())
request.META['HTTP_X_GITHUB_EVENT'] = 'ping'
return request
The issue has arisen because I want to assert that PayloadRequestFactory is generating valid requests for various passed arguments - so I'm trying to parse them and assert their validity but DRF's Request class doesn't seem to be able to achieve this - hence my question with a failing test.
So really my question is - how should I test this PayloadRequestFactory is generating the kind of request that I need?
"Yo dawg, I heard you like Request, cos' you put a Request inside a Request" XD
I'd do it like this:
from rest_framework.test import APIClient
client = APIClient()
response = client.post('/', {'github': 'payload'}, format='json')
self.assertEqual(response.data, {'github': 'payload'})
# ...or assert something was called, etc.
Hope this helps
Looking at the tests for APIRequestFactory in
DRF, stub
views
are created and then run through that view - the output is inspected for expected results.
Therefore a reasonable, but slightly long solution, is to copy this strategy to
assert that the PayloadRequestFactory is building valid requests, before then
pointing that at a full view.
The test above becomes:
from django.conf.urls import url
from django.test import TestCase, override_settings
from rest_framework.decorators import api_view
from rest_framework.response import Response
from rest_framework.test import APIRequestFactory
#api_view(['POST'])
def view(request):
"""
Testing stub view to return Request's data and GitHub event header.
"""
return Response({
'header_github_event': request.META.get('HTTP_X_GITHUB_EVENT', ''),
'request_data': request.data,
})
urlpatterns = [
url(r'^view/$', view),
]
#override_settings(ROOT_URLCONF='github.tests.test_payload_factories.test_roundtrip')
class TestRoundtrip(TestCase):
def test_round_trip(self):
"""
A DRF Request can be loaded via stub view
"""
request_factory = APIRequestFactory()
request = request_factory.post(
'/view/',
data={'hello': 'world'},
format='json',
)
result = view(request)
self.assertEqual(result.data['request_data'], {'hello': 'world'})
self.assertEqual(result.data['header_github_event'], '')
Which passes :D
Apologies if this seems rudimental as I am new to Python. The task I am trying to complete is to send a json object from an iPhone app to a python script that will process a stripe payment. The problem I have is I cannot figure out how to get Python to recognise the incoming json object to extract data from it and pass onto Stripe.
I have taken a step back to simplify the problem. I have a python script that attempts to post a json object with four value pairs to another function that should extract the values, create a new json object and return that object. I cannot get it to work and any help would be greatly appreciated as I've been stuck on this for a while. I am using Flask:
`
import json
import stripe
import smtplib
import requests
from flask import Flask, request, jsonify
#application.route('/run_post')
def run_post():
url = 'http://xxx.pythonanywhere.com/stripetest'
data = {'stripeAmount': '199', 'stripeCurrency': 'USD', 'stripeToken': '122', 'stripeDescription': 'Test post'}
headers = {'Content-Type' : 'application/json'}
r = requests.post(url, data, headers=headers)
#return json.dumps(r.json(), indent=4)
return r.text
#application.route('/stripetest', methods=["POST"])
def stripeTest():
if request.method == "POST":
json_dict = json.loads(request.body.raw)
stripeAmount = json_dict['stripeAmount']
stripeCurrency = json_dict['stripeCurrency']
stripeToken = json_dict['stripeToken']
stripeDescription = json_dict['stripeDescription']
data = "{'stripeAmountRet': " + stripeAmount + ", 'stripeCurrencyRet': " + stripeCurrency + ", 'stripeTokenRet': " + stripeToken + ", 'stripeDescriptionRet': " + stripeDescription + "}"
return jsonify(data)
else:
return """<html><body>
Something went horribly wrong
</body></html>"""
`
I get the following returned in the error log when I run this:
`
2015-03-19 21:07:47,148 :Starting new HTTP connection (1): xxx.pythonanywhere.com
2015-03-19 21:07:47,151 :Exception on /stripetest [POST]
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1687, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1360, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1358, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1344, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/var/www/xxx_pythonanywhere_com_wsgi.py", line 156, in stripeTest
json_dict = json.loads(request.body.raw)
File "/usr/local/lib/python2.7/dist-packages/werkzeug/local.py", line 336, in __getattr__
return getattr(self._get_current_object(), name)
AttributeError: 'Request' object has no attribute 'body'
`
You have a couple of issues with the code. First of all, you need to properly define the json data when you make the request from the requests library. You can do this as follows:
#application.route('/run_post')
def run_post():
url = 'http://xxx.pythonanywhere.com/stripetest'
data = {'stripeAmount': '199', 'stripeCurrency': 'USD', 'stripeToken': '122', 'stripeDescription': 'Test post'}
headers = {'Content-Type' : 'application/json'}
r = requests.post(url, data=json.dumps(data), headers=headers)
#return json.dumps(r.json(), indent=4)
return r.text
Notice that we call json.dumps instead of just passing the data directly. Otherwise, the incoming request is not interpreted as json data.
Next, in your receiving function, we change it as follows:
#application.route('/stripetest', methods=["POST"])
def stripeTest():
if request.method == "POST":
json_dict = request.get_json()
stripeAmount = json_dict['stripeAmount']
stripeCurrency = json_dict['stripeCurrency']
stripeToken = json_dict['stripeToken']
stripeDescription = json_dict['stripeDescription']
data = {'stripeAmountRet': stripeAmount, 'stripeCurrencyRet': stripeCurrency, 'stripeTokenRet': stripeToken, 'stripeDescriptionRet': stripeDescription}
return jsonify(data)
else:
return """<html><body>
Something went horribly wrong
</body></html>"""
A couple of things are changed. First, we read in the data by calling request.get_json(), which properly parses the incoming json data. Note from above that we needed to change how we actually made the request for it to parse the data properly. The next issue was how you returned the data. To properly jsonify the data to return, we put the data into a python dictionary rather than into a string.
If you're calling your function to process the stripe payment from somewhere else (i.e. not using the python requests library), another issue is that you may not be defining the json request properly for Flask to later interpret. If the issue still persists after making the above change to the processing function, post how you're making the json request elsewhere and I can take a look.
Let me know if this fixes your problems!
You should check the document Flask requests
It does not define a body, instead you should try with
request.get_json()
You just need to make sure you are specifying the correct mimetype which would be "application/json".
See request.get_json() method for more info
I am working on my first Flask application. Taking some code directly out of this, I am trying to make sure that a value is present in the user's cookies.
def after_this_request(f):
if not hasattr(g, 'after_request_callbacks'):
g.after_request_callbacks = []
g.after_request_callbacks.append(f)
return f
#app.after_request
def call_after_request_callbacks(response):
for callback in getattr(g, 'after_request_callbacks', ()):
response = callback(response)
return response
#app.before_request
def detect_unique_id():
unique_id = request.cookies.get('unique_id')
if unique_id is None:
unique_id = generate_unique_id()
#after_this_request
def remember_unique_id(response):
response.set_cookie('unique_id', unique_id)
g.unique_id = unique_id
I keep getting this error:
Traceback (most recent call last):
File "/..../env/lib/python2.7/site-packages/flask/app.py", line 1701, in __call__
return self.wsgi_app(environ, start_response)
File "/..../env/lib/python2.7/site-packages/flask/app.py", line 1690, in wsgi_app
return response(environ, start_response)
TypeError: 'NoneType' object is not callable
I am trying to understand the reason for this error. Please help.
The issue
remember_unique_id does not return the response object, but call_after_request_callbacks assigns the result of calling each callback added via the after_this_request decorator to result and then returns it. That is to say:
# This
for callback in getattr(g, 'after_request_callbacks', ()):
response = callback(response)
# translates to this
for callback in [remember_unique_id]:
response = callback(response)
# which translates to this
response = remember_unique_id(response)
# which translates to this
response = None
The solution
Either:
Update remember_unique_id to return the modified response object
Update call_after_request_callbacks to check the returned object and make sure it is not None:
for callback in getattr(g, 'after_request_callbacks', ()):
result = callback(response)
if result is not None:
response = result
Why does this happen?
Flask is a WSGI application under the covers and it expects response to be a WSGI application (that is, a callable object). When it is handling responses from view templates it runs some checks to make sure that it is being something it can use as a response object and if the returned value is not a WSGI application it converts it to one. It does not check that the response object hasn't been altered by the after_request decorators, and so when it tries to call the response object (which it assumes is a valid WSGI application at this point) you get the TypeError.