I am working on my first Flask application. Taking some code directly out of this, I am trying to make sure that a value is present in the user's cookies.
def after_this_request(f):
if not hasattr(g, 'after_request_callbacks'):
g.after_request_callbacks = []
g.after_request_callbacks.append(f)
return f
#app.after_request
def call_after_request_callbacks(response):
for callback in getattr(g, 'after_request_callbacks', ()):
response = callback(response)
return response
#app.before_request
def detect_unique_id():
unique_id = request.cookies.get('unique_id')
if unique_id is None:
unique_id = generate_unique_id()
#after_this_request
def remember_unique_id(response):
response.set_cookie('unique_id', unique_id)
g.unique_id = unique_id
I keep getting this error:
Traceback (most recent call last):
File "/..../env/lib/python2.7/site-packages/flask/app.py", line 1701, in __call__
return self.wsgi_app(environ, start_response)
File "/..../env/lib/python2.7/site-packages/flask/app.py", line 1690, in wsgi_app
return response(environ, start_response)
TypeError: 'NoneType' object is not callable
I am trying to understand the reason for this error. Please help.
The issue
remember_unique_id does not return the response object, but call_after_request_callbacks assigns the result of calling each callback added via the after_this_request decorator to result and then returns it. That is to say:
# This
for callback in getattr(g, 'after_request_callbacks', ()):
response = callback(response)
# translates to this
for callback in [remember_unique_id]:
response = callback(response)
# which translates to this
response = remember_unique_id(response)
# which translates to this
response = None
The solution
Either:
Update remember_unique_id to return the modified response object
Update call_after_request_callbacks to check the returned object and make sure it is not None:
for callback in getattr(g, 'after_request_callbacks', ()):
result = callback(response)
if result is not None:
response = result
Why does this happen?
Flask is a WSGI application under the covers and it expects response to be a WSGI application (that is, a callable object). When it is handling responses from view templates it runs some checks to make sure that it is being something it can use as a response object and if the returned value is not a WSGI application it converts it to one. It does not check that the response object hasn't been altered by the after_request decorators, and so when it tries to call the response object (which it assumes is a valid WSGI application at this point) you get the TypeError.
Related
I am trying to create a python api that works on the input data and run different scripts according to the provided input. I was using flask + gunicorn earlier with 8 workers. I got to know about asyncio library and I switched from flask to quart and gunicorn to hypercorn using the following [guide][1].
My functions look like :
#app.route("/pyscript", methods=["POST"])
##cross_origin(supports_credentials=True)
async def pyscript():
#check for pre defined token
try:
if(token == consts.authToken):
print("TOKEN VALIDATED")
else:
return jsonify({'message': 'token is invalid'})
except:
return jsonify({'message': 'token is invalid'})
json_all_data = await request.get_json(force=True)
print(json_all_data)
json_data = json_all_data['data']
json_data = json.dumps(json_data)
if json_data is None:
return "Bad request, invalid JSON Format inserted", 401
json_LOC = (json_all_data['LOC'])
json_LOI = (json_all_data['LOB'])
try:
if(json_LOC in keys and json_LOI in keys[json_LOC]):
chosen_operation_function = carriers.get(json_LOC).get(json_LOI)
return_UUID = uuid.uuid4()
app.add_background_task(run_task(json_data, json_LOC, json_LOI, chosen_operation_function, return_UUID))
out_int = {
"job_id" : return_UUID,
"status" : "Running",
"input" : json.loads(json_data)
}
return out_int, 202
else:
return jsonify({'message': 'LOC/LOB is invalid'})
except Exception as e:
print(e)
return e
run_task is a function in other file that is an async defined function.
Expected Output : I wanted to process the run_task in background while returning 202 response and a json data associated with it.
Whats happening : The value is returned in the postman request but the server shows the following error :
Traceback (most recent call last):
File "/home/user/.local/lib/python3.8/site-packages/quart/app.py", line 1396, in _wrapper
await copy_current_app_context(func)(*args, **kwargs)
File "/home/user/.local/lib/python3.8/site-packages/quart/ctx.py", line 333, in wrapper
return await app_context.app.ensure_async(func)(*args, **kwargs)
File "/home/user/.local/lib/python3.8/site-packages/quart/utils.py", line 55, in _wrapper
None, copy_context().run, partial(func, *args, **kwargs)
TypeError: the first argument must be callable
I am not able to identify the error and any help would be appreciated.
Thanks
[1]: https://pgjones.gitlab.io/quart/how_to_guides/flask_migration.html
I think you are calling add_background_task incorrectly, rather than,
app.add_background_task(run_task(json_data, json_LOC, json_LOI, chosen_operation_function, return_UUID))
It should be called as,
app.add_background_task(run_task, json_data, json_LOC, json_LOI, chosen_operation_function, return_UUID)
Note that run_task is not called in the second example in your code, instead it and the arguments to call it are passed to the add_background_task function.
When using a Scrapy Downloader Middleware, and you don't find what you need. Do you build a Response object and return that or return the responsevariable passed in with process_response?
I tried the latter but kept getting response has no attribute selector when used with FilesPipeline.
class CaptchaMiddleware(object):
def process_response(self, request, response, spider):
download_path = spider.settings['CAPTCHA_STORE']
# 1
captcha_images = parse_xpath(response, CAPTCHA_PATTERN, 'image')
if captcha_images:
for url in captcha_images:
url = response.urljoin(url)
print("Downloading %s" % url)
download_file(url, os.path.join(download_path, url.split('/')[-1]))
for image in os.listdir(download_path):
Image.open(image)
# 2
return response
If I return at #1, the FilesPipeline runs properly and download the files but if I return at #2, it returns an error response has no attribute selector
From the docs:
process_response(request, response, spider) process_response() should
either: return a Response object, return a Request object or raise a
IgnoreRequest exception.
If it returns a Response (it could be the same given response, or a
brand-new one), that response will continue to be processed with the
process_response() of the next middleware in the chain.
If it returns a Request object, the middleware chain is halted and the
returned request is rescheduled to be downloaded in the future. This
is the same behavior as if a request is returned from
process_request().
If it raises an IgnoreRequest exception, the errback function of the
request (Request.errback) is called. If no code handles the raised
exception, it is ignored and not logged (unlike other exceptions).
From the docs at https://doc.scrapy.org/en/latest/topics/request-response.html#textresponse-objects:
TextResponse objects adds encoding capabilities to the base Response
class, which is meant to be used only for binary data, such as images,
sounds or any media file.
Bare Response objects do not have a selector attribute, TextResponse and subclasses do:
In [1]: from scrapy.http import Response, TextResponse
In [2]: Response('http://example.org', body=b'<html><body><div>Something</div></body></html>').selector
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-2-6fdd116632d2> in <module>
----> 1 Response('http://example.org', body=b'<html><body><div>Something</div></body></html>').selector
AttributeError: 'Response' object has no attribute 'selector'
In [3]: TextResponse('http://example.org', body=b'<html><body><div>Something</div></body></html>').selector
Out[3]: <Selector xpath=None data='<html><body><div>Something</div></body><'>
I don't see a new response being created in the code, but from the beginning of the question ("Do you build a Response object and return that (...)") I suspect the snippet might be incomplete, and the response returned at #2 could be a manually created Response.
In a Django project of mine, I've written simple webhook logic that pings a url for certain payload if some condition is met. Here's the code:
#csrf_exempt
def webhook_event(request,*args,**kwargs):
if request.method == 'POST':
data = json.loads(request.body)
event_type = data['event']
# use CDN URL from webhook payload
if event_type == 'project.datafile_updated':
url = data['data']['cdn_url']
config_manager.set_obj(url)
return json.dumps({'success':True}), 200, {'ContentType':'application/json'}
return json.dumps({'success':False}), 400, {'ContentType':'application/json'}
else:
return render(request,"404.html",{})
I'm using the following to test it:
import requests
import json
# Update project_id
project_id = "<some_id>"
data = {"timestamp": 1463602412, "project_id": project_id, "data": {"cdn_url": "https://cdn.example.com/json/{0}.json".format(project_id), "origin_url": "https://example.s3.amazonaws.com/json/{0}.json".format(project_id), "revision": 15}, "event": "project.datafile_updated"}
r = requests.post("http://127.0.0.1:8000/ohook/", data=json.dumps(data), headers={'Content-Type': 'application/json'})
print r
It all works perfectly, but the tuple returned by webhook_event is giving me the following error:
Internal Server Error: /ohook/
Traceback (most recent call last):
File "/home/hassan/.virtualenvs/myenv/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 187, in get_response
response = middleware_method(request, response)
File "/home/hassan/.virtualenvs/myenv/local/lib/python2.7/site-packages/newrelic-2.56.0.42/newrelic/hooks/framework_django.py", line 328, in wrapper
return wrapped(*args, **kwargs)
File "/home/hassan/.virtualenvs/myenv/local/lib/python2.7/site-packages/django/middleware/clickjacking.py", line 30, in process_response
if response.get('X-Frame-Options', None) is not None:
AttributeError: 'tuple' object has no attribute 'get'
Can anyone help me diagnose this?
As the error says, you're returning a tuple: the JSON, an integer (presumably the status code) and a dict (presumably the headers).
Even if you fixed this, you can't just return JSON from a view; you have to return an instance of HttpResponse or one of its subclasses.
Instead of doing return json.dumps(...) in your POST blocks, you should use JsonResponse. This accepts a Python datastructure and returns a response containing the serialized JSON; as a bonus, it also sets the content-type appropriately.
if event_type == 'project.datafile_updated':
...
return JsonResponse({'success':True})
return JsonResponse({'success':False}, status=400)
(Note that render is explicitly a shortcut which renders a template and returns it as an HttpResponse; this isn't the case with json.dumps().)
I edited my question, because I realized I was combining two concepts (WSGIHandler and Middleware)
which I shouldn't have combined in this question.
I'm trying to get some basic understanding of what the WSGIHandler does.
django.core.handlers.wsgi.py
class WSGIHandler(base.BaseHandler):
initLock = Lock()
request_class = WSGIRequest
def __call__(self, environ, start_response):
#...
start_response(status, response_headers)
return response
An instance of WSGIHandler will be called with environ and start_response arguments.
environ will be used to create request object, which is an instance of WSGIRequest.
request = self.request_class(environ)
WSGIHandler.get_response will be used to create an HttpResponse object from request.
response = self.get_response(request)
My questions:
What does start_response do? By this time a response object has already been created. So what response does it start? Does it send some response somewhere? And where does it send the response sent by return response (i.e. what is it normally that calls it?)?
When you write a web app, you need to provide a callable to the server which the server will call when a request arrives. This callable is responsible for generating a response and sending the response back to server. The server in turns sends this response back to the browser.
In case of django, that callable is WSGIHandler. Since it has a __call__ defined, so this __call__ will be called by the server.
The arguments sent to __call__ are coming from server. start_response is also a callable which is implemented by the server. You do not need to worry about its implementation. So, __call__ of WSGIHandler calls this callable start_response.
start_response internally creates the header which later has to be sent back to the browser. But it only sets some attribute on the server, it doesn't sent the response back to the browser. It doesn't send any response anywhere.
After calling start_response, wsgi handler returns the actual response in line return response. When this line is executed, the server checks if the response is proper and then based on the response either sends the cookies as set in start_response call or raises some exeption.
Check http://www.python.org/dev/peps/pep-0333/ for more info.
For some reason I'm getting the error:
TypeError: search_products() takes at least 2 arguments (2 given)
The weird thing is that I make the same API call in two separate places -- one in a function which I placed in one of the Model classes. The other one in the page View. The one in the models class works fine, whereas the one in the View function is throwing back the error.
The following is my code in Views.py:
searchproducts=api.API().search_products(query="myproduct")
Again, when I write this exact same code in Models.py, all works fine.
My search_products function in the API class in api.py is as follows:
def search_products(self, category_id, query="", start=0, limit=10, filter=None, ranged_filters=None, sort_by=None):
How can I dig deeper to find the root of why this is happening?
Traceback:
/Users/me/Desktop/myenv2/lib/python2.7/site-packages/django/core/handlers/base.py in get_response
# Apply view middleware
for middleware_method in self._view_middleware:
response = middleware_method(request, callback, callback_args, callback_kwargs)
if response:
return response
try:
response = callback(request, *callback_args, **callback_kwargs) ...
except Exception, e:
# If the view raised an exception, run it through exception
# middleware, and if the exception middleware returns a
# response, use that. Otherwise, reraise the exception.
for middleware_method in self._exception_middleware:
response = middleware_method(request, e)
In your definition of search_products you have category_id as a required field, and you are not providing that as an argument when you are calling the method. Provide a default for category_id or pass in the appropriate argument to resolve your issue