Flask cache set method throwing KeyError? - python

In order to cache some data, I am calling cache.set method. However it is throwing KeyError. The error log:
File "D:\Sample_Project\SomeRouter.py", line 176, in run
FetchFeed._prepareCache(f_content, cache, _program, _sprint)
File "D:\Sample_Project\SomeRouter.py", line 197, in _prepareCache
cache.set(cKey, data[last_index:last_index + MAX_DATA_PER_PAGE])
File "c:\python\lib\site-packages\flask_caching\__init__.py", line 254, in set
return self.cache.set(*args, **kwargs)
File "c:\python\lib\site-packages\flask_caching\__init__.py", line 246, in cache
return app.extensions["cache"][self]
KeyError: <flask_caching.Cache object at 0x04F9C8D0>
The server module looks like :
cache_type = 'simple' if 'FLASK_ENV' in os.environ and os.environ['FLASK_ENV'] == 'development' else 'uwsgi'
cache = Cache(config={'CACHE_TYPE': cache_type})
app = Flask("MY_PROJECT")
cache.init_app(app)
# some api.route functions
# goes here ....
if __name__ == "__main__":
with app.app_context():
cache.clear()
app.run(host="0.0.0.0")
And SomeRouter module:
from server import cache
#staticmethod
def _prepareCache(data, cache, program):
total_records = len(data)
if total_records > 0:
cKey = FetchFeed \
._constructCacheKey(program)
cache.set(cKey, data)
else:
print("data size is empty.")
Note: I have removed unnecessary codes.
I also put breakpoints, and called cache.set(some_key, some_value) in server module itself. It is returning True, but same cache object is throwing KeyError when imported and used in SomeRouter module. Can it be that the way I am importing an object is wrong? I also tried importing the cache object right before using it, but it did not work. Any idea what is going on here?

The problem was I was accessing cache object outside request context i.e. in "SomeRouter" module, and because of which it was unaware that under which context it is been used.
In server module where request has been received, the cache knows about the app application, but during cache.set(cKey, data) in SomeRouter module, it throws KeyError. This error is justified as mentioned above.
The resolution is to push the application context as below:
from server import app, cache
# Using context
with app.app_context():
cache.set(cKey, data)
This will push a new application context (using the application of app).
All thanks to Mark Hildreth, for his great answer on context in flask

Related

RuntimeError: Working outside of application context Error in Flask Server

I am trying to use the flask server, but since I faced an error, I started debugging it and by removing many codes to simplify the code and find finally reached to the following error::
Traceback (most recent call last):
File "C:\Code\SportsPersonClassifier\server\server.py", line 18, in <module>
print(classify_image())
File "C:\Code\SportsPersonClassifier\server\server.py", line 10, in classify_image
response = jsonify(util.classify_image(util.get_b64_for_virat()))
File "E:\Users\Acer\anaconda3\lib\site-packages\flask\json\__init__.py", line 358, in jsonify
if current_app.config["JSONIFY_PRETTYPRINT_REGULAR"] or current_app.debug:
File "E:\Users\Acer\anaconda3\lib\site-packages\werkzeug\local.py", line 436, in __get__
obj = instance._get_current_object()
File "E:\Users\Acer\anaconda3\lib\site-packages\werkzeug\local.py", line 565, in _get_current_object
return self.__local() # type: ignore
File "E:\Users\Acer\anaconda3\lib\site-packages\flask\globals.py", line 52, in _find_app
raise RuntimeError(_app_ctx_err_msg)
RuntimeError: Working outside of application context.
This typically means that you attempted to use functionality that needed
to interface with the current application object in some way. To solve
this, set up an application context with app.app_context(). See the
documentation for more information.
This is my code:: in server.py
from flask import Flask, request, jsonify
import util
app = Flask(__name__)
#app.route('/classify_image', methods=['GET', 'POST'])
def classify_image():
response = jsonify(util.classify_image(util.get_b64_for_image()))
return response
if __name__ == "__main__":
print("Starting Python Flask Server For Sports Celebrity Image Classification")
util.load_saved_artifacts()
print(classify_image())
But the exact code works without any error, if I just remove jsonify() from the classify_image() function like this::
def classify_image():
response = util.classify_image(util.get_b64_for_image())
return response
If I write the classify_image function without jsonify it works as expected without error. I tried to solve the problem reading several StackOverflow answers but not working for my code. Please help me solve the problem with jsonify. Thank you.
As the error suggests, and as required by jsonify:
This requires an active request or application context
Calling the function this way via __main__ will not be within a Flask app context. Instead use app.run() to start the dev server, and then navigate to your route.

Flask/Python: How can I render error pages from a directory?

I have a directory in my flask app called error. This directory contains HTML documents named after error codes (without file extensions).
Whenever an Error happens I want to return the corresponding HTML file if it exists and render a template from error/template if it doesn't exist.
I already tried using #app.errorhandler(Exception) but it was never called (I used a print statement to test that).
Try to register the error_handler without a decorator.
from flask import Flask, render_template
from werkzeug.exceptions import HTTPException
def handle_handler(e):
code = 500
if isinstance(e, HTTPException):
code = e.code
return render_template("error.html", error_code=code), code
app = Flask('test')
app.config['TRAP_HTTP_EXCEPTIONS']=True
app.register_error_handler(Exception, default_handler)

How to properly use dask's upload_file() to pass local code to workers

I have functions in a local_code.py file that I would like to pass to workers through dask. I've seen answers to questions on here saying that this can be done using the upload_file() function, but I can't seem to get it working because I'm still getting a ModuleNotFoundError.
The relevant part of the code is as follows.
from dask.distributed import Client
from dask_jobqueue import SLURMCluster
from local_code import *
helper_file = '/absolute/path/to/local_code.py'
def main():
with SLURMCluster(**slurm_params) as cluster:
cluster.scale(n_workers)
with Client(cluster) as client:
client.upload_file(helper_file)
mapping = client.map(myfunc, data)
client.gather(mapping)
if __name__ == '__main__':
main()
Note, myfunc is imported from local_code, and there's no error importing it to map. The function myfunc also depends on other functions that are defined in local_code.
With this code, I'm still getting this error
distributed.protocol.pickle - INFO - Failed to deserialize b'\x80\x04\x95+\x00\x00\x00\x00\x00\x00\x00\x8c\x11local_code\x94\x8c\x$
Traceback (most recent call last):
File "/home/gallagher.r/.local/lib/python3.7/site-packages/distributed/protocol/pickle.py", line 61, in loads
return pickle.loads(x)
ModuleNotFoundError: No module named 'local_code'
Using upload_file() seems so straightforward that I'm not sure what I'm doing wrong. I must have it in the wrong place or not be understanding correctly what is passed to it.
I'd appreciate any help with this. Please let me know if you need any other information or if there's anything else I can supply from the error file.
The upload_file method only uploads the file to the currently available workers. If a worker arrives after you call upload_file then that worker won't have the provided file.
If your situation the easiest thing to do is probably to wait until all of the workers arrive before you call upload file
cluster.scale(n)
with Client(cluster) as client:
client.wait_for_workers(n)
client.upload_file(...)
Another option when you have workers going in/out is to use the Client.register_worker_callbacks to hook into whenever a new worker is registered/added. The one caveat is you will need to serialize your file(s) in the callback partial:
fname = ...
with open(fname, 'rb') as f:
data = f.read()
client.register_worker_callbacks(
setup=functools.partial(
_worker_upload, data=data, fname=fname,
)
)
def _worker_upload(dask_worker, *, data, fname):
dask_worker.loop.add_callback(
callback=dask_worker.upload_file,
comm=None, # not used
filename=fname,
data=data,
load=True)
This will also upload the file the first time the callback is registered so you can avoid calling client.upload_file entirely.

Blueprints in Flask "Attribute 'function' object has no attribute 'name'

Problem Description
Getting error message when trying to start Flask.
Traceback (most recent call last):
File "./run.py", line 3, in <module>
from app import app
File "/home/xxxxxx/xxxx.xxxxxxx.com/ClientTracker/app/__init__.py", line 13, in <module>
app.register_blueprint(admin)
File "/home/xxxxx/xxxxx.xxxxxxx.com/ClientTracker/env/local/lib/python2.7/site-packages/flask/app.py", line 65, in wrapper_func
return f(self, *args, **kwargs)
File "/home/xxxxx/xxxxx.xxxxxxx.com/ClientTracker/env/local/lib/python2.7/site-packages/flask/app.py", line 958, in register_blueprint
if blueprint.name in self.blueprints:
AttributeError: 'function' object has no attribute 'name'
This is a migration from a simpler hierarchy implementing Blueprints. I'm splitting out the function of the frontend and the admin panel.
I built this step by step and had both sides working fine.
Started migrating (functions and routes). After moving some code, I started getting an error message (basically the same as above, but different line).
Troubleshooting
Searched through related posts on stackO
Initially it was complaining about the second route statement I had. I
started removing code (rolling back) to what I (thought was) a known
good state. However I continued to have issues.
I have it down to the minimum code I believe I need but still getting
errors.
If I remove the registration in the init.py file, the frontend works
perfectly.
Code
#ClientTracker/run.py
#!env/bin/python
from app import app
app.run(host='0.0.0.0', port=8080, debug=False)
#ClientTracker/app/__init__.py
# Import flask and template operators
from flask import Flask, render_template
# Define the WSGI application object
app = Flask(__name__)
# Import a module / component using its blueprint handler variable (mod_auth)
#from app.mod_auth.controllers import mod_auth as auth_module
from app.admin.views import admin
from app.client.views import client
# Register blueprint(s)
app.register_blueprint(admin)
app.register_blueprint(client)
#ClientTracker/app/admin/views.py
from flask import render_template, request, Blueprint
from app import app
import MySQLdb
import datetime
admin = Blueprint(
'admin',
__name__,
url_prefix='/admin',
template_folder='templates',
static_folder='static'
)
#admin.route('/')
def admin():
return "ok"
I'm out of ideas.
Ok, so as seems to happen, I spend an hour looking, another 15 mins composing a question and then after I hit post, I find the answer.
I found a post (https://github.com/pallets/flask/issues/1327) that had the answer.
Basically, you cannot have a function name with the same name as your Blueprint name. Seems obvious now, but certainly stumped me for a while.
In thinking about it, my original "working" state had a dummy function name serving the '/'. When I rolled back, I didn't roll back far enough.
Replaced def admin(): with def admin1(): (will fix this better in prod) and all worked.
I hope this post helps someone else. Please still feel free to comment. As always, the group is always smarter than the individual. Lastly, thanks for reading this far. :-)
Your blueprint name is same with your function name, try to rename the function name instead.
Note that the blue print name and the function name can not be the same.
Make use if this tutorial to learn more about Blueprints, https://realpython.com/flask-blueprint/

web.py sessions: AttributeError: 'ThreadedDict' object has no attribute 'count'

I just ran this simple code snippet provided by the wiki, because I couldn't get sessions working:
import web
web.config.debug = False
urls = (
"/count", "count",
"/reset", "reset"
)
app = web.application(urls, locals())
session = web.session.Session(app, web.session.DiskStore('sessions'), initializer={'count': 0})
class count:
def GET(self):
session.count += 1
return str(session.count)
class reset:
def GET(self):
session.kill()
return ""
if __name__ == "__main__":
app.run()
But it results in this error:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/web/application.py", line 237, in process
return self.handle()
File "/usr/local/lib/python2.7/dist-packages/web/application.py", line 228, in handle
return self._delegate(fn, self.fvars, args)
File "/usr/local/lib/python2.7/dist-packages/web/application.py", line 411, in _delegate
return handle_class(cls)
File "/usr/local/lib/python2.7/dist-packages/web/application.py", line 387, in handle_class
return tocall(*args)
File "temp.py", line 12, in GET
session.count += 1
File "/usr/local/lib/python2.7/dist-packages/web/session.py", line 71, in __getattr__
return getattr(self._data, name)
AttributeError: 'ThreadedDict' object has no attribute 'count'
Is webpy not compatible with 2.7.3? I'm running this on the internal webserver of webpy. I'm using Ubuntu 12.04.
session.count += 1 is equal to session.count = session.count + 1 so session.count must exist for this to work.
Add the following check to make it work:
if 'count' not in session:
session.count = 0
session.count += 1
There is also another way which is even shown in the very simple session simple example of the docs:
try:
s.click += 1
except AttributeError:
s.click = 1
Ok for the trick of try...except also i'm not convinced it's the best way to do so (not clean at all).
Like previously said, Session constructor offer a way to initialize the session's variable.
I'm not really sure we can rely on the "very simple session simple example".
First, we hardly have any explication on the purpose of various variablse. For instance, what is the purpose the *db_parameter* dict ?
Last but not least, it needs a serious update. The provided code simply didn't work with the actual framework.
There is simply no web.ctx.session.
By the way, I implemented a simple counter like in the example.
The displayed error you had is due to a drastic change in the Session's API.
You cannot just call "counter" from your session.
That would be more smthg like that: session.store.store_instance.get('counter') .Where store_instance is either a shelf or a db.
Like i said, the official documentation needs a serious update.
That said, I noticed that this is not the same for the docstring. To progress i start Ipython and i see every posibilities I have.
I know it's pure guessing but naming is good so we can figure out what to do.
I will submit my example to the team of web.py so they can update the official doc.
Until my pull request get accepted on GitHub, i post the snippet of code illustrating the use of a simple incrementor:
import web
import shelve
urls = (
'/add', 'counter',
'/reset', 'reset'
)
shelf = shelve.open('session')
shelfStore = web.session.ShelfStore(shelf)
app = web.application(urls, globals())
s = web.session.Session(app, shelfStore)
class counter:
def GET(self):
numberToAdd = web.input().get('number')
if not numberToAdd:
numberToAdd = 1
try:
print numberToAdd
s.store.shelf["count"] += int(numberToAdd)
except Exception:
s.store.shelf["count"] = 1
return s.store.shelf.get("count")
class reset:
def GET(self):
s.store.shelf.clear()
if __name__ == "__main__":
app.run()
The problem is the python version. I had the same problem and I solved it when I executed 2.7 version of python. Just doing > python2.7 code.py and the sessions works perfectly.
It's a pity that the doc to web.py is very poor.

Categories