Is it possible to switch Flask session backends on a per-route basis?
My site stores all sessions in Redis with RedisSessionInterface. Each time a new visitor comes a new session gets written to Redis, hence there's a problem when there're thousands of visitors (real visitors or not) come within a few hours I run out of RAM because Redis eats it.
I really need sessions for internal pages only (/administration_area) accessible by me, so I don't want a session to be started each time a visitor hits the index page (/).
Is it possible to start a new session only on specific routes or can we change the RedisSessionInterface implementation somehow so it won't open a new session on every route?
Related
I'm trying to build a recommendation system with python using lightfm library and an api created with Flask framework.
My question is more design related than coding.
The webservice which will be called when a user logs in the website, recieves a json with userid and return a json with userid and 5 product sku to be recommended.
My desire is to save those recommendations in a DB. I want to do that because in this way I can see and comparing this table with other tables in DB and find out if a user has purchased the product that I recommended.
My concern (maybe it's stupid) is that everything will slow down if I open a connection to DB and write data in it.
Potentially the service can be called between 5k to 7k times per day.
Thanks
What I've understood from your explanation is that you will be comparing the actual selected data by the user and the ones you recommended. So, considering you are comparing every week once, it won't affect much of your processing.
Your concern is, would everything slow down if a DB connection is opened?
It won't slow down the service. Considering the usage of service of 5k times per day, other major factors are there which will slow the service down or will cause it to stop. Like when the number of users is too high, one python process will fail.
What you need to do here is, use a web application server like Gunicorn or uwsgi Using Gunicorn with Flask
This way, what gunicorn does is it starts multiple python processes running flask so it will support a high number of concurrent users.
I have an application (running in Docker and managed by Marathon) where I use server side flask sessions - FileSystemSessionInterface (permanent sessions).
My problem is that if the user waits too long to go to the next step, the session data is lost.
One of my assumptions was that this is because of Marathon, which performs health checks of the application by making an http get request every 2 seconds. This results in a new session file open on every request. And my assumption was that the maximum number of open files is reached. However, when I check in the docker container how many session files are open, the number is not that big, around 350 files.
Did anyone have this problem, any ideas on why my session data disappears?
Basically what I want to do is to regenerate every session with some new set of keys without having users to log in again. How can I do this?
edited for clarity
So let's assume we are using Redis as a backend for sessions and keeping cookies of it on the client-side. Cookie just consists of the session id. This session id corresponds to a session on the Redis. After we have initialized Session by writing Session(APP) in our application, for every request context, we can fetch the session of the current user by
from flask import session
After admin changes some general settings on the application, I am willing to regenerate the session of every current user which can be seen just for the current user by again
from flask import session
This is as far as I know.
For example, let's say there is a value on the user's session determined as
session['arbitrary_key'] = not_important_database_function()
After admin changes some stuff at application, I need to reload a key on the current user's session by
session['arbitrary_key'] = not_important_database_function()
Because after changes admin made, it will yield a different value. After that, I am changing session.modified as true. What I want to learn is how can I change the arbitrary_key on sessions of EVERY USER. Because I am lacking information on how to fetch every session and change them from Flask.
If I delete the sessions from Redis, users are required to reauthenticate. I don't want them to reauthenticate. I just want back-end sessions to be changed because I use some information inside of the user's session which needs to be fetched from Redis so I do not have to call not_important_database_function for every request.
I hope this is enough information for you to at least NOT answer but also NOT downvote so I can continue to seek a solution for my problem.
I am not sharing code snippets because no code snippet is helpful for the case in my opinion.
The question is rather old but it looks like many developers are interested in the answer. There are several approaches which come to mind:
1. Lazy calculations
You need a way to differentiate old and new session values. For example, storing version number in session. Then you can force clients to update their sessions when they are on your resource:
CURRENT_VERSION = '1.2.3'
#app.route('/some_route')
def some_handler():
if session.get('version', '0.0.0') < CURRENT_VERSION:
session['arbitrary_key'] = not_important_database_function()
session['version'] = CURRENT_VERSION
Pros: The method is easy to implement and it is agnostic to the way of storing session data (database, user-agent, etc.)
Cons: Session data is not updated instantly after deploy. You have to give up that some users' session data may not be updated at all.
2. Session storage update
You need to make some kind of a database migration for session storage. It is backend-dependable and will look different for different storages. For Redis it may look like this:
import json
import redis
r = redis.Redis() # Your connection settings here
for key in r.scan_iter():
raw_value = r.get(key)
session = json.loads(raw_value)
session['arbitrary_key'] = not_important_database_function()
r.set(key, json.dumps(session))
Pros: Session data for all users will be updated right after your service deployment.
Cons: The method implementations differ for different storages. It is not applicable if all session data is stored in user-agents.
3. Dropping session data
It is still an option though it is clearly stated in the question that you need to keep the users logged in. It may be a part of the policy of deploying new application versions: session key is regenerated on application deployment and all user sessions are invalidated.
Pros: You don't need to implement any logic to set new user session values.
Cons: There are no new user session values, the data is just wiped out.
Consider a web site that persists on disk the activitiy of its users, without requiring them to log-in/authenticate.
This would enable the user to return and find all their activity intact, even if the server had been relaunched.
Because the user session
from Flask import session
session['foo'] = 'bar'
is an ordinary dict, I'm assuming it gets wiped when the server is stopped and relaunched. It is hence not persistent if the user's two visits cross a server relaunch.
To do so using Flask, we'd use a database session
from flask.ext.sqlalchemy import SQLAlchemy
db = SQLAlchemy()
db.session.add(..)
db.session.commit()
and, since users are not logged-in, we'd distinguish between distinct users through their user sessions.
What is a unique ID that can be extracted from the user session to persist in a database session? The idea is that when a user returns, the cookie in their browser would identify them uniquely, which in turn would mean that the identifier we'd hash from the user session would remain intact.
From the terminology I assume you are using Flask -
So, the db.session does not refer to the "browsing" session, with date related to a single viewer- ratehr, it refers to a db connection session that is unique for that web view cycle (and not for the browsing session) - these are different things.
And as such, the code above do persist your object on the DB in a permanent, unique way. If you can't see them when restarting the Flask application it must be because you are using a transient database. Just adjust your configurations to use a permanent - in disk - database, instead of either an in-memory sqlite instance, or a database inside a docker container that is re-created every time.
The "session" you are thinking of, as "browsing" session does not exist out-of the box in Flask- you either roll yur own, or use one of the plug-ins such as https://pythonhosted.org/Flask-Session/ - this is what is related to PHP's "session".
Scenario:
Major web app w. Python+Flask
Flask login and Flask.session for basic session variables (user-id and session-id)
Flask.session and limitations? (Cookies)
Cookie based and basically persist only at the client side.
For some session variables that will be regularly read (ie, user permissions, custom application config) it feels awkward to carry all that info around in a cookie, at every single page request and response.
Database is too much?
Since the session can be identified at the server side by introducing unique session id at login, some server-side session variable management can be used. Reading this data at the server side from a database also feels like unnecessary overhead.
Question
What is the most efficient way to handle the session variables at the server side?
Perhaps that could be a memory-based solution, but I am worried that different Flask app requests could be executed at different threads that would not share the memory-stored session data, or cause conflicts in case of simultaneous reading-writing.
I am looking for advice and best practice for planning the basic level architecture.
Flask-Caching
What you need is a server-side caching package that's Flask-Caching.
A simple setup:
from flask import Flask
from flask_caching import Cache
app = Flask(__name__)
app.config['CACHE_TYPE'] = 'SimpleCache'
cache = Cache(app)
Then a explicitly use of a cached variable:
#app.route('/')
def load():
cache.set("foo", foo)
bar = cache.get("foo")
There is much more in Flask-Caching and that's the recommended approach by Flask.
In case of a multithread server with gunicorn from here you better use ['CACHE_TYPE'] = 'FileSystemCache'
Your instinct is correct, it's probably not the way to do it.
Session data should only be ephemeral information that is not too troublesome to lose and recreate. For example, the user will just have to login again to restore it.
Configuration data or anything else that's necessary on the server and that must survive a logout is not part of the session and should be stored in a DB.
Now, if you really need to easily keep this information client-side and it's not too much of a problem if it's lost, then use a session cookie for logged in/out state and a permanent cookie with a long lifespan for the rest of the configuration information.
If the information is too much size-wise, then the only option I can think of is to store the data, other than the logged in/out state, in a DB.