We Keep Coding

Flask & Flask-PyMongo "UserWarning: MongoClient opened before f
ork."

I am working on a Flask application with a MongoDB database.
I am using that database for my core application data as well as for Flask-Dance token storage (https://flask-dance.readthedocs.io/en/latest/storages.html). The code for the custom storage backend is in token_storages.py, below.
Everything is working perfectly in the Flask development server, but when I try running my prod server via uWSGI I get this error:
UserWarning: MongoClient opened before f
ork. Create MongoClient only after forking. See PyMongo's documentation for details: https://pymongo.readthedocs.io/en/stable/faq.html#is-
pymongo-fork-safeUserWarning: MongoClient opened before f
ork. Create MongoClient only after forking. See PyMongo's documentation for details: https://pymongo.readthedocs.io/en/stable/faq.html#is-
pymongo-fork-safe
All the research I have seen refers to setting connect=False when instantiating PyMongo, but this is having no effect. Per the docs, this setting should be default regardless (https://flask-pymongo.readthedocs.io/en/latest/).
The error doesn't actually show me on what line things are going wrong -- any guidance on how to find what it PyMongo regards as errant would be much appreciated.
All the database calls that I can find are made within the Flask routes, which I believe should work fine (see the example from storyboard_routes.py below.
I believe the below code should be enough to provide details on the app construction but please let me know if anything else should be included.
Finally I did find that setting lazy-apps = true in my uWSGI settings does work around this issue (in wsgi.ini) but I would rather solve the root issue if possible.
Thanks in advance!
This was suggested, but at least today it is inaccurate as connect=true is not the current default. I tried it nevertheless without success.
MongoClient opened before fork. Create MongoClient only Flask
Related Versions
Python 3.8.10
pymongo 3.12.0
Flask-PyMongo 2.3.0
Flask 2.0.1

Flask-Dance 5.0.0
my_app/init.py
from flask import Flask
from flask_session import Session # https://pythonhosted.org/Flask-Session
from . import app_config
from .util import misc
from .routes import graph_auth_routes
from .routes import google_auth_routes
from .routes import storyboard_routes
from .db import init_db
def create_app():
app = Flask(__name__)
app.config.from_object(app_config)
azure_oauth = misc.create_azure_dance_blueprint(app)
google_oauth = misc.create_google_dance_blueprint(app)
app.config['MONGO_CONNECT'] = False # This has no effect since already the default
init_db(app)
Session(app)
app.register_blueprint(graph_auth_routes.bp)
app.register_blueprint(google_auth_routes.bp)
app.register_blueprint(storyboard_routes.bp)
return app
app=create_app()
my_app/db.py
from flask_pymongo import PyMongo
mongo = PyMongo()
def init_db(app):
print("Initializing DB")
mongo.init_app(app)
return app
my_app/util/token_storages.py
from flask import session
from flask_dance.consumer.storage import BaseStorage
from ..db import mongo
class MongoStorage(BaseStorage):
def __init__(self, oauth_provider):
self.oauth_provider = oauth_provider
def get(self, blueprint):
print("getting token")
try:
token = mongo.db["tokens"].find_one(
{
"contact_id": session["contact_id"],
"oauth_provider": self.oauth_provider,
}
)["oauth"]["token"]
return token
except:
print("can't find token for %s" % self.oauth_provider)
return None
def set(self, blueprint, token):
print("setting token")
query = mongo.db["tokens"].update_one(
{
"contact_id": session["contact_id"],
"oauth_provider": self.oauth_provider,
},
{"$set": {"oauth.token": token, "oauth_provider": self.oauth_provider}},
upsert=True,
)
def delete(self, blueprint):
print("deleting token")
mongo.db["tokens"].delete_one(
{
"contact_id": session["contact_id"],
"oauth_provider": self.oauth_provider,
}
)
return None
my_pp/routes/storyboard_routes.py
#bp.route("/page2_365")
#misc.default_error_handler
def page2_365():
# Update if Google integration was performed
if request.args.get("gc_integration"):
filter = {"_id": session["contact_id"]}
new_value = {
"$set": {"gc_integration": misc.str2bool(request.args["gc_integration"])}
}
mongo.db["contacts"].update_one(filter, new_value)
wsgi.ini
[uwsgi]
module = lead_wizard:app
master = true
processes = 5
workers=1
socket = /tmp/lead_wizard.sock
chmod-socket = 666
vacuum = true
die-on-term = true

How to use pymongo.monitoring in a Flask project with mongoengine

I am trying to add some monitoring to a simple REST web service with flask and mongoengine and have come across what I think is a lack of understanding on my part of how imports and mongoengine is working in flask applications.
I'm following pymongo's documentation on monitoring : https://pymongo.readthedocs.io/en/3.7.2/api/pymongo/monitoring.html
I defined the following CommandListener in a separate file:
import logging
from pymongo import monitoring
log = logging.getLogger('my_logger')
class CommandLogger(monitoring.CommandListener):
def started(self, event):
log.debug("Command {0.command_name} with request id "
"{0.request_id} started on server "
"{0.connection_id}".format(event))
monitoring.register(CommandLogger())
I made an application_builder.py file to create my flask App, code looks something like this:
from flask_restful import Api
from flask import Flask
from command_logger import CommandLogger # <----
from db import initialize_db
from routes import initialize_routes
def create_app():
app = Flask(__name__)
api = Api(app)
initialize_db(app)
initialize_routes(api)
return app
The monitoring only seems to works if I import : CommandLogger in application_builder.py. I'd like to understand what is going on here, how does the import affect the monitoring registration?
Also I'd like to extract monitoring.register(CommandLogger()) as a function and call it at a latter stage in my code something like def register(): monitoring.register(CommandLogger())
But this doesn't seem to work, "registration' only works when it is in the same file as the CommandLogger class...

From the MongoEngine's doc, it seems important that the listener gets registered before connecting mongoengine
To use pymongo.monitoring with MongoEngine, you need to make sure that
you are registering the listeners before establishing the database
connection (i.e calling connect)

This worked for me. I'm just initializing/registering it the same way as I did other modules to avoid circular imports.
# admin/logger.py
import logging
from pymongo import monitoring
log = logging.getLogger()
log.setLevel(logging.DEBUG)
logging.basicConfig(level=logging.DEBUG)
class CommandLogger(monitoring.CommandListener):
# def methods...
class ServerLogger(monitoring.ServerListener):
# def methods
class HeartbeatLogger(monitoring.ServerHeartbeatListener):
# def methods
def initialize_logger():
monitoring.register(CommandLogger())
monitoring.register(ServerLogger())
monitoring.register(HeartbeatLogger())
monitoring.register(TopologyLogger())
# /app.py
from flask import Flask
from admin.toolbar import initialize_debugtoolbar
from admin.admin import initialize_admin
from admin.views import initialize_views
from admin.logger import initialize_logger
from database.db import initialize_db
from flask_restful import Api
from resources.errors import errors
app = Flask(__name__)
# imports requiring app
from resources.routes import initialize_routes
api = Api(app, errors=errors)
# Logger before db
initialize_logger()
# Database and Routes
initialize_db(app)
initialize_routes(api)
# Admin and Development
initialize_admin(app)
initialize_views()
initialize_debugtoolbar(app)
# /run.py
from app import app
app.run(debug=True)
then in any module...
from admin.logger import log
from db.models import User
# inside some class/view/queryset or however your objects are written...
log.info('Saving an item through MongoEngine...')
User(name='Foo').save()
What I'm trying to figure out now is how to integrate Flask DebuggerToolbar's Logging panel with the monitoring messages from these listeners...

Service Connection via Python

I am newbiee on pyhton development and trying to make the following example work.
The following code is for service authentication. Whenever I call localhost:3000/callback through browser, I am receiving code error because it is null.
When I create a webpage on auth0, then it puts all required information and then make the source code available to use. However, code is not included. I wonder what needs to inserted.
token = get_token.authorization_code(AUTH0_CLIENT_ID,
AUTH0_CLIENT_SECRET, code,
AUTH0_CALLBACK_URL)
auth0.v3.exceptions.Auth0Error: invalid_request: Missing required
parameter: code
.env
AUTH0_CLIENT_ID=xxxxxxxxxxx
AUTH0_DOMAIN=xxxxxx.auth0.com
AUTH0_CLIENT_SECRET=v-xxxxx2yVHntpn01RoEMMxhj6RLxxxxxxxxxx
AUTH0_CALLBACK_URL=http://localhost:3000/callback
API_IDENTIFIER={API_AUDIENCE}
server.py
"""Python Flask WebApp Auth0 integration example"""
from functools import wraps
from urllib.parse import urlparse
from os import environ as env, path
import json
from auth0.v3.authentication import GetToken
from auth0.v3.authentication import Users
from dotenv import load_dotenv
from flask import Flask
from flask import redirect
from flask import render_template
from flask import request
from flask import send_from_directory
from flask import session
import constants
load_dotenv(path.join(path.dirname(__file__), ".env"))
AUTH0_CALLBACK_URL = env[constants.AUTH0_CALLBACK_URL]
AUTH0_CLIENT_ID = env[constants.AUTH0_CLIENT_ID]
AUTH0_CLIENT_SECRET = env[constants.AUTH0_CLIENT_SECRET]
AUTH0_DOMAIN = env[constants.AUTH0_DOMAIN]
#APP.route('/callback')
def callback_handling():
code = request.args.get(constants.CODE_KEY)
get_token = GetToken(AUTH0_DOMAIN)
auth0_users = Users(AUTH0_DOMAIN)
#Receive exception
token = get_token.authorization_code(AUTH0_CLIENT_ID,
AUTH0_CLIENT_SECRET, code, AUTH0_CALLBACK_URL)
user_info = auth0_users.userinfo(token['access_token'])
session[constants.PROFILE_KEY] = json.loads(user_info)
return redirect('/dashboard')
if __name__ == "__main__":
APP.run(host='0.0.0.0', port=env.get('PORT', 3000))

Difficulty implementing server side session storage using redis and flask

I have a setup where a node.js app is making ajax requests to a flask based python server. Since ajax requests lack cookie data, I can't use the simple flask session object to persist data across requests. To remedy this, I'd like to implement a redis based server side implementation of a session storage system, but the solutions I've found so far do not work.
One solution I've tried is the following this snippet.
But this doesn't work. Is there more setup I need to do to configure redis beyond what is mentioned in the quickstart guide? Here is my attempt:
...
from flask import session
# Snippet code is copy pasted here verbatum
import session_interface
...
app = Flask(__name__)
app.session_interface = session_interface.RedisSessionInterface()
...
# Can't access this as session['key'] across requests
session['key'] = value
...
if __name__ == '__main__':
app.secret_key = '123456789012345678901234'
app.run(debug=True)
Another solution I've tried is importing the Flask-Session extention.
However, I can't get this to work either. The section I'm confused about is the following:
"We are not supplying something like SESSION_REDIS_HOST and SESSION_REDIS_PORT, if you want to use the RedisSessionInterface, you should configure SESSION_REDIS to your own redis.Redis instance. This gives you more flexibility, like maybe you want to use the same redis.Redis instance for cache purpose too, then you do not need to keep two redis.Redis instance in the same process."
What is meant by this section and how would I have figured this out? Here is my attempt to make this extension work:
...
from flask import session
from flask_session import Session
import redis
...
app = Flask(__name__)
SESSION_TYPE = 'redis'
app.config.from_object(__name__)
Session(app)
...
# Can't access this as session['key'] across requests
session['key'] = value
...
if __name__ == '__main__':
app.secret_key = '123456789012345678901234'
app.run(debug=True)
Has anyone successfully implemented manual session storage on a server running flask? Are there other options for getting this functionality?
Thanks for your input.

I think that's because you missed the URL configuration for your storage Redis, to check that, you can use Redis-CLI to see if there is anything being inserted into Redis.
I use this code and it worked:
from flask import Flask
from flask_session import Session
import redis
……
app = Flask(__name__)
app.config['SESSION_TYPE'] = 'redis'
app.config['SESSION_REDIS'] = redis.from_url('127.0.0.1:6379')
sess = Session()
sess.init_app(app)
def getSession():
return session.get('key', 'not set')
def setSession():
session.set('key')=123
return 'ok'
……

The following works for me:
...
from flask_session import Session
import redis
...
app = Flask(__name__)
SECRET_KEY = '123456789012345678901234'
SESSION_TYPE = 'redis'
SESSION_REDIS = redis.from_url('localhost:6379')
app.config.from_object(__name__)
sess = Session()
sess.init_app(app)
...
# Should be available accross requests now
session['key'] = value
...
if __name__ == '__main__':
app.run(debug=True)
Using sess.init_app(app) instead of Session(app) did the trick.

flask context inside context , for jsonfy, all in a single page

I'm having a single test for retrieve documents in a single page, i know it's not correct to do in a single page; but it's just to understand all this work like pure script, not for an api restful.
My problem is when i use:
print (jsonify({'result' : output}))
i've get this error:
RuntimeError: Working outside of request context.
This typically means that you attempted to use functionality that needed
an active HTTP request. Consult the documentation on testing for
information about how to avoid this problem.
when I replace this line by
print ( output)
have no erros and have the documents.
How i can to specify a context for jsonify ? inside another context ? because i'm already using
with app.app_context():
Here the code:
from flask import Flask
from flask import g
from flask import jsonify
from flask import request
from flask_pymongo import PyMongo
from flask import make_response
from bson.objectid import ObjectId
from flask import current_app
import sys
app = Flask(__name__)
app.config['MONGO_DBNAME'] = 'restdb'
app.config['MONGO_URI'] = 'mongodb://localhost:27017/crm1'
#app.errorhandler(404)
def not_found(error):
return make_response(jsonify({'error':'Notfound' }),404)
with app.app_context():
mongo = PyMongo(app)
star = mongo.db.accounts
output = []
for s in star.find():
output.append({'id': str(s['_id']) ,'firstname' : s['firstname'], 'lastname' : s['lastname']})
print (jsonify({'result' : output}))
#print ( output)
if __name__ == '__main__':
app.run(debug=True)

Jsonify Works with HttpResponse.
You can use python json module and print the output
Like:
import json
print(json.dumps(output))