We Keep Coding

Flask, Python and Socket.io: getting “RuntimeError: working outside of request context” when disconnecting from a threading.Timer

I'm trying to write a simple server that disconnects a user after a min of inactivity.
ive found a simple way of doing it with threading.Timer (restarting the timer every time there is an activity).
im getting RuntimeError when using disconnect in a Timer.
tried using app.app_context and app.test_request_context but either I don't know how and where to use them or it simply doesn't work.
server code:
from flask import Flask, request
from flask_socketio import SocketIO, emit, disconnect
from threading import Timer
app = Flask(__name__)
app.config['SECRET_KEY'] = 'secret!'
sio = SocketIO(app)
clients = {}
class Client:
def __init__(self, user, sid, client_time):
self.user = user
self.sid = sid
self.client_time = client_time
self.activity_timer = Timer(10, self.disc_after_60)
self.start_timer()
def disc_after_60(self):
disconnect(self.sid)
del clients[self.user]
def start_timer(self):
if self.activity_timer.is_alive():
self.activity_timer.cancel()
self.activity_timer.start()
else:
self.activity_timer.start()
#sio.on('register')
def handle_register(client_user, client_time):
clients[client_user] = Client(client_user, request.sid, client_time)
emit('message', ("SERVER", f"{client_user} has joined the server!"), broadcast=True)
client side I just connect using register.
the full error message:
Exception in thread Thread-8:
Traceback (most recent call last):
File "C:\Users\idshi\AppData\Local\Programs\Python\Python38-32\lib\threading.py", line 932, in _bootstrap_inner
self.run()
File "C:\Users\idshi\AppData\Local\Programs\Python\Python38-32\lib\threading.py", line 1254, in run
self.function(*self.args, **self.kwargs)
File "C:\Users\idshi\PycharmProjects\PyChat excersize\Server\fsserver.py", line 24, in disc_after_60
disconnect(self.sid)
File "C:\Users\idshi\AppData\Local\Programs\Python\Python38-32\lib\site-packages\flask_socketio\__init__.py", line 919, in disconnect
socketio = flask.current_app.extensions['socketio']
File "C:\Users\idshi\AppData\Local\Programs\Python\Python38-32\lib\site-packages\werkzeug\local.py", line 348, in __getattr__
return getattr(self._get_current_object(), name)
File "C:\Users\idshi\AppData\Local\Programs\Python\Python38-32\lib\site-packages\werkzeug\local.py", line 307, in _get_current_object
return self.__local()
File "C:\Users\idshi\AppData\Local\Programs\Python\Python38-32\lib\site-packages\flask\globals.py", line 52, in _find_app
raise RuntimeError(_app_ctx_err_msg)
RuntimeError: Working outside of application context.
This typically means that you attempted to use functionality that needed
to interface with the current application object in some way. To solve
this, set up an application context with app.app_context(). See the
documentation for more information.
I would be glad if someone can help me with this. Thanks in advance.

The disconnect() function needs to be called with an application context installed, as that's the only way to know what's the application instance.
Try this:
def disc_after_60(self):
with app.app_context():
disconnect(sid=self.sid, namespace='/')
del clients[self.user]

Synchronous fetch in Tornado

In Tornado's official documentation (https://www.tornadoweb.org/en/stable/guide/async.html) there is an example of synchronous fetch, but when I put it in get method of my IndexHandler it returns following error:
File "/home/stefan/.local/lib/python3.6/site-packages/tornado/web.py", line 1697, in _execute
result = method(*self.path_args, **self.path_kwargs)
File "ex1.py", line 19, in get
client = tornado.httpclient.HTTPClient()
File "/home/stefan/.local/lib/python3.6/site-packages/tornado/httpclient.py", line 107, in __init__
self._async_client = self._io_loop.run_sync(make_client)
File "/home/stefan/.local/lib/python3.6/site-packages/tornado/ioloop.py", line 526, in run_sync
self.start()
File "/home/stefan/.local/lib/python3.6/site-packages/tornado/platform/asyncio.py", line 148, in start
self.asyncio_loop.run_forever()
File "/usr/lib/python3.6/asyncio/base_events.py", line 428, in run_forever
'Cannot run the event loop while another loop is running')
RuntimeError: Cannot run the event loop while another loop is running
My code looks like:
import tornado.httpserver
import tornado.ioloop
import tornado.options
import tornado.web
import tornado.httpclient
from tornado.options import define, options
define("port", default=8000, help="run on the given port", type=int)
class IndexHandler(tornado.web.RequestHandler):
def get(self):
client = tornado.httpclient.HTTPClient()
response = client.fetch('https://www.google.com')
print(response.body)
if __name__ == "__main__":
tornado.options.parse_command_line()
app=tornado.web.Application(handlers=[(r"/", IndexHandler)],debug=True)
http_server = tornado.httpserver.HTTPServer(app)
http_server.listen(options.port)
tornado.ioloop.IOLoop.instance().start()
I believe I should make changes in main in order to make it work.

Because as you can see in your traceback, a "synchronous" fetch starts it's own ioloop under the hood. The easiest to fix is to rewrite your def get as a coroutine (either as async def get or with a #gen.coroutine decorator) and use asynchronous fetch
class IndexHandler(tornado.web.RequestHandler):
async def get(self):
client = tornado.httpclient.HTTPClient()
response = await client.fetch('https://www.google.com')
print(response.body)

python-logstash error thrown while running locally

If I import logstash, running locally I get the following error
Connected to pydev debugger (build 162.1812.1)
/home/vagrant/.envs/emailservice/lib/python3.4/site-packages/flask/exthook.py:71: ExtDeprecationWarning: Importing flask.ext.cache is deprecated, use flask_cache instead.
.format(x=modname), ExtDeprecationWarning
Traceback (most recent call last):
File "/home/vagrant/.pycharm_helpers/pydev/pydevd.py", line 1580, in <module>
globals = debugger.run(setup['file'], None, None, is_module)
File "/home/vagrant/.pycharm_helpers/pydev/pydevd.py", line 964, in run
pydev_imports.execfile(file, globals, locals) # execute the script
File "/home/vagrant/.pycharm_helpers/pydev/_pydev_imps/_pydev_execfile.py", line 18, in execfile
exec(compile(contents+"\n", file, 'exec'), glob, loc)
File "/emailService/app.py", line 59, in <module>
import logstash
File "/home/vagrant/.envs/emailservice/lib/python3.4/site-packages/logstash/__init__.py", line 2, in <module>
from event import Event
ImportError: No module named 'event'
Process finished with exit code 1
my app.py file looks mostly like this. I run it locally through a vagrant session. If I remove the import logstash from the local branch of the if, the application starts up fine and I get local console log output.
import logging
import os
import sys
from flask import Flask
from flask_restful import Api
from flask_cache import Cache
from flask_sqlalchemy import SQLAlchemy
from opbeat.contrib.flask import Opbeat
from tasks import make_celery
app = Flask(__name__)
app.secret_key = os.environ.get('SECRET_KEY', 'SUCHSECRETSWOW')
app.config.from_object(os.environ.get('APP_SETTINGS', 'config.DevelopmentConfig'))
cache = Cache(app)
db = SQLAlchemy(app)
api = Api(app)
celery = make_celery(app)
if len(app.config['OPBEAT_ORGANIZATION_ID']):
opbeat = Opbeat(
app,
organization_id=app.config['OPBEAT_ORGANIZATION_ID'],
app_id=app.config['OPBEAT_APP_ID'],
secret_token=app.config['OPBEAT_SECRET_TOKEN'],
)
#app.after_request
def after_request(response):
response.headers.add('Access-Control-Allow-Origin', '*')
response.headers.add('Access-Control-Allow-Headers', 'Content-Type,Authorization')
response.headers.add('Access-Control-Allow-Methods', 'GET,PUT,POST,DELETE')
return response
def clear_cache():
cache.clear()
def start_resources():
from emailService.api import HealthApi
api.add_resource(HealthApi, '/health')
def start_tasks():
from emailService.tasks import KickoffFetchEmailPeriodTask
if __name__ == '__main__':
if app.config.get('DEVELOPMENT', False):
#The reason this exists is purely because of my error.
import logstash
app.logger.setLevel(logging.DEBUG)
app.logger.addHandler(logging.StreamHandler())
else:
import logstash
app.logger = logging.getLogger('python-logstash-logger')
app.logger.setLevel(logging.INFO)
app.logger.addHandler(logstash.LogstashHandler('myhost.veryhost.suchhost', 5959, version=1))
app.logger.addHandler(logging.StreamHandler())
clear_cache()
start_tasks()
start_resources()
app.logger.debug('Starting app')
app.run(host='0.0.0.0', port=16600, debug=True, use_reloader=False)
All of the google searches result in a great big fat sum total of nothing helpful.

You're probably running into this issue, you have pip installed logstash instead of python-logstash
Run this and it should work afterwards:
> pip uninstall logstash
> pip install python-logstash

Delay in restarting Flask application

I have a Flask app where this is my main.py
from king_slayer.database import init_db, manager
from king_slayer.views import fetch_production
if __name__ == '__main__':
init_db()
manager.run()
Before the manger.run() gets executed I want to call
fetch_production()
so I did this.
from king_slayer.database import init_db, manager
from king_slayer.views import fetch_production
if __name__ == '__main__':
init_db()
with manager.app.test_request_context():
fetch_production()
manager.run()
It works.
The problem is when I kill the Flask app and then restart it, It wont startup for like 60-120 seconds. No errors, Nothing. The browser wont load anything. simple 'Problem loading page' appears and after sometime it works fine unless I again restart the Flask app.
If I remove
with manager.app.test_request_context():
fetch_production()
the delay doesn't happens.
P.S
If I dont use with manager.app.test_request_context():
Flask throws this error.
Traceback (most recent call last):
File "main.py", line 8, in <module>
fetch_production()
File "/home/jarvis/Development/kingslayer/king_slayer/king_slayer/views.py", line 156, in fetch_production
results={"msg": "Database Updated", }
File "/home/jarvis/Development/kingslayer/local/lib/python2.7/site-packages/flask/json.py", line 235, in jsonify
and not request.is_xhr:
File "/home/jarvis/Development/kingslayer/local/lib/python2.7/site-packages/werkzeug/local.py", line 338, in __getattr__
return getattr(self._get_current_object(), name)
File "/home/jarvis/Development/kingslayer/local/lib/python2.7/site-packages/werkzeug/local.py", line 297, in _get_current_object
return self.__local()
File "/home/jarvis/Development/kingslayer/local/lib/python2.7/site-packages/flask/globals.py", line 20, in _lookup_req_object
raise RuntimeError('working outside of request context')
RuntimeError: working outside of request context
What is happening ?
P.P.S fetch_production() does something like this
requests.get(config.SERVER, auth=HTTPBasicAuth(config.AUTH_USER, config.AUTH_PASSWORD))
makes a few function calls, functions that are defined in views.py and then creates an .ini file in the file system. Then in the end returns a return jsonify.
requests.get(config.SERVER, auth=HTTPBasicAuth(config.AUTH_USER, config.AUTH_PASSWORD))
is actually very fast and if the fetch_production() is called manually it doesn't take more than a few milliseconds.

Testing Tornado app for 4xx status code

Consider the following Tornado (v 4.0.2) application, which is a little bit modified version of official hello world example:
import tornado.ioloop
import tornado.web
class MainHandler(tornado.web.RequestHandler):
def get(self):
self.set_status(400)
self.write("Hello, world")
application = tornado.web.Application([
(r"/", MainHandler),
])
if __name__ == "__main__":
application.listen(8888)
tornado.ioloop.IOLoop.instance().start()
As you can see, the only difference here is set_status call in MainHandler. Now, I save this code into app.py, then I open tests.py and I put there this simple unit test:
import tornado.ioloop
from tornado.httpclient import HTTPRequest
from tornado.testing import AsyncHTTPTestCase, gen_test
from app import application
class SimpleTest(AsyncHTTPTestCase):
def get_app(self):
return application
def get_new_ioloop(self):
return tornado.ioloop.IOLoop.instance()
#gen_test
def test_bad_request(self):
request = HTTPRequest(url=self.get_url('/'))
response = yield self.http_client.fetch(request)
self.assertEqual(response.code, 400)
When I run this test with python -m tornado.test.runtests tests I get the following result:
E
======================================================================
ERROR: test_bad_request (tests.SimpleTest)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/tornado/testing.py", line 118, in __call__
result = self.orig_method(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/tornado/testing.py", line 494, in post_coroutine
timeout=timeout)
File "/usr/local/lib/python2.7/dist-packages/tornado/ioloop.py", line 418, in run_sync
return future_cell[0].result()
File "/usr/local/lib/python2.7/dist-packages/tornado/concurrent.py", line 109, in result
raise_exc_info(self._exc_info)
File "/usr/local/lib/python2.7/dist-packages/tornado/gen.py", line 631, in run
yielded = self.gen.throw(*sys.exc_info())
File "tests.py", line 18, in test_bad_request
response = yield self.http_client.fetch(request)
File "/usr/local/lib/python2.7/dist-packages/tornado/gen.py", line 628, in run
value = future.result()
File "/usr/local/lib/python2.7/dist-packages/tornado/concurrent.py", line 111, in result
raise self._exception
HTTPError: HTTP 400: Bad Request
----------------------------------------------------------------------
Ran 1 test in 0.022s
FAILED (errors=1)
[E 140929 12:55:59 testing:687] FAIL
Obviously this is correct, because the handler sets 400 status code. But how can I test my application for such case? I think 4xx codes are useful, so I don't want to give them up. However I'm new to Tornado and I wasn't able to find a way to test them. Is there any?

Try this:
#gen_test
def test_bad_request(self):
request = HTTPRequest(url=self.get_url('/'))
with self.assertRaises(tornado.httpclient.HTTPError) as context:
yield self.http_client.fetch(request)
self.assertEqual(context.exception.code, 400)
See the documentation for assertRaises.