I am currently running into an issue deploying a Flask app on Amazon's EB2 service. The Flask app works locally. When it is deployed, however, it only works for the first person who clicks the link. After that it throws the following error:
Internal Server Error The server encountered an internal error and was
unable to complete your request. Either the server is overloaded or
there is an error in the application.
The error it is throwing out concerns the Flask session - it becomes empty after routing from one site to another. I also noticed that the before_first_request function detailed below is ran only once, for the first user, and never again - which is even more bewildering.
Here's the minimal example:
from flask import Flask, render_template, request, session, url_for
application = Flask(__name__)
application.secret_key = "mysecretkey"
#application.before_first_request
def before_first_request():
""" these commands are run before the first request"""
# setup logging
application.logger.setLevel(logging.INFO)
application.logger.info('starting up Flask')
# clear session
session.clear()
# load in PID
session['pid'] = 123
# add parameters to the session
params = dict()
params['parameter'] = 0
session['params'] = params
application.logger.info(session) # it is printing the session as expected
return 'OK'
#application.route('/')
def main():
""" landing page """
application.logger.info(session) # empty
application.logger.info(application.secret_key) # as expected
params, results = session.pop('params'), session.pop('results') # throws out the error
return render_template('empty_template.jinja', args = session)
I am wondering if anyone might know what is going on how to resolve the issue?
I managed to solve it.
The error was that #before_first_request wrapper actually only ran once before first request ever made to the app. Hence, the session was actually only created and populated once.
I fixed that error by adding the call to before_first_request function at the top of the main function.
Related
I have a very basic Heroku web app where you can click on links (on the main page) to be redirected to other pages. I have done this to test the database page. All the pages work apart from when I click to view the database page. when I try I get an error message:
Internal Server Error
The server encountered an internal error and was unable to complete
your request.
Either the server is overloaded or there is an error in
the application.
I did try to play around with the procfile but in the end nothig worked. My current proctfile looks like this:
web: gunicorn flask-sqlalchemy-test-02.wsgi:application --log-file=-
To be honest I'm not too sure if its the procfile or the sytax in app.py which is causing me problems.
My app.py file:
import os
import psycopg2
from flask import Flask, render_template, g, url_for
app = Flask(__name__)
DATABASE_URL = os.environ.get('postgres://fikwczdiymxhwf:73bf42c2c8a15fa59b77e93654b6383e1cf4f85bdf0156818d1cf39a77815f13#ec2-54-243-47-196.compute-1.amazonaws.com:5432/d3uburco4fea1b'
#app.route('/')
def index():
return render_template("index.html")
#app.route('/page2')
def page_2():
return render_template("random_page_2.html")
#app.route('/hello')
def hello():
return render_template("hello.html")
#app.route('/view_database')
def view_db():
conn = psycopg2.connect(DATABASE_URL)
db = conn.cursor()
data = db.execute("SELECT * FROM example").fetchall()
db.close()
conn.close()
return render_template('view_database.html', data=data)
I expected to view the database in a form of an unordered list but recieved an error message instead:
Internal Server Error
The server encountered an internal error and was unable to complete your request.
Either the server is overloaded or there is an error in the application.
The way you are using os.environ.get() is wrong.
os.environ.get() is used to obtain environment variables that are exported by your OS, so DATABASE_URL returns None, you cannot connect to None URL, so internal server error.
**Correct Way : **
First, export the environment variable, if using Linux :
export DATABASE_URL=postgres://fikwczdiymxhwf:73bf42c2c8a15fa59b77e93654b6383e1cf4f85bdf0156818d1cf39a77815f13#ec2-54-243-47-196.compute-1.amazonaws.com:5432/d3uburco4fea1b
Then, in your code, replace that line as :
DATABASE_URL = os.environ.get('DATABASE_URL', '')
I am using:
Python 3.6.1
Flask 0.12.2
Section on session of Flask documentation says that:
This is a proxy.
and section on proxies elaborates that:
Some of the objects provided by Flask are proxies to other objects.
The reason behind this is that these proxies are shared between
threads and they have to dispatch to the actual object bound to a
thread behind the scenes as necessary. ...
If you need to get access
to the underlying object that is proxied, you can use the
_get_current_object() method
This all is pretty much straightforward.
But when I try the following:
from flask import (
Flask,
session,
)
app = Flask(__name__)
app.secret_key = 'some random secret key'
#app.route('/')
def index():
print("session ID is: {}".format(id(session)))
print("session._get_current_object() ID is: {}".format(id(session._get_current_object())))
print('________________________________')
return 'Check the console! ;-)'
each time I make a request to / — the value of id(session._get_current_object()) is different, while id(session) remains the same.
Following Flask documentation, quoted above, it should be the other way around. So why is this happening?
UPDATE
inspired by brunns's suggestion in the comments to his answer, that there is one underlying object per thread
Here is some code, to test assumption that there is one underlying session object (session._get_current_object()) per thread:
import threading
from flask import (
Flask,
session,
)
app = Flask(__name__)
app.secret_key = 'some random secret key'
#app.route('/')
def index():
print("session ID is: {}".format(id(session)))
print("session._get_current_object() ID is: {}".format(id(session._get_current_object())))
print("threading.current_thread().ident is: {}".format(threading.current_thread().ident))
print('________________________________')
return 'Check the console! ;-)'
Despite the expectations, threading.current_thread().ident) is never changed, while is id(session._get_current_object() is changing.
session is an object you have imported from the flask module. You only import it once, and it doesn't change, so nor will its id(). It's shared between threads, and it's a proxy to the underlying objects.
Each request may be run on a different thread, and each will have a different underlying object, so they may have different id()s.
Just to give a context here, I'm a node.JS developer, but I'm on a project that I need to work with Python using Flask framework.
The problem is, when a client request to an endpoint of my rest flask app, I need to emit an event using socket.IO, and get some data from the socket server, then this data is the response of the endpoint. But I didn't figured out how to send this, because flask needs a "return" statement saying what is the response, and my callback is in another context.
Sample of what I'm trying to do: (There's some comments explaining)
import socketio
import eventlet
from flask import Flask, request
sio = socketio.Server()
app = Flask(__name__)
#app.route('/test/<param>')
def get(param):
def ack(data):
print (data) #Should be the response
sio.emit('event', param, callback=ack) # Socket server call my ack function
#Without a return statement, the endpoint return 500
if __name__ == '__main__':
app = socketio.Middleware(sio, app)
eventlet.wsgi.server(eventlet.listen(('', 8000)), app)
Maybe, the right question here is: Is this possible?
I'm going to give you one way to implement what you want specifically, but I believe you have an important design flaw in this, as I explain in a comment above. In the way you have this coded, your socketio.Server() object will broadcast to all your clients, so will not be able to get a callback. If you want to emit to one client (hopefully not the same one that sent the HTTP request), then you need to add a room=client_sid argument to the emit. Or, if you are contacting a Socket.IO server, then you need to use a Socket.IO client here, not a server.
In any case, to block your HTTP route until the callback function is invoked, you can use an Event object. Something like this:
from threading import Event
from flask import jsonify
#app.route('/test/<param>')
def get(param):
ev = threading.Event()
result = None
def ack(data):
nonlocal result
nonlocal ev
result = {'data': data}
ev.set() # unblock HTTP route
sio.emit('event', param, room=some_client_sid, callback=ack)
ev.wait() # blocks until ev.set() is called
return jsonify(result)
I had a similar problem using FastAPI + socketIO (async version) and I was stuck at the exact same point. No eventlet so could not try out the monkey patching option.
After a lot of head bangings it turns out that, for some reason, adding asyncio.sleep(.1) just before ev.wait() made everything work smoothly. Without that, emitted event actually never reach the other side (socketio client, in my scenario)
I currently have a flask app that makes a call to S3 as well as an external API with the following structure before rendering the data in javascript:
from flask import Flask, render_template,make_response
from flask import request
import requests
import requests_cache
import redis
from boto3.session import Session
import json
app = Flask(__name__)
#app.route('/test')
def test1():
bucket_root = 'testbucket'
session = Session(
aws_access_key_id='s3_key',
aws_secret_access_key='s3_secret_key')
s3 = session.resource('s3')
bucket = s3.Bucket(bucket_root)
testvalues = json.dumps(s3.Object(bucket_root,'all1.json').get()['Body'].read())
r = requests.get(api_link)
return render_template('test_html.html',json_s3_test_response=r.content,
limit=limit, testvalues=testvalues)
#app.route('/test2')
def test2():
bucket_root = 'testbucket'
session = Session(
aws_access_key_id='s3_key',
aws_secret_access_key='s3_secret_key')
s3 = session.resource('s3')
bucket = s3.Bucket(bucket_root)
testvalues = json.dumps(s3.Object(bucket_root,'all2.json').get()['Body'].read())
r = requests.get(api_link)
return render_template('test_html.html',json_s3_test_response=r.content,
limit=limit, testvalues=testvalues)
#app.errorhandler(500)
def internal_error(error):
return "500 error"
#app.errorhandler(404)
def not_found(error):
return "404 error",404
#app.errorhandler(400)
def custom400(error):
return "400 error",400
//catch all?
#app.errorhandler(Exception)
def all_exception_handler(error):
return 'error', 500
Obviously I have a lot of inefficiencies here, but my main question is:
To me it seems like I'm calling S3 and the external API for each client, every time they refresh the page. This increases the chance for the app to crash due to timeouts (and my poor error handling) and diminishes performance. I would like to resolve this by periodically caching the S3 results (say every 10 mins) into a local redis server (already set up and running) as well as just pinging the external API just once from the server every few seconds before passing it onto ALL clients.
I have code that can store the data into redis every 10 mins in a regular python script, however, I'm not sure where to place this within the flask server? Do I put it as it's own function or keep the call to redis in the #app.route()?
Thank you everyone for your time and effort. Any help would be appreciated! I'm new to flask so some of this has been confusing.
When I run py.test --with-gae, I get the following error (I have pytest_gae plugin installed):
def get_current_session():
"""Returns the session associated with the current request."""
> return _tls.current_session
E AttributeError: 'thread._local' object has no attribute 'current_session'
gaesessions/__init__.py:50: AttributeError
I'm using pytest to test my google appengine application. The application runs fine when run in the localhost SDK or when deployed to GAE servers. I just can't figure out how to make pytest work with gaesessions.
My code is below:
test_handlers.py
from webtest import TestApp
import appengine_config
def pytest_funcarg__anon_user(request):
from main import app
app = appengine_config.webapp_add_wsgi_middleware(app)
return TestApp(app)
def test_session(anon_user):
from gaesessions import get_current_session
assert get_current_session()
appengine_config.py
from gaesessions import SessionMiddleware
def webapp_add_wsgi_middleware(app):
from google.appengine.ext.appstats import recording
app = recording.appstats_wsgi_middleware(app)
app = SessionMiddleware(app, cookie_key="replaced-with-this-boring-text")
return app
Relevant code from gaesessions:
# ... more code are not show here ...
_tls = threading.local()
def get_current_session():
"""Returns the session associated with the current request."""
return _tls.current_session
# ... more code are not show here ...
class SessionMiddleware(object):
"""WSGI middleware that adds session support.
``cookie_key`` - A key used to secure cookies so users cannot modify their
content. Keys should be at least 32 bytes (RFC2104). Tip: generate your
key using ``os.urandom(64)`` but do this OFFLINE and copy/paste the output
into a string which you pass in as ``cookie_key``. If you use ``os.urandom()``
to dynamically generate your key at runtime then any existing sessions will
become junk every time your app starts up!
``lifetime`` - ``datetime.timedelta`` that specifies how long a session may last. Defaults to 7 days.
``no_datastore`` - By default all writes also go to the datastore in case
memcache is lost. Set to True to never use the datastore. This improves
write performance but sessions may be occassionally lost.
``cookie_only_threshold`` - A size in bytes. If session data is less than this
threshold, then session data is kept only in a secure cookie. This avoids
memcache/datastore latency which is critical for small sessions. Larger
sessions are kept in memcache+datastore instead. Defaults to 10KB.
"""
def __init__(self, app, cookie_key, lifetime=DEFAULT_LIFETIME, no_datastore=False, cookie_only_threshold=DEFAULT_COOKIE_ONLY_THRESH):
self.app = app
self.lifetime = lifetime
self.no_datastore = no_datastore
self.cookie_only_thresh = cookie_only_threshold
self.cookie_key = cookie_key
if not self.cookie_key:
raise ValueError("cookie_key MUST be specified")
if len(self.cookie_key) < 32:
raise ValueError("RFC2104 recommends you use at least a 32 character key. Try os.urandom(64) to make a key.")
def __call__(self, environ, start_response):
# initialize a session for the current user
_tls.current_session = Session(lifetime=self.lifetime, no_datastore=self.no_datastore, cookie_only_threshold=self.cookie_only_thresh, cookie_key=self.cookie_key)
# create a hook for us to insert a cookie into the response headers
def my_start_response(status, headers, exc_info=None):
_tls.current_session.save() # store the session if it was changed
for ch in _tls.current_session.make_cookie_headers():
headers.append(('Set-Cookie', ch))
return start_response(status, headers, exc_info)
# let the app do its thing
return self.app(environ, my_start_response)
The problem is that your gae sessions is not yet called until the app is also called. The app is only called when you make a request to it. Try inserting a request call before you check for the session value. Check out the revised test_handlers.py code below.
def test_session(anon_user):
anon_user.get("/") # get any url to call the app to create a session.
from gaesessions import get_current_session
assert get_current_session()