I am trying to play around with deploying Flask application. When I run the code using docker swarm on Amazon's EC2 I started to get following errors:
Error message:
sqlalchemy.exc:OperationalError: (psycopg2.OperationalError) server closed the connection unexpectedly This probably means the server terminated abnormally before or while processing the request.[SQL: SELECT visitor."ID" AS "visitor_ID", visitor.username AS visitor_username, visitor.visits AS visitor_visits FROM visitor](Background on this error at: http://sqlalche.me/e/e3q8)
Stack trace
Traceback (most recent call last):
File "/usr/local/bin/gunicorn", line 8, in <module>
File "/usr/local/lib/python3.7/site-packages/gunicorn/app/wsgiapp.py", line 58, in run
File "/usr/local/lib/python3.7/site-packages/gunicorn/app/base.py", line 228, in run
File "/usr/local/lib/python3.7/site-packages/gunicorn/app/base.py", line 72, in run
File "/usr/local/lib/python3.7/site-packages/gunicorn/arbiter.py", line 202, in run
File "/usr/local/lib/python3.7/site-packages/gunicorn/arbiter.py", line 545, in manage_workers
File "/usr/local/lib/python3.7/site-packages/gunicorn/arbiter.py", line 616, in spawn_workers
File "/usr/local/lib/python3.7/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/base.py", line 140, in init_process
File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/sync.py", line 123, in run
File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/sync.py", line 67, in run_for_one
File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/sync.py", line 29, in accept
File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/sync.py", line 134, in handle
File "/usr/local/lib/python3.7/site-packages/gunicorn/workers/sync.py", line 175, in handle_request
File "/usr/local/lib/python3.7/site-packages/newrelic/api/wsgi_application.py", line 665, in _nr_wsgi_application_wrapper_
File "/usr/local/lib/python3.7/site-packages/newrelic/api/wsgi_application.py", line 193, in __init__
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2463, in __call__
File "/usr/local/lib/python3.7/site-packages/newrelic/api/wsgi_application.py", line 555, in _nr_wsgi_application_wrapper_
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 2446, in wsgi_app
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1949, in full_dispatch_request
File "/usr/local/lib/python3.7/site-packages/flask/app.py", line 1935, in dispatch_request
File "/usr/local/lib/python3.7/site-packages/newrelic/hooks/framework_flask.py", line 45, in _nr_wrapper_handler_
File "/app/src/views.py", line 5, in index
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3233, in all
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3389, in __iter__
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/orm/query.py", line 3414, in _execute_and_instances
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 982, in execute
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/sql/elements.py", line 293, in _execute_on_connection
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1101, in _execute_clauseelement
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1250, in _execute_context
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1476, in _handle_dbapi_exception
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/util/compat.py", line 152, in reraise
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/base.py", line 1246, in _execute_context
File "/usr/local/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 588, in do_execute
File "/usr/local/lib/python3.7/site-packages/newrelic/hooks/database_psycopg2.py", line 51, in execute
File "/usr/local/lib/python3.7/site-packages/newrelic/hooks/database_dbapi2.py", line 25, in execute
Initially I thought that this might be a result of not closing db connections after using them, but when I added db.session.close() the error still occurs.
Full code can be found here, the most important parts are:
views.py file:
from models import Visitor, db
def index():
visitors = Visitor.query.all()
response = [
{"id": visitor.ID, "username": visitor.username, "visits": visitor.visits}
for visitor in visitors
]
db.session.close()
return {"results": response}
def increment_visits(username: str) -> dict:
if username == "favicon.ico":
return {}
visitor = Visitor.query.filter_by(username=username).first()
if visitor is None:
visitor = Visitor(username=username, visits=1)
db.session.add(visitor)
else:
visitor.visits += 1
db.session.commit()
return {"id": visitor.ID, "username": visitor.username, "visits": visitor.visits}
main.py file:
import sys
from flask import Flask
from config import SQLALCHEMY_DATABASE_URI, HOST, PORT, DEBUG, DB_POOL_SIZE
from models import db
def make_app() -> Flask:
flask_app = Flask(__name__)
flask_app.config['SQLALCHEMY_DATABASE_URI'] = SQLALCHEMY_DATABASE_URI
flask_app.config['SQLALCHEMY_POOL_SIZE'] = DB_POOL_SIZE
add_urls(flask_app)
db.init_app(app=flask_app)
return flask_app
def add_urls(flask_app: Flask):
from views import index, increment_visits
flask_app.add_url_rule('/', view_func=index)
flask_app.add_url_rule('/<username>/', view_func=increment_visits)
app = make_app()
if __name__ == '__main__':
if 'createdb' in sys.argv:
app.app_context().push()
db.create_all()
else:
app.run(host=HOST, port=PORT, debug=DEBUG)
models.py:
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
class Visitor(db.Model):
ID = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(80), unique=True)
visits = db.Column(db.Integer(), default=1)
Docker services:
ubuntu#srv1:~$ docker stack ps --filter "desired-state=running" demo
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
frbdra27cdcb demo_web.1 gonczor/aws-simple-app:prod srv1 Running Running 7 hours ago
0by6yhvr9n4a demo_nginx.1 gonczor/aws-simple-app-nginx:prod srv1 Running Running 3 days ago
ym2t3we6r5b1 demo_db.1 postgres:11 srv1 Running Running 4 days ago
luwgpr3jnsj8 demo_web.2 gonczor/aws-simple-app:prod srv1 Running Running 7 hours ago
I have no clue where this error might come from, so I am happy to provide any further details.
EDIT
Output of requested command:
simple_app=# SELECT sum(numbackends) FROM pg_stat_database;
sum
-----
2
(1 row)
I think I found a solution, though I have very vague why it works. I found an answer in this reddit post, so here we go.
The error mainly occurred after a long idleness. When I was flooding the app with requests during benchmarking nothing happened. It seems that some connections were kept by SQLAlchemy for too long. I lowered the POOL_RECYCLE_TIME to 10 minutes and updated the settings format to avoid deprecation that will be introduced in version 3.0 of Flask-SQLAlchemy.
DB_POOL_SIZE = int(os.environ.get('DB_POOL_SIZE', '10'))
DB_POOL_RECYCLE = int(os.environ.get('DB_POOL_RECYCLE', '60'))
SQLALCHEMY_ENGINE_OPTIONS = {
'pool_size': DB_POOL_SIZE,
'pool_recycle': DB_POOL_RECYCLE,
}
As I say, I don't fully understand why this was a problem, but after introducing those changes no errors occurred. Anyway comments and links to materials explaining this issue are welcome.
Related
I have a flask rest api app with the following setup.
Packages installed
alembic==1.3.0
aniso8601==8.0.0
astroid==2.3.3
attrs==19.3.0
bcrypt==3.1.7
cffi==1.13.2
Click==7.0
colorama==0.4.1
Flask==1.1.1
Flask-Bcrypt==0.7.1
Flask-Migrate==2.5.2
flask-restplus==0.13.0
Flask-Script==2.0.6
Flask-SQLAlchemy==2.4.1
Flask-Testing==0.7.1
importlib-metadata==0.23
isort==4.3.21
itsdangerous==1.1.0
Jinja2==2.10.3
jsonschema==3.1.1
lazy-object-proxy==1.4.3
Mako==1.1.0
MarkupSafe==1.1.1
mccabe==0.6.1
more-itertools==7.2.0
mysqlclient==1.4.5
pycparser==2.19
PyJWT==1.7.1
pylint==2.4.3
PyMySQL==0.9.3
pyrsistent==0.15.5
python-dateutil==2.8.1
python-editor==1.0.4
pytz==2019.3
six==1.13.0
SQLAlchemy==1.3.11
Werkzeug==0.16.0
wrapt==1.11.2
zipp==0.6.0
I have following production config with debug enabled to true to know stack trace.
class ProductionConfig(Config):
DEBUG = True
user = 'xxxxxx#xxxxxx-mysqldbserver'
passs = 'xxxxxx'
host = 'xxxxxx-mysqldbserver.mysql.database.azure.com'
db = 'ifsc'
SQLALCHEMY_DATABASE_URI = 'mysql+pymysql://{0}:{1}#{2}/{3}'.format(user, passs, host, db)
I have __init__.py that initializes SQLAlchemy as follows
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
from flask_bcrypt import Bcrypt
from .config import config_by_name
db = SQLAlchemy()
flask_bcrypt = Bcrypt()
def create_app(config_name):
app = Flask(__name__)
app.config.from_object(config_by_name[config_name])
db.init_app(app)
flask_bcrypt.init_app(app)
return app
I have an app.py with the following code that actually runs the flask app.
app.py
from flask_cors import CORS, cross_origin
from api import blueprint
from api.main import create_app, db
from api.main.seed import seed_db
app = create_app(os.getenv('BOILERPLATE_ENV') or 'prod') # dev|prod
CORS(app)
app.register_blueprint(blueprint)
app.app_context().push()
if __name__ == '__main__':
app.run()
Problem: I'm facing the mysql connection issues with the following error on azure's db. I've tried setting max_allowed_packet server parameter to 1073741824. But still see this error.
Symptoms: For the fresh deployment, It responds and if the API is left idle for 5 to 10 mins and make the same request, it behaves with the following error. Again it starts working after 60 mins and the behavior continues.
Stack Trace:
sqlalchemy.exc.OperationalError sqlalchemy.exc.OperationalError:
(pymysql.err.OperationalError) (2006, "MySQL server has gone away
(ConnectionResetError(10054, 'An existing connection was forcibly
closed by the remote host', None, 10054, None))") [SQL: SELECT
bank_details.id AS bank_details_id, bank_details.bank AS
bank_details_bank, bank_details.ifsc AS bank_details_ifsc,
bank_details.branch AS bank_details_branch, bank_details.address AS
bank_details_address, bank_details.district AS bank_details_district,
bank_details.city AS bank_details_city, bank_details.state AS
bank_details_state, bank_details.phone AS bank_details_phone,
bank_details.micr AS bank_details_micr, bank_details.pin AS
bank_details_pin FROM bank_details WHERE bank_details.bank =
%(bank_1)s AND bank_details.city = %(city_1)s] [parameters: {'bank_1':
'axis bank', 'city_1': 'goa'}] (Background on this error at:
http://sqlalche.me/e/e3q8)
Traceback (most recent call last) File
"E:\sampled\sample_api\env\Lib\site-packages\pymysql\connections.py",
line 713, in _write_bytes self._sock.sendall(data) During handling of
the above exception, another exception occurred: File
"E:\sampled\sample_api\env\Lib\site-packages\sqlalchemy\engine\base.py",
line 1245, in _execute_context self.dialect.do_execute( File
"E:\sampled\sample_api\env\Lib\site-packages\sqlalchemy\engine\default.py",
line 581, in do_execute cursor.execute(statement, parameters) File
"E:\sampled\sample_api\env\Lib\site-packages\pymysql\cursors.py", line
170, in execute result = self._query(query) File
"E:\sampled\sample_api\env\Lib\site-packages\pymysql\cursors.py", line
328, in _query conn.query(q) File
"E:\sampled\sample_api\env\Lib\site-packages\pymysql\connections.py",
line 516, in query self._execute_command(COMMAND.COM_QUERY, sql) File
"E:\sampled\sample_api\env\Lib\site-packages\pymysql\connections.py",
line 771, in _execute_command self._write_bytes(packet) File
"E:\sampled\sample_api\env\Lib\site-packages\pymysql\connections.py",
line 716, in _write_bytes raise err.OperationalError( The above
exception was the direct cause of the following exception: File
"E:\sampled\sample_api\env\Lib\site-packages\flask\app.py", line 2463,
in
call return self.wsgi_app(environ, start_response) File "E:\sampled\sample_api\env\Lib\site-packages\flask\app.py", line 2449,
in wsgi_app response = self.handle_exception(e) File
"E:\sampled\sample_api\env\Lib\site-packages\flask_restplus\api.py",
line 584, in error_router return original_handler(e) File
"E:\sampled\sample_api\env\Lib\site-packages\flask\app.py", line 1866,
in handle_exception reraise(exc_type, exc_value, tb) File
"E:\sampled\sample_api\env\Lib\site-packages\flask_compat.py", line
38, in reraise raise value.with_traceback(tb) File
"E:\sampled\sample_api\env\Lib\site-packages\flask\app.py", line 2446,
in wsgi_app response = self.full_dispatch_request() File
"E:\sampled\sample_api\env\Lib\site-packages\flask\app.py", line 1951,
in full_dispatch_request rv = self.handle_user_exception(e) File
"E:\sampled\sample_api\env\Lib\site-packages\flask_restplus\api.py",
line 584, in error_router return original_handler(e) File
"E:\sampled\sample_api\env\Lib\site-packages\flask\app.py", line 1820,
in handle_user_exception reraise(exc_type, exc_value, tb) File
"E:\sampled\sample_api\env\Lib\site-packages\flask_compat.py", line
38, in reraise raise value.with_traceback(tb) File
"E:\sampled\sample_api\env\Lib\site-packages\flask\app.py", line 1949,
in full_dispatch_request rv = self.dispatch_request() File
"E:\sampled\sample_api\env\Lib\site-packages\flask\app.py", line 1935,
in dispatch_request return
self.view_functionsrule.endpoint File
"E:\sampled\sample_api\env\Lib\site-packages\flask_restplus\api.py",
line 325, in wrapper resp = resource(*args, **kwargs) File
"E:\sampled\sample_api\env\Lib\site-packages\flask\views.py", line 89,
in view return self.dispatch_request(*args, **kwargs) File
"E:\sampled\sample_api\env\Lib\site-packages\flask_restplus\resource.py",
line 44, in dispatch_request resp = meth(*args, **kwargs) File
"E:\sampled\sample_api\env\Lib\site-packages\flask_restplus\marshalling.py",
line 243, in wrapper resp = f(*args, **kwargs) File
"E:\sampled\sample_api\api\main\controller\ifsc_controller.py", line
37, in get banks = get_banks_by_name_city(bank, city) File
"E:\sampled\sample_api\api\main\service\ifsc_service.py", line 28, in
get_banks_by_name_city return Ifsc.query.filter_by(bank = name, city =
city).all() File
"E:\sampled\sample_api\env\Lib\site-packages\sqlalchemy\orm\query.py",
line 3211, in all return list(self) File
"E:\sampled\sample_api\env\Lib\site-packages\sqlalchemy\orm\query.py",
line 3367, in iter return self._execute_and_instances(context)
File
"E:\sampled\sample_api\env\Lib\site-packages\sqlalchemy\orm\query.py",
line 3392, in _execute_and_instances result =
conn.execute(querycontext.statement, self._params) File
"E:\sampled\sample_api\env\Lib\site-packages\sqlalchemy\engine\base.py",
line 982, in execute return meth(self, multiparams, params) File
"E:\sampled\sample_api\env\Lib\site-packages\sqlalchemy\sql\elements.py",
line 287, in _execute_on_connection return
connection._execute_clauseelement(self, multiparams, params) File
"E:\sampled\sample_api\env\Lib\site-packages\sqlalchemy\engine\base.py",
line 1095, in _execute_clauseelement ret = self._execute_context( File
"E:\sampled\sample_api\env\Lib\site-packages\sqlalchemy\engine\base.py",
line 1249, in _execute_context self._handle_dbapi_exception( File
"E:\sampled\sample_api\env\Lib\site-packages\sqlalchemy\engine\base.py",
line 1476, in _handle_dbapi_exception
util.raise_from_cause(sqlalchemy_exception, exc_info) File
"E:\sampled\sample_api\env\Lib\site-packages\sqlalchemy\util\compat.py",
line 398, in raise_from_cause reraise(type(exception), exception,
tb=exc_tb, cause=cause) File
"E:\sampled\sample_api\env\Lib\site-packages\sqlalchemy\util\compat.py",
line 152, in reraise raise value.with_traceback(tb) File
"E:\sampled\sample_api\env\Lib\site-packages\sqlalchemy\engine\base.py",
line 1245, in _execute_context self.dialect.do_execute( File
"E:\sampled\sample_api\env\Lib\site-packages\sqlalchemy\engine\default.py",
line 581, in do_execute cursor.execute(statement, parameters) File
"E:\sampled\sample_api\env\Lib\site-packages\pymysql\cursors.py", line
170, in execute result = self._query(query) File
"E:\sampled\sample_api\env\Lib\site-packages\pymysql\cursors.py", line
328, in _query conn.query(q) File
"E:\sampled\sample_api\env\Lib\site-packages\pymysql\connections.py",
line 516, in query self._execute_command(COMMAND.COM_QUERY, sql) File
"E:\sampled\sample_api\env\Lib\site-packages\pymysql\connections.py",
line 771, in _execute_command self._write_bytes(packet) File
"E:\sampled\sample_api\env\Lib\site-packages\pymysql\connections.py",
line 716, in _write_bytes raise err.OperationalError(
sqlalchemy.exc.OperationalError: (pymysql.err.OperationalError) (2006,
"MySQL server has gone away (ConnectionResetError(10054, 'An existing
connection was forcibly closed by the remote host', None, 10054,
None))") [SQL: SELECT bank_details.id AS bank_details_id,
bank_details.bank AS bank_details_bank, bank_details.ifsc AS
bank_details_ifsc, bank_details.branch AS bank_details_branch,
bank_details.address AS bank_details_address, bank_details.district AS
bank_details_district, bank_details.city AS bank_details_city,
bank_details.state AS bank_details_state, bank_details.phone AS
bank_details_phone, bank_details.micr AS bank_details_micr,
bank_details.pin AS bank_details_pin FROM bank_details WHERE
bank_details.bank = %(bank_1)s AND bank_details.city = %(city_1)s]
[parameters: {'bank_1': 'axis bank', 'city_1': 'goa'}] (Background on
this error at: http://sqlalche.me/e/e3q8) The debugger caught an
exception in your WSGI application. You can now look at the traceback
which led to the error. To switch between the interactive traceback
and the plaintext one, you can click on the "Traceback" headline. From
the text traceback you can also create a paste of it. For code
execution mouse-over the frame you want to debug and click on the
console icon on the right side.
You can execute arbitrary Python code in the stack frames and there
are some extra helpers available for introspection:
dump() shows all variables in the frame dump(obj) dumps all that's
known about the object
After 4 months of research and understanding how flask works, I made the following change by pushing the app_context to initialize the database resolved the issue.
with app.app_context():
db.init_app(app)
Point to note:
We must ensure that app context being pushed along with sqlalchemy database initialization. In the posted question, I've been initializing db and then I was pushing the app context later app.app_context().push().
References: Manually Push a Context
In MySQL server set these :
MAX_EXECUTION_TIME is used for long running queries.
SET SESSION MAX_EXECUTION_TIME=20000;
SET GLOBAL MAX_EXECUTION_TIME=200000;
Also set wait_timeout:
SET session wait_timeout=30000;
SET ##GLOBAL.wait_timeout=30000;
My app is working on my main PC. I tried to clone the app to my laptop (I tried to initialize it on PC in a different directory and the problem was the same so the problem isn't with the laptop) and when initializing the db with flask db init I got the following error:
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such table: race [SQL: 'SELECT race.id AS race_id, race.name AS race_name \nFROM race'] (Background on this error at: http://sqlalche.me/e/e3q8)
What I'm trying to do is in one of my forms to include a wtforms RadioField with choices based on the race table in db. So for every row in that table I want one choice in the selection. The important part of the form looks like this:
class RegistrationForm(FlaskForm):
race = RadioField('Race', validators=[InputRequired('Choose a race.')],
choices=races, coerce=int)
And the races variable is outside the form like this (before the form itself):
with create_app().app_context():
races = [(race.id, race.name.capitalize()) for race in Race.query.all()]
I figured that this last line is the problem from the call stack:
Traceback (most recent call last):
File "/home/dareon4/workspace/test/venv/bin/flask", line 11, in <module>
sys.exit(main())
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/flask/cli.py", line 894, in main
cli.main(args=args, prog_name=name)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/flask/cli.py", line 557, in main
return super(FlaskGroup, self).main(*args, **kwargs)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/flask/cli.py", line 411, in decorator
with __ctx.ensure_object(ScriptInfo).load_app().app_context():
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/flask/cli.py", line 372, in load_app
app = locate_app(self, import_name, name)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/flask/cli.py", line 235, in locate_app
__import__(module_name)
File "/home/dareon4/workspace/test/shapes.py", line 5, in <module>
app = create_app()
File "/home/dareon4/workspace/test/app/__init__.py", line 43, in create_app
from app.blueprints.auth import bp as auth_bp
File "/home/dareon4/workspace/test/app/blueprints/auth/__init__.py", line 7, in <module>
from . import routes
File "/home/dareon4/workspace/test/app/blueprints/auth/routes.py", line 13, in <module>
from .forms import LoginForm, RegistrationForm, ResetPasswordRequestForm, \
File "/home/dareon4/workspace/test/app/blueprints/auth/forms.py", line 19, in <module>
races = [(race.id, race.name.capitalize()) for race in Race.query.all()]
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/sqlalchemy/orm/query.py", line 2836, in all
return list(self)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/sqlalchemy/orm/query.py", line 2988, in __iter__
return self._execute_and_instances(context)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/sqlalchemy/orm/query.py", line 3011, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 948, in execute
return meth(self, multiparams, params)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/sqlalchemy/sql/elements.py", line 269, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1060, in _execute_clauseelement
compiled_sql, distilled_params
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1200, in _execute_context
context)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1413, in _handle_dbapi_exception
exc_info
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/sqlalchemy/util/compat.py", line 265, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/sqlalchemy/util/compat.py", line 248, in reraise
raise value.with_traceback(tb)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
context)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/sqlalchemy/engine/default.py", line 509, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such table: race [SQL: 'SELECT race.id AS race_id, race.name AS race_name \nFROM race'] (Background on this error at: http://sqlalche.me/e/e3q8)
So the problem is that I don't have the db initialized when I'm running the flask db init command and that one line needs the db to exist. This probably doesn't have a solution so I'm not asking how to correct this implementation, but how to do it correctly to have form based on rows from the db. I don't have hundreds of races so I could hardcode them in but I wanted to do it this way.
So after some googling I found that you can add the RadioField choices dynamically in the views like this:
#bp.route('/register', methods=['GET', 'POST'])
def register():
form = RegistrationForm()
form.race.choices = [(race.id, race.name.capitalize()) for race in Race.query.all()]
This is exactly what I was looking for.
I am trying to understand how to run django tests in parallel with in memory sqlite3.
I have django app with that structure:
gbook
order
...
tests
__init__.py
test_a1.py
test_b1.py
utils.py
test_a1.py and test_b1.py contains same code:
import time
from order import models
from .utils import BackendTestCase
class ATestCase(BackendTestCase):
def test_a(self):
time.sleep(1)
a = models.City.objects.count()
self.assertEqual(a, a)
class BTestCase(BackendTestCase):
def test_b(self):
time.sleep(1)
a = models.City.objects.count()
self.assertEqual(a, a)
utils.py is:
from django.test import TestCase, Client
from order import models
from django.conf import settings
from order.utils import to_hash
class BackendTestCase(TestCase):
fixtures = ['City.json', 'Agency.json']
def setUp(self):
self.client = Client()
self.lang_codes = (i[0] for i in settings.LANGUAGES)
...
settings_test.py:
from .settings import *
DEBUG = False
TEMPLATE_DEBUG = False
STATICFILES_STORAGE = 'django.contrib.staticfiles.storage.StaticFilesStorage'
PASSWORD_HASHERS = ['django.contrib.auth.hashers.MD5PasswordHasher',] # faster
DATABASES['default'] = {
'ENGINE': 'django.db.backends.sqlite3',
}
When I run test in single process, all goes well (about 4 sec):
python.exe manage.py test order --settings=gbook.settings_test
Then I trying to run tests in parallel:
python.exe manage.py test order --settings=gbook.settings_test --parallel=2
I get this trace (console):
Creating test database for alias 'default'...
Cloning test database for alias 'default'...
Cloning test database for alias 'default'...
System check identified no issues (0 silenced).
Process SpawnPoolWorker-2:
Process SpawnPoolWorker-1:
Traceback (most recent call last):
Traceback (most recent call last):
File "C:\python\Python36-32\lib\multiprocessing\process.py", line 258, in _bootstrap
self.run()
File "C:\python\Python36-32\lib\multiprocessing\process.py", line 258, in _bootstrap
self.run()
File "C:\python\Python36-32\lib\multiprocessing\process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "C:\python\Python36-32\lib\multiprocessing\process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "C:\python\Python36-32\lib\multiprocessing\pool.py", line 108, in worker
task = get()
File "C:\python\Python36-32\lib\multiprocessing\pool.py", line 108, in worker
task = get()
File "C:\python\Python36-32\lib\multiprocessing\queues.py", line 337, in get
return _ForkingPickler.loads(res)
File "C:\python\Python36-32\lib\multiprocessing\queues.py", line 337, in get
return _ForkingPickler.loads(res)
File "C:\kvk\develop\Python\gbook\order\tests\test_a1.py", line 2, in <module>
from order import models
File "C:\kvk\develop\Python\gbook\order\tests\test_a1.py", line 2, in <module>
from order import models
File "C:\kvk\develop\Python\gbook\order\models.py", line 79, in <module>
class Agency(models.Model):
File "C:\kvk\develop\Python\gbook\order\models.py", line 79, in <module>
class Agency(models.Model):
File "C:\python\venv\gbook\lib\site-packages\django\db\models\base.py", line 110, in __new__
app_config = apps.get_containing_app_config(module)
File "C:\python\venv\gbook\lib\site-packages\django\db\models\base.py", line 110, in __new__
app_config = apps.get_containing_app_config(module)
File "C:\python\venv\gbook\lib\site-packages\django\apps\registry.py", line 247, in get_containing_app_config
self.check_apps_ready()
File "C:\python\venv\gbook\lib\site-packages\django\apps\registry.py", line 247, in get_containing_app_config
self.check_apps_ready()
File "C:\python\venv\gbook\lib\site-packages\django\apps\registry.py", line 125, in check_apps_ready
raise AppRegistryNotReady("Apps aren't loaded yet.")
File "C:\python\venv\gbook\lib\site-packages\django\apps\registry.py", line 125, in check_apps_ready
raise AppRegistryNotReady("Apps aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.
django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.
From Pycharm trace is different:
...
Traceback (most recent call last):
File "C:\python\Python36-32\lib\unittest\suite.py", line 163, in _handleClassSetUp
setUpClass()
File "C:\python\venv\gbook\lib\site-packages\django\test\testcases.py", line 1036, in setUpClass
'database': db_name,
File "C:\python\venv\gbook\lib\site-packages\django\core\management\__init__.py", line 131, in call_command
return command.execute(*args, **defaults)
File "C:\python\venv\gbook\lib\site-packages\django\core\management\base.py", line 330, in execute
output = self.handle(*args, **options)
File "C:\python\venv\gbook\lib\site-packages\modeltranslation\management\commands\loaddata.py", line 61, in handle
return super(Command, self).handle(*fixture_labels, **options)
File "C:\python\venv\gbook\lib\site-packages\django\core\management\commands\loaddata.py", line 69, in handle
self.loaddata(fixture_labels)
File "C:\python\venv\gbook\lib\site-packages\django\core\management\commands\loaddata.py", line 109, in loaddata
self.load_label(fixture_label)
File "C:\python\venv\gbook\lib\site-packages\django\core\management\commands\loaddata.py", line 175, in load_label
obj.save(using=self.using)
File "C:\python\venv\gbook\lib\site-packages\django\core\serializers\base.py", line 205, in save
models.Model.save_base(self.object, using=using, raw=True, **kwargs)
File "C:\python\venv\gbook\lib\site-packages\django\db\models\base.py", line 838, in save_base
updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
File "C:\python\venv\gbook\lib\site-packages\django\db\models\base.py", line 905, in _save_table
forced_update)
File "C:\python\venv\gbook\lib\site-packages\django\db\models\base.py", line 955, in _do_update
return filtered._update(values) > 0
File "C:\python\venv\gbook\lib\site-packages\django\db\models\query.py", line 664, in _update
return query.get_compiler(self.db).execute_sql(CURSOR)
File "C:\python\venv\gbook\lib\site-packages\django\db\models\sql\compiler.py", line 1204, in execute_sql
cursor = super(SQLUpdateCompiler, self).execute_sql(result_type)
File "C:\python\venv\gbook\lib\site-packages\django\db\models\sql\compiler.py", line 899, in execute_sql
raise original_exception
File "C:\python\venv\gbook\lib\site-packages\django\db\models\sql\compiler.py", line 889, in execute_sql
cursor.execute(sql, params)
File "C:\python\venv\gbook\lib\site-packages\django\db\backends\utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "C:\python\venv\gbook\lib\site-packages\django\db\utils.py", line 94, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "C:\python\venv\gbook\lib\site-packages\django\utils\six.py", line 685, in reraise
raise value.with_traceback(tb)
File "C:\python\venv\gbook\lib\site-packages\django\db\backends\utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "C:\python\venv\gbook\lib\site-packages\django\db\backends\sqlite3\base.py", line 328, in execute
return Database.Cursor.execute(self, query, params)
django.db.utils.OperationalError: Problem installing fixture 'C:\kvk\develop\Python\gbook\order\fixtures\AirportInfo.json': Could not load order.AirportInfo(pk=2411): no such table: GB_AIRPORT_INFO
It seem like migrations not works for parallel, but why?
Docs says: "--parallel" Runs tests in separate parallel processes. Each process gets its own database. And I do not need to change my code for use it.
Please, help me to understand, what am i doing wrong.
multiprocessing.cpu_count() = 4
Django version 1.11.10
Python 3.6.5
Same issue as above with MacOS and Python 3.8+. You have to explicitly set import multiprocessing; multiprocessing.set_start_method('fork') at the top of your settings.py file. But be sure to understand the side effects before you do!
I ran into a similar issue trying to use the --parallel feature on Windows.
Django's documentation states
This feature isn’t available on Windows. It doesn’t work with the Oracle database backend either.
Running the same command on Linux completed with no issues.
Parallel running is still disabled on Windows as of today. You can track the ticket that keeps progress of this feature here: https://code.djangoproject.com/ticket/31169.
And here's the code block that disables this option on Windows:
def default_test_processes():
"""Default number of test processes when using the --parallel option."""
# The current implementation of the parallel test runner requires
# multiprocessing to start subprocesses with fork().
if multiprocessing.get_start_method() != 'fork':
return 1
try:
return int(os.environ['DJANGO_TEST_PROCESSES'])
except KeyError:
return multiprocessing.cpu_count()
Source: https://github.com/django/django/blob/59b4e99dd00b9c36d56055b889f96885995e4240/django/test/runner.py#L286-L295
In reply to #Menth, this is how I enable for testing only:
# near the top of settings.py
if "test" in sys.argv[1:]:
import multiprocessing
logging.info("Using multiproc for testing.")
multiprocessing.set_start_method("fork")
If I run in the shell:
mongo ds0219xx.mlab.com:219xx/dbname -u user -p pass
It works and allows me to connect to the database and pull information. But if I'm within my python application (Flask) and run this:
import pymongo
client = pymongo.MongoClient("mongodb://user:pass#ds0219xx.mlab.com:219xx/dbname")
db = client["dbname"]
db.users.insert_one({
"user1": "hello"
})
It gives me an:
pymongo.errors.OperationFailure: Authentication failed.
I'm pretty sure it's failing before it gets to the insert_one() call, but I'm not completely sure.
Thanks!
Edit:
By request, here is the full callback:
Traceback (most recent call last):
File "run.py", line 1, in <module>
from app import app
File "/Users/Derek/Documents/programming/shenalum/app/__init__.py", line 6, in <module>
from app import views
File "/Users/Derek/Documents/programming/shenalum/app/views.py", line 4, in <module>
from data import get_posts, get_user_info
File "/Users/Derek/Documents/programming/shenalum/app/data.py", line 9, in <module>
"user1": "hello"
File "/usr/local/lib/python2.7/site-packages/pymongo/collection.py", line 622, in insert_one
with self._socket_for_writes() as sock_info:
File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/contextlib.py", line 17, in __enter__
return self.gen.next()
File "/usr/local/lib/python2.7/site-packages/pymongo/mongo_client.py", line 718, in _get_socket
with server.get_socket(self.__all_credentials) as sock_info:
File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/contextlib.py", line 17, in __enter__
return self.gen.next()
File "/usr/local/lib/python2.7/site-packages/pymongo/server.py", line 152, in get_socket
with self.pool.get_socket(all_credentials, checkout) as sock_info:
File "/usr/local/Cellar/python/2.7.11/Frameworks/Python.framework/Versions/2.7/lib/python2.7/contextlib.py", line 17, in __enter__
return self.gen.next()
File "/usr/local/lib/python2.7/site-packages/pymongo/pool.py", line 541, in get_socket
sock_info.check_auth(all_credentials)
File "/usr/local/lib/python2.7/site-packages/pymongo/pool.py", line 306, in check_auth
auth.authenticate(credentials, self)
File "/usr/local/lib/python2.7/site-packages/pymongo/auth.py", line 436, in authenticate
auth_func(credentials, sock_info)
File "/usr/local/lib/python2.7/site-packages/pymongo/auth.py", line 416, in _authenticate_default
return _authenticate_scram_sha1(credentials, sock_info)
File "/usr/local/lib/python2.7/site-packages/pymongo/auth.py", line 188, in _authenticate_scram_sha1
res = sock_info.command(source, cmd)
File "/usr/local/lib/python2.7/site-packages/pymongo/pool.py", line 213, in command
read_concern)
File "/usr/local/lib/python2.7/site-packages/pymongo/network.py", line 99, in command
helpers._check_command_response(response_doc, None, allowable_errors)
File "/usr/local/lib/python2.7/site-packages/pymongo/helpers.py", line 196, in _check_command_response
raise OperationFailure(msg % errmsg, code, response)
pymongo.errors.OperationFailure: Authentication failed.
I figured it out. You can do this from the python file and it will work:
connection = pymongo.MongoClient(ab123456.mlab.com, 123456)
db = connection[databasename]
db.authenticate(database_user, database_pass)
Appending /?authSource=admin helped me.
Full Example:
uri = 'mongodb://username:password#r1.example.net:27017/?authSource=admin'
client = MongoClient(uri)
db = client.test
result = db.users.find()
for document in result:
print(document)
Version: pymongo 3.7.2
You need to authenticate after selecting the database for mongo < 4.0
So I have a Django project that doesn't use a database (the 'DATABASES' setting is commented out). I chose to use Django as there's a chance I will need the database functionality in the future. Anyway, I've been working on and off of tbhis project for a couple of months with no problems. I'm running Linux Mint and have had no troubles using the python manage.py runserver command so far.
Well, today I fired up the app and started the local server with no problems. I then tried to open the app in my browser and received the rather hideous error message:
Traceback (most recent call last):
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/contrib/sessions/backends/base.py", line 170, in _get_session
return self._session_cache
AttributeError: 'SessionStore' object has no attribute '_session_cache'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/core/handlers/base.py", line 87, in get_response
response = middleware_method(request)
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/contrib/auth/middleware.py", line 34, in process_request
if user and hasattr(user, 'get_session_auth_hash'):
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/utils/functional.py", line 224, in inner
self._setup()
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/utils/functional.py", line 357, in _setup
self._wrapped = self._setupfunc()
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/contrib/auth/middleware.py", line 23, in <lambda>
request.user = SimpleLazyObject(lambda: get_user(request))
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/contrib/auth/middleware.py", line 11, in get_user
request._cached_user = auth.get_user(request)
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/contrib/auth/__init__.py", line 151, in get_user
user_id = request.session[SESSION_KEY]
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/contrib/sessions/backends/base.py", line 49, in __getitem__
return self._session[key]
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/contrib/sessions/backends/base.py", line 175, in _get_session
self._session_cache = self.load()
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/contrib/sessions/backends/db.py", line 21, in load
expire_date__gt=timezone.now()
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/db/models/manager.py", line 92, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/db/models/query.py", line 351, in get
num = len(clone)
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/db/models/query.py", line 122, in __len__
self._fetch_all()
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/db/models/query.py", line 966, in _fetch_all
self._result_cache = list(self.iterator())
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/db/models/query.py", line 265, in iterator
for row in compiler.results_iter():
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/db/models/sql/compiler.py", line 700, in results_iter
for rows in self.execute_sql(MULTI):
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/db/models/sql/compiler.py", line 775, in execute_sql
sql, params = self.as_sql()
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/db/models/sql/compiler.py", line 100, in as_sql
out_cols, s_params = self.get_columns(with_col_aliases)
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/db/models/sql/compiler.py", line 246, in get_columns
col_aliases)
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/db/models/sql/compiler.py", line 328, in get_default_columns
r = '%s.%s' % (qn(alias), qn2(column))
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/db/models/sql/compiler.py", line 62, in __call__
r = self.connection.ops.quote_name(name)
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/db/backends/dummy/base.py", line 18, in complain
raise ImproperlyConfigured("settings.DATABASES is improperly configured. "
django.core.exceptions.ImproperlyConfigured: settings.DATABASES is improperly configured. Please supply the ENGINE value. Check settings documentation for more details.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/contrib/sessions/backends/base.py", line 170, in _get_session
return self._session_cache
AttributeError: 'SessionStore' object has no attribute '_session_cache'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/lib/python3.3/wsgiref/handlers.py", line 137, in run
self.result = application(self.environ, self.start_response)
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/contrib/staticfiles/handlers.py", line 64, in __call__
return self.application(environ, start_response)
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/core/handlers/wsgi.py", line 187, in __call__
response = self.get_response(request)
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/core/handlers/base.py", line 199, in get_response
response = self.handle_uncaught_exception(request, resolver, sys.exc_info())
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/core/handlers/base.py", line 236, in handle_uncaught_exception
return debug.technical_500_response(request, *exc_info)
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/views/debug.py", line 91, in technical_500_response
html = reporter.get_traceback_html()
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/views/debug.py", line 349, in get_traceback_html
c = Context(self.get_traceback_data(), use_l10n=False)
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/views/debug.py", line 307, in get_traceback_data
frames = self.get_traceback_frames()
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/views/debug.py", line 465, in get_traceback_frames
'vars': self.filter.get_traceback_frame_variables(self.request, tb.tb_frame),
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/views/debug.py", line 232, in get_traceback_frame_variables
cleansed[name] = self.cleanse_special_types(request, value)
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/views/debug.py", line 187, in cleanse_special_types
if isinstance(value, HttpRequest):
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/utils/functional.py", line 224, in inner
self._setup()
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/utils/functional.py", line 357, in _setup
self._wrapped = self._setupfunc()
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/contrib/auth/middleware.py", line 23, in <lambda>
request.user = SimpleLazyObject(lambda: get_user(request))
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/contrib/auth/middleware.py", line 11, in get_user
request._cached_user = auth.get_user(request)
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/contrib/auth/__init__.py", line 151, in get_user
user_id = request.session[SESSION_KEY]
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/contrib/sessions/backends/base.py", line 49, in __getitem__
return self._session[key]
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/contrib/sessions/backends/base.py", line 175, in _get_session
self._session_cache = self.load()
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/contrib/sessions/backends/db.py", line 21, in load
expire_date__gt=timezone.now()
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/db/models/manager.py", line 92, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/db/models/query.py", line 351, in get
num = len(clone)
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/db/models/query.py", line 122, in __len__
self._fetch_all()
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/db/models/query.py", line 966, in _fetch_all
self._result_cache = list(self.iterator())
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/db/models/query.py", line 265, in iterator
for row in compiler.results_iter():
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/db/models/sql/compiler.py", line 700, in results_iter
for rows in self.execute_sql(MULTI):
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/db/models/sql/compiler.py", line 775, in execute_sql
sql, params = self.as_sql()
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/db/models/sql/compiler.py", line 100, in as_sql
out_cols, s_params = self.get_columns(with_col_aliases)
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/db/models/sql/compiler.py", line 246, in get_columns
col_aliases)
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/db/models/sql/compiler.py", line 328, in get_default_columns
r = '%s.%s' % (qn(alias), qn2(column))
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/db/models/sql/compiler.py", line 62, in __call__
r = self.connection.ops.quote_name(name)
File "/home/peter/.virtualenvs/vis_it/lib/python3.3/site-packages/django/db/backends/dummy/base.py", line 18, in complain
raise ImproperlyConfigured("settings.DATABASES is improperly configured. "
django.core.exceptions.ImproperlyConfigured: settings.DATABASES is improperly configured. Please supply the ENGINE value. Check settings documentation for more details.
[28/Nov/2014 13:18:54] "GET / HTTP/1.1" 500 59
I've not touched the app since I last worked on it and it was working fine then. I really have no idea what this is talking about as I have no caches implemented and am not using a database... I've asked a few colleges who are familiar with Django and have no idea what this is either. Any Ideas? I've also tried setting DATABASES to an empty dict {} on the advice of a post online but to no effect. At a bit of a loss.
EDIT: Thought I should mention that in the interim since I last touched this project, I've since started and set up a new Django project that does use a database. Is it possible that this project is somehow cached and breaking the one I'm currently trying to work on?
EDIT2: I should point out that this webapp is currently live and working at http://mrcagney-maps.com. The code is exactly the same (having not touched it since I last pushed to the server). Really weird.
For future Googlers - I ran into this issue and the above solutions did not work for me. What did work for me was clearing/deleting my cookies in Chrome for the 127.0.0.1 URL. So go to Settings or press CMD+, then Cookies and other site data, then find 127.0.0.1 or localhost and delete those cookies. Refresh the local dev host page and the error should be gone. This has something to do with a corrupted session / cookie file.
I'm not sure why I started to get this error, it involved an upgrade though. I just deleted all the sessions and after logging back in all was well.
# from the shell but equivalent sql would work fine too
from django.contrib.sessions.models import Session
Session.objects.all().delete()
The error AttributeError: 'SessionStore' object has no attribute '_session_cache' can stem from the database not having a django_session table. However, since you are not using a table, you would need to make sure that you don't have the 'django.contrib.sessions.middleware.SessionMiddleware' in your MIDDLEWARE_CLASSES in the project's settings file. If it is in there, it will look for a database table which stores the sessions, causing the above error.
Here is what worked for me. Since there is no databases in your application. Admin page looks for the database be it default. So first lets create the default databases.
Shut down your servers and run
python manage.py makemigrations
python manage.py migrate
Now create the admin or superuser for your application. Fill username and password.
python manage.py createsuperuser
Now restart your server and go the admin page
python manage.py runserver
Throwed the same error for me after upgraded my wagtail version... So, for me worked even simpler solution or u didn't noticed already :D Actually I opened the browser > F12 > Storage Tab and Delete all Cookies and all other Cached Data
Users facing this issue via Postman please make sure that the value in the request header for Cache-Control is no-cache.
If not then clear the cookie data for the domain.
In normal case when we use
from django.http import JsonResponse
for sending response we have no need to uses sessions.
But when we use some package such as djangorestframework we force to use sessions.
Sessions need storage place in sqllite,mysql ,...or others
then we need run these commands:
python manage.py makemigrations
python manage.py migrate