When i run the dbManager.create_all() command, it runs with out errors but fails to create the tables. When i delete the database and run the create_all() command, i get the no such database as ##### error which i should get but when the database does exist, nothing happens.
Please can anyone see what i'm doing wrong?
from blogconfig import dbManager
class Art(dbManager.Model):
id = dbManager.Column(dbManager.Integer, primary_key = True)
title = dbManager.Column(dbManager.String(64), index = True, unique = True)
content = dbManager.Column(dbManager.Text(5000))
def __repr__(self):
return '<Art %r>' %(self.title)
EDIT
This is the shell command
from blogconfig import dbManager
>>> dbManager.create_all()
import models
>>> a = models.Art(title='des', content='asdfvhbdjbjdn')
>>> dbManager.session.add(a)
>>> dbManager.session.commit()
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/scoping.py", line 149, in do
return getattr(self.registry(), name)(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 721, in commit
self.transaction.commit()
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 354, in commit
self._prepare_impl()
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 334, in _prepare_impl
self.session.flush()
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1818, in flush
self._flush(objects)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1936, in _flush
transaction.rollback(_capture_exception=True)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/langhelpers.py", line 58, in __exit__
compat.reraise(exc_type, exc_value, exc_tb)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/session.py", line 1900, in _flush
flush_context.execute()
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 372, in execute
rec.execute(self)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/unitofwork.py", line 525, in execute
uow
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 64, in save_obj
table, insert)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/persistence.py", line 569, in _emit_insert_statements
execute(statement, params)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 662, in execute
params)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 761, in _execute_clauseelement
compiled_sql, distilled_params
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 874, in _execute_context
context)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1024, in _handle_dbapi_exception
exc_info
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 195, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 867, in _execute_context
context)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 324, in do_execute
cursor.execute(statement, parameters)
File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute
self.errorhandler(self, exc, value)
File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
sqlalchemy.exc.ProgrammingError: (ProgrammingError) (1146, "Table 'blog.art' doesn't exist") 'INSERT INTO art (title, content) VALUES (%s, %s)' ('des', 'asdfvhbdjbjdn')
dbManager will not know about the models you define in other modules unless they are imported before running create_all.
In a real application this shouldn't matter because running the flask app should set up the db and import views/blueprints to register them. Since the views use the models, the models are indirectly imported and are available to the dbManager.
Either import your models in the blogconfig module after creating the dbManager instance, or change the order of you shell commands to be
>>> from blogconfig import dbManager
>>> import models
>>> dbManager.create_all()
SQLAlchemy will only create tables, the database must already exist, which is why you're seeing the other error when you delete the database.
Related
I am trying to write to an in-memory SQLite database in multiple threads. According to SQLAlchemy docs this is possible. However, in practice I cannot reliably get it to work. I have tried with both scoped_session and without, yielding the same results. Strangely enough, if I pass echo=True to create_engine the code works every time, maybe because it forces the engine to synchronously print all statements to the screen. Code sample and exceptions are below.
EDIT: The more I look into this, the more I realize that SQLite is just not very good at concurrency for write operations. The section on High Concurrency says that while an unlimited amount of readers are allowed, there can only be one writer at any given time. A better approach here would be to queue all write operations instead of trying to achieve any level of concurrency. However I am still open to suggestions if anyone has managed to find a solution.
import threading
from sqlalchemy import *
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import create_engine
from sqlalchemy.pool import StaticPool
from sqlalchemy.orm import sessionmaker, scoped_session
in_memory_engine = create_engine('sqlite://', connect_args={'check_same_thread':False}, poolclass=StaticPool)
Base = declarative_base()
class Item(Base):
__tablename__ = 'items'
id = Column(Integer, primary_key=True, autoincrement=True)
data = Column(Integer)
Base.metadata.create_all(bind=in_memory_engine)
session_factory = sessionmaker(bind=in_memory_engine)
Session = scoped_session(session_factory)
def insert_items():
session = Session()
for i in range(1000):
session.add(Item(data=i))
session.commit()
Session.remove()
for i in range(10):
t = threading.Thread(target=insert_items)
t.start()
Exception in thread Thread-6:
Traceback (most recent call last):
File "c:\python\lib\site-packages\sqlalchemy\engine\base.py", line 1249, in _execute_context
cursor, statement, parameters, context
File "c:\python\lib\site-packages\sqlalchemy\engine\default.py", line 552, in do_execute
cursor.execute(statement, parameters)
sqlite3.OperationalError: cannot start a transaction within a transaction
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "c:\python\lib\threading.py", line 916, in _bootstrap_inner
self.run()
File "c:\python\lib\threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "<ipython-input-1-e1040f59c94e>", line 26, in insert_items
session.commit()
File "c:\python\lib\site-packages\sqlalchemy\orm\session.py", line 1027, in commit
self.transaction.commit()
File "c:\python\lib\site-packages\sqlalchemy\orm\session.py", line 494, in commit
self._prepare_impl()
File "c:\python\lib\site-packages\sqlalchemy\orm\session.py", line 473, in _prepare_impl
self.session.flush()
File "c:\python\lib\site-packages\sqlalchemy\orm\session.py", line 2459, in flush
self._flush(objects)
File "c:\python\lib\site-packages\sqlalchemy\orm\session.py", line 2597, in _flush
transaction.rollback(_capture_exception=True)
File "c:\python\lib\site-packages\sqlalchemy\util\langhelpers.py", line 68, in __exit__
compat.reraise(exc_type, exc_value, exc_tb)
File "c:\python\lib\site-packages\sqlalchemy\util\compat.py", line 153, in reraise
raise value
File "c:\python\lib\site-packages\sqlalchemy\orm\session.py", line 2557, in _flush
flush_context.execute()
File "c:\python\lib\site-packages\sqlalchemy\orm\unitofwork.py", line 422, in execute
rec.execute(self)
File "c:\python\lib\site-packages\sqlalchemy\orm\unitofwork.py", line 589, in execute
uow,
File "c:\python\lib\site-packages\sqlalchemy\orm\persistence.py", line 245, in save_obj
insert,
File "c:\python\lib\site-packages\sqlalchemy\orm\persistence.py", line 1138, in _emit_insert_statements
statement, params
File "c:\python\lib\site-packages\sqlalchemy\engine\base.py", line 988, in execute
return meth(self, multiparams, params)
File "c:\python\lib\site-packages\sqlalchemy\sql\elements.py", line 287, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "c:\python\lib\site-packages\sqlalchemy\engine\base.py", line 1107, in _execute_clauseelement
distilled_params,
File "c:\python\lib\site-packages\sqlalchemy\engine\base.py", line 1253, in _execute_context
e, statement, parameters, cursor, context
File "c:\python\lib\site-packages\sqlalchemy\engine\base.py", line 1473, in _handle_dbapi_exception
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "c:\python\lib\site-packages\sqlalchemy\util\compat.py", line 398, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "c:\python\lib\site-packages\sqlalchemy\util\compat.py", line 152, in reraise
raise value.with_traceback(tb)
File "c:\python\lib\site-packages\sqlalchemy\engine\base.py", line 1249, in _execute_context
cursor, statement, parameters, context
File "c:\python\lib\site-packages\sqlalchemy\engine\default.py", line 552, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) cannot start a transaction within a transaction
[SQL: INSERT INTO items (data) VALUES (?)]
[parameters: (780,)]
(Background on this error at: http://sqlalche.me/e/e3q8)
Exception in thread Thread-7:
Traceback (most recent call last):
File "c:\python\lib\site-packages\sqlalchemy\engine\base.py", line 1249, in _execute_context
cursor, statement, parameters, context
File "c:\python\lib\site-packages\sqlalchemy\engine\default.py", line 552, in do_execute
cursor.execute(statement, parameters)
sqlite3.InterfaceError: Error binding parameter 0 - probably unsupported type.
I would like to serialize many Gensim library Word2Vec models in MySQL database. For the purpose of my project I am utilizing: Python, Flask, SQLAlchemy, and MySQL+PyMySQL. Below you can see my Word2VecModel class which I am using in order to create word2vec table in MySQL DB:
from db.db import db
import db_models.influencer
class Word2VecModel(db.Model):
__tablename__ = "word2vec"
m_id = db.Column(db.Integer, primary_key=True)
m_username = db.Column(db.String(20), db.ForeignKey("influencer.m_username"))
m_binary = db.Column(db.LargeBinary)
influencer = db.relationship("InfluencerModel")
def __init__(self, m_binary, m_username):
self.m_binary = m_binary
self.m_username = m_username
def __repr__(self):
return '<Word2Vec {}>'.format(self.m_name)
#classmethod
def find_by_username(cls, m_username):
return cls.query.filter_by(name=m_username).first()
def save_to_db(self):
db.session.add(self)
db.session.commit()
def delete_from_db(self):
db.session.delete(self)
db.session.commit()
Whenever I try to execute:
word2vec_model = Word2VecModel(pickle.dumps(word2vec), username)
word2vec_model .save_to_db()
I get the following error message:
Traceback (most recent call last):
File "/home/stefan/.virtualenvs/dl4cv/lib/python3.6/site-packages/pymysql/connections.py", line 705, in _write_bytes
self._sock.sendall(data)
BrokenPipeError: [Errno 32] Broken pipe
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/stefan/.virtualenvs/dl4cv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1278, in _execute_context
cursor, statement, parameters, context
File "/home/stefan/.virtualenvs/dl4cv/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 593, in do_execute
cursor.execute(statement, parameters)
File "/home/stefan/.virtualenvs/dl4cv/lib/python3.6/site-packages/pymysql/cursors.py", line 163, in execute
result = self._query(query)
File "/home/stefan/.virtualenvs/dl4cv/lib/python3.6/site-packages/pymysql/cursors.py", line 321, in _query
conn.query(q)
File "/home/stefan/.virtualenvs/dl4cv/lib/python3.6/site-packages/pymysql/connections.py", line 504, in query
self._execute_command(COMMAND.COM_QUERY, sql)
File "/home/stefan/.virtualenvs/dl4cv/lib/python3.6/site-packages/pymysql/connections.py", line 773, in _execute_command
self.write_packet(sql[:packet_size])
File "/home/stefan/.virtualenvs/dl4cv/lib/python3.6/site-packages/pymysql/connections.py", line 634, in write_packet
self._write_bytes(data)
File "/home/stefan/.virtualenvs/dl4cv/lib/python3.6/site-packages/pymysql/connections.py", line 710, in _write_bytes
"MySQL server has gone away (%r)" % (e,))
pymysql.err.OperationalError: (2006, "MySQL server has gone away (BrokenPipeError(32, 'Broken pipe'))")
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "app.py", line 117, in <module>
word2vec.save_to_db()
File "/home/stefan/PycharmProjects/NLP - Influencer Text Analysis/src/flask_api/db_models/word2vec.py", line 25, in save_to_db
db.session.commit()
File "/home/stefan/.virtualenvs/dl4cv/lib/python3.6/site-packages/sqlalchemy/orm/scoping.py", line 163, in do
return getattr(self.registry(), name)(*args, **kwargs)
File "/home/stefan/.virtualenvs/dl4cv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 1042, in commit
self.transaction.commit()
File "/home/stefan/.virtualenvs/dl4cv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 504, in commit
self._prepare_impl()
File "/home/stefan/.virtualenvs/dl4cv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 483, in _prepare_impl
self.session.flush()
File "/home/stefan/.virtualenvs/dl4cv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2523, in flush
self._flush(objects)
File "/home/stefan/.virtualenvs/dl4cv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2664, in _flush
transaction.rollback(_capture_exception=True)
File "/home/stefan/.virtualenvs/dl4cv/lib/python3.6/site-packages/sqlalchemy/util/langhelpers.py", line 69, in __exit__
exc_value, with_traceback=exc_tb,
File "/home/stefan/.virtualenvs/dl4cv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 178, in raise_
raise exception
File "/home/stefan/.virtualenvs/dl4cv/lib/python3.6/site-packages/sqlalchemy/orm/session.py", line 2624, in _flush
flush_context.execute()
File "/home/stefan/.virtualenvs/dl4cv/lib/python3.6/site-packages/sqlalchemy/orm/unitofwork.py", line 422, in execute
rec.execute(self)
File "/home/stefan/.virtualenvs/dl4cv/lib/python3.6/site-packages/sqlalchemy/orm/unitofwork.py", line 589, in execute
uow,
File "/home/stefan/.virtualenvs/dl4cv/lib/python3.6/site-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj
insert,
File "/home/stefan/.virtualenvs/dl4cv/lib/python3.6/site-packages/sqlalchemy/orm/persistence.py", line 1136, in _emit_insert_statements
statement, params
File "/home/stefan/.virtualenvs/dl4cv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1014, in execute
return meth(self, multiparams, params)
File "/home/stefan/.virtualenvs/dl4cv/lib/python3.6/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/home/stefan/.virtualenvs/dl4cv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1133, in _execute_clauseelement
distilled_params,
File "/home/stefan/.virtualenvs/dl4cv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1318, in _execute_context
e, statement, parameters, cursor, context
File "/home/stefan/.virtualenvs/dl4cv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1512, in _handle_dbapi_exception
sqlalchemy_exception, with_traceback=exc_info[2], from_=e
File "/home/stefan/.virtualenvs/dl4cv/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 178, in raise_
raise exception
File "/home/stefan/.virtualenvs/dl4cv/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1278, in _execute_context
cursor, statement, parameters, context
File "/home/stefan/.virtualenvs/dl4cv/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 593, in do_execute
cursor.execute(statement, parameters)
File "/home/stefan/.virtualenvs/dl4cv/lib/python3.6/site-packages/pymysql/cursors.py", line 163, in execute
result = self._query(query)
File "/home/stefan/.virtualenvs/dl4cv/lib/python3.6/site-packages/pymysql/cursors.py", line 321, in _query
conn.query(q)
File "/home/stefan/.virtualenvs/dl4cv/lib/python3.6/site-packages/pymysql/connections.py", line 504, in query
self._execute_command(COMMAND.COM_QUERY, sql)
File "/home/stefan/.virtualenvs/dl4cv/lib/python3.6/site-packages/pymysql/connections.py", line 773, in _execute_command
self.write_packet(sql[:packet_size])
File "/home/stefan/.virtualenvs/dl4cv/lib/python3.6/site-packages/pymysql/connections.py", line 634, in write_packet
self._write_bytes(data)
File "/home/stefan/.virtualenvs/dl4cv/lib/python3.6/site-packages/pymysql/connections.py", line 710, in _write_bytes
"MySQL server has gone away (%r)" % (e,))
Killed
I've read somewhere that increasing MySQL connection timeout might help, but not in my case. Here is my DB engine configuration:
app.config['SQLALCHEMY_TRACK_MODIFICATIONS'] = False
app.config['SQLALCHEMY_DATABASE_URI'] = 'mysql+pymysql://root:''#localhost/********'
app.config['PROPAGATE_EXCEPTIONS'] = True
app.config["SQLALCHEMY_ENGINE_OPTIONS"] = {"pool_recycle": 120}
I want to access an oracle database through sqlalchemy in flask development. An engine is created by sqlalchemy create_engine()
MemoryError is reported when engine.connect() is called.
If setting a wrong password in the linkword, it would report:
sqlalchemy.exc.DatabaseError: (cx_Oracle.DatabaseError) ORA-01017: invalid username/password; logon denied.
With the correct password and user name, MemoryError would appear.
It is very strange when i copy the codes into jupyter or a new project (not a flask project)in pycharm, the codes work well. I wonder if it is possible the flask has problems.
from sqlalchemy import create_engine
#test_bp.route('/testtd')
def testtd():
#the linkword has no problem because these codes works in Jupyter or a new project
linkword = 'blablabla'
engine = create_engine(linkword)
#here i got a problem. memoryError was reported.
conn = engine.connect()
ret = conn.execute('select table_name from user_tables').fetchall()
print(ret)
conn.close()
Errors reported by pycharm are as follows:
File "E:\Projects\Python\DAM2\blueprints\test.py", line 19, in testtd
conn = engine.connect()
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\engine\base.py", line 2196, in connect
return self._connection_cls(self, **kwargs)
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\engine\base.py", line 103, in __init__
else engine.raw_connection()
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\engine\base.py", line 2296, in raw_connection
self.pool.unique_connection, _connection
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\engine\base.py", line 2265, in _wrap_pool_connect
return fn()
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\pool\base.py", line 303, in unique_connection
return _ConnectionFairy._checkout(self)
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\pool\base.py", line 760, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\pool\base.py", line 492, in checkout
rec = pool._do_get()
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\pool\impl.py", line 139, in _do_get
self._dec_overflow()
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\util\langhelpers.py", line 68, in __exit__
compat.reraise(exc_type, exc_value, exc_tb)
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\util\compat.py", line 153, in reraise
raise value
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\pool\impl.py", line 136, in _do_get
return self._create_connection()
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\pool\base.py", line 308, in _create_connection
return _ConnectionRecord(self)
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\pool\base.py", line 437, in __init__
self.__connect(first_connect_check=True)
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\pool\base.py", line 649, in __connect
).exec_once(self.connection, self)
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\event\attr.py", line 287, in exec_once
self(*args, **kw)
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\event\attr.py", line 297, in __call__
fn(*args, **kw)
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\util\langhelpers.py", line 1443, in go
return once_fn(*arg, **kw)
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\engine\strategies.py", line 199, in first_connect
dialect.initialize(c)
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\dialects\oracle\cx_oracle.py", line 832, in initialize
super(OracleDialect_cx_oracle, self).initialize(connection)
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\dialects\oracle\base.py", line 1140, in initialize
super(OracleDialect, self).initialize(connection)
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\engine\default.py", line 297, in initialize
connection
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\dialects\oracle\base.py", line 1239, in _get_default_schema_name
connection.execute("SELECT USER FROM DUAL").scalar()
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\engine\base.py", line 982, in execute
return self._execute_text(object_, multiparams, params)
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\engine\base.py", line 1155, in _execute_text
parameters,
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\engine\base.py", line 1248, in _execute_context
e, statement, parameters, cursor, context
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\engine\base.py", line 1468, in _handle_dbapi_exception
util.reraise(*exc_info)
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\util\compat.py", line 153, in reraise
raise value
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\engine\base.py", line 1244, in _execute_context
cursor, statement, parameters, context
File "C:\Users\GAC\.virtualenvs\DAM2-SwV_IMXP\lib\site-packages\sqlalchemy\engine\default.py", line 552, in do_execute
cursor.execute(statement, parameters)
MemoryError
how can i fix the codes?
My app is working on my main PC. I tried to clone the app to my laptop (I tried to initialize it on PC in a different directory and the problem was the same so the problem isn't with the laptop) and when initializing the db with flask db init I got the following error:
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such table: race [SQL: 'SELECT race.id AS race_id, race.name AS race_name \nFROM race'] (Background on this error at: http://sqlalche.me/e/e3q8)
What I'm trying to do is in one of my forms to include a wtforms RadioField with choices based on the race table in db. So for every row in that table I want one choice in the selection. The important part of the form looks like this:
class RegistrationForm(FlaskForm):
race = RadioField('Race', validators=[InputRequired('Choose a race.')],
choices=races, coerce=int)
And the races variable is outside the form like this (before the form itself):
with create_app().app_context():
races = [(race.id, race.name.capitalize()) for race in Race.query.all()]
I figured that this last line is the problem from the call stack:
Traceback (most recent call last):
File "/home/dareon4/workspace/test/venv/bin/flask", line 11, in <module>
sys.exit(main())
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/flask/cli.py", line 894, in main
cli.main(args=args, prog_name=name)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/flask/cli.py", line 557, in main
return super(FlaskGroup, self).main(*args, **kwargs)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/click/core.py", line 697, in main
rv = self.invoke(ctx)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/click/core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/click/core.py", line 895, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/click/core.py", line 535, in invoke
return callback(*args, **kwargs)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/click/decorators.py", line 17, in new_func
return f(get_current_context(), *args, **kwargs)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/flask/cli.py", line 411, in decorator
with __ctx.ensure_object(ScriptInfo).load_app().app_context():
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/flask/cli.py", line 372, in load_app
app = locate_app(self, import_name, name)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/flask/cli.py", line 235, in locate_app
__import__(module_name)
File "/home/dareon4/workspace/test/shapes.py", line 5, in <module>
app = create_app()
File "/home/dareon4/workspace/test/app/__init__.py", line 43, in create_app
from app.blueprints.auth import bp as auth_bp
File "/home/dareon4/workspace/test/app/blueprints/auth/__init__.py", line 7, in <module>
from . import routes
File "/home/dareon4/workspace/test/app/blueprints/auth/routes.py", line 13, in <module>
from .forms import LoginForm, RegistrationForm, ResetPasswordRequestForm, \
File "/home/dareon4/workspace/test/app/blueprints/auth/forms.py", line 19, in <module>
races = [(race.id, race.name.capitalize()) for race in Race.query.all()]
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/sqlalchemy/orm/query.py", line 2836, in all
return list(self)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/sqlalchemy/orm/query.py", line 2988, in __iter__
return self._execute_and_instances(context)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/sqlalchemy/orm/query.py", line 3011, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 948, in execute
return meth(self, multiparams, params)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/sqlalchemy/sql/elements.py", line 269, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1060, in _execute_clauseelement
compiled_sql, distilled_params
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1200, in _execute_context
context)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1413, in _handle_dbapi_exception
exc_info
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/sqlalchemy/util/compat.py", line 265, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/sqlalchemy/util/compat.py", line 248, in reraise
raise value.with_traceback(tb)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
context)
File "/home/dareon4/workspace/test/venv/lib64/python3.6/site-packages/sqlalchemy/engine/default.py", line 509, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such table: race [SQL: 'SELECT race.id AS race_id, race.name AS race_name \nFROM race'] (Background on this error at: http://sqlalche.me/e/e3q8)
So the problem is that I don't have the db initialized when I'm running the flask db init command and that one line needs the db to exist. This probably doesn't have a solution so I'm not asking how to correct this implementation, but how to do it correctly to have form based on rows from the db. I don't have hundreds of races so I could hardcode them in but I wanted to do it this way.
So after some googling I found that you can add the RadioField choices dynamically in the views like this:
#bp.route('/register', methods=['GET', 'POST'])
def register():
form = RegistrationForm()
form.race.choices = [(race.id, race.name.capitalize()) for race in Race.query.all()]
This is exactly what I was looking for.
The issue I am having is with the mixing of sqlalchemy sessions with flask-sqlalchemy session. So for context, I am running a flask-restful app on the background and I have a file (runnable.py) with a main method which is supposed to run periodically for an average period of 4 weeks.
This file uses its own session (pure sqlalchemy session) and connects to this app (which uses flask-sqlalchemy session).
After a few hours (sometimes 1 day), I get this error
Traceback (most recent call last):
File "/home/ubuntu/cron/runnable.py", line 781, in <module>
main()
File "/home/ubuntu/cron/runnable.py", line 677, in main
db_process_attendee(attendee, eid, event_id)
File "/home/ubuntu/cron/runnable.py", line 78, in db_process_attendee
update_attendee_information(attendee, attributes, eid, event_id)
File "/home/ubuntu/cron/runnable.py", line 117, in update_attendee_information
update_attendee_profile(attendee, id_attendee_login, eid, event_id)
File "/home/ubuntu/cron/runnable.py", line 291, in update_attendee_profile
generate_company_tags_lower.generate_company_tags(id_attendee_login)
File "/home/ubuntu/cron/app/util/generate_company_tags_lower.py", line 28, in generate_company_tags
filter(a_p.id_attendee_login == id_attendee_login).\
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2426, in scalar
ret = self.one()
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2395, in one
ret = list(self)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2438, in __iter__
return self._execute_and_instances(context)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/orm/query.py", line 2453, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 729, in execute
return meth(self, multiparams, params)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/sql/elements.py", line 322, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 826, in _execute_clauseelement
compiled_sql, distilled_params
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 958, in _execute_context
context)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 1159, in _handle_dbapi_exception
exc_info
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/util/compat.py", line 199, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/base.py", line 951, in _execute_context
context)
File "/usr/local/lib/python2.7/dist-packages/sqlalchemy/engine/default.py", line 436, in do_execute
cursor.execute(statement, parameters)
File "/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 134, in execute
result = self._query(query)
File "/usr/local/lib/python2.7/dist-packages/pymysql/cursors.py", line 282, in _query
conn.query(q)
File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 767, in query
self._execute_command(COMMAND.COM_QUERY, sql)
File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 957, in _execute_command
self._write_bytes(prelude + sql[:chunk_size-1])
File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 916, in _write_bytes
raise err.OperationalError(2006, "MySQL server has gone away (%r)" % (e,))
sqlalchemy.exc.OperationalError: (OperationalError) (2006, "MySQL server has gone away (error(110, 'Connection timed out'))") 'SELECT a_p.id_attendee_profile AS a_p_id_attendee_profile, a_p.id_attendee_login AS a_p_id_attendee_login \nFROM a_p \nWHERE a_p.id_attendee_login = %s' (128368,)
I tried the following suggestions but they did not work:
• In my app config, adding "pymysql" to the SQLALCHEMY_DATABASE_URI
• Doing a db.session.commit() when I enter generate_company_tags(id_attendee_login) before writing a sql select query. I thought it would flush the session.
Btw, the sqlalchemy pool recycle value is set to less than the max in AWS RDS.
Any suggestions on how I can go solving this?