We Keep Coding

sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unknown database "mydb"

I have a problem to use flask_alchemy for my unit test function
In the production environment I use a postgresql database
"SQLALCHEMY_DATABASE_URI": "postgresql://login:passwd#dburl:1234/mydatabase",
To work with the postgresql schema, in my entity definition i declare a _table_args to specify a schema
class MyTable(Base):
__tablename__ = 'my_tablename'
__table_args__ = {'schema': 'mydbschema'}
my_id = Column('my_id', Date, primary_key=True)
....
But in my unittest i would like to use a memory database
"SQLALCHEMY_DATABASE_URI": "sqlite://",
When i run my function i have this error :
E sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unknown database "mydbschema"
Somebody knows a workarround in this case ?

I know that OP found a workaround (though, I'm not sure how it works), but for those who still searching for proper solution - you need to use ATTACH DATABASE statement in order to select specific schema, for example if you are using pytest you can use following fixture to create test's setup:
from sqlalchemy import create_engine
#pytest.fixture
def setup_db():
engine = create_engine('sqlite:///:memory:')
with engine.connect() as conn:
conn.execute('ATTACH DATABASE \':memory:\' AS mydbschema;')
yield conn
Of course, you can define fixture scope by your needs.
Similarly, you can use DETACH DATABASE statement to detach and dissociate a named database from a database connection (if necessary), which will destroy DB in the case of in-memory databases.

I found a workaround ...
class MyTable(Base):
__tablename__ = 'my_tablename'
__table_args__ = {'schema': 'mydbschema'}
my_id = Column('my_id', Date, primary_key=True) if os.environ.get('TESTVAR') is None else {}
....

Is there a way to get the engine from the SQLAlchemy object?

I have the following code:
db = SQLAlchemy(
engine_options={ 'connect_args': { 'connect_timeout': 60 }}
)
basis_engine = create_engine(database_stages["dev"]["dev_basis"])
meta_data = MetaData()
meta_data.reflect(bind=basis_engine)
I've created the engine just to be able to access directly a single table, and that adds a great amount of overhead to the app when I start it up (It takes much more time). So, is there a way to get the engine from the SQLAlchemy object and avoid create_engine?

The db object (an instance of SQLAlchemy) has a get_engine method that will return an engine.

How to get cursor in SQLAlchemy

I am newbie in Python Flask. In my project we are creating db object using below code.
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:////tmp/test.db'
db = SQLAlchemy(app)
I want to get cursor object from db. Can someone please help me on it.
I know using connection object we can get cursor. But can we get cursor from db object which is created by above way? Thanks.

Finally got answer from Flask documentation, we can get cursor from db object using,
from sqlalchemy import create_engine
engine = create_engine('your_connection_string')
connection = engine.raw_connection()
cursor = connection.cursor()

It's ok in this way.
db = SQLAlchemy(app)
session = db.session()
cursor = session.execute(sql).cursor

create cursor from sqlalchemy orm session
from sqlalchemy.orm import sessionmaker
engine=create_engine(url)
session=sessionmaker(bind= engine)()
curs=session.connection().connection.cursor()

You don't have a cursor in Flask SQLAlchemy because it's an ORM. Methods exist to perform actions on the database directly, unlike packages like SQLite. Check the documentation for more info on these.
To execute SQL queries directly, you'd have to run them through the terminal directly through whichever RDBMS you're using (MySQL, PostgreSQL, etc.)

ProgrammingError, Flask with postgres and sqlalchemy

I am trying to get this setup to work, the database is created correctly, but trying to insert data I get the following error:
On sqlite:
sqlalchemy.exc.OperationalError
OperationalError: (sqlite3.OperationalError) no such column: Author [SQL: u'SELECT count(*) AS count_1 \nFROM (SELECT Author) AS anon_1']
On postgres:
sqlalchemy.exc.ProgrammingError
ProgrammingError: (psycopg2.ProgrammingError) column "author" does not exist
LINE 2: FROM (SELECT Author) AS anon_1
^
[SQL: 'SELECT count(*) AS count_1 \nFROM (SELECT Author) AS anon_1']
edit: Perhaps this has to do with it: I don't understand why it says "anon_1", as I am using credentials clearly?
I have inspected postgres and sqlite and the tables are created correctly. It seems to be an ORM configuration error, as it only seems to happend on inspecting or creating entries, any suggestion would be welcome!
class Author(CommonColumns):
__tablename__ = 'author'
author = Column(String(200))
author_url = Column(String(2000))
author_icon = Column(String(2000))
comment = Column(String(5000))
registerSchema('author')(Author)
SETTINGS = {
'SQLALCHEMY_TRACK_MODIFICATIONS': True,
'SQLALCHEMY_DATABASE_URI': 'sqlite:////tmp/test.db',
# 'SQLALCHEMY_DATABASE_URI': 'postgresql://xxx:xxx#localhost/test',
}
application = Flask(__name__)
# bind SQLAlchemy
db = application.data.driver
Base.metadata.bind = db.engine
db.Model = Base
db.create_all()
if __name__ == "__main__":
application.run(debug=True)

What is the query you're using to insert data?
I think the error messages may be a bit more opaque than they need to be because you're using Author/author in three very similar contexts:
the Class name
the table name
the column name
For easier debugging, the first thing I'd do is temporarily make each one unique (AuthorClass, author_table, author_column) so you can check which 'Author' is actually being referred to by the error message.
Since you're using the ORM, I suspect the underlying issue is that your insert statement uses Author (the object) when it should actually be using Author.author (the attribute/column name). The SELECT statements are complaining that they can't find the column 'author', but because you use author for both the table and column name, it's unclear what's actually being passed into the SQL statement.

How to handle multiple requests without SQLAlchemy crashing/raising exceptions?

Context: I'm working on a Flask app, running on CherryPy, DB handled using SQLAlchemy ORM.
Problem:
The app runs fine and does everything I want, however, if I have a page which fetches some data from DB and displays, and I press and hold "Ctrl + R" or "F5". That is, continuously refresh a page, making that many DB requests. First few goes fine, and then it breaks.
The following errors are logged:
(OperationalError) (2013, 'Lost connection to MySQL server during query')
Can't reconnect until invalid transaction is rolled back (original cause:
InvalidRequestError: Can't reconnect until invalid transaction is rolled back)
This result object does not return rows. It has been closed automatically.
(ProgrammingError) (2014, "Commands out of sync; you can't run this command now")
There's also another error which bothers me (but not logged this time), it's
dictionary changed size during iteration
This happens when I'm iterating through a query, using values obtained to populate a dictionary. The dictionary is local (scope of the dict) to the function.
More info:
How I am handling sessions:
A new session is created when you enter any page, use that session to perform all the DB transactions, and the session is closed right before rendering the HTML. Technically, that means, the scope of session is the same as the HTTP request.
I do a session.rollback() only when there's an exception raised during updating table or inserting into a table. No rollback() during any query() operations.
I'm pretty sure I've made some silly mistakes or am not doing things the right way.
Unlimited refreshes like that is not really a probably scenario, but can't be overlooked.
Also, I think the behavior would be similar when there a lot of users using it at the same time.
How the SQLAlchemy engine, sessionmaker was handled:
sql_alchemy_engine = create_engine(self.db_string, echo=False, encoding="utf8", convert_unicode=True, pool_recycle=9)
sqla_session = sessionmaker(bind=sql_alchemy_engine)
It's done only ONCE like it's recommended in the SQLA documentation, and a new session is created and returned sqla_session() whenever required.

If you're using Flask, you should be using flask-sqlalchemy, and let the Flask request context manage your session, and not handling your engine and sessions by hand. This is how SQLAlchemy recommends it:
Most web frameworks include infrastructure to establish a single Session, associated with the request, which is correctly constructed and torn down corresponding torn down at the end of a request. Such infrastructure pieces include products such as Flask-SQLAlchemy, for usage in conjunction with the Flask web framework, and Zope-SQLAlchemy, for usage in conjunction with the Pyramid and Zope frameworks. SQLAlchemy strongly recommends that these products be used as available.
http://docs.sqlalchemy.org/en/rel_0_9/orm/session.html?highlight=flask
Then you create your engine simply by:
from flask import Flask
from flask.ext.sqlalchemy import SQLAlchemy
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'your db uri'
db = SQLAlchemy(app)
Or, if you're using app factory:
from flask import Flask
from flask.ext.sqlalchemy import SQLAlchemy
db = SQLAlchemy()
def create_app():
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'your db uri'
db.init_app(app)
With that, the base declarative model you should be using will be at db.Model and the session you should be using will be at db.session.