We Keep Coding

sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such table: items

I don't understand why I have this error. Please explain the error.
I used official documentation.
I run Pipenv virtual env:
python 3.8.2
sqlalchemy 1.3.16
You can try run this code too.
import enum
from sqlalchemy import create_engine, Column, Integer, String, Enum
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker
engine = create_engine('sqlite:///:memory:', echo=True)
Base = declarative_base()
Session = sessionmaker(bind=engine)
session = Session()
class Type(str, enum.Enum):
ONE = "one"
TWO = "two"
class Item(Base):
__tablename__ = 'items'
id = Column(Integer, primary_key=True, index=True)
name = Column(String, unique=True, index=True)
type = Column(Enum(Type), default=Type.ONE, nullable=False)
item = Item(name="item_name", type="one")
session.add(item)
print(Item.__table__)
session.commit()
for name in session.query(Item.name):
print(name)

I added:
engine = create_engine('sqlite:///:memory:', echo=True)
Base = declarative_base()
Base.metadata.create_all(bind=engine)
Session = sessionmaker(bind=engine)
session = Session()
It creates the tables in the database (there are ways of using SQLAlchemy with preexisting tables, so an explicit instruction isd necessary).

In case it ends up helping someone, my issue was due to using an in memory sqlite db to unittest an API. Pytest would set up the database and tables with one connection while the API would create its own separate connection when the test were hitting endpoints. This ultimately solved it for me.
engine = create_engine("sqlite:///:memory:", poolclass=StaticPool, connect_args={'check_same_thread': False})
Ref: https://docs.sqlalchemy.org/en/14/dialects/sqlite.html#using-a-memory-database-in-multiple-threads

Django: create a separate set of models in in-memory SQLite DB on the flight

What I'm trying to do:
I have a general huge set of models in my Django (v2.3.3) application in PostgreSQL DB, but now for a specific task I need to create a rather complicated aggregation of DB objects. And it would be much easier to work with them if I could only in that thread/process that handles this specific web-request create an in-memory SQLite DB, define a new set of model-classes there (without foreign keys to the global set of models of course), create some objects in that DB, do my calculations and kill that DB upon the response serving. For consistency I would like to use Django models for this small DB as well.
Is this possible? Or do you have some better ideas how to approach this?

Eventually I've ended up with SQLAlchemy + in-memory SQLite solution:
from sqlalchemy import Column, Integer, String, ForeignKey, create_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship, sessionmaker, backref
Base = declarative_base()
engine = create_engine('sqlite:///:memory:')
Session = sessionmaker(bind=engine)
session = Session()
class Node(Base):
"""A node entity in sankey diagram."""
__tablename__ = 'node'
id = Column(Integer, primary_key=True)
name = Column(String) # unique - might include parts from event.extra_data
event = Column(String) # not unique - totally equeals to event.event
Base.metadata.create_all(engine)

How do I know whether SQLAlchemy had to create tables when create_all() is called?

I'd like to log whether or not SQLAlchemy has to create any database tables when create_all() is called, however I don't see any documentation on a return value from create_all(). How do I go about this?
I've tried setting up a simple in-memory database using the following code and called create_all(). With echo set to true, I can see that the table is created as expected, but myreturn has a type of NoneType. If I call create_all() a second time, the table isn't created, and myreturn is still NoneType.
from sqlalchemy import create_engine, Column, Integer, String, ForeignKey
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker, relationship
Base = declarative_base()
class User(Base):
__tablename__ = 'testtable'
id = Column('id', Integer, primary_key=True)
name = Column('name', String, unique=True)
engine = create_engine('sqlite:///:memory:', echo=True)
myreturn = Base.metadata.create_all(bind=engine)
Is there a way to identify whether the tables are created by create_all() or do I need to create additional logic to verify this in the database directly before invoking create_all() ?

You can use the after_create event. The event handler will be passed a keyword arg called tables which is a collection of any tables that were created within the create_all() method call.
from sqlalchemy import event
#event.listens_for(Base.metadata, 'after_create')
def receive_after_create(target, connection, tables, **kw):
"listen for the 'after_create' event"
if tables:
print('A table was created')
else:
print('A table was not created')

Flask sqlAlchemy: Creating models for the existing database structure

I am new to sqlAlchemy and I wonder if there is a way create a class that would be mapped to the existing table in DB without specifying any columns of the table (but all columns could be accessed as attributes of the object)?

This is how I generated models for flask-sqlalchemy which use MS SQL.
flask-sqlacodegen --flask mssql+pymssql://<username>:<password>#<server>/<database> --tables <table_names>> db_name.py
you have to install flask-sqlacodegen and pymssql though.

Have recently came across this issue. Try steps below:
from sqlalchemy import create_engine, MetaData, Table
from sqlalchemy.orm import mapper, sessionmaker
class User(object): # class which can can act as ORM class
pass
dbPath = 'places.sqlite'
engine = create_engine('sqlite:///%s' % dbPath, echo=True) # create engine
metadata = MetaData(engine)
user_table= Table('user_existing_class', metadata, autoload=True) # create a Table object
mapper(User, user_table) # map Table to ORM class
Session = sessionmaker(bind=engine)
session = Session()
res = session.query(User).all()
res[1].name

sqlalchemy existing database query

I am using SQLAlchemy as ORM for a python project. I have created few models/schema and it is working fine. Now I need to query a existing MySQL database, no insert/update just the select statement.
How can I create a wrapper around the tables of this existing database? I have briefly gone through the sqlalchemy docs and SO but couldn't find anything relevant. All suggest execute method, where I need to write the raw sql queries, while I want to use the SQLAlchemy query method in same way as I am using with the SA models.
For example if the existing db has table name User then I want to query it using the dbsession ( only the select operation, probably with join)

You seem to have an impression that SQLAlchemy can only work with a database structure created by SQLAlchemy (probably using MetaData.create_all()) - this is not correct. SQLAlchemy can work perfectly with a pre-existing database, you just need to define your models to match database tables. One way to do that is to use reflection, as Ilja Everilä suggests:
from sqlalchemy import Table
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class MyClass(Base):
__table__ = Table('mytable', Base.metadata,
autoload=True, autoload_with=some_engine)
(which, in my opinion, would be totally fine for one-off scripts but may lead to incredibly frustrating bugs in a "real" application if there's a potential that the database structure may change over time)
Another way is to simply define your models as usual taking care to define your models to match the database tables, which is not that difficult. The benefit of this approach is that you can map only a subset of database tables to you models and even only a subset of table columns to your model's fields. Suppose you have 10 tables in the database but only interested in users table from where you only need id, name and email fields:
import sqlalchemy as sa
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class User(Base):
id = sa.Column(sa.Integer, primary_key=True)
name = sa.Column(sa.String)
email = sa.Column(sa.String)
(note how we didn't need to define some details which are only needed to emit correct DDL, such as the length of the String fields or the fact that the email field has an index)
SQLAlchemy will not emit INSERT/UPDATE queries unless you create or modify models in your code. If you want to ensure that your queries are read-only you may create a special user in the database and grant that user SELECT privileges only. Alternatively/in addition, you may also experiment with rolling back the transaction in your application code.

You can access an existing table using the automap extension:
from sqlalchemy.ext.automap import automap_base
from sqlalchemy.orm import Session
Base = automap_base()
Base.prepare(engine, reflect=True)
Users = Base.classes.users
session = Session(engine)
res = session.query(Users).first()

Create a table with autoload enabled that will inspect it. Some example code:
from sqlalchemy.sql import select
from sqlalchemy import create_engine, MetaData, Table
CONN_STR = '…'
engine = create_engine(CONN_STR, echo=True)
metadata = MetaData()
cookies = Table('cookies', metadata, autoload=True,
autoload_with=engine)
cols = cookies.c
with engine.connect() as conn:
query = (
select([cols.created_at, cols.name])
.order_by(cols.created_at)
.limit(1)
)
for row in conn.execute(query):
print(row)

Other answers don't mention what to do if you have a table with no primary key, so I thought I would address this. Assuming a table called Customers that has columns for CustomerId, CustomerName, CustomerLocation you could do;
from sqlalchemy.ext.automap import automap_base
from sqlalchemy import create_engine, MetaData, Column, String, Table
from sqlalchemy.orm import Session
Base = automap_base()
conn_str = '...'
engine = create_engine(conn_str)
metadata = MetaData()
# you only need to define which column is the primary key. It can automap the rest of the columns.
customers = Table('Customers',metadata, Column('CustomerId', String, primary_key=true), autoload=True, autoload_with=engine)
Base.prepare()
Customers= Base.classes.Customers
session = Session(engine)
customer1 = session.query(Customers).first()
print(customer1.CustomerName)

Assume we have a Postgresql database named accounts. And we already have a table named users.
import sqlalchemy as sa
psw = "verysecret"
db = "accounts"
# create an engine
pengine = sa.create_engine('postgresql+psycopg2://postgres:' + psw +'#localhost/' + db)
from sqlalchemy.ext.declarative import declarative_base
# define declarative base
Base = declarative_base()
# reflect current database engine to metadata
metadata = sa.MetaData(pengine)
metadata.reflect()
# build your User class on existing `users` table
class User(Base):
__table__ = sa.Table("users", metadata)
# call the session maker factory
Session = sa.orm.sessionmaker(pengine)
session = Session()
# filter a record
session.query(User).filter(User.id==1).first()
Warning: Your table should have a Primary Key defined. Otherwise, Sqlalchemy won't like it.