I don't understand why I have this error. Please explain the error.
I used official documentation.
I run Pipenv virtual env:
python 3.8.2
sqlalchemy 1.3.16
You can try run this code too.
import enum
from sqlalchemy import create_engine, Column, Integer, String, Enum
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker
engine = create_engine('sqlite:///:memory:', echo=True)
Base = declarative_base()
Session = sessionmaker(bind=engine)
session = Session()
class Type(str, enum.Enum):
ONE = "one"
TWO = "two"
class Item(Base):
__tablename__ = 'items'
id = Column(Integer, primary_key=True, index=True)
name = Column(String, unique=True, index=True)
type = Column(Enum(Type), default=Type.ONE, nullable=False)
item = Item(name="item_name", type="one")
session.add(item)
print(Item.__table__)
session.commit()
for name in session.query(Item.name):
print(name)
I added:
engine = create_engine('sqlite:///:memory:', echo=True)
Base = declarative_base()
Base.metadata.create_all(bind=engine)
Session = sessionmaker(bind=engine)
session = Session()
It creates the tables in the database (there are ways of using SQLAlchemy with preexisting tables, so an explicit instruction isd necessary).
In case it ends up helping someone, my issue was due to using an in memory sqlite db to unittest an API. Pytest would set up the database and tables with one connection while the API would create its own separate connection when the test were hitting endpoints. This ultimately solved it for me.
engine = create_engine("sqlite:///:memory:", poolclass=StaticPool, connect_args={'check_same_thread': False})
Ref: https://docs.sqlalchemy.org/en/14/dialects/sqlite.html#using-a-memory-database-in-multiple-threads
Related
I am working with SQLAlchemy and I am getting confused between Engine, Metadata, Base, and Sessions.
I did find some useful links: here, here and here, but I think it would help to know why the code below is not working.
import sqlalchemy as sqlalc
from sqlalchemy import Column, INTEGER, Integer,ForeignKey, Table, VARCHAR, TIMESTAMP, MetaData, create_engine, inspect, cast, select, SmallInteger
from sqlalchemy.orm import relationship, backref, sessionmaker
from sqlalchemy.ext.declarative import declarative_base
url = 'mysql://info#127.0.0.1/info'
engine = create_engine(url, echo=True)
metadata = MetaData()
metadata.bind = engine
metadata.create_all(engine)
connection = engine.connect()
Base = declarative_base()
Base.metadata.create_all()
Session = sessionmaker(bind=engine)
session = Session()
When I execute, I get the following error:
sqlalchemy.exc.UnboundExecutionError: MetaData object is not bound to an Engine or Connection. Execution can not proceed without a database to execute against.
Definitions
Engine: an abstraction for a database connection, or connection pool
Metadata: an object that knows about database objects, primarily tables
Base: the base class for ORM models
Session: an object that keeps track of new, removed and changed ORM model instances while they are in use; it sends changes to the database (Session.commit()) or reverts them (Session.rollback())
Code
This code
metadata = MetaData()
metadata.bind = engine
metadata.create_all(engine)
"works" because you have bound (or associated) an engine with your MetaData instance: the engine enables connection to a database to create any tables that have been registered with the MetaData instance. However you should be aware that binding like this is deprecated in SQLAlchemy v1.4. The preferred approach is to pass the engine to create_all like this:
metadata = MetaData()
metadata.create_all(engine)
This code
Base = declarative_base()
Base.metadata.create_all()
does not work because there is no engine associated with Base.metadata. As before, the preferred approach is to pass an engine to create_all:
Base.metadata.create_all(engine)
In both cases, executing create_all will do nothing unless tables have been registered with the metadata instances. For core, this usually happens through table definitions:
metadata = MetaData()
t = Table(
't',
metadata, # <- this will associate the table with the metadata
Column(...),
)
In the ORM, (using declarative mapping), the association is made by inheriting from Base:
Base = orm.declarative_base()
class MyModel(Base): # <- this will register mytable with Base's metadata
__tablename__ = 'mytable'
...
I'm trying to use schema_translate_map to change a schema:
Base = declarative_base()
class DataAccessLayer():
def __init__(self):
conn_string = "mysql+mysqlconnector://root:root#localhost/"
self.engine = create_engine(conn_string)
Session = sessionmaker()
Session.configure(bind=self.engine)
self.session = Session()
def change_schema(self):
self.session.connection(execution_options={"schema_translate_map": {"belgarath": "belgarath_test"}})
class Player(Base):
__tablename__ = "player"
__table_args__ = {'schema': "belgarath"}
id_ = Column(Integer, primary_key=True)
dal = DataAccessLayer()
dal.change_schema()
qry = dal.session.query(Player.id_)
print(qry)
However, the SQL comes out as:
SELECT belgarath.player.id_ AS belgarath_player_id_
FROM belgarath.player
Instead of:
SELECT belgarath_test.player.id_ AS belgarath_test_player_id_
FROM belgarath_test.player
Where am I going wrong?
Try what happens if you simply append .all() to your qry:
from sqlalchemy import Integer
from sqlalchemy import Column
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class DataAccessLayer():
def __init__(self):
conn_string = "sqlite:///:memory:"
#conn_string = "mysql+mysqlconnector://root:root#localhost/"
self.engine = create_engine(conn_string)
Session = sessionmaker()
Session.configure(bind=self.engine)
self.session = Session()
def change_schema(self):
self.session.connection(execution_options={"schema_translate_map": {"belgarath": "belgarath_test"}})
class Player(Base):
__tablename__ = "player"
__table_args__ = {'schema': "belgarath"}
id_ = Column(Integer, primary_key=True)
dal = DataAccessLayer()
dal.change_schema()
qry = dal.session.query(Player.id_)
print(qry.all())
Output (without trace):
OperationalError: (sqlite3.OperationalError) no such table: belgarath_test.player
[SQL: SELECT belgarath_test.player.id_ AS belgarath_player_id_
FROM belgarath_test.player]
(Background on this error at: http://sqlalche.me/e/13/e3q8)
I'm not an expert, but I guess this might be related to the following issue:
the schema translate feature takes place within the compiler and this is plainly wrong. the schema assignment should be taking place after the SQL is generated so that we only need one cache key. This is along the lines of #5002 however I think even the existing cache key mechanism used with baked etc. needs to pull the schema translate out of the compiler entirely for 1.4 and add it to the translations which occur from the ExecutionContext, along with the expanding parameter sets of logic. schema translate is intended to service many hundreds / thousands of schemas so having this occur pre-cache has to change.
I guess that Query API is not aware of execution_options in connection. Try not to mix this two approaches.
dal = DataAccessLayer()
dal.change_schema()
qry = dal.session.connection().execute(Player.__table__.select())
print(qry)
Result:
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such table: belgarath_test.player
[SQL: SELECT belgarath_test.player.id_
FROM belgarath_test.player]
(Background on this error at: http://sqlalche.me/e/13/e3q8)
I'm newer in SQLAlchemy I use some examples to create table and insert information to it and it's working 100% .
But what I didn't find is some example for how can I update & delete some information from the database.
What I'm doing is :
from sqlalchemy import Column, ForeignKey, Integer, String
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker
from sqlalchemy import create_engine
Base = declarative_base()
## create
class Person(Base):
__tablename__ = 'person'
id = Column(Integer, primary_key=True)
name = Column(String(250), nullable=False)
engine = create_engine('sqlite:///database.db')
Base.metadata.create_all(engine)
## insert
Base.metadata.bind = engine
DBSession = sessionmaker(bind=engine)
session = DBSession()
new_person = Person(name='new person')
session.add(new_person)
session.commit()
## fetch
getperson = session.query(Person).first()
print getperson.name
# this will print : new person
# I need some example to how can I update and delete this : new person
So in this code it'll print "new person" my question is how can I update or delete it ?
Here's some example on each CRUD operation in sqlalchemy (ommiting Create, Read as you already know how to perform those):
First, necessary imports and configs for any operation:
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
# Category, Item, User are my tables
from database_setup import Base, Category, Item, User
# Generating session to connect to the db's ORM
engine = create_engine('sqlite:///catalogwithusers.db') # my db
Base.metadata.bind = engine
DBSession = sessionmaker(bind = engine)
session = DBSession()
Then peforming an update:
# Get the item filtering by it's id using a one() query on Item table
# If query is not empty, update the attributes, add query to session and commit
q = session.query(Item).filter_by(id=item_id).one()
if q != []:
q.name = edited_name
q.description = edited_description
session.add(q)
session.commit()
Finally, performing a deletion:
# Again get the item similarly to the example above
# Then if query returned results, use the delete method and commit
q = session.query(Item).filter_by(id=item_id).one()
if q != []:
session.delete(q)
session.commit()
These examples are taken from here. I suggest you have a look. ORM Creation is inside database_setup.py and CRUD ops are performed inside project.py and populatecatalog.py.
I have a little problem regarding "arranging of code, which depends on each other" when setting an sqlalchemy mapping to a sqlite database in python.
The goal is to write a script, whcih satisfies the following conditions:
Gets a filename parameter as command line argument.
Based on the filename it should create an absolute path to the SQLite database.
It should connect to the database and create an engine
It shall reflect the tables in this databases.
It should monkey patch the column id in the table mytable as a primary key column, since the table doesn't habe a primary key and sqlalchemy requires one.
So I came up with this:
from sqlalchemy import create_engine, Column, Integer
from sqlalchemy.orm import sessionmaker
from sqlalchemy.ext.declarative import declarative_base
def create_path(file_name):
# generate absolute path to file_name
path_to_file = create_path("my_file_name.sqlite")
engine = create_engine('sqlite:///{path}'.format(path=path_to_file), echo=False)
Base = declarative_base()
Base.metadata.create_all(engine)
class MyTable(Base):
__tablename__ = 'my_table'
__table_args__ = {'autoload': True}
id = Column(Integer, primary_key=True) # Monkey patching the id column as primary key.
def main(argv):
# parse file_name here from argv
Session = sessionmaker(bind=engine)
session = Session()
for row in session.query(MyTable).all():
print row
return "Stop!"
if __name__ == '__main__':
sys.exit(main())
But this is a doomed construction and I don't see how I could rearrange my code without breaking the dependencies.
To be able to create MyClass I need Base to be defined before MyClass.
To be able to create Base I need engine to be defined before Base.
To be able to create engine I need path_to_file to be defined before engine.
To be able to create path_to_file outside of main() I need create_file() to be defined before path_to_file.
And so on...
Hopefully you see where I am stuck...
Any suggestions?
Edit: By the way, the code works, but only with a hardcoded filename in the top of the script.
I do not see why you could not use the declarative mapping still complety. I think the key to the proper dependencies is to know what to put in the module and what in the script/function. With python, you can easily define the class inside the run_script function.
from sqlalchemy import create_engine, Column, Integer
from sqlalchemy.orm import sessionmaker
from sqlalchemy.ext.declarative import declarative_base
def create_path(filename):
import os
fn = os.path.abspath(os.path.join(os.path.dirname(__file__), 'sqlite_files', filename))
return fn
def run_script(filename):
path_to_file = create_path(filename)
engine = create_engine('sqlite:///{path}'.format(path=path_to_file), echo=True)
Session = sessionmaker(bind=engine)
Base = declarative_base(bind=engine)
# object model
class MyTable(Base):
__tablename__ = 'my_table'
__table_args__ = {'autoload': True}
id = Column(Integer, primary_key=True) # Monkey patching the id column as primary key
# script itself
session = Session()
for row in session.query(MyTable).all():
print row
return "Stop!"
def main(argv):
assert argv, "must specify a database file name"
run_script(argv[0])
if __name__ == '__main__':
import sys
sys.exit(main(sys.argv))
I solved my problem using the classical mapping approach of SQLAlchemy:
from sqlalchemy import create_engine, Column, Integer, MetaData, Table
from sqlalchemy.orm import sessionmaker, mapper
class MyTable(object):
pass
def main():
path_to_file = create_path("my_file_name.sqlite")
engine = create_engine('sqlite:///{path}'.format(path=path_to_file), echo=False)
metadata = MetaData()
metadata.bind = engine
my_table = Table('mytable',
metadata,
Column('id', Integer, primary_key=True), # Monkey patching the id column as primary key.
autoload=True,
autoload_with=engine)
mapper(MyTable, my_table)
Session = sessionmaker(bind=engine)
session = Session()
# Do Stuff!
I don't have the capability to do any testing at my current location. But I suggest moving all of that code to main() as follows. Also, MyTable subclasses declarative_base.
from sqlalchemy import create_engine, Column, Integer
from sqlalchemy.orm import sessionmaker from sqlalchemy.ext.declarative import declarative_base
class MyTable(declarative_base):
__tablename__ = 'my_table'
__table_args__ = {'autoload': True}
id = Column(Integer, primary_key=True) # Monkey patching the id column as primary key.
def create_path(file_name):
# generate absolute path to file_name
def main(argv): # parse file_name here from argv
path_to_file = create_path("my_file_name.sqlite")
engine = create_engine('sqlite:///{path}'.format(path=path_to_file), echo=False)
Base = MyTable()
Base.metadata.create_all(engine)
Session = sessionmaker(bind=engine)
session = Session()
for row in session.query(MyTable).all():
print row
return "Stop!"
if __name__ == '__main__':
sys.exit(main())
I access a a postgres table using SQLAlchemy. I want a query to have eagerloading.
from sqlalchemy.orm import sessionmaker, scoped_session, eagerload
from settings import DATABASE_USER, DATABASE_PASSWORD, DATABASE_HOST, DATABASE_PORT, DATABASE_NAME
from sqlalchemy import create_engine
from sqlalchemy import Table, Column, Integer, String, Boolean, MetaData, ForeignKey
from sqlalchemy.orm import mapper
from sqlalchemy.ext.declarative import declarative_base
def create_session():
engine = create_engine('postgres://%s:%s#%s:%s/%s' % (DATABASE_USER, DATABASE_PASSWORD, DATABASE_HOST, DATABASE_PORT, DATABASE_NAME), echo=True)
Session = scoped_session(sessionmaker(bind=engine))
return Session()
Base = declarative_base()
class Zipcode(Base):
__tablename__ = 'zipcode'
zipcode = Column(String(6), primary_key = True, nullable=False)
city = Column(String(30), nullable=False)
state = Column(String(30), nullable=False)
session = create_session()
query = session.query(Zipcode).options(eagerload('zipcode')).filter(Zipcode.state.in_(['NH', 'ME']))
#query = session.query(Zipcode.zipcode).filter(Zipcode.state.in_(['NH', 'ME']))
print query.count()
This fails with
AttributeError: 'ColumnProperty' object has no attribute 'mapper'
One without eagerloading returns the records correctly.
I am new to SQLAlchemy. I am not sure what the problem is. Any pointers?
You can only eager load on a relation property. Not on the table itself. Eager loading is meant for loading objects from other tables at the same time as getting a particular object. The way you load all the objects for a query will be simply adding all().
query = session.query(Zipcode).options(eagerload('zipcode')).filter(Zipcode.state.in_(['NH', 'ME'])).all()
The query will now be a list of all objects (rows) in the table and len(query) will give you the count.