We Keep Coding

SQLAlchemy: relationship collection lazy loading

As per the SQLAlchemy documentation on relationship loading:
When the given collection or reference is first accessed on a particular object, an additional SELECT statement is emitted such that the requested collection is loaded.
How do I achieve loading behavior such that only the single elements of a relationship collection that I access are loaded, rather than the entire collection all at once?
I have heard of deferred column loading; this would be more like "deferred row loading". Rather than deferring loading of attributes, I'd like to defer loading of relationship collection elements.
Desired use case:
# Persist instance.
coln = Collection([1, 2, 3])
session.add(coln)
session.commit()
# Test lazy loading.
print('data' in coln.__dict__)
# Lazy loads the entire collection. I'd like only one element.
print(coln.data[1])
# Will output: "True 3". I'd like: "True 1".
print('data' in coln.__dict__, len(coln.__dict__['data']))
Class definitions and other backwork:
from sqlalchemy import Column, Integer, ForeignKey
from sqlalchemy import create_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship
Base = declarative_base()
engine = create_engine('sqlite:///:memory:')
# Define classes.
class Collection(Base):
__tablename__ = 'collection'
id = Column(Integer, primary_key=True)
data = relationship('Element')
def __init__(self, list_):
self.data = [Element(e) for e in list_]
class Element(Base):
__tablename__ = 'element'
id = Column(Integer, primary_key=True)
parent_id = Column(Integer, ForeignKey('collection.id'))
value = Column(Integer)
def __init__(self, value):
self.value = value
def __repr__(self):
return 'Element({})'.format(self.value)
# Create schema.
Base.metadata.create_all(engine)
# Create session.
from sqlalchemy.orm import sessionmaker
Session = sessionmaker(bind=engine)
session = Session()

Use the lazy parameter with dynamic value:
data = relationship('Element', lazy='dynamic')
https://docs.sqlalchemy.org/en/13/orm/collections.html#dynamic-relationship

How to fetch referenced entities with SQLAlchemy?

I have a question concerning the mapping of entities in SQLAlchemy.
I have a transient object, which already contains foreign keys to some persistent objects. I want that SQLAlchemy fetches the referenced objects and assigns them to their relationship-attributes. From the SQLAlchemy documentation, I thought that I have to use the merge-operation on the session to achieve this. But in my configuration, it doesn't work.
This is a minimum example demonstrating my problem:
# -*- coding: utf-8 -*-
from sqlalchemy import create_engine
from sqlalchemy.orm import mapper
from sqlalchemy import Table, Column, Integer, String, MetaData, ForeignKey
from sqlalchemy.orm import relationship, sessionmaker
class User(object):
def __init__(self, id, name, fullname, password, best_friend_id=None):
self.id = id
self.name = name
self.fullname = fullname
self.password = password
self.best_friend_id = best_friend_id
def __repr__(self):
return "<User('%s','%s', '%s')>" % (self.name, self.fullname, self.password)
class Dog(object):
def __init__(self, id, name):
self.id = id
self.name = name
def __repr__(self):
return "<User('%s','%s', '%s')>" % (self.name, self.fullname, self.password)
engine = create_engine('sqlite:///:memory:', echo=True)
Session = sessionmaker(bind=engine)
session = Session()
metadata = MetaData()
dogs_table = Table('dogs', metadata,
Column('id', Integer, primary_key=True),
Column('name', String),
)
users_table = Table('users', metadata,
Column('id', Integer, primary_key=True),
Column('name', String),
Column('fullname', String),
Column('password', String),
Column('best_friend_id', Integer, ForeignKey('dogs.id'))
)
metadata.create_all(engine)
mapper(User, users_table, properties={'best_friend': relationship(Dog, uselist=False)})
mapper(Dog, dogs_table)
dog = Dog(id=1, name='Hasso')
lordling = User(id=2, name='John', fullname='Miller', password='very_secret', best_friend_id=1)
session.add(dog)
session.commit()
merged_lordling = session.merge(lordling)
print str(merged_lordling.best_friend.name)
I expect that merged_lordling.best_friend contains the dog 'Hasso'. But it is still None.

I was bit by this same problem recently. Once you established a relationship, you should simply assign your Dog instance to User.best_friend directly, not explicitly using the foreign key. I don't know why exactly that happens, but while investigating a similar problem I realized that if you do that, SQLAlchemy doesn't populate the relationship property until you flushed all the related instances.
So, instead of:
dog = Dog(id=1, name='Hasso')
lordling = User(id=2, name='John', fullname='Miller', password='very_secret',
best_friend_id=1)
session.add(dog)
Simply do:
dog = Dog(id=1, name='Hasso')
lordling = User(id=2, name='John', fullname='Miller', password='very_secret',
best_friend=dog)
session.add(lordling)
Or even:
lordling = User(id=2, name='John', fullname='Miller', password='very_secret',
best_friend=Dog(id=1, name='Hasso'))
session.add(lordling)
As a general rule, avoid using the foreign key columns directly when you have a relationship established. Embrace the ORM, and only assign or query directly from foreign keys when you really have no other choice. I learned that the hard way.

sqlalchemy: column_prefix causes issues accessing model attributes

I went searching w/o result in a way to get the integer value or the boolean value from an object model created via sqlalchemy,
I mean i can add it and it works flawless but i cant get the integer value or the boolean value all i get when i tried to print it is the object name:
from sqlalchemy import create_engine, MetaData, Table, Column,Integer,String,Boolean,Sequence
from sqlalchemy.orm import mapper, sessionmaker
from sqlalchemy.ext.declarative import declarative_base
import json
class Bookmarks(object):
pass
#----------------------------------------------------------------------
engine = create_engine('postgresql://u:p#localghost/asd', echo=True)
Base = declarative_base()
class Tramo(Base):
__tablename__ = 'tramos'
__mapper_args__ = {'column_prefix':'tramos'}
id = Column(Integer, primary_key=True)
nombre = Column(String)
tramo_data = Column(String)
estado = Column(Boolean,default=True)
def __init__(self,nombre,tramo_data):
self.nombre=nombre
self.tramo_data=tramo_data
def __repr__(self):
return "[id:%s][nombre:%s][tramo:%s]" % (getattr(self, 'id'), self.nombre,self.tramo_data)
Session = sessionmaker(bind=engine)
session = Session()
tabla = Tramo.__table__
metadata = Base.metadata
metadata.create_all(engine)
b=Tramo('tramo1','adadas')
session.add(b)
session.commit()
print b
print b.id
its prints
[id:tramos.id][nombre:tramo1][tramo:adadas]
tramos.id
i cant get to print the id value, looks like the object column is in there but it doesn't return the value ot the property
i even use
session.refresh(b)
after the add but the result is the same.

According to the documentation Naming All Columns with a Prefix:
...prefix to the mapped attribute names relative to the
(table) column name ...
Since you define the mapped attributes in your class, I do not think it does what you desire.
Solution-1: remove the 'column_prefix':'tramos' from your __mapper_args__
Solution-2: print b.tramosid will print its id. You would need to change the __repr__ accordingly:
def __repr__(self):
return "[id:%s][nombre:%s][tramo:%s]" % (getattr(self, 'tramosid'), self.nombre, self.tramo_data)

How to automatically add a SQLAlchemy object to the session?

I have a SQLAlchemy table class created using the Declarative method:
mysqlengine = create_engine(dsn)
session = scoped_session(sessionmaker(bind=mysqlengine))
Base = declarative_base()
Base.metadata.bind = mysqlengine
class MyTable(Base):
__table_args__ = {'autoload' : True}
Now, when using this table within the code I would like to not have to use the session.add method in order to add each new record to the active session so instead of:
row = MyTable(1, 2, 3)
session.add(row)
session.commit()
I would like to have:
row = MyTable(1, 2, 3)
session.commit()
Now, I know of this question already: Possible to add an object to SQLAlchemy session without explicit session.add()?
And, I realize you can force this behavior by doing the following:
class MyTable(Base):
def __init__(self, *args, **kw):
super(MyTable, self).__init__(*args, **kw)
session.add(self)
However, I do not want to bloat my code containing 30 tables with this method. I also know that Elixir ( http://elixir.ematia.de/trac/wiki ) does this so it must be possible in some sense.

Super simple. Use an event:
from sqlalchemy import event, Integer, Column, String
from sqlalchemy.orm import scoped_session, sessionmaker, mapper
from sqlalchemy.ext.declarative import declarative_base
Session = scoped_session(sessionmaker())
#event.listens_for(mapper, 'init')
def auto_add(target, args, kwargs):
Session.add(target)
Base = declarative_base()
class A(Base):
__tablename__ = "a"
id = Column(Integer, primary_key=True)
data = Column(String)
a1 = A(data="foo")
assert a1 in Session()

How can I select all rows with sqlalchemy?

I am trying to get all rows from a table.
In controller I have:
meta.Session.query(User).all()
The result is [, ], but I have 2 rows in this table.
I use this model for the table:
import hashlib
import sqlalchemy as sa
from sqlalchemy import orm
from allsun.model import meta
t_user = sa.Table("users",meta.metadata,autoload=True)
class Duplicat(Exception):
pass
class LoginExistsException(Exception):
pass
class EmailExistsException(Exception):
pass
And next, in the same file:
class User(object):
def loginExists(self):
try:
meta.Session
.query(User)
.filter(User.login==self.login)
.one()
except orm.exc.NoResultFound:
pass
else:
raise LoginExistsException()
def emailExists(self):
try:
meta
.Session
.query(User)
.filter(User.email==self.email)
.one()
except orm.exc.NoResultFound:
pass
else:
raise EmailExistsException()
def save(self):
meta.Session.begin()
meta.Session.save(self)
try:
meta.Session.commit()
except sa.exc.IntegrityError:
raise Duplicat()
orm.mapper(User, t_user)

You can easily import your model and run this:
from models import User
# User is the name of table that has a column name
users = User.query.all()
for user in users:
print user.name

I use the following snippet to view all the rows in a table. Use a query to find all the rows. The returned objects are the class instances. They can be used to view/edit the values as required:
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import create_engine, Sequence
from sqlalchemy import String, Integer, Float, Boolean, Column
from sqlalchemy.orm import sessionmaker
Base = declarative_base()
class MyTable(Base):
__tablename__ = 'MyTable'
id = Column(Integer, Sequence('user_id_seq'), primary_key=True)
some_col = Column(String(500))
def __init__(self, some_col):
self.some_col = some_col
engine = create_engine('sqlite:///sqllight.db', echo=True)
Session = sessionmaker(bind=engine)
session = Session()
for class_instance in session.query(MyTable).all():
print(vars(class_instance))
session.close()