How close can I get to defining a model in SQLAlchemy like:
class Person(Base):
pass
And just have it dynamically pick up the field names? anyway to get naming conventions to control the relationships between tables? I guess I'm looking for something similar to RoR's ActiveRecord but in Python.
Not sure if this matters but I'll be trying to use this under IronPython rather than cPython.
It is very simple to automatically pick up the field names:
from sqlalchemy import Table
from sqlalchemy.orm import MetaData, mapper
metadata = MetaData()
metadata.bind = engine
person_table = Table(metadata, "tablename", autoload=True)
class Person(object):
pass
mapper(Person, person_table)
Using this approach, you have to define the relationships in the call to mapper(), so no auto-discovery of relationships.
To automatically map classes to tables with same name, you could do:
def map_class(class_):
table = Table(metadata, class_.__name__, autoload=True)
mapper(class_, table)
map_class(Person)
map_class(Order)
Elixir might do everything you want.
AFAIK sqlalchemy intentionally decouples database metadata and class layout.
You may should investigate Elixir (http://elixir.ematia.de/trac/wiki): Active Record Pattern for sqlalchemy, but you have to define the classes, not the database tables.
Related
New to sqlalchemy and somewhat novice with programing and python. I had wanted to query a table. It seems I can use the all() function when querying but cannot filter without creating a class.
1.) Can I filter without creating a class and using the declarative api? Is the filtering example stated below incorrect?
2.) When would it be appropriate to use declarative api in sqlalchemy and when would it not be appropriate?
import sqlalchemy as sql
from sqlalchemy import Table, Column, Integer, String, MetaData, ForeignKey
from sqlalchemy.orm import sessionmaker
from sqlalchemy.orm import sessionmaker
db = sql.create_engine('postgresql://postgres:password#localhost:5432/postgres')
engine = db.connect()
meta = MetaData(engine)
session = sessionmaker(bind=engine)
session = session()
files = Table('files',meta,
Column('file_id',Integer,primary_key=True),
Column('file_name',String(256)),
Column('query',String(256)),
Column('results',Integer),
Column('totalresults',Integer),
schema='indeed')
session.query(files).all() #ok
session.query(files).filter(files.file_name = 'test.json') #not ok
If you want to filter by a Table construct, it should be:
session.query(files).filter(files.c.file_name == 'test.json')
You need to create mapped classes if you want to use the ORM features of SQLAlchemy. For example, with the code you currently have, in order to do an update you have to do
session.execute(files.update().values(...))
As opposed to:
file = session.query(File).first()
file.file_name = "new file name"
session.commit()
The declarative API happens to be the easiest way of constructing mapped classes, so use it if you want to use the ORM.
Filter using declarative api this way:
session.query(files).filter(files.file_name == 'test.json').all()
You can also use raw sql queries (docs).
Whether using declarative api or not may depend on your queries complexity, because sometimes sqlalchemy doesn't optimize them right way.
Actually i'm designing a large application for desktop with python 3.4.
I choosed the Port and Adapter architecture as know as Hexagonal architecture.
The main purpose it to work with reusable component.
To organize the code and put some rules we will use the Zope Component Architecture (ZCA)
So to do some POC, i'm creating the database component.
But the fact to work with an ORM block me. I mean i designed some database component which looked like :
-IDatabase
-IDatabaseConfig
-IEntity
-IKey
-IReader
-IWriter
...
and the implementation.
SQLAlchemy management lot of thing and i don't know how to make my component reusable.
i found this code :
#Reflect each database table we need to use, using metadata
class Customer(Base):
__table__ = Table('Customers', metadata, autoload=True)
orders = relationship("Order", backref="customer")
class Shipper(Base):
__table__ = Table('Shippers', metadata, autoload=True)
orders = relationship("Order", backref="shipper")
class Product(Base):
__table__ = Table('Products', metadata, autoload=True)
supplier = relationship('Supplier', backref='products')
category = relationship('Category', backref='products')
But this code is realy hardly coupled with SQLAlchemy i guess.
So what is the approach i should use with my architecture?
As the Entities must be the center of the application (Domain layer) it will have a problem with that solution if i need to change my database component and don't use SQLAlchemy ?
i'm open to all suggestions.
I use the ORM as entity objects and put Adapters over it:
somewhere in a interfaces.py the API is defined:
from zope.interface import Interface
class IEntity(Interface):
def pi_ing():
'''makes the entity go "pi-ing" '''
somewhere the database model is defined:
class EntityA(Base):
# .. table and relationship definitions are here
somewhere else the API is implemented:
from zope.interface import implementer
from zope.component import adapter, getGlobalSiteManager
class EntityAPI(object):
def __init__(self, instance):
self.instance = instance
def pi_ing(self):
# make "pi_ing"
#implementer(IEntityAProtocol)
#adapter(int)
def get_entity_a_by_id(id):
return EntityAPI(session().query(EntityA).get(id))
getGlobalSiteManager().registerAdapter(get_entity_a_by_id)
Now everything is in place. Somewhere in the business logic of the code
when you get the id of an entity_a you can just do:
from interfaces import IEntityAProtocol
def stuff_which_goes_splat():
# ...
entity_id = got_from_somewhere()
IEntityProtocol(entity_id).pi_ing()
Now you have a complete separate implementation of interface, database entity and API logic. The nice thing is, you don't have to adapt only such primitive stuff like an int ID of an object, you can adapt anything, as long as you can implement a direct transformation from your adaptee to the database entity.
caveat: of course, relative to the point in execution time, the getGlobalSiteManager().registerAdapter(get_entity_a_by_id) must have been called already.
I want to be able to load data automatically upon creation of tables using SQLAlchemy.
In django, you have fixtures which allow you to easily pre-populate your database with data upon creation of a table. This I found useful especially when you have basic "lookup" tables e.g. product_type, student_type which contain just a few rows or even a table like currencies which will load all the currencies of the world without you having to key them in over and over again when you destroy your models/classes.
My current app isn't using django. I have SQLAlchemy. How can I achieve the same thing? I want the app to know that the database is being created for the first time and hence it populates some tables with data.
I used the event listener to prepopulate database with data upon creation of a table.
Let's say you have ProductType model in your code:
from sqlalchemy import event, Column, Integer, String
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class ProductType(Base):
__tablename__ = 'product_type'
id = Column(Integer, primary_key=True)
name = Column(String(100))
First, you need to define a callback function, which will be executed when the table is created:
def insert_data(target, connection, **kw):
connection.execute(target.insert(), {'id': 1, 'name':'spam'}, {'id':2, 'name': 'eggs'})
Then you just add the event listener:
event.listen(ProductType.__table__, 'after_create', insert_data)
The short answer is no, SQLAlchemy doesn't provide the same feature as dumpdata and loaddata like Django.
There is https://github.com/kvesteri/sqlalchemy-fixtures that might be useful for you but the workflow is different.
Starting from an existing (SQLite) database with foreign keys, can SQLAlchemy automatically build relationships?
SQLAlchemy classes are automatically created via __table_args__ = {'autoload': True}.
The goal would be to easily access data from related tables without having to add all the relationships one by one by hand (i.e. without using sqlalchemy.orm.relationship() and sqlalchemy.orm.backref).
[Update] As of SQLAlchemy 0.9.1 there is Automap extension for doing that.
For SQLAlchemy < 0.9.0 it is possible to use sqlalchemy reflection.
SQLAlchemy reflection loads foreign/primary keys relations between tables. But doesn't create relations between mapped classes. Actually reflection doesn't create mapped classes for you - you have to specify mapped class name.
Actually I think that reflection support for loading foreign keys is a great helper and time saving tool. Using it you can build a query using joins without need to specify which columns to use for a join.
from sqlalchemy import *
from sqlalchemy import create_engine, orm
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship
metadata = MetaData()
Base = declarative_base()
Base.metadata = metadata
db = create_engine('<db connection URL>',echo=False)
metadata.reflect(bind=db)
cause_code_table = metadata.tables['cause_code']
ndticket_table = metadata.tables['ndticket']
sm = orm.sessionmaker(bind=db, autoflush=True, autocommit=True, expire_on_commit=True)
session = orm.scoped_session(sm)
q = session.query(ndticket_table,cause_code_table).join(cause_code_table)
for r in q.limit(10):
print r
Also when I was using reflection to run queries to existing database - I had to define only mapped classes names, table bindings, relations, BUT there were no need to define table columns for these relations.
class CauseCode(Base):
__tablename__ = "cause_code"
class NDTicket(Base):
__tablename__ = "ndticket"
cause_code = relationship("CauseCode", backref = "ndticket")
q = session.query(NDTicket)
for r in q.limit(10):
print r.ticket_id, r.cause_code.cause_code
Overall SQLAlchemy reflection is already powerful tool and save me time, so adding relations manually is a small overhead for me.
If I would have to develop functionality that will add relations between mapped objects using existing foreign keys, I would start from using reflection with inspector. Using get_foreign_keys() method gives all information required to build relations - referred table name, referred column name and column name in target table. And would use this information for adding property with relationship into mapped class.
insp = reflection.Inspector.from_engine(db)
print insp.get_table_names()
print insp.get_foreign_keys(NDTicket.__tablename__)
>>>[{'referred_table': u'cause_code', 'referred_columns': [u'cause_code'], 'referred_schema': None, 'name': u'SYS_C00135367', 'constrained_columns': [u'cause_code_id']}]
As of SQLAlchemy 0.9.1 the (for now experimental) Automap extension would seem to do just that: http://docs.sqlalchemy.org/en/rel_0_9/orm/extensions/automap.html
There are two (three, but I'm not counting Elixir, as its not "official") ways to define a persisting object with SQLAlchemy:
Explicit syntax for mapper objects
from sqlalchemy import Table, Column, Integer, String, MetaData, ForeignKey
from sqlalchemy.orm import mapper
metadata = MetaData()
users_table = Table('users', metadata,
Column('id', Integer, primary_key=True),
Column('name', String),
)
class User(object):
def __init__(self, name):
self.name = name
def __repr__(self):
return "<User('%s')>" % (self.name)
mapper(User, users_table) # <Mapper at 0x...; User>
Declarative syntax
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
name = Column(String)
def __init__(self, name):
self.name = name
def __repr__(self):
return "<User('%s')>" % (self.name)
I can see that while using the mapper objects, I separate completely the ORM definition from the business logic, while using the declarative syntax, whenever I modify the business logic class, I can edit right there the database class (which ideally should be edited little).
What I'm not completely sure, is which approach is more maintainable for a business application?
I haven't been able to find a comparative between the two mapping methods, to be able to decide which one is a better fit for my project.
I'm leaning towards using the "normal" way (i.e. not the declarative extension) as it allows me to "hide", and keep out of the business view all the ORM logic, but I'd like to hear compelling arguments for both approaches.
"What I'm not completely sure, is which approach is more maintainable for a business application?"
Can't be answered in general.
However, consider this.
The Django ORM is strictly declarative -- and people like that.
SQLAlchemy does several things, not all of which are relevant to all problems.
SQLAlchemy creates DB-specific SQL from general purpose Python. If you want to mess with the SQL, or map Python classes to existing tables, then you have to use explicit mappings, because your focus is on the SQL, not the business objects and the ORM.
SQLAlchemy can use declarative style (like Django) to create everything for you. If you want this, then you are giving up explicitly writing table definitions and explicitly messing with the SQL.
Elixir is an alternative to save you having to look at SQL.
The fundamental question is "Do you want to see and touch the SQL?"
If you think that touching the SQL makes things more "maintainable", then you have to use explicit mappings.
If you think that concealing the SQL makes things more "maintainable", then you have to use declarative style.
If you think Elixir might diverge from SQLAlchemy, or fail to live up to it's promise in some way, then don't use it.
If you think Elixir will help you, then use it.
In our team we settled on declarative syntax.
Rationale:
metadata is trivial to get to, if needed: User.metadata.
Your User class, by virtue of subclassing Base, has a nice ctor that takes kwargs for all fields. Useful for testing and otherwise. E.g.: user=User(name='doe', password='42'). So no need to write a ctor!
If you add an attribute/column, you only need to do it once. "Don't Repeat Yourself" is a nice principle.
Regarding "keeping out ORM from business view": in reality your User class, defined in a "normal" way, gets seriously monkey-patched by SA when mapper function has its way with it. IMHO, declarative way is more honest because it screams: "this class is used in ORM scenarios, and may not be treated just as you would treat your simple non-ORM objects".
I've found that using mapper objects are much simpler then declarative syntax if you use sqlalchemy-migrate to version your database schema (and this is a must-have for a business application from my point of view). If you are using mapper objects you can simply copy/paste your table declarations to migration versions, and use simple api to modify tables in the database. Declarative syntax makes this harder because you have to filter away all helper functions from your class definitions after copying them to the migration version.
Also, it seems to me that complex relations between tables are expressed more clearly with mapper objects syntax, but this may be subjective.
as of current (2019), many years later, sqlalchemy v1.3 allows a hybrid approach with the best of both worlds
https://docs.sqlalchemy.org/en/13/orm/extensions/declarative/table_config.html#using-a-hybrid-approach-with-table
metadata = MetaData()
users_table = Table('users', metadata,
Column('id', Integer, primary_key=True),
Column('name', String),
)
# possibly in a different file/place the orm-declaration
Base = declarative_base(metadata)
class User(Base):
__table__ = Base.metadata.tables['users']
def __str__():
return "<User('%s')>" % (self.name)