I have some problems with setting up the dictionary collection in Python's SQLAlchemy:
I am using declarative definition of tables. I have Item table in 1:N relation with Record table. I set up the relation using the following code:
_Base = declarative_base()
class Record(_Base):
__tablename__ = 'records'
item_id = Column(String(M_ITEM_ID), ForeignKey('items.id'))
id = Column(String(M_RECORD_ID), primary_key=True)
uri = Column(String(M_RECORD_URI))
name = Column(String(M_RECORD_NAME))
class Item(_Base):
__tablename__ = 'items'
id = Column(String(M_ITEM_ID), primary_key=True)
records = relation(Record, collection_class=column_mapped_collection(Record.name), backref='item')
Now I want to work with the Items and Records. Let's create some objects:
i1 = Item(id='id1')
r = Record(id='mujrecord')
And now I want to associate these objects using the following code:
i1.records['source_wav'] = r
but the Record r doesn't have set the name attribute (the foreign key). Is there any solution how to automatically ensure this? (I know that setting the foreign key during the Record creation works, but it doesn't sound good for me).
Many thanks
You want something like this:
from sqlalchemy.orm import validates
class Item(_Base):
[...]
#validates('records')
def validate_record(self, key, record):
assert record.name is not None, "Record fails validation, must have a name"
return record
With this, you get the desired validation:
>>> i1 = Item(id='id1')
>>> r = Record(id='mujrecord')
>>> i1.records['source_wav'] = r
Traceback (most recent call last):
[...]
AssertionError: Record fails validation, must have a name
>>> r.name = 'foo'
>>> i1.records['source_wav'] = r
>>>
I can't comment yet, so I'm just going to write this as a separate answer:
from sqlalchemy.orm import validates
class Item(_Base):
[...]
#validates('records')
def validate_record(self, key, record):
record.name=key
return record
This is basically a copy of Gunnlaugur's answer but abusing the validates decorator to do something more useful than exploding.
You have:
backref='item'
Is this a typo for
backref='name'
?
Related
I have simple classes mapped to python dictionies, WHen trying to add them to an in memory SQLite Db using SQLalchemy i get troubles running filtered queries.
As the objects were already in dicts I added a class method to my mapped objects, that could create an instance of the mapped class from the dict.
here is the set up:
from sqlalchemy import Column, Integer, String, text
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import create_engine
from sqlalchemy.orm.session import sessionmaker
Base = declarative_base()
class User(Base):
__tablename__ = 'Users'
mailbox = Column(String, primary_key=True)
uid = Column(Integer, primary_key=True)
name = Column(String)
#classmethod
def from_dict(cls, dict):
cls.mailbox = dict["mailbox"]
cls.uid = dict["uid"]
cls.name = dict["name"]
return cls(mailbox=cls.mailbox,
uid=cls.uid,
name=cls.name)
The classmethod of User returns the User class. All the other set up is pretty normal.
For demonstration i created two lists, one of mapped User objects. One of dictionaries with User Data. Despite the types, the info inside each is identical.
def dict_user_list():
return [{'mailbox':"inbox", 'uid':1, 'name':"tim"},
{'mailbox':"inbox", 'uid':2, 'name':"mark"},
{'mailbox':"inbox", 'uid':3, 'name':"susan"},
{'mailbox':"mybox", 'uid':1, 'name':"geoff"},
{'mailbox':"mybox", 'uid':2, 'name':"kim"},
{'mailbox':"mybox", 'uid':3, 'name':"trev"},
{'mailbox':"mybox", 'uid':4, 'name':"rick"}]
def user_list():
return [User(mailbox="inbox", uid=1, name="tim"),
User(mailbox="inbox", uid=2, name="mark"),
User(mailbox="inbox", uid=3, name="susan"),
User(mailbox="mybox", uid=1, name="geoff"),
User(mailbox="mybox", uid=2, name="kim"),
User(mailbox="mybox", uid=3, name="trev"),
User(mailbox="mybox", uid=4, name="rick")]
Here is the rest of the setup:
ENGINE = create_engine('sqlite://')
Base.metadata.create_all(ENGINE)
Base.metadata.bind = ENGINE
session_maker = sessionmaker()
session = session_maker()
Now if i add the Users via the .from_dict class method:
session.add_all([User.from_dict(user) for user in dict_user_list()])
Then run a query and filter:
query = session.query(User)
print("inbox filter: ", query.filter(User.mailbox == 'inbox').count())
print("mybox filter: ", query.filter(User.mailbox == 'mybox').count())
print("inbox filter_by: ", query.filter_by(mailbox='inbox').count())
print("mybox filter_by: ", query.filter_by(mailbox='mybox').count())
The output is:
inbox filter: 0
mybox filter: 7
inbox filter_by: 0
mybox filter_by: 7
This is wrong. But i can get the correct answers by using text()
print("inbox filter: ", query.filter(text('Users.mailbox = "inbox"')).count())
print("mybox filter: ", query.filter(text('Users.mailbox = "mybox"')).count())
The output gives the correct results:
inbox filter: 3
mybox filter: 4
If I change my .add_all() argument to create my instances via User(....):
session.add_all([user for user in user_list()])
Then run mu queries and filters again without using text():
query = session.query(User)
print("inbox filter: ", query.filter(User.mailbox == 'inbox').count())
print("mybox filter: ", query.filter(User.mailbox == 'mybox').count())
print("inbox filter_by: ", query.filter_by(mailbox='inbox').count())
print("mybox filter_by: ", query.filter_by(mailbox='mybox').count())
The output is correct:
inbox filter: 3
mybox filter: 4
inbox filter_by: 3
mybox filter_by: 4
Obviously the solution is not to invoke instances of my mapped class from a dict. But As i spend so long trying to work out what was going on here I would love to know what is going on here, Is it a lack of understanding of SQLalchemy? a lack of understanding of classmethods? or just a lack of understanding of python in general?
I've inherited a SQLAlchemy project and need some assistance figuring out why one relationship does not appear to be behaving like all the others. Our orm models are generated by a script and as such I would assume all would either work or not.
When I try to insert on the table with the FK I receive the below error message:
Foreign key associated with column 'RenewalBatchGroup.renewalbatchfrequencyid' could not find table 'RenewalBatchFrequency' with which to generate a foreign key to target column 'renewalbatchfrequencyid'
In the below code, as it is, I get the NoReferencedTableError exception. However if I change the import from Frequency importing Group to the other way around, it does work.
IE:
Remove the following from RenewalBatchFrequency.py
"from app.orm.LROCustomer.dbo.RenewalBatchGroup import RenewalBatchGroup"
and add the following to RenewalBatchGroup.py
from app.orm.LROCustomer.dbo.RenewalBatchFrequency import RenewalBatchFrequency
I need to know why this relationship is behaving differently from other relationships like the RenewalBatch relationship. It's setup the exact same way.
RenewalBatchGroup.py
from sqlalchemy.dialects.mssql import *
from sqlalchemy import Column, ForeignKey
from sqlalchemy.orm import relationship
from app.DatabaseCore import Base
from Utils.DateTimeUtils import getDateTimeString
from app.orm.LROCustomer.dbo.RenewalBatch import RenewalBatch
from app.orm.LROCustomer.dbo.RenewalBatchGroupProperty import RenewalBatchGroupProperty
class RenewalBatchGroup(Base):
__tablename__ = 'RenewalBatchGroup'
__table_args__ = {'implicit_returning': False} # http://docs.sqlalchemy.org/en/latest/dialects/mssql.html#triggers
batchleadtime = Column(SMALLINT)
batchsize = Column(SMALLINT)
groupname = Column(VARCHAR(400))
offerstatusid = Column(INTEGER)
renewalbatchfrequencyid = Column(INTEGER, ForeignKey('RenewalBatchFrequency.renewalbatchfrequencyid'))
renewalbatchfrequencyvalue = Column(VARCHAR(100))
renewalbatchgroupid = Column(INTEGER, primary_key=True)
updateuser = Column(VARCHAR(254))
RenewalBatches = relationship("RenewalBatch", backref="RenewalBatchGroup")
RenewalBatchGroupPropertys = relationship("RenewalBatchGroupProperty", backref="RenewalBatchGroup")
def jsonify(self):
return {"batchleadtime": self.batchleadtime,
"batchsize": self.batchsize,
"groupname": self.groupname,
"offerstatusid": self.offerstatusid,
"renewalbatchfrequencyid": self.renewalbatchfrequencyid,
"renewalbatchfrequencyvalue": self.renewalbatchfrequencyvalue,
"renewalbatchgroupid": self.renewalbatchgroupid,
"updateuser": self.updateuser
}
def update(self, row):
self.batchleadtime = row.batchleadtime
self.batchsize = row.batchsize
self.groupname = row.groupname
self.offerstatusid = row.offerstatusid
self.renewalbatchfrequencyid = row.renewalbatchfrequencyid
self.renewalbatchfrequencyvalue = row.renewalbatchfrequencyvalue
self.updateuser = row.updateuser
RenewalBatchFrequency.py
from sqlalchemy.dialects.mssql import *
from sqlalchemy import Column, ForeignKey
from sqlalchemy.orm import relationship
from app.DatabaseCore import Base
from Utils.DateTimeUtils import getDateTimeString
from app.orm.LROCustomer.dbo.RenewalBatchGroup import RenewalBatchGroup
class RenewalBatchFrequency(Base):
__tablename__ = 'RenewalBatchFrequency'
__table_args__ = {'implicit_returning': False} # http://docs.sqlalchemy.org/en/latest/dialects/mssql.html#triggers
name = Column(VARCHAR(100))
renewalbatchfrequencyid = Column(INTEGER, primary_key=True)
RenewalBatchGroups = relationship("RenewalBatchGroup", backref="RenewalBatchFrequency")
def jsonify(self):
return {"name": self.name,
"renewalbatchfrequencyid": self.renewalbatchfrequencyid
}
def update(self, row):
self.name = row.name
If any other information is needed just let me know.
Thank you.
As mentioned above, I can solve the issue by reversing the import statement. That's contrary to all the other relationships. I can only assume there is a circular reference somehow with the other relationships that SQLAlchemy is working out.
The way that I solved this issue is to put all of my model imports into the __ init__.py file and thus every model has access to all models. Now when I generate a new model I just need to make sure it's added to the __ init__.py file and I should be good to go.
A very frustrating experience.
I am trying to create a program that loads in over 100 tables from a database so that I can change all appearances of a user's user id.
Rather than map all of the tables individually, I decided to use a loop to map each of the tables using an array of objects. This way, the table definitions can be stored in a config file and later updated.
Here is my code so far:
def init_model(engine):
"""Call me before using any of the tables or classes in the model"""
meta.Session.configure(bind=engine)
meta.engine = engine
class Table:
tableID = ''
primaryKey = ''
pkType = sa.types.String()
class mappedClass(object):
pass
WIW_TBL = Table()
LOCATIONS_TBL = Table()
WIW_TBL.tableID = "wiw_tbl"
WIW_TBL.primaryKey = "PORTAL_USERID"
WIW_TBL.pkType = sa.types.String()
LOCATIONS_TBL.tableID = "locations_tbl"
LOCATIONS_TBL.primaryKey = "LOCATION_CODE"
LOCATIONS_TBL.pkType = sa.types.Integer()
tableList = ([WIW_TBL, LOCATIONS_TBL])
for i in tableList:
i.tableID = sa.Table(i.tableID.upper(), meta.metadata,
sa.Column(i.primaryKey, i.pkType, primary_key=True),
autoload=True,
autoload_with=engine)
orm.mapper(i.mappedClass, i.tableID)
The error that this code returns is:
sqlalchemy.exc.ArgumentError: Class '<class 'changeofname.model.mappedClass'>' already has a primary mapper defined. Use non_primary=True to create a non primary Mapper. clear_mappers() will remove *all* current mappers from all classes.
I cant use clear_mappers as it wipes all of the classes and the entity_name scheme doesn't seem to apply here.
It seems that every object wants to use the same class, although they all should have their own instance of it.
Does anyone have any ideas?
Well, in your case it *is the same Class you try to map to different Tables. To solve this, create a class dynamically for each Table:
class Table(object):
tableID = ''
primaryKey = ''
pkType = sa.types.String()
def __init__(self):
self.mappedClass = type('TempClass', (object,), {})
But I would prefer slightly cleaner version:
class Table2(object):
def __init__(self, table_id, pk_name, pk_type):
self.tableID = table_id
self.primaryKey = pk_name
self.pkType = pk_type
self.mappedClass = type('Class_' + self.tableID, (object,), {})
# ...
WIW_TBL = Table2("wiw_tbl", "PORTAL_USERID", sa.types.String())
LOCATIONS_TBL = Table2("locations_tbl", "LOCATION_CODE", sa.types.Integer())
I am trying to write a logging system, which uses dynamic classes to make tables. Getting the classes created, and the tables created seems to be working fine, but trying to put entries into them is lead to an error message regarding mapping, below is the sample code and the error message.
Base = declarative_base()
#my init function
def tableinit(self,keyargs):
self.__dict__ = dict(keyargs)
#table creation
tableName = "newTable"
columnsDict["__tablename__"] = tableName
columnsDict["__init__"] = tableinit
columnsDict["id"] = Column("id",Integer, autoincrement = True, nullable = False, primary_key=True)
columnsDict["pid"] = Column("pid",Integer, ForeignKey('someparenttable.id')) #someparenttable is created with a hard coded class
newTable = type(tableName,(Base,),columnsDict)
tableClassDict[tableName]=newTable
#when doing an entry
newClassInst = subEntryClassDict[tableName]
newEntry = newClassInst(dataDict)
entryList.append(newEntry) # this is called in a for loop with the entries for someparenttable's entries also
self.session.add_all(entryList) # at this point the error occurs
The error:
UnmappedInstanceError: Class 'newTable' is mapped, but this instance lacks instrumentation. This occurs when the instance is created before sqlalchemy.orm.mapper(module.newTable) was called.
This is easier if you create a function to return a class that you set up normally. I've tried something like this and it works:
def getNewTable( db, table ):
class NewTable( Base ):
__tablename__ = table
__table_args__ = { 'schema': db }
id = Column( ...
return NewTable
newClassInst = getNewTable( 'somedb', 'sometable' )
newRow = newClassInst( data )
This problem is caused by lack of instruments function interfaces for the orm as the error description says. And it is actually caused by self.__dict__ = dict(keyargs) I think.
So this can be solved by reconstruct the init, which do not modify the injected functions by ORM.
Turn this
#my init function
def tableinit(self,keyargs):
self.__dict__ = dict(keyargs)
To
#my init function
def tableinit(self,**kwargs):
self.__dict__.update(kwargs)
I'm trying to make some generic apps using Sql Alchemy, such as tags or rating for any model. But I couldn't find any help in the docs. I really liked what I could do with the django contenttypes framework ? Is there any similar functionality in Sql Alchemy ?
I once wrote some example code about something similar to this (see http://taketwoprogramming.blogspot.com/2009/08/reusable-sqlalchemy-models.html).
The basic idea is that you can create a model like this:
#commentable
class Post(Base):
__tablename__ = 'posts'
id = sa.Column(sa.Integer, primary_key=True)
text = sa.Column(sa.String)
...where commentable is defined like this...
class BaseComment(object):
pass
def build_comment_model(clazz):
class_table_name = str(class_mapper(clazz).local_table)
metadata = clazz.metadata
comment_class_name = clazz.__name__ + 'Comment'
comment_class = type(comment_class_name, (BaseComment,), {})
comment_table_name = class_table_name + '_comments'
comment_table = sa.Table(comment_table_name, metadata,
sa.Column('id', sa.Integer, primary_key=True),
sa.Column(class_table_name + '_id',
sa.Integer,
sa.ForeignKey(class_table_name + '.id')),
sa.Column('text', sa.String),
sa.Column('name', sa.String(100)),
sa.Column('url', sa.String(255)),
)
mapper(comment_class, comment_table)
return comment_class, comment_table
def commentable(clazz):
comment_class, comment_table = build_comment_model(clazz)
clazz.Comment = comment_class
setattr(clazz, 'comments', relation(comment_class))
def add_comment(self, comment):
self.comments.append(comment)
setattr(clazz, 'add_comment', add_comment)
return clazz
Basically, the commentable decorator dynamically creates a new type and table, along with some helper methods to the decorated class. This is the test I used to test that the code works, which shows some example of how it would work...
class TestModels(SATestCase):
def test_make_comment(self):
p = Post()
p.text = 'SQLAlchemy is amazing!'
text = 'I agree!'
name = 'Mark'
url = 'http://www.sqlalchemy.org/'
c = Post.Comment()
c.text = text
c.name = name
c.url = url
p.add_comment(c)
Session.add(p)
# This is a method I use to force the reload of the objects from
# the database to make sure that when I test them, I'm actually
# pulling from the database rather than just getting the data
# of the object still in the session.
p = self.reload(p)
self.assertEquals(len(p.comments), 1)
c = p.comments[0]
self.assertEquals(c.text, text)
self.assertEquals(c.name, name)
self.assertEquals(c.url, url)
I wrote this awhile ago, but I don't think there's anything in SQLA that will do this kind of thing for you, but you can create something similar without too much trouble. In my example, I created new mapped classes and methods to use it on the fly in a class decorator.
I never really made much use out of it, but it might give you some ideas.