I have data for a particular entity partitioned across multiple identical tables, often separated chronologically or by numeric range. For instance, I may have a table called mytable for current data, a mytable_2013 for last year's data, mytable_2012, and so on.
Only the current table is ever written to. The others are only consulted. With SQLAlchemy, is there any way I can specify the table to query from when using the declarative model?
Use mixins and change table names by an object property.
class Node(Base):
__tablename__ = 'node'
nid = Column(Integer, primary_key=True)
uuid = Column(String(128))
vid = Column(Integer)
class Node1(Node):
__tablename__ = 'node_1'
class Node2(Node):
__tablename__ = 'node_2'
As requested, re-posting as answer:
Please take a look at this answer to Mapping lots of similar tables in SQLAlchemy in the Concrete Table Inheritance section.
In your case you can query MyTable only when working with the current data, and do a polymorphic search on all tables when you need the whole history.
Related
I have data for a particular entity partitioned across multiple identical tables, often separated chronologically or by numeric range. For instance, I may have a table called mytable for current data, a mytable_2013 for last year's data, mytable_2012, and so on.
Only the current table is ever written to. The others are only consulted. With SQLAlchemy, is there any way I can specify the table to query from when using the declarative model?
Use mixins and change table names by an object property.
class Node(Base):
__tablename__ = 'node'
nid = Column(Integer, primary_key=True)
uuid = Column(String(128))
vid = Column(Integer)
class Node1(Node):
__tablename__ = 'node_1'
class Node2(Node):
__tablename__ = 'node_2'
As requested, re-posting as answer:
Please take a look at this answer to Mapping lots of similar tables in SQLAlchemy in the Concrete Table Inheritance section.
In your case you can query MyTable only when working with the current data, and do a polymorphic search on all tables when you need the whole history.
I want to create 200+ tables using declarative base on the fly. I learnt it's not possible, so my idea was to create a common table and rename it 200+ times.
class Movie(Base):
id = Column(Integer, primary_key=True)
title = Column(String)
release_date = Column(Date)
name=Column(String)
__tablename__ = 'titanic'
def __init__(self, newname,title, release_date):
self.title = title
self.release_date = release_date
What is the code to change the table name from "titanic" to "wild"?
In Postgresql it is
ALTER TABLE table_name
RENAME TO new_table_name;
I am not finding a solution in sqlalchemy.
There are no foreign keys to this table.
The objective of this question is to rename an existing table thru a solution (if) available in sqlalchemy, not in a purely python way (as mentioned in the other question).
The easiest way to rename a table is to create a new table, dumping the data into it with an INSERT INTO statement.
More from the web:
You must issue the appropriate ALTER statements to your database to change the name of the table. As far as the Table metadata itself, you can attempt to set table.name = 'newname', and re-place the Table object within metadata.tables with its new name, but this may have lingering side effects regarding foreign keys that reference the old name. In general, the pattern is not supported - its intended that a SQLAlchemy application runs with a fixed database structure (only new tables can be added on the fly).
(Source)
I'm attempting to query and filter on a one-to-many relationship and cannot seem to figure out how to do this.
Here are my mappings (trimmed for brevity):
class Bug(Base):
__tablename__ = 'bug'
id = Column('bug_id', Integer, primary_key=True)
tags = relationship('Tag', backref='bug')
class Tag(Base):
id = Column('tag_id', Integer, primary_key=True)
name = Column('tag_name', String)
bug_id = Column('bug_id', ForeignKey('bug.bug_id'))
I want to be able to find all bugs that do not have tag with name "foo".
You can use the any() operator on the relationship.
bugs_without_foo = session.query(Bug).filter(
db.not_(Bug.tags.any(Tag.name == 'foo'))
).all()
It's nicer to look at, but it could be less efficient over very large data sets than the subquery from Dan Lenski's answer.
I am not sure exactly what the Tag table is supposed to represent for you, but it is odd that your schema associates each Tag with exactly one Bug. If you want to tag multiple Bugs with a tag of the same name, you will be creating multiple rows in the Tag class with the same name. This would seem to violate the 3rd normal form.
The standard way to describe a tag cloud in a database would be to use a many-to-many relationship with a secondary "association" table that associates (bug,tag) pairs. The SQLAlchemy docs have a very nice tutorial on this pattern.
If you stick with your schema as-is, there are several ways to do it.
Client-side filtering
This is obviously inefficient but it is easy to understand. You go through the bugs one by one, go through their tags one by one, and eliminate the bugs where tag.name=="foo":
non_foo_bugs = [ bug for bug in session.query(Bug)
if not any(tag.name=="foo" for tag in bug.tag) ]
Two queries
Find all distinct bugs that are tagged "foo", and then find the complement of that set.
This version uses exactly two queries of the database:
foo_bugs = [t.bug_id for t in session.query(Tag).filter_by(name="foo").distinct()]
session.query(Bug).filter(~Bug.id.in_(foo_bugs))
One query with a subquery
Same as the above, but make foo_bugs a subquery, since there's no reason to fetch its contents on the client side:
foo_bugs = session.query(Tag.bug_id).filter_by(name="foo").distinct().subquery()
session.query(Bug).filter(~Bug.id.in_(foo_bugs))
This would be an uncorrelated subquery, so from the server point of view it should be optimized just about the same as two separate queries.
I'm new to Python, as well as SQL Alchemy, but not the underlying development and database concepts. I know what I want to do and how I'd do it manually, but I'm trying to learn how an ORM works.
I have two tables, Images and Keywords. The Images table contains an id column that is its primary key, as well as some other metadata. The Keywords table contains only an id column (foreign key to Images) and a keyword column. I'm trying to properly declare this relationship using the declarative syntax, which I think I've done correctly.
Base = declarative_base()
class Keyword(Base):
__tablename__ = 'Keywords'
__table_args__ = {'mysql_engine' : 'InnoDB'}
id = Column(Integer, ForeignKey('Images.id', ondelete='CASCADE'),
primary_key=True)
keyword = Column(String(32), primary_key=True)
class Image(Base):
__tablename__ = 'Images'
__table_args__ = {'mysql_engine' : 'InnoDB'}
id = Column(Integer, primary_key=True, autoincrement=True)
name = Column(String(256), nullable=False)
keywords = relationship(Keyword, backref='image')
This represents a many-to-many relationship. One image can have many keywords, and one keyword can relate back to many images.
I want to do a keyword search of my images. I've tried the following with no luck.
Conceptually this would've been nice, but I understand why it doesn't work.
image = session.query(Image).filter(Image.keywords.contains('boy'))
I keep getting errors about no foreign key relationship, which seems clearly defined to me. I saw something about making sure I get the right 'join', and I'm using 'from sqlalchemy.orm import join', but still no luck.
image = session.query(Image).select_from(join(Image, Keyword)).\
filter(Keyword.keyword == 'boy')
I added the specific join clause to the query to help it along, though as I understand it, I shouldn't have to do this.
image = session.query(Image).select_from(join(Image, Keyword,
Image.id==Keyword.id)).filter(Keyword.keyword == 'boy')
So finally I switched tactics and tried querying the keywords and then using the backreference. However, when I try to use the '.images' iterating over the result, I get an error that the 'image' property doesn't exist, even though I did declare it as a backref.
result = session.query(Keyword).filter(Keyword.keyword == 'boy').all()
I want to be able to query a unique set of image matches on a set of keywords. I just can't guess my way to the syntax, and I've spent days reading the SQL Alchemy documentation trying to piece this out myself.
I would very much appreciate anyone who can point out what I'm missing.
It appears that I was still getting the wrong version of join, even importing the one under sqlalchemy.orm. I did this to resolve the problem:
from sqlalchemy.orm.util import join as join_
image = session.query(Image).select_from(join_(Image, Keyword)).\
filter(Keyword.keyword == 'boy')
Is that really the "most right" solution, or am I missing some nuance of Python? Since I'm still learning, I'd like to do things the "most right" way as advised by those with more experience. Thanks.
I'm trying to use association proxy for tags, in a very similar scenario to the example in the docs. Here is a subset of my schema (it's a blog), using declarative:
class Tag(Base):
__tablename__ = 'tags'
id = Column(Integer, primary_key=True)
tag = Column(Unicode(255), unique=True, nullable=False)
class EntryTag(Base):
__tablename__ = 'entrytags'
entry_id = Column(Integer, ForeignKey('entries.id'), key='entry', primary_key=True)
tag_id = Column(Integer, ForeignKey('tags.id'), key='tag', primary_key=True)
class Entry(Base):
__tablename__ = 'entries'
id = Column(Integer, primary_key=True)
subject = Column(Unicode(255), nullable=False)
# some other fields here
_tags = relation('Tag', backref='entries', secondary=EntryTag.__table__)
tags = association_proxy('_tags','tag')
Here's how I'm trying to use it:
>>> e = db.query(Entry).first()
>>> e.tags
[u'foo']
>>> e.tags = [u'foo', u'bar'] # really this is from a comma-separated input
db.commit()
Traceback (most recent call last):
[...]
sqlalchemy.exc.IntegrityError: (IntegrityError) duplicate key value violates unique constraint "tags_tag_key"
'INSERT INTO tags (id, tag) VALUES (%(id)s, %(tag)s)' {'tag': 'bar', 'id': 11L}
>>> map(lambda t:(t.id,t.tag), db.query(Tag).all())
[(1, u'foo'), (2, u'bar'), (3, u'baz')]
The tag u'bar' already existed with id 2; why didn't SQLAlchemy just attach that one instead of trying to create it? Is my schema wrong somehow?
Disclaimer: it's been ages since I used SQLAlchemy so this is more of a guess than anything.
It looks like you're expecting SQLAlchemy to magically take the string 'bar' and look up the relevant Tag for it when performing the insert on the many-to-many table. I expect this is invalid, because the field in question ('tag') is not a primary key.
Imagine a similar situation where your Tag table is actually Comment, also with an id and a text field. You'd expect to be able to add Comments to an Entry with the same e.comments = ['u'Foo', 'u'Bar'] syntax that you've used above, but you'd want it to just perform INSERTs, not check for existing comments with the same content.
So that is probably what it's doing here, but it hits the uniqueness constraint on your tag name and fails, assuming that you're attempting to do the wrong thing.
How to fix it? Making tags.tag the primary key is arguably the correct thing to do, although I don't know how efficient that is nor how well SQLAlchemy handles it. Failing that, try querying for Tag objects by name before assigning them to the entry. You may have to write a little utility function that takes a unicode string and either returns an existing Tag or creates a new one for you.
I've never used SQLAlchemy 0.5 yet (my last app using it was 0.4 based) but I can see one quirk in your code: you should modify the association_proxy object, not reassign it.
Try doing something like:
e.tags.append(u"bar")
Instead of
e.tags = ...
If that doesn't work, try pasting a complete working example for those tables (including the imports, please!) and I'll give you some more advice.