We Keep Coding

How to bulk-associate an object to multiple objects that have ManyToManyField?

I have a model that looks like this:
class Keyword(models.Model):
name = models.CharField(unique=True)
class Post(models.Model):
title = models.CharField()
keywords = models.ManyToManyField(
Keyword, related_name="posts_that_have_this_keyword"
)
Now I want to migrate all Posts of a wrongly named Keyword to a new properly named Keyword. And there are multiple wrongly named Keywords.
I can do the following but it leads to a number of SQL queries.
for keyword in Keyword.objects.filter(is_wrongly_named=True).iterator():
old = keyword
new, _ = Keyword.objects.get_or_create(name='some proper name')
for note in old.notes_that_have_this_keyword.all():
note.keywords.add(old)
old.delete()
Is there a way I can achieve this while minimizing the SQL queries executed?
I prefer Django ORM solution to a raw SQL one, because I jumped right into the Django ORM without studying deep into SQL, not so familiar with SQL.
Thank you.

If you want to perform bulk operations with M2M relationships I suggest that you act directly on the table that joins the two objects. Django allows you to access this otherwise anonymous table by using the through attribute on the M2M attribute on an object.
So, to get the table that joins Keywords and Posts you could reference either Keyword.posts_that_have_this_keyword.through or Post.keywords.through. I'd suggest you assign a nicely named variable to this like:
KeywordPost = Post.keywords.through
Once you get a hold onto that table bulk operations can be performed.
bulk remove bad entries
KeywordPost.objects.filter(keyword__is_wrongly_named=True).delete()
bulk create new entries
invalid_keyword_posts = KeywordPost.objects.filter(keyword__is_wrongly_named=True)
post_ids_to_update = invalid_keyword_posts.values_list("post_id", flat=True)
new_keyword_posts = [KeywordPost(post_id=p_id, keyword=new_keyword) for p_id in post_ids_to_update]
KeywordPost.objects.bulk_create(new_keyword_posts)
Basically you get access to all the features that the ORM provides on this join table. You should be able to achieve much better performance that way.
You can read up more on the through attribute here: https://docs.djangoproject.com/en/3.0/ref/models/fields/#django.db.models.ManyToManyField.through
Good luck!

How do I retrieve a field from a Many-To-Many table?

I need to retrieve a value from a Many-To-Many query. Let's say I have 3 models: Toy, Part, and ToyParts
ToyParts has a field called "part_no". I need to be able to get the value of this.
class Toy(models.Model):
parts = models.ManyToManyField(Part, through="ToyParts")
class Part(models.Model):
pass
class ToyParts(models.Model):
toy = models.ForeignKey(Toy, ...)
part = models.ForeignKey(Part, ...)
part_no = models.CharField(...)
I've tried using:
toy.parts.all().first().part_no
which obviously doesn't work as Part does not have a field called "part_no"
I've also tried just simply using:
ToyParts.objects.filter(toy=..., part=...)
but that adds additional queries.
How would I be able to get part_no without querying ToyParts directly?

I've tried using: toy.parts.all().first().part_no
The part_no field is declared on the model ToyParts. You therefore need to get an instance of ToyParts to access this field. Assuming you have a Toy instance you can use the reverse relation to ToyParts, which defaults to toyparts_set, as follows:
toy.toyparts_set.first().part_no
How would I be able to get part_no without querying ToyParts directly?
You can't. If you want to reduce the number of queries you can use select_related:
for tp in toy.toyparts_set.select_related('part').all():
print(tp.part_no, tp.part.id)
In this example tp.part doesn't require an extra query as the part instance is already fetched by select_related.

Flask SQLAlchemy get columns from two joined mapped entities in query result

I have a table, MenuOptions which represents any option found in a dropdown in my app. Each option can be identified by the menu it is part of (e.g. MenuOptions.menu_name) and the specific value of that option (MenuOptions.option_value).
This table has relationships all across my db and doesn't use foreign keys, so I'm having trouble getting it to mesh with SQLAlchemy.
In SQL it would be as easy as:
SELECT
*
FROM
document
JOIN
menu_options ON menu_options.option_menu_name = 'document_type'
AND menu_options.option_value = document.document_type_id
to define this relationship. However I'm running into trouble when doing this in SQLAlchemy because I can't map this relationship cleanly without foreign keys. In SQLAlchemy the best I've done so far is:
the_doc = db.session.query(Document, MenuOptions).filter(
Document.id == document_id
).join(
MenuOptions,
and_(
MenuOptions.menu_name == text('"document_type"'),
MenuOptions.value == Document.type_id
)
).first()
Which does work, and does return the correct values, but returns them as a list of two separate model objects such that I have to reference the mapped Document properties via the_doc[0] and the mapped MenuOptions properties via the_doc[1]
Is there a way I can get this relationship returned as a single query object with all the properties on it without using foreign keys or any ForeignKeyConstraint in my model? I've tried add_columns and add_entity but I get essentially the same result.

you can use with_entities
entities = [getattr(Document, c) for c in Document.__table__.columns.keys()] + \
[getattr(MenuOptions, c) for c in MenuOptions.__table__.columns.keys()]
session.query(Document, MenuOptions).filter(
Document.id == document_id
).join(
MenuOptions,
and_(
MenuOptions.menu_name == text('"document_type"'),
MenuOptions.value == Document.type_id
)
).with_entities(*entities)

I ended up taking a slightly different approach using association_proxy, but if you ended up here from google then this should help you. In the following example, I store a document_type_id in the document table and hold the corresponding values for that id in a table called menu_options. Normally you would use foreign keys for this, but our menu_options is essentially an inhouse lookup table, and it contains relationships to several other tables so foreign keys are not a clean solution.
By first establishing a relationship via the primaryjoin property, then using associationproxy, I can immediately load the document_type based on the document_type_id with the following code:
from sqlalchemy import and_
from sqlalchemy.ext.associationproxy import association_proxy
class Document(db.Model):
__tablename__ = "document"
document_type_id = db.Column(db.Integer)
document_type_proxy = db.relationship(
"MenuOptions",
primaryjoin=(
and_(
MenuOptions.menu_name=='document_type',
foreign('Document.document_type_id')==MenuOptions.value
)
),
lazy="immediate"
viewonly=True
)
If all you need is a mapped relationship without the use of foreign keys within your database, then this will do just fine. If, however, you want to be able to access the remote attribute (in this case the document_type) directly as an attribute on the initial class (in this case Document) then you can use association_proxy to do so by simply passing the name of the mapped relationship and the name of the remote property:
document_type = association_proxy("document_type_proxy", "document_type")

SQLAlchemy: Filter by objects in a member list

Consider the following classes,
class Person:
id
name
jobs
class Job:
name
person
person_id (FK)
where Person.jobs refers to class Job objects. Now I'd like to perform the following query,
# for a given person p
Job.query.filter(Job.notin_(p.jobs))
Is it possible in SQLAlchemy?
Please not that I do not want to write Job.query.filter(Job.person_id != p.id).

Nope !
As the documentation about in_ explains, as of version 1.2 of SQLAlchemy, you can not have list of instances of object, only :
list of ID
empty list
select
bound params
As of version 1.2, only what you describe as "do not want" is the right answer :
Job.query.filter(Job.id.notin_([j.id for j in person.jobs]))

Get Table Name By Table Class In Sqlalchemy

I want to pass a class to a function, and don't like to pass the name again.
class TableClass(Base):
__table__ = Table('t1', metadata, autoload=True)
def get_name(TableClass):
print TableClass.GetTableName() # print 't1'
get_name(TableClass)
So, I search it with google, and there is no answer.

According To:
How to discover table properties from SQLAlchemy mapped object
I can use this:
print TableClass.__table__.name

Independent on whether you use declarative extension or not, you can use the Runtime Inspection API:
def get_name(TableClass):
from sqlalchemy import inspect
mapper = inspect(TableClass)
print mapper.tables[0].name
Please note that a class can have multiple tables mapped to it, for example when using Inheritance.

print TableClass.__tablename__
works for me

In SQLAlchemy you can fetch table information with tableclass attributes.
In your example
print TableClass.__tablename__ # Prints 't1'
According to #Aylwyn Lake 's Finding
print TableClass.__table__.name

I just hit this issue myself, passing the object around and didn't want to pass the string of the tablename as well..
Turns out, it's just a property of the Table object, eg:
table = new Table("somename",meta)
...
print("My table is called: " + table.name)

None of the answers worked for me.
This did:
For Class Object:
TableClass.sa_class_manager.mapper.mapped_table.name
For Instance Object:
tableObj.sa_instance_state.mapper.mapped_table.name