Django setting many_to_many object while doing a bulk_create - python

I am using Django 1.9 and am trying the bulk_create to create many new model objects and associate them with a common related many_to_many object.
My models are as follows
#Computational Job object
class OT_job(models.Model):
is_complete = models.BooleanField()
is_submitted = models.BooleanField()
user_email = models.EmailField()
#Many sequences
class Seq(models.Model):
sequence=models.CharField(max_length=100)
ot_job = models.ManyToManyField(OT_job)
I have thousands of Seq objects that are submitted and have to be associated with their associated job. Previously I was using an iterator and saving them in a for loop. But after reading realized that Django 1.9 has bulk_create.
Currently I am doing
DNASeqs_list = []
for a_seq in some_iterable_with_my_data:
# I create new model instances and add them to the list
DNASeqs_list.append(Seq(sequence=..., ))
I now want to bulk_create these sequence and associate them with the current_job_object.
created_dnaseqs = Seq.objects.bulk_create(DNASeqs_list)
# How do I streamline this part below
for a_seq in created_dnaseqs:
# Had to call save here otherwise got an error
a_seq.save()
a_seq.ot_job.add(curr_job_obj)
I had to call "a_seq.save()" in for loop because I got an error in the part where I was doing "a_seq.ot_job.add(curr_job_obj)" which said
....needs to have a value for field "seq" before this many-to-many relationship can be used.
Despite reading the other questions on this topic , I am still confused because unlike others I do not have a custom "through" model. I am confused with how best to associate the OT_Job with many Seqs with minimal hits to database.

From the docs https://docs.djangoproject.com/en/1.9/ref/models/querysets/#bulk-create:
If the model’s primary key is an AutoField it does not retrieve and set the primary key attribute, as save() does.
It does not work with many-to-many relationships.
bulk_create literally will just create the objects, it does not retrieve the PK into the variable as save does. You would have to re-query the db to get your newly created objects, and then create the M2M relationships, but it sounds like that would not be appropriate and that your current method is currently the best solution.

Related

how to create a relation model with forms?

Hey i need to create a model from form. It's different way if i need to create a some another model, that have a relations with created object model from form. its must work like that -> i come on site, get form for create a model object and save new object. an then -> i have another model with relations with model from form. and i need, to create relation model object automaticly - when django taked new object from forms.
tried any can help me. i make it this way, but this time i have problem. -> i have manytomany field in my relation model, and i have manytomany field ->users from form. and i cant create a relation model with this instance :( Traceback:
TypeError at /ru/center/add/
Direct assignment to the forward side of a many-to-many set is prohibited. Use users_center.set() instead.
but i tired to try it(( help please, how i may do it?
views.py
for x in form['users_center'].data:
name_center = form['name_center'].data
fullname_center = form['fullname_center'].data
ogrn = form['ogrn_center'].data
head_center = form['head_center'].data # user id
many_users = form['users_center'].data
user_center = x # user id
new_center = Center.objects.create(name_center=name_center,
fullname_center=fullname_center,
ogrn_center=ogrn,
head_center=User.objects.get(id=head_center),
users_center=User.objects.get(id=int(x)))
new_center.save()
models.py
users_center = models.ManyToManyField(User,
related_name='center',
# through=CenterDetails,
default=None, blank=True,
verbose_name=_("Сотрудники"))
There is a join table implied by the many-to-many relationship between two models. The error is letting you know that you must set the relationship after creating a User object instead of trying to assign the relationship while creating the User object. You can use set() or add() to do this.
So try doing:
new_center = Center.objects.create(
name_center=name_center,
fullname_center=fullname_center,
ogrn_center=ogrn,
head_center=User.objects.get(id=head_center),
)
users_center=User.objects.get(id=int(x))
new_center.users_center.add(users_center)
Additionally it may be useful to rename your many to many field as a plural to indicate the relationship. Maybe users_centers instead. Since it seems like users can have many centers, and centers can have many users. That's up to you though, not required for it to work.

Bulk clear ManyToMany after object traversal

I have the following data models:
class ArticleSet(Model):
pass
class Article(Model):
article_set = ForeignKey(ArticleSet, related_name = 'articles')
attachments = ManyToManyField(Attachment, related_name='articles')
Now I have an ArticleSet and want to clear (not delete!) all attachments from them. The following will work:
for article in article_set.articles.all():
article.attachments.clear()
However, this will cause a database query per article in the article_set. Is there a way to do the same in a single call to the database? I am using Django 1.11.
p.s. This question is related to How do I remove multiple objects in a ManyToMany relationship based on a filter?, but in that question there is no model traversal before deleting relations.
You can access the ManyToMany table by using the through attribute. By deleting items from that table you are removing the relationship. So try:
m2m_model = Article.attachments.through
m2m_model.objects.filter(article__article_set=article_set).delete()
That will run in one query.

Django filter related field using related model's custom manager

How can I apply annotations and filters from a custom manager queryset when filtering via a related field? Here's some code to demonstrate what I mean.
Manager and models
from django.db.models import Value, BooleanField
class OtherModelManager(Manager):
def get_queryset(self):
return super(OtherModelManager, self).get_queryset().annotate(
some_flag=Value(True, output_field=BooleanField())
).filter(
disabled=False
)
class MyModel(Model):
other_model = ForeignKey(OtherModel)
class OtherModel(Model):
disabled = BooleanField()
objects = OtherModelManager()
Attempting to filter the related field using the manager
# This should only give me MyModel objects with related
# OtherModel objects that have the some_flag annotation
# set to True and disabled=False
my_model = MyModel.objects.filter(some_flag=True)
If you try the above code you will get the following error:
TypeError: Related Field got invalid lookup: some_flag
To further clarify, essentially the same question was reported as a bug with no response on how to actually achieve this: https://code.djangoproject.com/ticket/26393.
I'm aware that this can be achieved by simply using the filter and annotation from the manager directly in the MyModel filter, however the point is to keep this DRY and ensure this behaviour is repeated everywhere this model is accessed (unless explicitly instructed not to).
How about running nested queries (or two queries, in case your backend is MySQL; performance).
The first to fetch the pk of the related OtherModel objects.
The second to filter the Model objects on the fetched pks.
other_model_pks = OtherModel.objects.filter(some_flag=...).values_list('pk', flat=True)
my_model = MyModel.objects.filter(other_model__in=other_model_pks)
# use (...__in=list(other_model_pks)) for MySQL to avoid a nested query.
I don't think what you want is possible.
1) I think you are miss-understanding what annotations do.
Generating aggregates for each item in a QuerySet
The second way to generate summary values is to generate an
independent summary for each object in a QuerySet. For example, if you
are retrieving a list of books, you may want to know how many authors
contributed to each book. Each Book has a many-to-many relationship
with the Author; we want to summarize this relationship for each book
in the QuerySet.
Per-object summaries can be generated using the annotate() clause.
When an annotate() clause is specified, each object in the QuerySet
will be annotated with the specified values.
The syntax for these annotations is identical to that used for the
aggregate() clause. Each argument to annotate() describes an aggregate
that is to be calculated.
So when you say:
MyModel.objects.annotate(other_model__some_flag=Value(True, output_field=BooleanField()))
You are not annotation some_flag over other_model.
i.e. you won't have: mymodel.other_model.some_flag
You are annotating other_model__some_flag over mymodel.
i.e. you will have: mymodel.other_model__some_flag
2) I'm not sure how familiar SQL is for you, but in order to preserve MyModel.objects.filter(other_model__some_flag=True) possible, i.e. to keep the annotation when doing JOINS, the ORM would have to do a JOIN over subquery, something like:
INNER JOIN
(
SELECT other_model.id, /* more fields,*/ 1 as some_flag
FROM other_model
) as sub on mymodel.other_model_id = sub.id
which would be super slow and I'm not surprised they are not doing it.
Possible solution
don't annotate your field, but add it as a regular field in your model.
The simplified answer is that models are authoritative on the field collection and Managers are authoritative on collections of models. In your efforts to make it DRY you made it WET, cause you alter the field collection in your manager.
In order to fix it, you would have to teach the model about the lookup and need to do that using the Lookup API.
Now I'm assuming that you're not actually annotating with a fixed value, so if that annotation is in fact reducible to fields, then you may just get it done, because in the end it needs to be mapped to database representation.

Using Django 1.4, How can I save TimeStampedModel objects without automatically updating the modified field?

I am in the process of running a Datamigration using Django and South. I have already added a new field to my model with a Schemamigration, and now I am in the process of populating the field for all the objects of that Model.
The problem is that when I call the save() method on my objects in the datamigration, it is automatically updating the modified field that is on each object and all the objects are ending up with the same modified date. I would like to be able to preserve the modified date from before the datamigration if possible.
Currently my datamigration looks like this:
class Migration(DataMigration):
def forwards(self, orm):
for w in orm.Writer.objects.all():
w.type = 'outside'
if w.managed_by != None:
if w.managed_by.writer != None:
if w.id == w.managed_by.writer.id:
w.type = 'client'
w.save()
Is there a way to only save the values in the type field, without updating the modified date?
You can update your object to only change the fields that you desire by using the update() method available for your model's queryset (see https://docs.djangoproject.com/en/1.4/topics/db/queries/#updating-multiple-objects-at-once for additional details).
Though the documentation references using this feature for multiple objects, you can filter the update query to only target a single object by restricting to the PK of the object you're working with:
orm.Writer.objects.filter(pk=w.pk).update(type='client')

Django ForeignKey which does not require referential integrity?

I'd like to set up a ForeignKey field in a django model which points to another table some of the time. But I want it to be okay to insert an id into this field which refers to an entry in the other table which might not be there. So if the row exists in the other table, I'd like to get all the benefits of the ForeignKey relationship. But if not, I'd like this treated as just a number.
Is this possible? Is this what Generic relations are for?
This question was asked a long time ago, but for newcomers there is now a built in way to handle this by setting db_constraint=False on your ForeignKey:
https://docs.djangoproject.com/en/dev/ref/models/fields/#django.db.models.ForeignKey.db_constraint
customer = models.ForeignKey('Customer', db_constraint=False)
or if you want to to be nullable as well as not enforcing referential integrity:
customer = models.ForeignKey('Customer', null=True, blank=True, db_constraint=False)
We use this in cases where we cannot guarantee that the relations will get created in the right order.
EDIT: update link
I'm new to Django, so I don't now if it provides what you want out-of-the-box. I thought of something like this:
from django.db import models
class YourModel(models.Model):
my_fk = models.PositiveIntegerField()
def set_fk_obj(self, obj):
my_fk = obj.id
def get_fk_obj(self):
if my_fk == None:
return None
try:
obj = YourFkModel.objects.get(pk = self.my_fk)
return obj
except YourFkModel.DoesNotExist:
return None
I don't know if you use the contrib admin app. Using PositiveIntegerField instead of ForeignKey the field would be rendered with a text field on the admin site.
This is probably as simple as declaring a ForeignKey and creating the column without actually declaring it as a FOREIGN KEY. That way, you'll get o.obj_id, o.obj will work if the object exists, and--I think--raise an exception if you try to load an object that doesn't actually exist (probably DoesNotExist).
However, I don't think there's any way to make syncdb do this for you. I found syncdb to be limiting to the point of being useless, so I bypass it entirely and create the schema with my own code. You can use syncdb to create the database, then alter the table directly, eg. ALTER TABLE tablename DROP CONSTRAINT fk_constraint_name.
You also inherently lose ON DELETE CASCADE and all referential integrity checking, of course.
To do the solution by #Glenn Maynard via South, generate an empty South migration:
python manage.py schemamigration myapp name_of_migration --empty
Edit the migration file then run it:
def forwards(self, orm):
db.delete_foreign_key('table_name', 'field_name')
def backwards(self, orm):
sql = db.foreign_key_sql('table_name', 'field_name', 'foreign_table_name', 'foreign_field_name')
db.execute(sql)
Source article
(Note: It might help if you explain why you want this. There might be a better way to approach the underlying problem.)
Is this possible?
Not with ForeignKey alone, because you're overloading the column values with two different meanings, without a reliable way of distinguishing them. (For example, what would happen if a new entry in the target table is created with a primary key matching old entries in the referencing table? What would happen to these old referencing entries when the new target entry is deleted?)
The usual ad hoc solution to this problem is to define a "type" or "tag" column alongside the foreign key, to distinguish the different meanings (but see below).
Is this what Generic relations are for?
Yes, partly.
GenericForeignKey is just a Django convenience helper for the pattern above; it pairs a foreign key with a type tag that identifies which table/model it refers to (using the model's associated ContentType; see contenttypes)
Example:
class Foo(models.Model):
other_type = models.ForeignKey('contenttypes.ContentType', null=True)
other_id = models.PositiveIntegerField()
# Optional accessor, not a stored column
other = generic.GenericForeignKey('other_type', 'other_id')
This will allow you use other like a ForeignKey, to refer to instances of your other model. (In the background, GenericForeignKey gets and sets other_type and other_id for you.)
To represent a number that isn't a reference, you would set other_type to None, and just use other_id directly. In this case, trying to access other will always return None, instead of raising DoesNotExist (or returning an unintended object, due to id collision).
tablename= columnname.ForeignKey('table', null=True, blank=True, db_constraint=False)
use this in your program

Categories