Hi I have list of model objects: my_objects, which should be saved in a databse.
This model has order_with_respect_to property in its Meta class.
When I try to bulk_create this list I got:
null value in column "_order" violates not-null constraint" during bulk_create
When I just iterate over elements and invoke save() on every each of them. Everything is fine, but such sequential database access doesn't satisfy me...
I've tried to invoke signals.pre_save.send function, but this didn't change the situation.
This worked when I've invoked _save_table, on every signle element from my_objects, but _save_table is the heaviest part of save() method, so I gained nothing...
Is there a possibility to save batch of django objects with only one database connection?
I'm using postgresql.
It's just a field and you can set "_order" manually or calculate that before bulk_create.
# Model
class Product(models.Model):
name = models.CharField(max_length=255)
# Bulk create example
# Data
product_data_list = [{"name": "Apple"}, {"name": "Potato"}]
# Add "_order" field for each product
for index, product_data in enumerate(product_data_list):
product_data["_order"] = index
# Create
Product.objects.bulk_create(Product(**product_data) for product_data in product_data_list)
From the docs:
https://docs.djangoproject.com/en/1.8/ref/models/querysets/#bulk-create
If the model’s primary key is an AutoField it does not retrieve and set the primary key attribute, as save() does.
I am guessing that your id was on autoincrement and now it isn't being saved, which is being referenced by _order.
Related
I am a bit confused how Django handles '_id' property when we use ORM with some models that use foreign key.
For example:
class CartItem(models.Model):
user = models.ForeignKey('accounts.CustomUser', related_name='carts', on_delete=models.CASCADE, verbose_name='User')
product = models.ForeignKey('pizza.Product', related_name='carts', on_delete=models.CASCADE, verbose_name=_('Product'))
quantity = models.SmallIntegerField(verbose_name=_('Quantity'))
And when I use ORM with 'filter' I can easily use something like:
CartItem.objects.filter(user=1, product=1, quantity=1)
And Django kind of 'see' that I refer to 'id', but when I use exacly the same line of code, but with 'create' instead of 'filter':
CartItem.objects.create(user=1, product=1, quantity=1)
Then it throws an error saying:
Cannot assign "1": "CartItem.user" must be a "CustomUser" instance.
And to create it I need to use:
CartItem.objects.create(user_id=1, product_id=1, quantity=1)
Why is that? Is there some rule here that I don't understand?
This is the database representation of the ForeignKey [Django-doc]. A reference to model object is represented as:
Behind the scenes, Django appends "_id" to the field name to create its database column name. In the above example, the database table for the Car model will have a manufacturer_id column. (You can change this explicitly by specifying db_column) However, your code should never have to deal with the database column name, unless you write custom SQL. You’ll always deal with the field names of your model object.
So you could say that Django will construct a "twin" column, with an _id suffix. This column has the same type as the type of the primary key of the model you target, and that column will thus contain the primary key of the model object you use. Note that you can use a different field to which you target by specifying the to_field=… parameter [Django-doc].
The ForeignKey itself thus does not exists at the database, that is the logic of Django that will use the primary of that object, and store that in a column with, by default, an _id suffix.
I am very new to Django. I want to link a model which has 2 field 'username' and 'password'. I want to make 'username' field as as Foreign in another model. But as per Django we can only pass the whole Model Object, who is referring to as it's foreign key.
am I wrong somewhere? please give me any solution regarding this basic problem.
No you can link to any unique field of a Django model. So if your models look like:
class Target(models.Model):
name = models.CharField(max_length=128, unique=True)
class SourceModel(models.Model):
target = models.ForeignKey(Target, to_field='name', on_delete=models.CASCADE)
You can assign the value of the target column to the target_id then. So for example:
Target.objects.create(name='target1')
SourceModel.objects.create(target_id='target1')
So you do not need to pass a Target object itself. You can use the …_id "twin" field to use the target column value. The database will normally enforce referential integrity, and thus will prevent passing a non-existing value to the foreign key column.
I am using Django 1.9 and am trying the bulk_create to create many new model objects and associate them with a common related many_to_many object.
My models are as follows
#Computational Job object
class OT_job(models.Model):
is_complete = models.BooleanField()
is_submitted = models.BooleanField()
user_email = models.EmailField()
#Many sequences
class Seq(models.Model):
sequence=models.CharField(max_length=100)
ot_job = models.ManyToManyField(OT_job)
I have thousands of Seq objects that are submitted and have to be associated with their associated job. Previously I was using an iterator and saving them in a for loop. But after reading realized that Django 1.9 has bulk_create.
Currently I am doing
DNASeqs_list = []
for a_seq in some_iterable_with_my_data:
# I create new model instances and add them to the list
DNASeqs_list.append(Seq(sequence=..., ))
I now want to bulk_create these sequence and associate them with the current_job_object.
created_dnaseqs = Seq.objects.bulk_create(DNASeqs_list)
# How do I streamline this part below
for a_seq in created_dnaseqs:
# Had to call save here otherwise got an error
a_seq.save()
a_seq.ot_job.add(curr_job_obj)
I had to call "a_seq.save()" in for loop because I got an error in the part where I was doing "a_seq.ot_job.add(curr_job_obj)" which said
....needs to have a value for field "seq" before this many-to-many relationship can be used.
Despite reading the other questions on this topic , I am still confused because unlike others I do not have a custom "through" model. I am confused with how best to associate the OT_Job with many Seqs with minimal hits to database.
From the docs https://docs.djangoproject.com/en/1.9/ref/models/querysets/#bulk-create:
If the model’s primary key is an AutoField it does not retrieve and set the primary key attribute, as save() does.
It does not work with many-to-many relationships.
bulk_create literally will just create the objects, it does not retrieve the PK into the variable as save does. You would have to re-query the db to get your newly created objects, and then create the M2M relationships, but it sounds like that would not be appropriate and that your current method is currently the best solution.
How to replace default primary key in Django model with custom primary key field?
I have a model with no primary key defined at first since django automatically adds an id field by default as primary field.
#models.py
from django.db import models
class Event(models.Model):
title = models.CharField(max_length=50, unique=True)
description = models.CharField(max_length=150)
I added some objects into it from django shell.
>>e = Event('meeting', 'Contents about meeting')
>>e.save()
>>e = Event('party', 'Contents about party')
>>e.save()
Then I require to add custom character field as primary into this model.
class Event(models.Model):
event-id = models.CharField(max_length=50, primary_key=True)
...
Running makemigrations:
$ python manage.py makemigrations
You are trying to add a non-nullable field 'event-id' to event without a default; we can't do that (the database needs something to populate existing rows).
Please select a fix:
1) Provide a one-off default now (will be set on all existing rows)
2) Quit, and let me add a default in models.py
Select an option: 1
Please enter the default value now, as valid Python
The datetime and `django.utils.timezone modules` are available, so you can do e.g. timezone.now()
>>> 'meetings'
Migrations for 'blog':
0002_auto_20141201_0301.py:
- Remove field id from event
- Add field event-id to event
But while running migrate it threw an error:
.virtualenvs/env/local/lib/python2.7/site-packages/django/db/backends/sqlite3/base.py", line 485, in execute
return Database.Cursor.execute(self, query, params)
django.db.utils.IntegrityError: UNIQUE constraint failed: blog_event__new.event-id
In my experience (using Django 1.8.* here), I've seen similar situations when trying to update the PK field for models that already exist, have a Foreign Key relationship to another model, and have associated data in the back-end table.
You didn't specify if this model is being used in a FK relation, but it seems this is the case.
In this case, the error message you're getting is because the data that already exists needs to be made consistent with the changes you're requesting --i.e. a new field will be the PK. This implies that the current PK must be dropped for django to 'replace' them. (Django only supports a single PK field per model, as per docs[1].)
Providing a default value that matches currently existing data in the related table should work.
For example:
class Organization(models.Model):
# assume former PK field no longer here; name is the new PK
name = models.CharField(primary_key=True)
class Product(models.Model):
name = models.CharField(primary_key=True)
organization = models.ForeignKey(Organization)
If you're updating the Organization model and products already exist, then existing product rows must be updated to refer to a valid Organization PK value. During the migration, you'd want to choose one of the existing Organization PKs (e.g. 'R&D') to update the existing products.
[1] https://docs.djangoproject.com/en/1.8/topics/db/models/#automatic-primary-key-fields
Django has already established an auto incrementing integer id as primary key in your backend as and when u made the previous model.
When u were trying to run the new model , An attempt was made to recreate a new primary key column that failed.
Another reason is,When u made the field,Django was expecting a unique value be explicitly defined for each new row which it couldn't found,hence the reason.
As told in previous answer you can re-create the migration and then try doing it again.It should work.. cheers :-)
The problem is that you made the field unique, then attempted to use the same value for all the rows in the table. I'm not sure if there's a way to programmatically provide the key, but you could do the following:
Delete the migration
Remove the primary_key attribute from the field
Make a new migration
Apply it
Fill in the value for all your rows
Add the primary_key attribute to the field
Make a new migration
Apply it
It's bruteforce-ish, but should work well enough.
Best of luck!
I have a model consisting of two ForeignKey fields as below. (This is a ManytoMany through field)
class EntityConceptLink(models.Model):
entity = models.ForeignKey(Entity)
standard_concept = models.ForeignKey(StandardConcept)
other fields...
I am trying to create objects like so:
EntityConceptLink.objects.get_or_create(
entity_id=entity_id, # passing in an integer, should be PK of Entity
standard_concept=concept) # passing in a model instance
The problem is, when I pass in an entity_id corresponding to an nonexistent Entity, the above code somehow nonetheless saves the model instance. It's only later when I try to do entityconceptlinkinstance.entity that a DoesNotExist: Entity matching query does not exist is raised.
Shouldn't the model fail validation during the attempt to save? Am I doing something wrong here?
Yeah I think below code should work fine.
EntityConceptLink.objects.get_or_create(
entity__id=entity_id, # Believe `id` would be the primary key
standard_concept=concept)
In the above case it will raise error if entity model do not find the required id. See the use double __ instead of single _ in the entity id.