We Keep Coding

Using a single ManyToMany relation table instead of ManyToMany & ForeignKey field on multiple models?

I have a Django application that handles data analysis workflows, with database models that look something like this:
class Workflow(models.Model):
execution_id = models.UUIDField()
class WorkflowItem(models.Model):
workflow = models.ForeignKey(Workflow)
type = models.CharField(choices=["input", "output"])
files = models.ManyToManyField(File)
class File(models.Model):
path = models.CharField()
class FileMetadata(models.Model):
metadata = models.JSONField()
file = models.ForeignKey(File)
version = models.IntegerField()
A given Workflow will have many WorkflowItem's, which correspond to File's which can be used by WorkflowItem's across many Workflow's. Each File can have many associated FileMetadata's, of which the entry with the max version value is typically used for a given operation.
As the application has been growing, its getting tedious to build out all the different combinations of logic needed to find the entries in one table based on a given entry in another table just by using each tables' Foreign Key interface (Workflow <-> WorkflowItem <-> File <-> FileMetadata).
I am considering just building a table that holds all the foreign keys for every relationship in a single place. Something like this:
class WorkflowFile(models.Model):
workflow = models.ForeignKey(Workflow)
workflow_item = models.ForeignKey(WorkflowItem)
file = models.ForeignKey(File)
file_metadata = models.ForeignKey(FileMetadata)
However, I am not sure if this is a good idea or not. Its not clear to me if implementing a table like this is advantageous compared to just following all the foreign key relationships individually per-table. Its also not clear to me how I should set up such a table through Django, and if the new requirement for manually entering values into this table all the time would outweigh the reduced need for unique query logic every time I want to query these relationships. My end-goal is to provide a simpler, more consistent way to get all of the items in the relationship based on any of the other items in the relationship.
This question seems similar in premise, but I am not clear that the problem or proposed solution is relevant to what I am looking for here.

Not sure this will actually answer your question but if you want to go the way with multiple FK's then you may consider using through table in combination with m2m changed signal to add proper FK's to this model after adding M2M records to WorkflowItem.
It'll be something like:
from django.db.models.signals import m2m_changed
class WorkflowItem(models.Model):
workflow = models.ForeignKey(Workflow)
type = models.CharField(choices=["input", "output"])
files = models.ManyToManyField(File, through=IntermediateTable)
class IntermediateTable(models.Model):
file = models.ForeignKey(File, related_name='file')
workflow_item = models.ForeignKey(WorkflowItem, related_name='worflowitem')
workflow = models.ForeignKey(Workflow, null=True)
file_metadata = models.ForeignKey(FileMetadata)
def workflow_item_changed(sender, **kwargs):
sender.workflow = sender.workflow_item.workflow
...
sender.save()
m2m_changed.connect(workflow_item_changed, sender=WorkflowItem.files.through)

Django: join two table on foreign key to third table?

I have three models
class A(Model):
...
class B(Model):
id = IntegerField()
a = ForeignKey(A)
class C(Model):
id = IntegerField()
a = ForeignKey(A)
I want get the pairs of (B.id, C.id), for which B.a==C.a. How do I make that join using the django orm?

Django allows you to reverse the lookup in much the same way that you can use do a forward lookup using __:
It works backwards, too. To refer to a “reverse” relationship, just use the lowercase name of the model.
This example retrieves all Blog objects which have at least one Entry whose headline contains 'Lennon':
Blog.objects.filter(entry__headline__contains='Lennon')
I think you can do something like this, with #Daniel Roseman's caveat about the type of result set that you will get back.
ids = B.objects.prefetch_related('a', 'a__c').values_list('id', 'a__c__id')
The prefetch related will help with performance in older versions of django if memory serves.

Django ManyToMany Field

I need to make a smart menu, for which I need a ManyToMany relation.
My model is:
from django.db import models
class Health_plan(models.Model):
a = models.IntegerField ()
b = models.IntegerField ()
class Doctors_list(models.Model):
name = models.CharField(max_length=30)
hp_id = models.ManyToManyField(Health_plan)
def __unicode__(self):
return self.name
How do I make this relation in the database ? I was thinking in puting the health_plans (a,b) as columns, and the doctors as rows, with 0s and 1s to identify their covered health_plans.
Someone told me this was a misuse of a ManyToManyField, I don't know wich step to take.
Help appreciated

The approach of puting the health_plans as columns is not necessarily wrong, but it implies that you have a fixed number of health plans and that you will never add a new one.
The traditional approach for many-to-many relationships in relational databases is to introduce a table in the middle. This table will just contain the association between a doctor and a health plan.
If you have a Doctor table that contains:
id name
1 foo
2 bar
And a HealthPlan table:
id model
1 a
2 b
You then add a table Doctor_HealthPlan that is like:
doctor_id healthplan_id
1 2
2 1
2 2
The ManyToMany field type in django will automatically create this table for you. Your code is correct, but you should probably rename hp_id to something like health_plans, since it is a proxy that allows you to access the list of health plans associated to a doctor.

Django's ORM already takes care of the intermediate table so you don't have to "make this relation(ship) in the database", but given your question you obviously need to learn about proper relational model normalisation - if you don't understand the relational model you won't get nowhere with Django's ORM, nor with any other sql stuff FWIW.
For the record, in the relational model, a many to many relationship is modeled as a relation ("table" in SQL) with foreign keys on both other tables, ie:
health_plan(#health_plan_id, name, ...)
doctor(#doctor_id, firstname, lastname, ...)
doctors_health_plans(#health_plan_id, #doctor_id)
So your django models should be:
class HealthPlan(models.Model):
# no need to define an 'id' field,
# the ORM provides one by default
name = models.CharField(....)
class Doctor(models.Model):
firstname = models.CharField(....)
lastname = models.CharField(....)
health_plans = models.ManyToManyField(HealthPlan, related_name="doctors")
Then you'll be able to get all HealthPlans for a Doctor :
doc = Doctor.objects.get(pk=xxxx)
doc.health_plans.all()
and all Doctors for an HealthPlan:
plan = HealthPlan.objects.get(pk=xxxx)
plan.doctors.all()
The FineManual(tm) is your friend as usual...

You just need to save the two models first then add the healthplan instance to the doctors list. Django will handle the rest for you .
For example :
doctor_list = Doctors_list(name="Bwire")
health_plan.save()
doctor_list.save()
#Then add the plan to the doctors list.
doctor_list.hp_id.add(health_plan)

Django creates the tabels for you. In your project folder run:
python manage.py syncdb
Health_plan and Doctors_list are both tables.
'a' and 'b' are columns in Health_plan. 'Name' and 'hp_id' are columns in Doctors_list.
Django will create a column for id in each table. Django will also create a table "Doctor_list_Health_plan" to store the relation information.
Django models are Python classes, so the Python naming conventions apply. Use HealthPlan and Doctor (CapitalizeWord singular).
Your field names are a bit abstract. I suggest you use more descriptive names. Eg:
class HealthPlan(models.Model):
name = models.CharField()
extra_care = models.BooleanField()

Django ORM - join a lot of tables by user_id

I have about 5 tables with user foreign key, i.e.:
class Passport(models.Model):
user = models.OneToOneField(User)
...
Also, classes like UserProfile, Company, UserOptions, NotifySettings, etc. I need to get dict with joined values for user-summary page. Also i need to join to this union this summary stats:
rent_sums = WriteOff.objects.filter(created_at__range=(rent_start, rent_finish), write_off_type='rent').values('user').\
annotate(rent_amount=Sum('amount')).order_by()
How can i do it without manualy update of a result dict?

you can do this by executing a custom SQL from django but the main problem I see here is the way the models are been handled, it would be a lot easier if you mixed all this user foreign keys to be in the UserProfile, for example:
class UserProfile(models.Model):
user = fields.OneToOneField(User)
company = fields.ForeignKey(Company)
options = fields.ForeignKey(Options)
notification_settings = fields.ForeignKey(NotifySettings)
...
This way you can use the ORM django brings with more ease. In my opinion it would be faster for you to create a migration instead of this huge query.

Django ForeignKey which does not require referential integrity?

I'd like to set up a ForeignKey field in a django model which points to another table some of the time. But I want it to be okay to insert an id into this field which refers to an entry in the other table which might not be there. So if the row exists in the other table, I'd like to get all the benefits of the ForeignKey relationship. But if not, I'd like this treated as just a number.
Is this possible? Is this what Generic relations are for?

This question was asked a long time ago, but for newcomers there is now a built in way to handle this by setting db_constraint=False on your ForeignKey:
https://docs.djangoproject.com/en/dev/ref/models/fields/#django.db.models.ForeignKey.db_constraint
customer = models.ForeignKey('Customer', db_constraint=False)
or if you want to to be nullable as well as not enforcing referential integrity:
customer = models.ForeignKey('Customer', null=True, blank=True, db_constraint=False)
We use this in cases where we cannot guarantee that the relations will get created in the right order.
EDIT: update link

I'm new to Django, so I don't now if it provides what you want out-of-the-box. I thought of something like this:
from django.db import models
class YourModel(models.Model):
my_fk = models.PositiveIntegerField()
def set_fk_obj(self, obj):
my_fk = obj.id
def get_fk_obj(self):
if my_fk == None:
return None
try:
obj = YourFkModel.objects.get(pk = self.my_fk)
return obj
except YourFkModel.DoesNotExist:
return None
I don't know if you use the contrib admin app. Using PositiveIntegerField instead of ForeignKey the field would be rendered with a text field on the admin site.

This is probably as simple as declaring a ForeignKey and creating the column without actually declaring it as a FOREIGN KEY. That way, you'll get o.obj_id, o.obj will work if the object exists, and--I think--raise an exception if you try to load an object that doesn't actually exist (probably DoesNotExist).
However, I don't think there's any way to make syncdb do this for you. I found syncdb to be limiting to the point of being useless, so I bypass it entirely and create the schema with my own code. You can use syncdb to create the database, then alter the table directly, eg. ALTER TABLE tablename DROP CONSTRAINT fk_constraint_name.
You also inherently lose ON DELETE CASCADE and all referential integrity checking, of course.

To do the solution by #Glenn Maynard via South, generate an empty South migration:
python manage.py schemamigration myapp name_of_migration --empty
Edit the migration file then run it:
def forwards(self, orm):
db.delete_foreign_key('table_name', 'field_name')
def backwards(self, orm):
sql = db.foreign_key_sql('table_name', 'field_name', 'foreign_table_name', 'foreign_field_name')
db.execute(sql)
Source article

(Note: It might help if you explain why you want this. There might be a better way to approach the underlying problem.)
Is this possible?
Not with ForeignKey alone, because you're overloading the column values with two different meanings, without a reliable way of distinguishing them. (For example, what would happen if a new entry in the target table is created with a primary key matching old entries in the referencing table? What would happen to these old referencing entries when the new target entry is deleted?)
The usual ad hoc solution to this problem is to define a "type" or "tag" column alongside the foreign key, to distinguish the different meanings (but see below).
Is this what Generic relations are for?
Yes, partly.
GenericForeignKey is just a Django convenience helper for the pattern above; it pairs a foreign key with a type tag that identifies which table/model it refers to (using the model's associated ContentType; see contenttypes)
Example:
class Foo(models.Model):
other_type = models.ForeignKey('contenttypes.ContentType', null=True)
other_id = models.PositiveIntegerField()
# Optional accessor, not a stored column
other = generic.GenericForeignKey('other_type', 'other_id')
This will allow you use other like a ForeignKey, to refer to instances of your other model. (In the background, GenericForeignKey gets and sets other_type and other_id for you.)
To represent a number that isn't a reference, you would set other_type to None, and just use other_id directly. In this case, trying to access other will always return None, instead of raising DoesNotExist (or returning an unintended object, due to id collision).

tablename= columnname.ForeignKey('table', null=True, blank=True, db_constraint=False)
use this in your program