We Keep Coding

Pulling several Django models together into a single list

I have a MySQL database with four related tables: project, unit, unit_equipment, and equipment. A project can have many units; a unit can have many related equipment entries. A single unit can only belong to one project, but there is a many-to-many between equipment and unit (hence the unit_equipment bridge table in the DB). I'm using Django and trying to create a view (or a list?) that shows all 3 models on the same page, together. So it would list all projects, all units, and all equipment. Ideally, the display would be like this:
Project --------- Unit ------------- Equipment
Project 1 first_unit some_equipment1, some_equipment2
Project 1 second_unit more_equipment1, more_equipment2
Project 2 another_unit some_equipment1, more_equipment1
Project 2 and_another_unit some_equipment2, more_equipment2
but at this point I'd also be happy with just having a separate line for each piece of equipment, if comma-separating them is a pain.
Although it seems straightforward to create a form where I can add a new project and add related unit and equipment data (using the TabularInline class), I cannot for the life of me figure out how to bring this data together and just display it. I just want a "master list" of everything in the database, basically.
Here's the code I have so far:
models.py
class Project(models.Model):
name = models.CharField(max_length=255, blank=True, null=True)
class Meta:
managed = False
db_table = 'project'
def __str__(self):
return self.name
class Unit(models.Model):
project = models.ForeignKey(Project, models.DO_NOTHING, blank=True, null=True)
name = models.CharField(max_length=255, blank=True, null=True)
class Meta:
managed = False
db_table = 'unit'
def __str__(self):
return self.name
class UnitEquipment(models.Model):
unit = models.ForeignKey(Unit, models.DO_NOTHING, blank=True, null=True)
equipment = models.ForeignKey(Equipment, models.DO_NOTHING, blank=True, null=True)
class Meta:
managed = False
db_table = 'unit_equipment'
class Equipment(models.Model):
name = models.CharField(max_length=100, blank=True, null=True)
description = models.CharField(max_length=255, blank=True, null=True)
class Meta:
managed = False
db_table = 'equipment'
def __str__(self):
return self.name
views.py
def project_detail_view(request):
obj = Project.objects.all()
context = {'object': obj}
return render(request, "project/project_detail.html", context)
urls.py
urlpatterns = [
path('project/', project_detail_view),
path('', admin.site.urls),
]
admin.py
class UnitTabularInLine(admin.TabularInline):
model = Unit
extra = 0
class ProjectAdmin(admin.ModelAdmin):
inlines = [UnitTabularInLine]
class Meta:
model = Project
# a list of displayed columns name.
list_display = ['name']
# define search columns list, then a search box will be added at the top of list page.
search_fields = ['name']
# define filter columns list, then a filter widget will be shown at right side of list page.
list_filter = ['name']
# define model data list ordering.
ordering = ('name')
I think I need to somehow add more entries to the list_display in the admin file, but every time I try to add unit or equipment it throws an error. I've also tried adding more attributes to Project, but I can't seem to get the syntax right, and I'm never sure which model class I'm supposed to make it.
I've also looked at FormSets, but I cannot get my head around how to alter my current code to get it to work.
How do I get these models together into a unified view?

You don't need to edit the admin view to add your own view: which you may find you are able to do in this case to get your data displayed exactly as you want.
If you do want to show the related object values in the admin list, then you can use lookups and custom columns: however in this case your list would be based upon the Unit.
# You don't need an explicit UnitEquipment model here: you can
# use a simple ManyToManyField
class Unit(models.Model):
project = ...
name = ...
equipment = models.ManyToManyField(Equipment, related_name='units')
def equipment_list(admin, instance):
return ', '.join([x.name for x in instance.equimpent.all()])
class UnitAdmin(admin.ModelAdmin):
class Meta:
model = Unit
list_display = ['project__name', 'name', equipment_list]
def get_queryset(self, request):
return super().get_queryset(request)\
.select_related('project')\
.prefetch_related('equipment')
Note that you need to have the queryset override, otherwise there will be a bunch of extra queries as each unit also requires fetching the project and list of equipment for that unit.
There's also a further improvement you can make to your queries: you could aggregate the related equipment names using a Subquery annotation, and prevent the second query (that fetches all related equipment items for the units in the queryset). This would replace the prefetch_related()

Thanks to #Matthew Schinckel, I was able to find my way to the answer. Here's what my files look like now (only edited the Unit class in models.py):
models.py
class Unit(models.Model):
project = models.ForeignKey(Project, models.DO_NOTHING, blank=True, null=True)
name = models.CharField(max_length=255, blank=True, null=True)
equipment = models.ManyToManyField(Equipment, related_name='units')
class Meta:
managed = False
db_table = 'unit'
def __str__(self):
return self.name
def equipment_list(self):
return ', '.join([x.name for x in self.equipment.all()])
admin.py
class UnitAdmin(admin.ModelAdmin):
class Meta:
model = Unit
# a list of displayed columns name.
list_display = ('project', 'name', 'equipment_list')
# define search columns list, then a search box will be added at the top of list page.
search_fields = ['project']
# define filter columns list, then a filter widget will be shown at right side of list page.
list_filter = ['project', 'name']
# define model data list ordering.
ordering = ('project', 'name')
def get_queryset(self, request):
return super().get_queryset(request)\
.select_related('project')\
.prefetch_related('equipment')
So the changes I made were:
1. Make list_display a tuple instead of a list.
2. Throw def equipment_list(self) into the Unit class (so it's callable as an attribute of Unit) and pass (self) instead of (admin, instance) (I kept getting an error that was looking for the instance argument).

Slow loading Django Admin change/add

I'm making a classic single view application, mapping multiple
datasources.
Django-admin is paginated so there's no impact when I view my list, the problem is when I want to change/add it is .
Using the debug-toolbar my queries look fine, I don't think they take a long time.
I tried to use a suggestion here Django admin change form load quite slow and created a form, but this had no impact.
When is use exclude = ['e_vehicle','e_product'] it's no surprise that add/change screens load instantly.
Any thoughts please
model.py
class Product_Mapping(Trackable):
product_mapping_id = models.AutoField(primary_key=True)
s_product = models.OneToOneField(sProduct, on_delete=models.CASCADE)
e_fund_manager = models.ForeignKey(eManager, models.DO_NOTHING, blank=True, null=True)
e_product = models.ForeignKey(eProduct, models.DO_NOTHING, blank=True, null=True)
e_vehicle = models.ForeignKey(eVehicle, models.DO_NOTHING, blank=True, null=True)
eManager has around 3K
eProduct has around 17K (has fkey to eManager)
eVehicle has around 25K (has fkey to eProduct)
form.py
class MappingProductForm(forms.ModelForm):
s_product = forms.ChoiceField(required=False,
choices=sProduct.objects.values_list('final_publications_product_id', 'product_name'))
e_fund_manager = forms.ChoiceField(required=False,
choices=eManager.objects.values_list('e_fund_manager_id', 'manager_name'))
e_product = forms.ChoiceField(required=False,
choices=eProduct.objects.values_list('e_product_id', 'product_name'))
e_vehicle = forms.ChoiceField(required=False,
choices=eVehicle.objects.values_list('e_vehicle_id', 'formal_vehicle_name'))
class Meta:
model = Product_Mapping
fields = '__all__'
admin.py
#admin.register(Product_Mapping)
class ChampProductMappingAdmin(admin.ModelAdmin):
form = MappingProductForm

It can be seen that there are too many values in e_product and e_vehicle tables. You are using ChoiceField it means inside HTML dropdown there would be ~17K-25K options this would slow down rendering and sometimes hang on client side.
Solution
edit your forms.py
from django.contrib.admin.widgets import ForeignKeyRawIdWidget
class MappingProductForm(forms.ModelForm):
def __init__(self, *args, **kwargs):
super(MappingProductForm, self).__init__(*args, **kwargs)
rel_model = self.Meta.model
rel_eprod = rel_model._meta.get_field('e_product').rel
rel_eprod = rel_model._meta.get_field('e_vehicle').rel
self.fields['e_product'].widget = ForeignKeyRawIdWidget(rel_eprod, admin_site=admin.site)
self.fields['e_vehicle'].widget = ForeignKeyRawIdWidget(rel_eprod, admin_site=admin.site)
s_product = forms.ChoiceField(required=False,
choices=sProduct.objects.values_list('final_publications_product_id', 'product_name'))
e_fund_manager = forms.ChoiceField(required=False,
choices=eManager.objects.values_list('e_fund_manager_id', 'manager_name'))
e_product = forms.ModelChoiceField(required=False,
choices=eProduct.objects.all())
e_vehicle = forms.ModelChoiceField(required=False,
choices=eVehicle.objects.all()
)
class Meta:
model = Product_Mapping
fields = '__all__'
This would keep raw id as widget. You can get something like below.
You can add other entries by hitting search button besides input.
This is better option when you have lots of choices in ModelChoiceField

ModelForm with user-defined fields and choices from foreign keys

In my app, I have Study as the central model. Studys have multiple Strata, and each Strata has multiple Levels. Users create Allocations for a study by choosing one level for each Strata linked to that study, like:
class Study(models.Model):
name = models.CharField(max_length=100)
class Stratum(models.Model):
study = models.ForeignKey(Study, on_delete=models.CASCADE, related_name='strata')
name = models.CharField(max_length=100)
class Level(models.Model):
stratum = models.ForeignKey(Stratum, on_delete=models.CASCADE, related_name='levels')
label = models.CharField(max_length=100)
class Allocation(models.Model):
study = models.ForeignKey(Study, on_delete=models.CASCADE, related_name='allocations')
code = models.CharField(blank=False, max_length=100)
levels = models.ManyToManyField(Level, related_name='allocations')
In order to create fields for the allocation creation form, I am currently finding all the Strata and associated levels in the form's constructor, but hiding the strata as the user doesn't interact with them:
class AllocationForm(forms.ModelForm):
class Meta:
model = Allocation
fields = ('code',)
def __init__(self, *args, **kwargs):
study = kwargs.pop('study')
super(AllocationForm, self).__init__(*args, **kwargs)
strata = Stratum.objects.filter(study=study)
for stratum in strata:
self.fields[stratum.name] = forms.IntegerField(
widget=forms.HiddenInput()
)
self.fields[stratum.name].initial = stratum.id
self.fields[stratum.name].disabled = True
self.fields[stratum.name + '_level'] = forms.ModelChoiceField(
queryset=Level.objects.filter(stratum=stratum)
)
Is this a safe and sensible way to attach the associated objects to the form? I worry that I will lose track of the connection between Strata and Levels when trying to create the allocation. Is this something that would be better performed in the view?

Django Model - Charfield with choices from a row in another model?

I have created a settings table in Django as per the below:-
class Settings(models.Model):
name = models.CharField(max_length=200)
class Meta:
verbose_name = "Settings"
verbose_name_plural = "Settings"
def __str__(self):
return self.name
class SettingChoices(models.Model):
setting = models.ForeignKey(Settings, on_delete=models.PROTECT)
choice = models.CharField(max_length=200)
class Meta:
verbose_name = "Setting Choices"
verbose_name_plural = "Setting Choices"
def __str__(self):
return '{0} - {1}'.format(self.setting, self.choice)
and a sample use of this would be:-
Setting = Circuit Type:
Choices:
DSL
4G
Fibre
then in another model I want to be able to reference this as set of choices
class Circuits(models.Model):
site_data = models.ForeignKey(SiteData, verbose_name="Site", on_delete=models.PROTECT)
order_no = models.CharField(max_length=200, verbose_name="Order No")
expected_install_date = models.DateField()
install_date = models.DateField(blank=True, null=True)
circuit_type = models.CharField(max_length=100, choices=*** here I would get model settings - Circuit Type - Choices ***)
currently I use a list in settings.py but its not fluid, I need my users to be able to alter these settings not for me to manually edit a list in settings.py and push changes each time
I attempted the below:
functions.py
def settings_circuit_types():
from home.models import SettingChoices
type_data = SettingChoices.objects.filter(setting__name='CIRCUIT_TYPES')
circuit_types = []
for c in type_data:
circuit_types.append(c.choice)
return circuit_types
models.py
from app.functions import settings_circuit_types
CIRCUIT_CHOICES = settings_circuit_types()
...
circuit_type = models.CharField(max_length=100, choices= CIRCUIT_CHOICES)
but this has thrown an error
dango.core.exceptions.AppRegistryNotReady: Models aren't loaded yet.
which is understandable, im wondering if what im trying to achieve is possible by other means?
Thanks

So this is a better way of doing this as i mentioned it in comment section:
1 - You don't need Settings and SettingChoices. They are basically the same so you can combine them into one model called Setting:
class Setting(models.Model):
name = models.CharField(max_length=200)
# if you need anything that you might think you need another model for,
# just think this way, add a BooleanField.
# for example if you have a setting available only for admins:
only_admin = models.BooleanField(default=False)
# then when you're going to make a from, just filter the options;
# options = setting.objects.filter(only_admin=False)
class Meta:
verbose_name = "Settings"
verbose_name_plural = "Settings"
def __str__(self):
return self.name
2 - And for Circuits model you just need a simple ForeignKey field:
class Circuits(models.Model):
site_data = models.ForeignKey(SiteData, verbose_name="Site", on_delete=models.PROTECT)
order_no = models.CharField(max_length=200, verbose_name="Order No")
expected_install_date = models.DateField()
install_date = models.DateField(blank=True, null=True)
circuit_type = models.ForeignKey(Setting, null=False, blank=False)
Now when you want to make a form for users to fill:
forms.py:
class CircuitsForm(forms.ModelForm):
class Meta:
model = Circuits
fields = ('install_date', 'circuit_type') # or other fields.
# and to filter which choices are available to choose from:
def __init__(self, *args, **kwargs):
super(CircuitsForm, self).__init__(*args, **kwargs)
self.fields["circuit_type"].queryset = setting.objects.filter(only_admin=False)
This way you have a safe and easy way to make forms for both users and your admins.
You can edit the admin panel itself or just make a url just for admin users with a form like this.
Also if you aren't that kind of people who uses django to render their form you can simply get the available choices in your view and pass it to the template like this:
settings = setting.objects.filter(only_admin=False)
and render it in a template like this:
<select name="circuit_type">
{% for setting in settings %}
<option value="{{ setting.pk }}">{{ setting.name }}</option>
{% endfor %}
</select>
Now you will have only choices you want them to show up in form and even if user tries to mess with template code and add more options, form won't allow them to be accepted and it will raise an error for it.

Additional variable passed to django formset

I have a quite peculiar issue, regarding creating a formset out of a certain form. The thing is, that the form has to has an undisclosed number of fields, so I want to pass a variable to that form, which will take a proper model from database, and create proper fields within form's init method.
StrangeForm(forms.Form):
def __init__(self, id, *args, **kwargs):
super(StrangeForm, self).__init__(*args, **kwargs)
# get the form's base entry
self.certain_entry = Certain_Entry.objects.get(id=id)
# create a hidden input to keep the id
self.fields["id"] = forms.HiddenInput()
self.fields["id"].initial = id
# initiate text field
self.fields["text_field"] = forms.CharField(widget=forms.TextInput(attrs={'style': 'width:95%', }),
required=False)
self.fields["text_field"].initial = self.certain_entry.text
# create and initiate fields for undisclosed number of subentries
subordinates = Certain_Entry_Subordinates.objects.filter(foreign_key=self.certain_entry)
for each in subordinates:
the_name = "%d_%s_%s" % (each.further.id, each.further.name)
self.fields[the_name] = forms.ChoiceField(choices=(("Ok", "Ok"), ("Not Ok", "Not Ok"), ("Maybe" "Maybe"),
required=False)
self.fields[the_name].initial = each.option
The models are as follows:
class Certain_Entry(models.Model):
text = models.TextField(max_length=128, null=True, blank=True)
class Certain_Entry_Subordinates(models.Model):
certain_entry = models.ForeignKey(Certain_Entry, null=False, blank=False)
option = models.TextField(max_length=8, null=True, blank=True)
further = models.ForeignKey(Further, null=False, blank=False)
class Further(models.Model):
name = models.TextField(max_length=32, null=True, blank=True)
So as you see, I need to pass this ID into the form. When it's a single form, it's all OK, but I can't find any information as to how create a formset with forms that require a variable. Any ideas?
PS: As to WHY... Don't ask, I just need to do it that way, trust me. I'm an engineer.