I'm trying to set up a reusable set of data models which I can include in multiple apps, something like this (I'm using users as an example here, but the actual one is a peewee backend for the Authlib library):
# mixins.py
class UserMixin(peewee.Model):
username = peewee.CharField()
password = peewee.CharField()
def set_password(self):
# do stuff
...
Once that mixin's created, I should be able to import it like this, defining only the additional fields (the defaults will already be there from the mixin)
# models.py
db = peewee.SqliteDatabase(config.get('DATABASE_FILE'))
class BaseModel(peewee.model):
class Meta:
database = db
class User(BaseModel, UserMixin):
email = peewee.CharField()
...
I've seen people do this with SQLAlchemy, but when I use this strategy with peewee it doesn't seem to save the fields properly:
if UserMixin inherits from peewee.Model, it says "unable to resolve import hierarchy" (probably since we're importing from peewee.Model multiple times)
if UserMixin is just an object, then peewee doesn't seem to handle its fields properly: they all end up as unbound instances and don't get saved in the database.
My question: is there an "official way" to create reusable model mixins with fields in peewee?
I've seen other projects (such as flask-login) use mixins, but those are generally additional functions like set_password in this example, and not ones that define the fields themselves.
I have a few potential alternate solutions, like
Define the models themselves, rather than mixins, in the shared file, and override their .Meta.database separately for each models.py entry
Define only the other functions in the mixin; let the fields be defined separately each time in models.py
Use the shared code as a file to copy-paste from rather than importing directly.
But there's probably some cleaner way of doing this?
Here's a simple example:
from peewee import *
db = SqliteDatabase(':memory:')
class Base(Model):
class Meta:
database = db
class UserModelMixin(Model):
username = TextField()
class User(UserModelMixin, Base):
pass
print(User._meta.fields)
#{'id': <AutoField: User.id>, 'username': <TextField: User.username>}
I think the problem was the ordering of your mixins.
My app_templ models definition:
models.py
class TableName(models.Model):
name = models.CharField(max_length=100)
#
class TableAbstract(models.Model):
...
class Meta:
abstract = True
It can be used by other apps:
app1 / models.py
from app_templ.models import TableAbstract
class Table1(TableAbstract):
...
app2 / models.py
from app_templ.models import TableAbstract
class Table2(TableAbstract):
...
and so on...
It is necessary for me that in TableName, names of models (tables) of successors registered.
How to make it by means of coding only in the app_templ app?
Technically, what you are describing sounds fine. You are defining an abstract model and then using it to create several models. You do need to import it, and to specify that you want to create these tables (using your above examples). You should think carefully about why you are using the same model multiple times in different apps (should this actually be one app?), but in theory it is fine.
I don't quite understand your first definition, you should probably define your model something like this:
class TableBaseClass(models.Model):
name = models.CharField(max_length=100)
class Meta:
abstract = True
abstract = True will mean that the model is not created in your database (docs) so for clarity, you could store this file in a location distinct from your regular model classes that create tables.
This code:
from app_templ.models import TableAbstract
class Table1(TableAbstract):
...
should be in models.py in your app
From the example of Django Book, I understand if I create models as following:
from xxx import B
class A(models.Model):
b = ManyToManyField(B)
The Django would create a new table(A_B) beyond Table A, which has three columns:
id
a_id
b_id
But now I want to add a new column in the Table A_B, thus would be very easy if I use normal SQL, but now anyone can help me how to do? I can't find any useful information in this book.
It's very easy using django too! You can use through to define your own manytomany intermediary tables
Documentation provides an example addressing your issue:
Extra fields on many-to-many relationships
class Person(models.Model):
name = models.CharField(max_length=128)
def __unicode__(self):
return self.name
class Group(models.Model):
name = models.CharField(max_length=128)
members = models.ManyToManyField(Person, through='Membership')
def __unicode__(self):
return self.name
class Membership(models.Model):
person = models.ForeignKey(Person, on_delete=models.CASCADE)
group = models.ForeignKey(Group, on_delete=models.CASCADE)
date_joined = models.DateField()
invite_reason = models.CharField(max_length=64)
As #dm03514 has answered it is indeed very easy to add column to M2M table via
defining explicitly the M2M through model and adding the desired field there.
However if you would like to add some column to all m2m tables - such approach
wouldn't be sufficient, because it would require to define the M2M through
models for all ManyToManyField's that have been defined across the project.
In my case I wanted to add a "created" timestamp column to all M2M tables that
Django generates "under the hood" without the necessity of defining a separate
model for every ManyToManyField field used in the project. I came up with a
neat solution presented bellow. Cheers!
Introduction
While Django scans your models at startup it creates automatically an implicit
through model for every ManyToManyField that does not define it explicitly.
class ManyToManyField(RelatedField):
# (...)
def contribute_to_class(self, cls, name, **kwargs):
# (...)
super().contribute_to_class(cls, name, **kwargs)
# The intermediate m2m model is not auto created if:
# 1) There is a manually specified intermediate, or
# 2) The class owning the m2m field is abstract.
# 3) The class owning the m2m field has been swapped out.
if not cls._meta.abstract:
if self.remote_field.through:
def resolve_through_model(_, model, field):
field.remote_field.through = model
lazy_related_operation(resolve_through_model, cls, self.remote_field.through, field=self)
elif not cls._meta.swapped:
self.remote_field.through = create_many_to_many_intermediary_model(self, cls)
Source: ManyToManyField.contribute_to_class()
For creation of this implicit model Django uses the
create_many_to_many_intermediary_model() function, which constructs new class
that inherits from models.Model and contains foreign keys to both sides of the
M2M relation. Source: django.db.models.fields.related.create_many_to_many_intermediary_model()
In order to add some column to all auto generated M2M through tables you will
need to monkeypatch this function.
The solution
First you should create the new version of the function that will be used to
patch the original Django function. To do so just copy the code of the function
from Django sources and add the desired fields to the class it returns:
# For example in: <project_root>/lib/monkeypatching/custom_create_m2m_model.py
def create_many_to_many_intermediary_model(field, klass):
# (...)
return type(name, (models.Model,), {
'Meta': meta,
'__module__': klass.__module__,
from_: models.ForeignKey(
klass,
related_name='%s+' % name,
db_tablespace=field.db_tablespace,
db_constraint=field.remote_field.db_constraint,
on_delete=CASCADE,
),
to: models.ForeignKey(
to_model,
related_name='%s+' % name,
db_tablespace=field.db_tablespace,
db_constraint=field.remote_field.db_constraint,
on_delete=CASCADE,
),
# Add your custom-need fields here:
'created': models.DateTimeField(
auto_now_add=True,
verbose_name='Created (UTC)',
),
})
Then you should enclose the patching logic in a separate function:
# For example in: <project_root>/lib/monkeypatching/patches.py
def django_m2m_intermediary_model_monkeypatch():
""" We monkey patch function responsible for creation of intermediary m2m
models in order to inject there a "created" timestamp.
"""
from django.db.models.fields import related
from lib.monkeypatching.custom_create_m2m_model import create_many_to_many_intermediary_model
setattr(
related,
'create_many_to_many_intermediary_model',
create_many_to_many_intermediary_model
)
Finally you have to perform patching, before Django kicks in. Put such code in
__init__.py file located next to your Django project settings.py file:
# <project_root>/<project_name>/__init__.py
from lib.monkeypatching.patches import django_m2m_intermediary_model_monkeypatch
django_m2m_intermediary_model_monkeypatch()
Few other things worth mentioning
Remember that this does not affect m2m tables that have been created in the
db in the past, so if you are introducing this solution in a project that
already had ManyToManyField fields migrated to db, you will need to prepare a
custom migration that will add your custom columns to the tables which were
created before the monkeypatch. Sample migration provided below :)
from django.db import migrations
def auto_created_m2m_fields(_models):
""" Retrieves M2M fields from provided models but only those that have auto
created intermediary models (not user-defined through models).
"""
for model in _models:
for field in model._meta.get_fields():
if (
isinstance(field, models.ManyToManyField)
and field.remote_field.through._meta.auto_created
):
yield field
def add_created_to_m2m_tables(apps, schema_editor):
# Exclude proxy models that don't have separate tables in db
selected_models = [
model for model in apps.get_models()
if not model._meta.proxy
]
# Select only m2m fields that have auto created intermediary models and then
# retrieve m2m intermediary db tables
tables = [
field.remote_field.through._meta.db_table
for field in auto_created_m2m_fields(selected_models)
]
for table_name in tables:
schema_editor.execute(
f'ALTER TABLE {table_name} ADD COLUMN IF NOT EXISTS created '
'timestamp with time zone NOT NULL DEFAULT now()',
)
class Migration(migrations.Migration):
dependencies = []
operations = [migrations.RunPython(add_created_to_m2m_tables)]
Remember that the solution presented only affects the tables that Django
creates automatically for ManyToManyField fields that do not define the
through model. If you already have some explicit m2m through models you will
need to add your custom-need columns there manually.
The patched create_many_to_many_intermediary_model function will apply also
to the models of all 3rd-party apps listed in your INSTALLED_APPS setting.
Last but not least, remember that if you upgrade Django version the original
source code of the patched function may change (!). It's a good idea to setup a
simple unit test that will warn you if such situation happens in the future.
To do so modify the patching function to save the original Django function:
# For example in: <project_root>/lib/monkeypatching/patches.py
def django_m2m_intermediary_model_monkeypatch():
""" We monkey patch function responsible for creation of intermediary m2m
models in order to inject there a "created" timestamp.
"""
from django.db.models.fields import related
from lib.monkeypatching.custom_create_m2m_model import create_many_to_many_intermediary_model
# Save the original Django function for test
original_function = related.create_many_to_many_intermediary_model
setattr(
create_many_to_many_intermediary_model,
'_original_django_function',
original_function
)
# Patch django function with our version of this function
setattr(
related,
'create_many_to_many_intermediary_model',
create_many_to_many_intermediary_model
)
Compute the hash of the source code of the original Django function and prepare
a test that checks whether it is still the same as when you patched it:
def _hash_source_code(_obj):
from inspect import getsourcelines
from hashlib import md5
source_code = ''.join(getsourcelines(_obj)[0])
return md5(source_code.encode()).hexdigest()
def test_original_create_many_to_many_intermediary_model():
""" This test checks whether the original Django function that has been
patched did not changed. The hash of function source code is compared
and if it does not match original hash, that means that Django version
could have been upgraded and patched function could have changed.
"""
from django.db.models.fields.related import create_many_to_many_intermediary_model
original_function_md5_hash = '69d8cea3ce9640f64ce7b1df1c0934b8' # hash obtained before patching (Django 2.0.3)
original_function = getattr(
create_many_to_many_intermediary_model,
'_original_django_function',
None
)
assert original_function
assert _hash_source_code(original_function) == original_function_md5_hash
Cheers
I hope someone will find this answer useful :)
Under the hood, Django creates automatically a through model. It is possible to modify this automatic model foreign key column names.
I could not test the implications on all scenarios, so far it works properly for me.
Using Django 1.8 and onwards' _meta api:
class Person(models.Model):
pass
class Group(models.Model):
members = models.ManyToManyField(Person)
Group.members.through._meta.get_field('person').column = 'alt_person_id'
Group.members.through._meta.get_field('group' ).column = 'alt_group_id'
# Prior to Django 1.8 _meta can also be used, but is more hackish than this
Group.members.through.person.field.column = 'alt_person_id'
Group.members.through.group .field.column = 'alt_group_id'
Same as question I needed a custom models.ManyToManyField to add some columns to specific M2M relations.
My answer is Base on #Krzysiek answer with a small change, I inherit a class from models.ManyToManyField and monkeypatch its contribute_to_class method partially with unittest.mock.patch to use a custom create_many_to_many_intermediary_model instead of original one, This way I can control which M2M relations can have custom columns and also 3rd-party apps won't affected as #Krzysiek mentioned in its answer
from django.db.models.fields.related import (
lazy_related_operation,
resolve_relation,
make_model_tuple,
CASCADE,
_,
)
from unittest.mock import patch
def custom_create_many_to_many_intermediary_model(field, klass):
from django.db import models
def set_managed(model, related, through):
through._meta.managed = model._meta.managed or related._meta.managed
to_model = resolve_relation(klass, field.remote_field.model)
name = "%s_%s" % (klass._meta.object_name, field.name)
lazy_related_operation(set_managed, klass, to_model, name)
to = make_model_tuple(to_model)[1]
from_ = klass._meta.model_name
if to == from_:
to = "to_%s" % to
from_ = "from_%s" % from_
meta = type(
"Meta",
(),
{
"db_table": field._get_m2m_db_table(klass._meta),
"auto_created": klass,
"app_label": klass._meta.app_label,
"db_tablespace": klass._meta.db_tablespace,
"unique_together": (from_, to),
"verbose_name": _("%(from)s-%(to)s relationship")
% {"from": from_, "to": to},
"verbose_name_plural": _("%(from)s-%(to)s relationships")
% {"from": from_, "to": to},
"apps": field.model._meta.apps,
},
)
# Construct and return the new class.
return type(
name,
(models.Model,),
{
"Meta": meta,
"__module__": klass.__module__,
from_: models.ForeignKey(
klass,
related_name="%s+" % name,
db_tablespace=field.db_tablespace,
db_constraint=field.remote_field.db_constraint,
on_delete=CASCADE,
),
to: models.ForeignKey(
to_model,
related_name="%s+" % name,
db_tablespace=field.db_tablespace,
db_constraint=field.remote_field.db_constraint,
on_delete=CASCADE,
),
# custom-need fields here:
"is_custom_m2m": models.BooleanField(default=False),
},
)
class CustomManyToManyField(models.ManyToManyField):
def contribute_to_class(self, cls, name, **kwargs):
############################################################
# Inspired by https://stackoverflow.com/a/60421834/9917276 #
############################################################
with patch(
"django.db.models.fields.related.create_many_to_many_intermediary_model",
wraps=custom_create_many_to_many_intermediary_model,
):
super().contribute_to_class(cls, name, **kwargs)
Then I use my CustomManyToManyField instead of models.ManyToMany When I want my m2m table have custom fields
class MyModel(models.Model):
my_m2m_field = CustomManyToManyField()
Note that new custom columns may not add if m2m field already exist, and you have to add them manualy or with script by migration as #Krzysiek mentioned.
I would like to mix a field into an existing model which I would rather not edit (it comes from a third party project and I would rather leave the project untouched). I have created a simple example which illustrates what I am trying but unable to do:
In an empty Django project I have created apps app1 and app2 (they are in that order in settings). They look like the following:
app1.models.py:
from django.db import models
from app2.models import BlogPost
class BlogPostExtend(models.Model):
custom_field_name = models.CharField(max_length=200)
class Meta:
abstract = True
BlogPost.__bases__ = (BlogPostExtend,)+BlogPost.__bases__ # this prevents MRO error
app2.models.py:
from django.db import models
class BlogPost(models.Model):
field_name = models.CharField(max_length=100)
Unfortunately this does not result in custom_field_name being created in the database when I syncdb, although at the command line if I type BlogPost.custom_field_name it does recognize it as a CharField. I know that in this simple case I could have BlogPost inherit from BlogPostExtend, but in the real use case I cannot edit BlogPost.
This is a very simplified example but it illustrates what I am trying to do.
Thanks!
Mixins work great with adding attributes and methods, but not fields.
In app1.models.py, do this instead:
from django.db import models
from app2.models import BlogPost
custom_field_name = models.CharField(max_length=200)
custom_field_name.contribute_to_class(BlogPost, "custom_field_name")
I think also the app1 app should come after app2 in INSTALLED_APPS for this to work.
Here is an explanation on contribute_to_class
I want to add few fields to every model in my django application. This time it's created_at, updated_at and notes. Duplicating code for every of 20+ models seems dumb. So, I decided to use abstract base class which would add these fields. The problem is that fields inherited from abstract base class come first in the field list in admin. Declaring field order for every ModelAdmin class is not an option, it's even more duplicate code than with manual field declaration.
In my final solution, I modified model constructor to reorder fields in _meta before creating new instance:
class MyModel(models.Model):
# Service fields
notes = my_fields.NotesField()
created_at = models.DateTimeField(auto_now_add=True)
updated_at = models.DateTimeField(auto_now=True)
class Meta:
abstract = True
last_fields = ("notes", "created_at", "updated_at")
def __init__(self, *args, **kwargs):
new_order = [f.name for f in self._meta.fields]
for field in self.last_fields:
new_order.remove(field)
new_order.append(field)
self._meta._field_name_cache.sort(key=lambda x: new_order.index(x.name))
super(MyModel, self).__init__(*args, **kwargs)
class ModelA(MyModel):
field1 = models.CharField()
field2 = models.CharField()
#etc ...
It works as intended, but I'm wondering, is there a better way to acheive my goal?
I was having the very same problem, but I found these solutions to be problematic, so here's what I did:
class BaseAdmin(admin.ModelAdmin):
def get_fieldsets(self, request, obj = None):
res = super(BaseAdmin, self).get_fieldsets(request, obj)
# I only need to move one field; change the following
# line to account for more.
res[0][1]['fields'].append(res[0][1]['fields'].pop(0))
return res
Changing the fieldset in the admin makes more sense to me, than changing the fields in the model.
If you mainly need the ordering for Django's admin you could also create your "generic"-admin class via sub-classing Django's admin class. See http://docs.djangoproject.com/en/dev/intro/tutorial02/#customize-the-admin-form for customizing the display of fields in the admin.
You could overwrite the admin's __init__ to setup fields/fieldsets on creation of the admin instance as you wish. E.g. you could do something like:
class MyAdmin(admin.ModelAdmin):
def __init__(self, model, admin_site):
general_fields = ['notes', 'created_at', 'updated_at']
fields = [f.name for f in self.model._meta.fields if f.name not in general_fields]
self.fields = fields + general_fields
super(admin.ModelAdmin, self).__init__(model, admin_site)
Besides that i think it's not a good practice to modify the (private) _field_name_cache!
I ALSO didn't like the other solutions, so I instead just modified the migrations files directly.
Whenever you create a new table in models.py, you will have to run "python manage.py makemigrations" (I believe this in Django >= v1.7.5). Once you do this, open up the newly created migrations file in your_app_path/migrations/ directory and simply move the rows to the order you want them to be in. Then run "python manage.py migrate". Voila! By going into "python manage.py dbshell" you can see that the order of the columns is exactly how you wanted them!
Downside to this method: You have to do this manually for each table you create, but fortunately the overhead is minimal. And this can only be done when you're creating a new table, not to modify an existing one.