We Keep Coding

Django model serialization problem with default fields

Inside of my app model, I use IntegerRangeField fields:
from django.db import models
from django.contrib.postgres.fields import IntegerRangeField
from django.contrib.postgres.validators import RangeMinValueValidator, RangeMaxValueValidator
from psycopg2.extras import NumericRange
class MyModel(models.Model):
...
field = IntegerRangeField(default=NumericRange(400, 600), validators=[
RangeMinValueValidator(1),
RangeMaxValueValidator(1000)
])
...
The "default" attributes are used in the admin panel UI only, and are not needed anywhere else.
If I add them after migration, they work smoothly. However, if I add them before I run makemigrations, I get this message:
ValueError: Cannot serialize: NumericRange(400, 600, '[)') There are
some values Django cannot serialize into migration files.
I don't even want the default values to be saved to my PostgreSQL database, I just want to not have to remove and bring them back every time I run makemigrations.
Any ideas?
(Didn't work: a custom object with "lower" and "higher" attributes, a single integer, a string, a tuple)
Python: 3.6.6, Django: 2.1.2, PostgreSQL: 11.0

Try to move default value calculation into separate function:
def get_default_range():
return NumericRange(400, 600)
class MyModel(models.Model):
field = IntegerRangeField(default=get_default_range, validators=[
RangeMinValueValidator(1),
RangeMaxValueValidator(1000)
])
In this case migration was successfully generated:
operations = [
migrations.AddField(
model_name='comment',
name='field',
field=django.contrib.postgres.fields.ranges.IntegerRangeField(
default=play.models.get_default_range,
validators=[django.contrib.postgres.validators.RangeMinValueValidator(1),
django.contrib.postgres.validators.RangeMaxValueValidator(1000)]),
),
]

I was able to solve this problem using the string representation of the range:
IntegerRangeField(default='[400, 600]')
django==3.0.5
psycopg2==2.8.5

EDIT I should point out that the original question was 2 years old, but at least in django 3.1, their is a serializer that you must register separately.
You need to register the serializer that is provided by django.
from psycopg2.extras import NumericRange
from django.contrib.postgres.serializers import RangeSerializer
from django.db.migrations.writer import MigrationWriter
MigrationWriter.register_serializer(NumericRange, RangeSerializer)
This piece was not in the documentation, but then you can add your defaults as you'd expect:
class AgeDivision(models.Model):
name = models.CharField(max_length=50, unique=True)
age_range = fields.IntegerRangeField(
unique=True, blank=True, default=NumericRange(None, None))
as for where to put this, it just needs to go along side any module that is only loaded once. The documentation didn't specify where to put custom serializers (as least that I could find), but I'd say put them in the migrations/__init__.py file for any app that requires the serializer. here's the documentation on migration serialization: https://docs.djangoproject.com/en/3.1/topics/migrations/#custom-serializers

Django Custom Migration Not Executing

So I added a new "status" field to a django database table. This field needed a default value, so I defaulted it to "New", but I then added a custom migration file that calls the save() method on all of the objects in that table, as I have the save() overridden to check a different table and pull the correct status from that. However, after running this migration, all of the statuses are still set to "New", so it looks like the save isn't getting executed. I tested this by manually calling the save on all the objects after running the migration, and the statuses are updated as expected.
Here's the table model in models.py:
class SOS(models.Model):
number = models.CharField(max_length=20, unique=True)
...
# the default="New" portion is missing here because I have a migration to remove it after the custom migration (shown below) that saves the models
status = models.CharField(max_length=20)
def save(self, *args, **kwargs):
self.status = self.history_set.get(version=self.latest_version).status if self.history_set.count() != 0 else "New"
super(SOS, self).save(*args, **kwargs)
And here is the migration:
# Generated by Django 2.0.5 on 2018-05-23 13:50
from django.db import migrations, models
def set_status(apps, schema_editor):
SOS = apps.get_model('sos', 'SOS')
for sos in SOS.objects.all():
sos.save()
class Migration(migrations.Migration):
dependencies = [
('sos', '0033_auto_20180523_0950'),
]
operations = [
migrations.RunPython(set_status),
]
So it seems pretty clear to me that I'm doing something wrong with the migration, but I matched it exactly to what I see in the Django Documentation and I also compared it to this StackOverflow answer, and I can't see what I'm doing wrong. There are no errors when I run the migrations, but the custom one I wrote does run pretty much instanteously, which seems strange, as when I do the save manually, it takes about 5 seconds to save all 300+ entries.
Any suggestions?
P.S. Please let me know if there are any relevant details I neglected to include.

When you run migrations and get Model from apps you can not use custom managers or custom save or create or something like that. This model only have the fields and that's all. If you want to achieve what you want you should add your logic into you migrations like this:
# comment to be more than 6 chars...
def set_status(apps, schema_editor):
SOS = apps.get_model('sos', 'SOS')
for sos in SOS.objects.all():
if sos.history_set.exists():
sos.status = sos.history_set.get(version=sos.latest_version).status
else:
sos.status = "New"
sos.save()

What is the correct way to reference nested django apps in ManytoManyFields

I'm trying to update an old django app from 1.1.x to 1.8 LTS, which has involved updating paths, as I apps move apps around.
However, I'm unable to generate migrations for one app, and I can't see how to reference a namespaced app model correctly (assuming that's the problem)
If I've moved files from PROJECT/news/models to PROJECT/site_name/news/models, how should I be referencing these models in in foreign keys or ManyToManyFields?
My app
I have a projects app I want to make migrations for. Projects in some_org/projects, and listed in installed apps like so:
INSTALLED_APPS = (
'some_org.maps',
'some_org.library',
'some_org.extras',
'some_org.news',
'some_org.projects',
'some_org.members',
'some_org.comments',
)
All the apps with the namespace are within the some_org directory.
Here's an abridged view of the models file in the projects app:
# some_org/projects/models.py
from some_org.library import Paper
class Project(models.Model):
name = models.CharField(max_length=30)
def get_children(self):
return ProjectPage.objects.filter(level=1, publish=True, project=self)
def has_library(self):
return Paper.objects.filter(projects=self).count() > 0
Calling ./manage.py makemigrations library, gives me this error:
ValueError: Lookup failed for model referenced by field library.Paper.projects: projects.Project
When I look in the Paper model, it looks like this:
class Paper(models.Model):
# snip
# NewsLinkSubject, Projects et al used to in an app on
# the project root, like `./app_name/models.py`, but is now
# in `some_org/app_name/models.py`
subjects = models.ManyToManyField("news.NewsLinkSubject", blank=True)
projects = models.ManyToManyField("projects.Project", blank=True,)
country = models.ForeignKey("maps.Country", null=True, blank=True)
I initially wonder if the label for the app is wrong, and try the projects ManytoMany field to:
projects = models.ManyToManyField("some_org.projects.Project", blank=True,)
This gives a different error:
ERRORS:
library.Paper.subjects: (fields.E300) Field defines a relation with model some_org.projects.Project', which is either not installed, or is abstract.
As far as I can tell the app is installed, and the models aren't abstract.
I'm pretty stumped - what am I doing wrong, and how can I fix this so I can make migrations for these apps?
I'm using Django 1.8.17, and Python 2.7.13.

You should define an app_label in the Meta class for your models. Whatever you put in there becomes the first part of the name you use in the ManyToManyField definition.

Django Evolution error: 'ConnectionRouter' object has no attribute 'allow_syncdb'

I've searched for the solution on internet and here and didn't find one. It seems like something needs to be changed or added, or maybe I just did something wrong.
I recently started a new project with Django and although I know a little bit of web programming and python I'm totally new to Django.
Everything was ok until I decided to add a new column to my MySQL DB. So, I with easy_install I installed Django Evolution in my VirtualEnv, then I added "django_evolution" in my INSTALLED_APPS.
After this, I ran 'python manage.py syncdb'
Then I added a new field (preview) in my Models file:
from django.db import models
# Create your models here.
class posts(models.Model):
author = models.CharField(max_length = 30)
title = models.CharField(max_length = 100)
bodytext = models.TextField()
preview = models.TextField()
timestamp = models.DateTimeField()
After this I ran 'python manage.py evolve --hint --execute' command.
In the terminal I got an error: ConnectionRouter' object has no attribute 'allow_syncdb'
When I'm trying to access the page I get: 1054, "Unknown column 'myblog_posts.preview' in 'field list'"
It seems like for some reason a preview column can't be added to the DB with Django Evolution.
What I'm doing wrong?

Renaming a django model class-name and corresponding foreign keys with south, without loosing the data

Following is my model:
class myUser_Group(models.Model):
name = models.CharField(max_length=100)
class Channel(models.Model):
name = models.CharField(max_length=100)
description = models.CharField(max_length=1000)
belongs_to_group = models.ManyToManyField(myUser_Group)
class Video(models.Model):
video_url = models.URLField(max_length=300)
belongs_to_channel = models.ManyToManyField(Channel)
description = models.CharField(max_length=1000)
tags = TagField()
class UserProfile(models.Model):
user = models.OneToOneField(User)
class User_History(models.Model):
date_time = models.DateTimeField()
user = models.ForeignKey(UserProfile, null=True, blank=True)
videos_watched = models.ManyToManyField(Video)
I just wanted to remove the underscores from all the class names so that User_History looks UserHistory, also the foreign keys should be updated. I tried using south but could not find it in the documentaion.
One way is export the data, uninstall south, delete migration, rename the table and then import data again. Is there any other way to do it?

You can do this using just South.
For this example I have an app called usergroups with the following model:
class myUser_Group(models.Model):
name = models.CharField(max_length=100)
which I assume is already under migration control with South.
Make the model name change:
class MyUserGroup(models.Model):
name = models.CharField(max_length=100)
and create an empty migration from south
$ python manage.py schemamigration usergroups model_name_change --empty
This will create a skeleton migration file for you to specify what happens. If we edit it so it looks like this (this file will be in the app_name/migrations/ folder -- usergroups/migrations/ in this case):
import datetime
from south.db import db
from south.v2 import SchemaMigration
from django.db import models
class Migration(SchemaMigration):
def forwards(self, orm):
# Change the table name from the old model name to the new model name
# ADD THIS LINE (using the correct table names)
db.rename_table('usergroups_myuser_group', 'usergroups_myusergroup')
def backwards(self, orm):
# Provide a way to do the migration backwards by renaming the other way
# ADD THIS LINE (using the correct table names)
db.rename_table('usergroups_myusergroup', 'usergroups_myuser_group')
models = {
'usergroups.myusergroup': {
'Meta': {'object_name': 'MyUserGroup'},
'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
'name': ('django.db.models.fields.CharField', [], {'max_length': '100'})
}
}
complete_apps = ['usergroups']
In the forwards method we are renaming the database table name to match what the django ORM will look for with the new model name. We reverse the change in backwards to ensure the migration can be stepped back if required.
Run the migration with no need to import/export the exisiting data:
$ python manage.py migrate
The only step remaining is to update the foreign key and many-to-many columns in the models that refer to myUser_Group and change to refer to MyUserGroup.

mmcnickle's solution may work and seems reasonable but I prefer a two step process. In the first step you change the table name.
In your model make sure you have your new table name in:
class Meta:
db_table = new_table_name'
Then like mmcnickle suggested, create a custom migration:
python manage.py schemamigration xyz migration_name --empty
You can read more about that here:
https://docs.djangoproject.com/en/dev/ref/models/options/
Now with your custom migration also add the line to rename your table forward and backwards:
db.rename_table("old_table_name","new_table_name")
This can be enough to migrate and change the table name but if you have been using the Class Meta custom table name before then you'll have to do a bit more. So I would say as a rule, just to be safe do a search in your migration file for "old_table_name" and change any entries you find to the new table name. For example, if you were previously using the Class Meta custom table name, you will likely see:
'Meta': {'object_name': 'ModelNameYouWillChangeNext', 'db_table': "u'old_table_name'"},
So you'll need to change that old table name to the new one.
Now you can migrate with:
python manage.py migrate xyz
At this point your app should run since all you have done is change the table name and tell Django to look for the new table name.
The second step is to change your model name. The difficulty of this really depends on your app but basically you just need to change all the code that references the old model name to code that references the new model name. You also probably need to change some file names and directory names if you have used your old model name in them for organization purposes.
After you do this your app should run fine. At this point your task is pretty much accomplished and your app should run fine with a new model name and new table name. The only problem you will run into using South is the next time you create a migration using it's auto detection feature it will try to drop the old table and create a new one from scratch because it has detected your new model name. To fix this you need to create another custom migration:
python manage.py schemamigration xyz tell_south_we_changed_the_model_name_for_old_model_name --empty
The nice thing is here you do nothing since you have already changed your model name so South picks this up. Just migrate with "pass" in the migrate forwards and backwards:
python manage.py migrate xyz
Nothing is done and South now realizes it is up to date. Try:
python manage.py schemamigration xyz --auto
and you should see it detects nothing has changed