Django : ProgrammingError: column "id" does not exist - python

I try to use postgresql database (before I had SQLite) but I have a message when I execute python manage.py migrate :
Operations to perform:
Apply all migrations: sessions, admin, sites, auth, contenttypes, main_app
Running migrations:
Rendering model states... DONE
Applying main_app.0004_auto_20161002_0034...Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 353, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 345, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 348, in run_from_argv
self.execute(*args, **cmd_options)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 399, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/migrate.py", line 200, in handle
executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 92, in migrate
self._migrate_all_forwards(plan, full_plan, fake=fake, fake_initial=fake_initial)
File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 121, in _migrate_all_forwards
state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 198, in apply_migration
state = migration.apply(state, schema_editor)
File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/migration.py", line 123, in apply
operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/operations/fields.py", line 201, in database_forwards
schema_editor.alter_field(from_model, from_field, to_field)
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/base/schema.py", line 482, in alter_field
old_db_params, new_db_params, strict)
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/postgresql/schema.py", line 110, in _alter_field
new_db_params, strict,
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/base/schema.py", line 634, in _alter_field
params,
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/base/schema.py", line 110, in execute
cursor.execute(sql, params)
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 79, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python2.7/dist-packages/django/db/utils.py", line 95, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
django.db.utils.ProgrammingError: column "id" does not exist
LINE 1: ..._app_projet" ALTER COLUMN "id" TYPE integer USING "id"::inte...
So I guess this error is about my model 'Projet' but this model has ID field ! I see that in 0001_initial.py :
...
migrations.CreateModel(
name='Projet',
fields=[
('id', models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID')),
('name', models.CharField(db_index=True, max_length=30)),
('presentation', models.TextField(max_length=2500, null=True)),
...
My migration main_app.0004_auto_20161002_0034 :
class Migration(migrations.Migration):
dependencies = [
('main_app', '0003_auto_20161002_0033'),
]
operations = [
migrations.AlterField(
model_name='projet',
name='id',
field=models.AutoField(auto_created=True, primary_key=True, serialize=False, verbose_name='ID'),
),
]
Projet model is like that :
class Group(models.Model) :
name = models.CharField(max_length=30, db_index=True)
class Meta :
abstract = True
unique_together = (("id", "name"),)
class Projet(Group) :
presentation = models.TextField(max_length=2500, null=True)
Realy I don't understand why I have this error... my settings configuration database is good, all others models work, ...
If I forgot something, please don't hesitate to tell me ! I need realy your helps !

Your model definition doesn't identify any of the fields as a primary key, therefore Django assumes a primary key column named id. However this column doesn't actually exist in the table.
Change your model definition to designate one of the fields as a primary key, and Django won't try to look for an id column.

class Group(models.Model) :
name = models.CharField(max_length=30, unique=True)
class Meta:
abstract = True
class Projet(Group) :
presentation = models.CharField(max_length=2500, blank=True)
Remove your migration and create a new one.
Just adding my two cents.
Nullable fields:
Avoid using null on string-based fields such as CharField and
TextField because empty string values will always be stored as empty
strings, not as NULL. If a string-based field has null=True, that
means it has two possible values for “no data”: NULL, and the empty
string. In most cases, it’s redundant to have two possible values for
“no data;” the Django convention is to use the empty string, not NULL.
See https://docs.djangoproject.com/en/1.10/ref/models/fields/#null
max_length:
If you specify a max_length attribute, it will be reflected in the
Textarea widget of the auto-generated form field. However it is not
enforced at the model or database level. Use a CharField for that.
https://docs.djangoproject.com/en/1.10/ref/models/fields/#textfield

if the id column is missing, just add the column in Postgres itself. Go to pgadmin than to the table and there create the column id.
Than back to Django makemigrations than migrate. If this will fail, delete everything in the migration folder of the app without init.py and delete everything in the folder pycache
than makemigrations followed by migrate and it should work.

I have that error happen when I try to use a third party library to upload data to the table with the keyword 'replace'.
For instance if I use pyodbc/sqlalchemy code and pandas together like
df.to_sql(table_name, engine, if_exists='replace')
where table_name is a table in my model.py file.
I do this on initial load as well as periodically for dashboards where I want to use external sources without keeping existing data.
What pandas does is create a primary key column called 'index' instead of id on replace. So, then when I try to use a queryset, it says it can't find the id field because its' name got changed.
I don't know if there is a way using pandas to change the index to id as the default. Perhaps a comment can add flavor. My data doesn't have a true primary key because I have a lot of redundant data in the data for different levels of aggregation.

Do this:
You need to delete the table from the database.
DROP TABLE <table>;
Perform migration:
python manage.py migrate
If this does not help, do the migration using peewee:
/var/venv3.8/bin/pw_migrate migrate --database=postgresql://postgres#127.0.0.1:5432/<db>

Related

Django + SQLite + ForeignKey('self') = Failed Migration

In Django 1.9 using SQLite as the database backend, I am getting an error when trying to apply migrations after modifying a model to use multi-table inheritance instead of the OneToOneField relationship it was previously using.
Particularly, the problem seems to be due to including a ForeignKey('self') in the model.
models.py
Here is the app from which an initial migration is successfully made and applied:
from django.db import models
from django.contrib.auth.models import User
class Customer(models.Model):
account = models.OneToOneField(User)
parent = models.ForeignKey('self', null=True)
models.py (modified)
The app is then modified to inherit from the User model rather than linking to it:
from django.db import models
from django.contrib.auth.models import User
class Customer(User):
parent = models.ForeignKey('self', null=True)
At this point a ./manage.py makemigrations <app> succeeds but then applying the migration fails with:
Error
Traceback (most recent call last):
File "./manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/home/piranha/.virtualenvs/Python_3.5-Django_1.9/lib/python3.5/site-packages/django/core/management/__init__.py", line 353, in execute_from_command_line
utility.execute()
File "/home/piranha/.virtualenvs/Python_3.5-Django_1.9/lib/python3.5/site-packages/django/core/management/__init__.py", line 345, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/piranha/.virtualenvs/Python_3.5-Django_1.9/lib/python3.5/site-packages/django/core/management/base.py", line 348, in run_from_argv
self.execute(*args, **cmd_options)
File "/home/piranha/.virtualenvs/Python_3.5-Django_1.9/lib/python3.5/site-packages/django/core/management/base.py", line 399, in execute
output = self.handle(*args, **options)
File "/home/piranha/.virtualenvs/Python_3.5-Django_1.9/lib/python3.5/site-packages/django/core/management/commands/migrate.py", line 200, in handle
executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
File "/home/piranha/.virtualenvs/Python_3.5-Django_1.9/lib/python3.5/site-packages/django/db/migrations/executor.py", line 92, in migrate
self._migrate_all_forwards(plan, full_plan, fake=fake, fake_initial=fake_initial)
File "/home/piranha/.virtualenvs/Python_3.5-Django_1.9/lib/python3.5/site-packages/django/db/migrations/executor.py", line 121, in _migrate_all_forwards
state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
File "/home/piranha/.virtualenvs/Python_3.5-Django_1.9/lib/python3.5/site-packages/django/db/migrations/executor.py", line 198, in apply_migration
state = migration.apply(state, schema_editor)
File "/home/piranha/.virtualenvs/Python_3.5-Django_1.9/lib/python3.5/site-packages/django/db/migrations/migration.py", line 123, in apply
operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
File "/home/piranha/.virtualenvs/Python_3.5-Django_1.9/lib/python3.5/site-packages/django/db/migrations/operations/fields.py", line 62, in database_forwards
field,
File "/home/piranha/.virtualenvs/Python_3.5-Django_1.9/lib/python3.5/site-packages/django/db/backends/sqlite3/schema.py", line 221, in add_field
self._remake_table(model, create_fields=[field])
File "/home/piranha/.virtualenvs/Python_3.5-Django_1.9/lib/python3.5/site-packages/django/db/backends/sqlite3/schema.py", line 181, in _remake_table
self.create_model(temp_model)
File "/home/piranha/.virtualenvs/Python_3.5-Django_1.9/lib/python3.5/site-packages/django/db/backends/base/schema.py", line 250, in create_model
to_column = field.remote_field.model._meta.get_field(field.remote_field.field_name).column
File "/home/piranha/.virtualenvs/Python_3.5-Django_1.9/lib/python3.5/site-packages/django/db/models/options.py", line 582, in get_field
raise FieldDoesNotExist('%s has no field named %r' % (self.object_name, field_name))
django.core.exceptions.FieldDoesNotExist: Customer has no field named 'id'
Again, the problem seems to only occur when using SQLite and including the self-referencing ForeignKey. I think it is occurring because SQLite can't ALTER columns resulting in Django rebuilding the table, but the self-referencing ForeignKey is causing the rebuild to look for the 'id' column which no longer exists in the new model.
I've tried manually adjusting the migration operations every which way and even adding a migrations.RunPython with apps.get_model() to use the historical version of the model. Nothing seems to work.
How can I adjust the following migration to avoid the error:
Migration
# -*- coding: utf-8 -*-
# Generated by Django 1.9.2 on 2016-09-07 01:16
from __future__ import unicode_literals
from django.conf import settings
import django.contrib.auth.models
from django.db import migrations, models
import django.db.models.deletion
class Migration(migrations.Migration):
dependencies = [
migrations.swappable_dependency(settings.AUTH_USER_MODEL),
('base', '0001_initial'),
]
operations = [
migrations.AlterModelOptions(
name='customer',
options={'verbose_name': 'user', 'verbose_name_plural': 'users'},
),
migrations.AlterModelManagers(
name='customer',
managers=[
('objects', django.contrib.auth.models.UserManager()),
],
),
migrations.RemoveField(
model_name='customer',
name='account',
),
migrations.RemoveField(
model_name='customer',
name='id',
),
migrations.AddField(
model_name='customer',
name='user_ptr',
field=models.OneToOneField(auto_created=True, default=None, on_delete=django.db.models.deletion.CASCADE, parent_link=True, primary_key=True, serialize=False, to=settings.AUTH_USER_MODEL),
preserve_default=False,
),
]
And, yes, I know I'll ultimately need to break the migration into multiple phases and do a data migration to put something useful in the new user_ptr field.
Ok, it took me a while to work out: but basically, I don't think you can do this:
class Customer(User):
parent = models.ForeignKey('self', null=True)
Instead, it should be something more like (untested:)
class Customer(User):
parent = models.ForeignKey('admin.User', null=True)
If you want to make sure that data integrity is actually a Customer, you could put a pre-validate in save (or something, if you're using a save-pipeline of some kind)
If you include a data migration step to preserve that value as well, perhaps through using a temporary value to store it in (or dump and reinsert the objects).

ValueError on foreignkey default value

I deleted my db.sqlite3 and the file on my migration folder except __init__.py but including the __pycache__ folder.
My app's models are multiple files in a folder with an __init__.py file with some import and an __all__ variable (I don't know if this matter).
From the prompt I give the first migrate command and it look to work (migrating sessions, admin, auth, registration, contenttypes. There isn't my app 'core').
So i give the first makemigrations command and again the migrate command.
This time it give an error:
ValueError: invalid literal for int() with base 10: 'basic'
In a model there's an ForeignKey fields with default='basic'. basic will be an object of the related class but actually it don't exist (before I create the database, then I will populate it). I think it's normal.
Anyway I change in default=1 and it works (but 1 will not be an object, so it's wrong!)
My model:
Class Payment(models.Model):
name = models.CharField(max_length=32)
Class ProfileUser(models.Model):
payment = models.ForeignKey(Payment, blank=True, null=True,
default='basic', on_delete=models.PROTECT)
#etc
There is a way to use 'basic'? I prefer to not use id=1 because I'm not sure to know the id that basic will have (remember that now it don't exist yet).
Btw in other models I have alike situations and they appear to works...
The Whole error:
(myvenv) c:\Python34\Scripts\possedimenti\sitopossedimenti>manage.py migrate
Operations to perform:
Apply all migrations: contenttypes, core, registration, admin, auth, sessions
Running migrations:
Rendering model states... DONE
Applying core.0001_initial...Traceback (most recent call last):
File "C:\Python34\Scripts\possedimenti\sitopossedimenti\manage.py", line 10, i
n <module>
execute_from_command_line(sys.argv)
File "c:\Python34\Scripts\possedimenti\myvenv\lib\site-packages\django\core\ma
nagement\__init__.py", line 353, in execute_from_command_line
utility.execute()
File "c:\Python34\Scripts\possedimenti\myvenv\lib\site-packages\django\core\ma
nagement\__init__.py", line 345, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "c:\Python34\Scripts\possedimenti\myvenv\lib\site-packages\django\core\ma
nagement\base.py", line 348, in run_from_argv
self.execute(*args, **cmd_options)
File "c:\Python34\Scripts\possedimenti\myvenv\lib\site-packages\django\core\ma
nagement\base.py", line 399, in execute
output = self.handle(*args, **options)
File "c:\Python34\Scripts\possedimenti\myvenv\lib\site-packages\django\core\ma
nagement\commands\migrate.py", line 200, in handle
executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
File "c:\Python34\Scripts\possedimenti\myvenv\lib\site-packages\django\db\migr
ations\executor.py", line 92, in migrate
self._migrate_all_forwards(plan, full_plan, fake=fake, fake_initial=fake_ini
tial)
File "c:\Python34\Scripts\possedimenti\myvenv\lib\site-packages\django\db\migr
ations\executor.py", line 121, in _migrate_all_forwards
state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_
initial)
File "c:\Python34\Scripts\possedimenti\myvenv\lib\site-packages\django\db\migr
ations\executor.py", line 198, in apply_migration
state = migration.apply(state, schema_editor)
File "c:\Python34\Scripts\possedimenti\myvenv\lib\site-packages\django\db\migr
ations\migration.py", line 123, in apply
operation.database_forwards(self.app_label, schema_editor, old_state, projec
t_state)
File "c:\Python34\Scripts\possedimenti\myvenv\lib\site-packages\django\db\migr
ations\operations\fields.py", line 62, in database_forwards
field,
File "c:\Python34\Scripts\possedimenti\myvenv\lib\site-packages\django\db\back
ends\sqlite3\schema.py", line 221, in add_field
self._remake_table(model, create_fields=[field])
File "c:\Python34\Scripts\possedimenti\myvenv\lib\site-packages\django\db\back
ends\sqlite3\schema.py", line 103, in _remake_table
self.effective_default(field)
File "c:\Python34\Scripts\possedimenti\myvenv\lib\site-packages\django\db\back
ends\base\schema.py", line 210, in effective_default
default = field.get_db_prep_save(default, self.connection)
File "c:\Python34\Scripts\possedimenti\myvenv\lib\site-packages\django\db\mode
ls\fields\related.py", line 912, in get_db_prep_save
return self.target_field.get_db_prep_save(value, connection=connection)
File "c:\Python34\Scripts\possedimenti\myvenv\lib\site-packages\django\db\mode
ls\fields\__init__.py", line 728, in get_db_prep_save
prepared=False)
File "c:\Python34\Scripts\possedimenti\myvenv\lib\site-packages\django\db\mode
ls\fields\__init__.py", line 968, in get_db_prep_value
value = self.get_prep_value(value)
File "c:\Python34\Scripts\possedimenti\myvenv\lib\site-packages\django\db\mode
ls\fields\__init__.py", line 976, in get_prep_value
return int(value)
ValueError: invalid literal for int() with base 10: 'basic'
Thank You
PS: I'm using python 3.4 Django 1.9 Windows7
In your Payment model you have not explicitly designated a primary key. So django will create one for you. It will be an integer auto increment field.
Class Payment(models.Model):
name = models.CharField(max_length=32)
Now when you define Payment as a foreign key for ProfileUser, django thinks that it should be the primary key field in Payment that ought to be used as the reference.
Class ProfileUser(models.Model):
payment = models.ForeignKey(Payment, blank=True, null=True,
default='basic', on_delete=models.PROTECT)
#etc
In order to enforce this reference, the payment field needs to be an integer. And you cannot insert 'basic' into an integer field.
One solution is to use the
ForeignKey.to_field option to explicitly refer to a different field in the Payment class.
ForeignKey.to_field¶ The field on the related object that the relation
is to. By default, Django uses the primary key of the related object.
Thus your class would become
Class ProfileUser(models.Model):
payment = models.ForeignKey(Payment, blank=True, null=True,
default='basic', on_delete=models.PROTECT,
to_field = 'name' )
#etc
Now the payment field here becomes a CharField. Note that it would be more efficient to use an integer field as a foreign key in most cases.
Another possibility is for you to define the name field in Payment as it's primary key.
Create a function to return the id (put it before the ProfileUser defenition).
def get_basic_id():
payment = Payment.options.get(name='basic')
return payment.id
Class ProfileUser(models.Model):
payment = models.ForeignKey(Payment, blank=True, null=True,
default=get_basic_id(), on_delete=models.PROTECT)
Though if you don't know what id 'basic' will have, are you certain that it will be in the database? One way around this would be to populate the payment table using fixtures.
https://docs.djangoproject.com/en/1.9/howto/initial-data/

Django Models Relations to User Model

I'm new to the forum and I've got a problem.
in Django im trying to create a UserProfile class by referencing it via a OnetoOneField to an User Object. When i try to migrate, I get:
"You are trying to add a non-nullable field 'id' to author without a default; we can't do that (the database needs something to populate existing rows).
Please select a fix:
1) Provide a one-off default now (will be set on all existing rows)
2) Quit, and let me add a default in models.py"
I understand, that there are some fields in the User model (for example "id") that are not allowed to be null.
When I try to set a default it will makemigrations, but migrate fails with a syntax error.
My Question is how do I allow Null, or set a default that works, without writing a custom User class, and is that even possible ?
I'd really appreciate any help with this.
Here is my Code:
from django.db import models
from django.contrib.auth.models import User as User_django
class User(User_django):
pass
class Author(models.Model):
user = models.OneToOneField(User, null=True)
class BaseElement(models.Model):
author=models.ForeignKey(Author, null=True)
upvotes = models.PositiveIntegerField(default=0)
downvotes = models.PositiveIntegerField(default=0)
created = models.DateTimeField(auto_now_add=True, null=True)
def vote_up(self):
self.upvotes.value = self.upvotes.value + 1
def vote_down(self):
self.downvotes.value = self.downvotes.value + 1
class Meta:
abstract = True
class Post(BaseElement):
content = models.TextField(max_length=240)
tags = models.CharField(max_length=40, blank=True)
said_by = models.CharField(max_length=40)
said_at = models.CharField(max_length=40, blank=True)
def get_comments(self):
return self.comment_set.all()
def get_comments_anzahl(self):
return self.comment_set.all().count()
class Comment(BaseElement):
content = models.TextField(max_length=140)
commented_post = models.ForeignKey(Post, related_query_name="comment", null=True)
EDIT:
This happens if I try to set a default and try to migrate:
You are trying to add a non-nullable field 'id' to author without a
default; we can't do that (the database needs something to populate
existing rows).
Please select a fix:
1) Provide a one-off default now (will be set on all existing rows)
2) Quit, and let me add a default in models.py
Select an option: 1
Please enter the default value now, as valid Python
The datetime and django.utils.timezone modules are available, so you can
do e.g. timezone.now()
>>> 1
Migrations for 'post':
0003_auto_20160505_1237.py:
- Change Meta options on author
- Change managers on author
- Remove field user_ptr from author
- Add field id to author
- Add field user to author
Operations to perform:
Apply all migrations: contenttypes, admin, sessions, post, auth
Running migrations:
Rendering model states... DONE
Applying post.0003_auto_20160505_1237...Traceback (most recent call last):
File "/home/jaidmin/.conda/envs/cyf/lib/python3.5/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/home/jaidmin/.conda/envs/cyf/lib/python3.5/site-packages/django/db/backends/sqlite3/base.py", line 323, in execute
return Database.Cursor.execute(self, query, params)
sqlite3.OperationalError: near ")": syntax error
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/home/jaidmin/.conda/envs/cyf/lib/python3.5/site-packages/django /core/management/__init__.py", line 353, in execute_from_command_line
utility.execute()
File "/home/jaidmin/.conda/envs/cyf/lib/python3.5/site-packages/django/core/management/__init__.py", line 345, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/jaidmin/.conda/envs/cyf/lib/python3.5/site-packages/django/core/management/base.py", line 348, in run_from_argv
self.execute(*args, **cmd_options)
File "/home/jaidmin/.conda/envs/cyf/lib/python3.5/site-packages/django/core/management/base.py", line 399, in execute
output = self.handle(*args, **options)
File "/home/jaidmin/.conda/envs/cyf/lib/python3.5/site-packages/django/core/management/commands/migrate.py", line 200, in handle
executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
File "/home/jaidmin/.conda/envs/cyf/lib/python3.5/site-packages/django/db/migrations/executor.py", line 92, in migrate
self._migrate_all_forwards(plan, full_plan, fake=fake, fake_initial=fake_initial)
File "/home/jaidmin/.conda/envs/cyf/lib/python3.5/site-packages/django/db/migrations/executor.py", line 121, in _migrate_all_forwards
state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
File "/home/jaidmin/.conda/envs/cyf/lib/python3.5/site-packages/django/db/migrations/executor.py", line 198, in apply_migration
state = migration.apply(state, schema_editor)
File "/home/jaidmin/.conda/envs/cyf/lib/python3.5/site-packages/django/db/migrations/migration.py", line 123, in apply
operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
File "/home/jaidmin/.conda/envs/cyf/lib/python3.5/site-packages/django/db/migrations/operations/fields.py", line 121, in database_forwards
schema_editor.remove_field(from_model, from_model._meta.get_field(self.name))
File "/home/jaidmin/.conda/envs/cyf/lib/python3.5/site-packages/django/db/backends/sqlite3/schema.py", line 247, in remove_field
self._remake_table(model, delete_fields=[field])
File "/home/jaidmin/.conda/envs/cyf/lib/python3.5/site-packages/django/db/backends/sqlite3/schema.py", line 197, in _remake_table
self.quote_name(model._meta.db_table),
File "/home/jaidmin/.conda/envs/cyf/lib/python3.5/site-packages/django/db/backends/base/schema.py", line 110, in execute
cursor.execute(sql, params)
File "/home/jaidmin/.conda/envs/cyf/lib/python3.5/site-packages/django/db/backends/utils.py", line 79, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File "/home/jaidmin/.conda/envs/cyf/lib/python3.5/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/home/jaidmin/.conda/envs/cyf/lib/python3.5/site-packages/django/db/utils.py", line 95, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/home/jaidmin/.conda/envs/cyf/lib/python3.5/site-packages/django/utils/six.py", line 685, in reraise
raise value.with_traceback(tb)
File "/home/jaidmin/.conda/envs/cyf/lib/python3.5/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File "/home/jaidmin/.conda/envs/cyf/lib/python3.5/site-packages/django/db/backends/sqlite3/base.py", line 323, in execute
return Database.Cursor.execute(self, query, params)
django.db.utils.OperationalError: near ")": syntax error
EDIT:
It seems like I solved it by creating a new project and step by step copying the models and migrating. Still wondering what was going on there, because I flushed the database and deleted the migration files multiple times and it was still throwing errors.
Thanks for the help !
Take a look at this:
https://docs.djangoproject.com/es/1.9/topics/db/examples/one_to_one/
Your code is a ONE-to-ONE with user. When you set it, it implies CASCADE delete, and therefore can never be null.
if you change to One-to-Many, and just ensure that you only insert one object for the foreign key, it'll work. Otherwise, change your cascade rule.
I will say, however, that's not a good idea, because if you make it null, what separates it from other records. Consider a second PK field.
Best of luck.
Just import django's built-in User model like this:
from django.contrib.auth.models import User
and delete this line:
class User(User_django):
pass
from your code as it is completely unnecessary.
UPDATE: Also it looks like your problem has nothing to do with the User model. I think somehow you added an id field to your Author model and deleted it (maybe after an unsuccessful migration). If this is the case, then you could edit your 0003_auto_20160505_1237.py. To do that open it and remove the lines which look something like this:
migrations.AddField(
model_name='author',
name='id',
field=models.someFieldType(options)),
),
and try again.

Resolving ValidationError: [u"'' value has an invalid date format. It must be in YYYY-MM-DD format."] in Django 1.9.2?

Earlier I created two fields & migrated everything. after that I tried to add three fields title,about,birthdate into the model.
I created a model like this :
from __future__ import unicode_literals
from django.utils import timezone
from django.db import models
# Create your models here.
class APP1Model(models.Model):
name = models.CharField(max_length=120)
percentage = models.CharField(max_length=120)
title = models.CharField(max_length=100,default='Title')
birth_date = models.DateTimeField(blank=True, null=True)
about = models.TextField(max_length=100,null=True,default='About Yourself')
def __unicode__(self):
return self.name
But when I try to migrate in python shell, it is showing a validation error like this:
Operations to perform:
Apply all migrations: admin, contenttypes, auth, app1, sessions
Running migrations:
Applying app1.0005_auto_20160217_1346...Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 353, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 345, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 348, in run_from_argv
self.execute(*args, **cmd_options)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 399, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/migrate.py", line 200, in handle
executor.migrate(targets, plan, fake=fake, fake_initial=fake_initial)
File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 92, in migrate
self._migrate_all_forwards(plan, full_plan, fake=fake, fake_initial=fake_initial)
File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 121, in _migrate_all_forwards
state = self.apply_migration(state, migration, fake=fake, fake_initial=fake_initial)
File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 198, in apply_migration
state = migration.apply(state, schema_editor)
File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/migration.py", line 123, in apply
operation.database_forwards(self.app_label, schema_editor, old_state, project_state)
File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/operations/fields.py", line 62, in database_forwards
field,
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/sqlite3/schema.py", line 221, in add_field
self._remake_table(model, create_fields=[field])
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/sqlite3/schema.py", line 103, in _remake_table
self.effective_default(field)
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/base/schema.py", line 210, in effective_default
default = field.get_db_prep_save(default, self.connection)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/fields/__init__.py", line 728, in get_db_prep_save
prepared=False)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/fields/__init__.py", line 1301, in get_db_prep_value
value = self.get_prep_value(value)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/fields/__init__.py", line 1296, in get_prep_value
return self.to_python(value)
File "/usr/local/lib/python2.7/dist-packages/django/db/models/fields/__init__.py", line 1273, in to_python
params={'value': value},
django.core.exceptions.ValidationError: [u"'' value has an invalid date format. It must be in YYYY-MM-DD format."]
How to rectify this? I tried all solution i read here but it doesn't work?
I am using Django: 1.9.2
My migration File
from __future__ import unicode_literals
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('app1', '0004_auto_20160217_0427'),
]
operations = [
migrations.AddField(
model_name='app1model',
name='about',
field=models.TextField(default='About Yourself', max_length=100, null=True),
),
migrations.AddField(
model_name='app1model',
name='birth_date',
field=models.DateField(blank=True, default='', null=True),
),
migrations.AddField(
model_name='app1model',
name='title',
field=models.CharField(default='', max_length=100),
),
]
I went through the same problem some months back.I Just deleted the birthdate field changes in all the migration Files inside migration folder. Then I replaced the birthdate with this code:-
birthdate = models.DateTimeField(blank=True, null=True)
Then after applying migration ,it works fine...
It seems you have passed DateTimeField
birth_date = models.DateTimeField(blank=True, null=True)
in your APP1Model(model) and migration shows DateFields
models.DateField(blank=True, default='', null=True)
first correct your Model and don't pass default='' in DateField,
Use DateField instead of DatiTimeField for birthdate
then remove your migration file app1.0005_auto_20160217_1346
and run makemigrations and migrate your app it'll work fine.
Changing the DateField to DateTimeField is not a bad idea. Although, there is a DateField and a DateTimeField, which are different for some reasons.
This error happens when a default value is applied to DateField in models.py in the wrong format.
According to your terminal output, the error occurs in migration file 0005_auto_20160217_1346.py. You must find that migration file in 'app1/0005_auto_20160217_1346.py`, edit it and look for:
...
migrations.AddField(
model_name='app1model',
name='birth_date',
field=models.DateField(blank=True, default='<jamming-date>', null=True),
),
...
Now, you can see the DateField with the default attribute containing a <jamming-date> with format %d-%m-%Y, for example. Django needs a %Y-%m-%d format.
In order to solve this problem, delete the default attribute, save the file and make migrations again.
Just remove default=' ' in migrations file and run migrate .
Before:
migrations.AddField(
model_name='formd',
name='cash_payment_date',
field=models.DateField(blank=True, default='', help_text='Date of Cash Payment', null=True),
),
After:
migrations.AddField(
model_name='formd',
name='cash_payment_date',
field=models.DateField(blank=True, help_text='Date of Cash Payment', null=True),
),
Removed default=''
Ran makemigrations
I had this same error after trying to change the default property on a DateTime from blank '' (set in my initial migration file) to django.utils.timezone.now (in a later migration). You need to go back and edit your 0001_Initial.py file and change the default value to whatever you want it to be now. It seems like when applying migrations Django is referencing back to that initial file for validation and throwing the error, which can be a bit misleading, took me a few hours trying to sort out why... hopefully I save someone some time :)

Django Db Utils IntegrityError: (1062, "Duplicate entry '' for key 'slug'") [duplicate]

I'm trying to follow the tangowithdjango book and must add a slug to update the category table. However I'm getting an error after trying to migrate the databases.
http://www.tangowithdjango.com/book17/chapters/models_templates.html#creating-a-details-page
I didn't provide a default value for the slug, so Django asked me to provide one and as the book instructed I type in ''.
It's worth noticing that instead of using sqlite as in the original book I'm using mysql.
models.py
from django.db import models
from django.template.defaultfilters import slugify
# Create your models here.
class Category(models.Model):
name = models.CharField(max_length=128, unique=True)
views = models.IntegerField(default=0)
likes = models.IntegerField(default=0)
slug = models.SlugField(unique=True)
def save(self, *args, **kwargs):
self.slug = slugify(self.name)
super(Category, self).save(*args, **kwargs)
class Meta:
verbose_name_plural = "Categories"
def __unicode__(self):
return self.name
class Page(models.Model):
category = models.ForeignKey(Category)
title = models.CharField(max_length=128)
url = models.URLField()
views = models.IntegerField(default=0)
def __unicode__(self):
return self.title
The command prompt
sudo python manage.py migrate
Operations to perform:
Apply all migrations: admin, rango, contenttypes, auth, sessions
Running migrations:
Applying rango.0003_category_slug...Traceback (most recent call last):
File "manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 385, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python2.7/dist-packages/django/core/management/__init__.py", line 377, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 288, in run_from_argv
self.execute(*args, **options.__dict__)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/base.py", line 338, in execute
output = self.handle(*args, **options)
File "/usr/local/lib/python2.7/dist-packages/django/core/management/commands/migrate.py", line 160, in handle
executor.migrate(targets, plan, fake=options.get("fake", False))
File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 63, in migrate
self.apply_migration(migration, fake=fake)
File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/executor.py", line 97, in apply_migration
migration.apply(project_state, schema_editor)
File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/migration.py", line 107, in apply
operation.database_forwards(self.app_label, schema_editor, project_state, new_state)
File "/usr/local/lib/python2.7/dist-packages/django/db/migrations/operations/fields.py", line 37, in database_forwards
field,
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/mysql/schema.py", line 42, in add_field
super(DatabaseSchemaEditor, self).add_field(model, field)
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/schema.py", line 411, in add_field
self.execute(sql, params)
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/schema.py", line 98, in execute
cursor.execute(sql, params)
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 81, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 65, in execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python2.7/dist-packages/django/db/utils.py", line 94, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/utils.py", line 65, in execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python2.7/dist-packages/django/db/backends/mysql/base.py", line 128, in execute
return self.cursor.execute(query, args)
File "/usr/local/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 205, in execute
self.errorhandler(self, exc, value)
File "/usr/local/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
django.db.utils.IntegrityError: (1062, "Duplicate entry '' for key 'slug'")
Let's analyse it step by step:
You're adding slug field with unique = True, that means: each record must have different value, there can't be two records with same value in slug
You're creating migration: django asks you for default value for fields that exists already in database, so you provided '' (empty string) as that value.
Now django is trying to migrate your database. In database we have at least 2 records
First record is migrated, slug column is populated with empty string. That's good because no other record is having empty string in slug field
Second record is migrated, slug column is populated with empty string. That fails, because first record already have empty string in slug field. Exception is raised and migration is aborted.
That's why your migration fails. All you should do is to edit migration, copy migrations.AlterField operation twice, in first operation remove unique=True. Between that operations you should put migrations.RunPython operation and provide 2 parameters into that: generate_slugs and migrations.RunPython.noop.
Now you must create inside your migration function BEFORE migration class, name that function generate_slugs. Function should take 2 arguments: apps and schema_editor. In your function put at first line:
Category = apps.get_model('your_app_name', 'Category')
and now use Category.objects.all() to loop all your records and provide unique slug for each of them.
If you have more than one category in your table, then you cannot have unique=True and default='', because then you will have more than one category with slug=''. If your tutorial says to do this, then it's bad advice, although it might work in SQLite.
The correct approach to add a unique field to a model is:
Delete your current migration that isn't working.
Add the slug field, with unique=False. Create a new migration and run it.
Set a unique slug for every category. It sounds like the rango populate script might do this. Alternatively, you could write a migration to set the slugs, or even set them manually in the Django admin.
Change the slug field to unique=True. Create a new migration and run it.
If that's too difficult, then you could delete all your categories from your database except one. Then your current migration will run without having problems with the unique constraint. You can add the categories again afterwards.
You must have rows in your table already with empty slugs, which is a violation of the mysql unique constraint you created. You can update them manually by running manage.py dbshell to get to the mysql client, then updating the offending rows, e.g.
update table rango_category set slug = name where slug = '';
(assuming the rows with blank slugs have names). Or you can delete the rows with
delete from rango_category where slug = '';
After that, you should be able to run your migrations.

Categories