We Keep Coding

Django Bug with long name in db_column?

When i use a long name in db_columns at a field in Models.py, django does not work correct. It truncates the name, and add random letters/numbers at the end.
Like this: db_column='my_loooooooooooooooooong_column_name'
And when i try queryset, django returns:
'table name'.'my_looooooooooooooo6E4': invalid identifier.
My scenario in detail:
I have a legacy database in Oracle.
The table name in database: PALAVRA_CHAVE_ENTREGA_VALOR
With 3 Fields: PCEV_CD_PALAVRA_CHAVE_ENTREGA_VALOR (Primary Key), PACH_CD_PALAVRA_CHAVE, ENVA_CD_ENTREGA_VALOR
In my models.py:
class PalavraChaveEntregaValor(models.Model):
pcev_cd_palavra_chave_entrega_valor = models.BigIntegerField(primary_key=True, db_column='pcev_cd_palavra_chave_entrega_valor')
pach_cd_palavra_chave = models.BigIntegerField()
enva_cd_entrega_valor = models.BigIntegerField()
class Meta:
managed = False
db_table = 'palavra_chave_entrega_valor'
When i run in shell (python manage.py shell) this command:
PalavraChaveEntregaValor.objects.all()
I got output: DatabaseError: ORA-00904: "PALAVRA_CHAVE_ENTREGA_VALOR"."PCEV_CD_PALAVRA_CHAVE_ENTRA6E4": invalid identifier
I made a test, changed the long name PCEV_CD_PALAVRA_CHAVE_ENTREGA_VALOR to PCEV_CD, and everthing works fine..
Is there a limitation of characters in db_columns at django? Is there a workaround for this? If not, i will have to create a lot of Views in Oracle Database with shorter names of columns only for django work.. Change the current table column names is not an option.

Using the SQL initialization hook with ManytoManyField

I'm fairly new to Django and I'm trying to add some 'host' data to 'record' using django's hook for using SQL to initialise (a SQL file in lowercase in the app folder & sql subfolder)
Here's the models:
class Record(models.Model):
species = models.TextField(max_length = 80)
data=models.TextField(max_length = 700)
hosts = models.ManyToManyField('Host')
class Host(models.Model):
hostname = models.TextField()
I've used a ManyToManyField as each record should be able to have multiple hosts, and hosts should be 'reusable': ie be able to appear in many records.
When I'm trying to insert via SQL I have
INSERT INTO myapp_record VALUES ('Species name', 'data1', XYZ);
I'm not sure what to put for XYZ (the ManytoMany) if I wanted hosts 1, 2 and 3 for example
Separating them by commas doesn't work obviously, and I tried a tuple and neither did that.
Should I be trying to insert into the intermediary table Django makes? Does that have a similar hook to the one I'm using? If not, how can I execute SQL inserts on this table?

The use of initial SQL data files is deprecated. Instead, you should be using a data migration, which might look something like this:
from django.db import models, migrations
def create_records(apps, schema_editor):
# We can't import the Person model directly as it may be a newer
# version than this migration expects. We use the historical version.
Record = apps.get_model("yourappname", "Record")
Host = apps.get_model("yourappname", "Host")
host1 = Host.objects.get(hostname='host1')
record = Record.objects.create(name='Species name', data='Data')
record.hosts.add(host1)
...etc...
class Migration(migrations.Migration):
dependencies = [
('yourappname', '0001_initial'),
]
operations = [
migrations.RunPython(create_records),
]

Django manage.py sqlclear omit some tables

I wrote python script for dropping tables in all Django apps. (using settings.INSTALLED_APP)
https://gist.github.com/1520683
My django project creates 41 tables after running manage.py syncdb, but my script says only 40 tables will be dropped. So, I examined the result of sqlall and result of sqlclear. And I revealed sqlclear omits one table that stores ManyToManyField relationship.
I knew that drop database is much simpler than the above script. But I confused why django admin or manage script omit some tables while running sql commands.
Below model creates common_userbook_purchasedBooks table while running syncdb, but not in sqlclear command.
class UserBook(models.Model):
user = models.OneToOneField(User)
purchasedBooks = models.ManyToManyField(Book)
Added) So, I'm using an alternative approach for this.
https://gist.github.com/1520810

lqez, I gues this issue related to you local environment, because for Django 1.3.1, Python 2.7.2
for models
from django.contrib.auth.models import User
from django.db import models
class Book(models.Model):
name = models.CharField(max_length=10)
class UserBook(models.Model):
user = models.OneToOneField(User)
purchasedBooks = models.ManyToManyField(Book)
when I run (.env)testme$ ./manage.py sqlclear testapp output looks like
sqlite3
BEGIN;
DROP TABLE "testapp_userbook";
DROP TABLE "testapp_userbook_purchasedBooks";
DROP TABLE "testapp_book";
COMMIT;
postgresql_psycopg2
BEGIN;
ALTER TABLE "testapp_userbook_purchasedBooks" DROP CONSTRAINT "userbook_id_refs_id_8bda4b0";
DROP TABLE "testapp_userbook";
DROP TABLE "testapp_userbook_purchasedBooks";
DROP TABLE "testapp_book";
COMMIT;
mysql
BEGIN;
ALTER TABLE `testapp_userbook_purchasedBooks` DROP FOREIGN KEY `userbook_id_refs_id_8bda4b0`;
DROP TABLE `testapp_userbook`;
DROP TABLE `testapp_userbook_purchasedBooks`;
DROP TABLE `testapp_book`;
COMMIT;
Also your script can be a little bit improved using introspection:
from django.db import connection
cursor = connection.cursor()
connection.introspection.get_table_list(cursor)
[u'auth_group', u'auth_group_permissions', u'auth_message', u'auth_permission', u'auth_user', u'auth_user_groups', u'auth_user_user_permissions', u'django_content_type', u'django_session', u'django_site', u'testapp_book', u'testapp_userbook', u'testapp_userbook_purchasedBooks']

Renaming a django model class-name and corresponding foreign keys with south, without loosing the data

Following is my model:
class myUser_Group(models.Model):
name = models.CharField(max_length=100)
class Channel(models.Model):
name = models.CharField(max_length=100)
description = models.CharField(max_length=1000)
belongs_to_group = models.ManyToManyField(myUser_Group)
class Video(models.Model):
video_url = models.URLField(max_length=300)
belongs_to_channel = models.ManyToManyField(Channel)
description = models.CharField(max_length=1000)
tags = TagField()
class UserProfile(models.Model):
user = models.OneToOneField(User)
class User_History(models.Model):
date_time = models.DateTimeField()
user = models.ForeignKey(UserProfile, null=True, blank=True)
videos_watched = models.ManyToManyField(Video)
I just wanted to remove the underscores from all the class names so that User_History looks UserHistory, also the foreign keys should be updated. I tried using south but could not find it in the documentaion.
One way is export the data, uninstall south, delete migration, rename the table and then import data again. Is there any other way to do it?

You can do this using just South.
For this example I have an app called usergroups with the following model:
class myUser_Group(models.Model):
name = models.CharField(max_length=100)
which I assume is already under migration control with South.
Make the model name change:
class MyUserGroup(models.Model):
name = models.CharField(max_length=100)
and create an empty migration from south
$ python manage.py schemamigration usergroups model_name_change --empty
This will create a skeleton migration file for you to specify what happens. If we edit it so it looks like this (this file will be in the app_name/migrations/ folder -- usergroups/migrations/ in this case):
import datetime
from south.db import db
from south.v2 import SchemaMigration
from django.db import models
class Migration(SchemaMigration):
def forwards(self, orm):
# Change the table name from the old model name to the new model name
# ADD THIS LINE (using the correct table names)
db.rename_table('usergroups_myuser_group', 'usergroups_myusergroup')
def backwards(self, orm):
# Provide a way to do the migration backwards by renaming the other way
# ADD THIS LINE (using the correct table names)
db.rename_table('usergroups_myusergroup', 'usergroups_myuser_group')
models = {
'usergroups.myusergroup': {
'Meta': {'object_name': 'MyUserGroup'},
'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
'name': ('django.db.models.fields.CharField', [], {'max_length': '100'})
}
}
complete_apps = ['usergroups']
In the forwards method we are renaming the database table name to match what the django ORM will look for with the new model name. We reverse the change in backwards to ensure the migration can be stepped back if required.
Run the migration with no need to import/export the exisiting data:
$ python manage.py migrate
The only step remaining is to update the foreign key and many-to-many columns in the models that refer to myUser_Group and change to refer to MyUserGroup.

mmcnickle's solution may work and seems reasonable but I prefer a two step process. In the first step you change the table name.
In your model make sure you have your new table name in:
class Meta:
db_table = new_table_name'
Then like mmcnickle suggested, create a custom migration:
python manage.py schemamigration xyz migration_name --empty
You can read more about that here:
https://docs.djangoproject.com/en/dev/ref/models/options/
Now with your custom migration also add the line to rename your table forward and backwards:
db.rename_table("old_table_name","new_table_name")
This can be enough to migrate and change the table name but if you have been using the Class Meta custom table name before then you'll have to do a bit more. So I would say as a rule, just to be safe do a search in your migration file for "old_table_name" and change any entries you find to the new table name. For example, if you were previously using the Class Meta custom table name, you will likely see:
'Meta': {'object_name': 'ModelNameYouWillChangeNext', 'db_table': "u'old_table_name'"},
So you'll need to change that old table name to the new one.
Now you can migrate with:
python manage.py migrate xyz
At this point your app should run since all you have done is change the table name and tell Django to look for the new table name.
The second step is to change your model name. The difficulty of this really depends on your app but basically you just need to change all the code that references the old model name to code that references the new model name. You also probably need to change some file names and directory names if you have used your old model name in them for organization purposes.
After you do this your app should run fine. At this point your task is pretty much accomplished and your app should run fine with a new model name and new table name. The only problem you will run into using South is the next time you create a migration using it's auto detection feature it will try to drop the old table and create a new one from scratch because it has detected your new model name. To fix this you need to create another custom migration:
python manage.py schemamigration xyz tell_south_we_changed_the_model_name_for_old_model_name --empty
The nice thing is here you do nothing since you have already changed your model name so South picks this up. Just migrate with "pass" in the migrate forwards and backwards:
python manage.py migrate xyz
Nothing is done and South now realizes it is up to date. Try:
python manage.py schemamigration xyz --auto
and you should see it detects nothing has changed

Django - How to rename a model field using South?

I would like to change a name of specific fields in a model:
class Foo(models.Model):
name = models.CharField()
rel = models.ForeignKey(Bar)
should change to:
class Foo(models.Model):
full_name = models.CharField()
odd_relation = models.ForeignKey(Bar)
What's the easiest way to do this using South?

You can use the db.rename_column function.
class Migration:
def forwards(self, orm):
# Rename 'name' field to 'full_name'
db.rename_column('app_foo', 'name', 'full_name')
def backwards(self, orm):
# Rename 'full_name' field to 'name'
db.rename_column('app_foo', 'full_name', 'name')
The first argument of db.rename_column is the table name, so it's important to remember how Django creates table names:
Django automatically derives the name of the database table from the name of your model class and the app that contains it. A model's database table name is constructed by joining the model's "app label" -- the name you used in manage.py startapp -- to the model's class name, with an underscore between them.
In the case where you have a multi-worded, camel-cased model name, such as ProjectItem, the table name will be app_projectitem (i.e., an underscore will not be inserted between project and item even though they are camel-cased).

Here's what I do:
Make the column name change in your model (in this example it would be myapp/models.py)
Run ./manage.py schemamigration myapp renaming_column_x --auto
Note renaming_column_x can be anything you like, it's just a way of giving a descriptive name to the migration file.
This will generate you a file called myapp/migrations/000x_renaming_column_x.py which will delete your old column and add a new column.
Modify the code in this file to change the migration behaviour to a simple rename:
class Migration(SchemaMigration):
def forwards(self, orm):
# Renaming column 'mymodel.old_column_name' to 'mymodel.new_column_name'
db.rename_column(u'myapp_mymodel', 'old_column_name', 'new_column_name')
def backwards(self, orm):
# Renaming column 'mymodel.new_column_name' to 'mymodel.old_column_name'
db.rename_column(u'myapp_mymodel', 'new_column_name', 'old_column_name')

I didn't know about db.rename column, sounds handy, however in the past I have added the new column as one schemamigration, then created a datamigration to move values into the new field, then a second schemamigration to remove the old column

Django 1.7 introduced Migrations so now you don't even need to install extra package to manage your migrations.
To rename your model you need to create empty migration first:
$ manage.py makemigrations <app_name> --empty
Then you need to edit your migration's code like this:
from django.db import models, migrations
class Migration(migrations.Migration):
dependencies = [
('yourapp', 'XXXX_your_previous_migration'),
]
operations = [
migrations.RenameField(
model_name='Foo',
old_name='name',
new_name='full_name'
),
migrations.RenameField(
model_name='Foo',
old_name='rel',
new_name='odd_relation'
),
]
And after that you need to run:
$ manage.py migrate <app_name>

Just change the model and run makemigrations in 1.9
Django automatically detects that you've deleted and created a single field, and asks:
Did you rename model.old to model.new (a IntegerField)? [y/N]
Say yes, and the right migration gets created. Magic.

Add south to your installed apps in project setting file.
Comment out the added/modified field/table.
$ manage.py Schemamigration <app_name> --initial
$ manage.py migrate <app_name> --Fake
Un-comment the field and write the modified one
$ manage.py Schemamigration --auto
$ manage.py migrate <app_name>
If you are using 'pycharm', then you can use 'ctrl+shift+r' instead of 'manage.py' , and 'shift ' for parameters.