We Keep Coding

Retrieving data from Mongodb and show it on front-end using django

I have a mongodb database named as world, which has two collection from before city, languages. I want to show the data of my collection on the web, how can i do it.
currently i know to create collection in models.py and migrate it. like;
first we have to edit databases[] in setting.py
DATABASES = {
'default': {
'ENGINE': 'djongo',
'NAME': 'world',
}
}
in models.py i creted a class and migrated it using python manage.py migrate
class Destination(models.Model):
name= models.CharField(max_length=100)
img=models.ImageField(upload_to='pics')
desc=models.TextField()
and i'm able to retrieve data from Destination by using below code in views.py
from django.shortcuts import render
from .models import Destination
def index(request):
dests=Destination.objects.all()
return render(request,'index.html',{'dests':dests})
My question is my collection/class is already available in the database ( city, language) and i'm not creating it contrary to Destination which was defined by me. then how to show data of city collection of world database on the front-end.
kindly looking for help.

If I got you properly, then you have a MongoDB database called world.
There you stored city and languages before you started to set up Django.
Then you added a Destination model, thus created a new collection.
For now, you're looking for a way how to get city and languages collections data similar way as you do with Destination.
So there are multiple ways how you could handle it:
Create Django models for city and languages collections (define fields that you have in existing collections):
class City(models.Model):
field1 = ...
field2 = ...
class Meta:
db_table = 'city' # important, should be your existing collection name
class Language(models.Model):
field3 = ...
field4 = ...
class Meta:
db_table = 'languages' # important, should be your existing collection name
Now you're ready to use City and Language the same way as you do with the Destination model.
Use PyMongo (this is already installed as you're using Djongo). So your snipped will look something like:
from django.shortcuts import render
from .models import Destination
import pymongo
# default localhost connection URL
MONGO_URL = 'mongodb://localhost:27017'
connection = pymongo.MongoClient(MONGO_URL)
mongo_db = connection.world
def index(request):
collection_city = db['city']
collection_languages = db['languages']
cities = list(collection_city.find())
languages = list(collection_languages.find())
dests=Destination.objects.all()
return render(
request,
'index.html',
{
'dests': dests,
'cities': cities,
'languages': languages
}
)
I'd use option 1, as it allows you to keep the project coherent.

Django models default function run for every current object

models.py
class Subscription(models.Model):
#... many fields ...
# I added this field when I already had many objects
uniqueSubscriptionId = models.CharField(default=generateUniqueSubscription, max_length=30)
generateUniqueSubscription
from django.utils.crypto import get_random_string
def generateUniqueSubscription():
return get_random_string(20)
The Problem is that, when I run migrations, all of my old objects get the same uniqueSubscriptionId. I want each and every single old object to get a unique uniqueSubscriptionId.
How can I do that?

Here's what I did:
models.py
def updateOldSubscriptionObjs(apps, schema_editor):
old_subscription_model = apps.get_model("app_label", "Profile")
for obj in old_subscription_model.objects.all():
obj.uniqueSubscriptionId = generateUniqueSubscription()
obj.save()
class Subscription(models.Model):
#... many fields ...
# I added this field when I already had many objects
uniqueSubscriptionId = models.CharField(default=generateUniqueSubscription, max_length=30)
Then I ran makemigrations:
python manage.py makemigrations
Then edited the latest migration file:
class Migration(migrations.Migration):
dependencies = [
# forget this
]
operations = [
# .... many things ...
migrations.RunPython(updateOldProfileObjs)
]
Then ran migrate:
python manage.py migrate
And voila, all old objects got updated, and also, any new object will also get updated as I specified default.
If you are lazy like me, and don't want to do these things, then open django python shell:
python manage.py shell
and then execute this function in shell:
def updateOldSubscriptionObjs():
for obj in Subscription.objects.all():
obj.uniqueSubscriptionId = generateUniqueSubscription()
obj.save()
I wish if there was some built-in django feature for this.

Editing models.py file and migrating changes to database in the same python code

I am looking to edit a models.py file for an app, and then migrate the changes to the database (postgreSQL) in the same python code.
In the view definition that is supposed to perform this, I am doing the following:
Open models.py
Append the new table's model to the existing models
Close models.py
Run makemigrations
Run sqlmigrate
Run migrate
It works fine up to Step no.3. The new table's model is being added to models.py, but 'makemigrations' doesn't take in the latest model that was added to models.py.
Effectively, it is not taking the latest copy of models.py for the migrations, it is using the one that existed BEFORE the view was called.
Is there a way to use the latest (appended with the required model) models.py file for the migration commands?
#Here we are attempting to create a user's portfolio table as soon as his registration is complete
#Step1: Adding schema to model.py
modeladdscript = '\n\nclass '+ username + '(models.Model):'
modeladdscript = modeladdscript + '''
stock_id = models.CharField(max_length = 20)
stock_name = models.CharField(max_length = 100)
qty = models.IntegerField(default = 0)
investment = models.FloatField()
hold = models.CharField(max_length = 50)'''
#Step 2: Getting the location of models.py in fileloc
fileloc ='portfolio\models.py' #Need to figure out how to get location of portfolio/models.py
#Step 3: Opening models.py and appending the model add script
with open(fileloc,"a") as f:
f.write(modeladdscript)
#Step 4: Closing models.py
f.close()
#Works absolutely fine till here
# Make migrations
call_command('makemigrations') #This command is not taking the model that has been added above at the time of user creation.
# Remove all the terminal colors, less parsing later.
os.environ['DJANGO_COLORS'] = 'nocolor'
# Capturing the list of applied and unapplied migrations
out = StringIO()
call_command('showmigrations', stdout=out)
# Clean up the line, and remove all applied migrations
work = [line.strip() for line in out.getvalue().split('\n') if not line.startswith(' [X]')]
# Keep looping until we run out of lines.
while work:
app_name = work.pop(0)
while work and work[0].startswith('['):
migration_name = work.pop(0)[4:]
app_path = apps.get_app_config(app_name).module.__file__
# Print the whole path to the migration file that corresponds to this migration
print('--', os.path.abspath(os.path.join(app_path, '../migrations/', '{}.py'.format(migration_name))))
call_command('sqlmigrate', app_name, migration_name[:8])
# Applying all pending migrations
call_command('migrate')

remove orphaned file from a model

I have the following model:
class Class(models.Model):
title = models.CharField(max_length = 60)
video = models.FileField(
upload_to = class_files_custom_upload,
validators = [
FileExtensionValidator(['mp4', 'webm', 'mpg', 'mpeg', 'ogv']),
]
)
section = models.ForeignKey(Section, on_delete = models.CASCADE)
created = models.DateTimeField(auto_now_add = True)
class Meta:
verbose_name = 'clase'
verbose_name_plural = 'clases'
ordering = ['created']
def __str__(self):
return self.title
I create an instance of this model, but if I update the video field with another file of any instance, the previous saved file is orphaned and the file takes up space and I want to avoid it, deleting the file.
To do this I customize the file load, putting a callable in the upload_to:
def class_files_custom_upload(instance, filename):
try:
old_instance = Class.objects.get(id = instance.id)
old_instance.video.delete()
except Class.DoesNotExist:
pass
return os.path.join('courses/videos', generate_secure_filename(filename))
In this way I achieve my goal. But I have several models that save multimedia files, and for each one I have to customize the file load, practically doing a function almost equal to class_files_custom_upload, and the code repeats and this is not optimal at all.
I tried to create a reusable function that meets the goal of the class_files_custom_upload function, in various fields like ImageField and FileField, but I can't do it since the function receives only 2 parameters, instance and filename, which is too little data to achieve it.
The only way I managed to create that "function" that meets the goal and is reusable, was to create a validator:
def delete_orphaned_media_file(value):
old_instance = value.instance.__class__.objects.get(pk = value.instance.pk)
media_file_field = getattr(old_instance, value.field.name)
if not media_file_field.name == value.name: media_file_field.delete()
And it works, but after all it is a "validator", a "validator" is supposed to validate a field, not "that". My question is, is it good practice to do this?
Is there a better alternative to my solution? but that this alternative meets the objective of being reusable.
Any suggestion helps my learning, thanks.

One of the problem is that, two or more FileFields can refer to the same file. In the database a FileField stores the location of the file, so two or more columns can have the same file, therefore, just removing the old one is not (completely) safe.
You can for example make use of django-unused-media. You install this with:
$ pip install django-unused-media
Next you add this to the installed apps:
# settings.py
INSTALLED_APPS = [
# …,
'django_unused_media',
# …
]
Next you can run:
python3 manage.py cleanup_unused_media
this will look for files that are no longer referenced, and clean these up interactively.
You can also make a scheduled task (for example with cron), that runs with the --no-input flag:
python3 manage.py cleanup_unused_media --no-input

Renaming a django model class-name and corresponding foreign keys with south, without loosing the data

Following is my model:
class myUser_Group(models.Model):
name = models.CharField(max_length=100)
class Channel(models.Model):
name = models.CharField(max_length=100)
description = models.CharField(max_length=1000)
belongs_to_group = models.ManyToManyField(myUser_Group)
class Video(models.Model):
video_url = models.URLField(max_length=300)
belongs_to_channel = models.ManyToManyField(Channel)
description = models.CharField(max_length=1000)
tags = TagField()
class UserProfile(models.Model):
user = models.OneToOneField(User)
class User_History(models.Model):
date_time = models.DateTimeField()
user = models.ForeignKey(UserProfile, null=True, blank=True)
videos_watched = models.ManyToManyField(Video)
I just wanted to remove the underscores from all the class names so that User_History looks UserHistory, also the foreign keys should be updated. I tried using south but could not find it in the documentaion.
One way is export the data, uninstall south, delete migration, rename the table and then import data again. Is there any other way to do it?

You can do this using just South.
For this example I have an app called usergroups with the following model:
class myUser_Group(models.Model):
name = models.CharField(max_length=100)
which I assume is already under migration control with South.
Make the model name change:
class MyUserGroup(models.Model):
name = models.CharField(max_length=100)
and create an empty migration from south
$ python manage.py schemamigration usergroups model_name_change --empty
This will create a skeleton migration file for you to specify what happens. If we edit it so it looks like this (this file will be in the app_name/migrations/ folder -- usergroups/migrations/ in this case):
import datetime
from south.db import db
from south.v2 import SchemaMigration
from django.db import models
class Migration(SchemaMigration):
def forwards(self, orm):
# Change the table name from the old model name to the new model name
# ADD THIS LINE (using the correct table names)
db.rename_table('usergroups_myuser_group', 'usergroups_myusergroup')
def backwards(self, orm):
# Provide a way to do the migration backwards by renaming the other way
# ADD THIS LINE (using the correct table names)
db.rename_table('usergroups_myusergroup', 'usergroups_myuser_group')
models = {
'usergroups.myusergroup': {
'Meta': {'object_name': 'MyUserGroup'},
'id': ('django.db.models.fields.AutoField', [], {'primary_key': 'True'}),
'name': ('django.db.models.fields.CharField', [], {'max_length': '100'})
}
}
complete_apps = ['usergroups']
In the forwards method we are renaming the database table name to match what the django ORM will look for with the new model name. We reverse the change in backwards to ensure the migration can be stepped back if required.
Run the migration with no need to import/export the exisiting data:
$ python manage.py migrate
The only step remaining is to update the foreign key and many-to-many columns in the models that refer to myUser_Group and change to refer to MyUserGroup.

mmcnickle's solution may work and seems reasonable but I prefer a two step process. In the first step you change the table name.
In your model make sure you have your new table name in:
class Meta:
db_table = new_table_name'
Then like mmcnickle suggested, create a custom migration:
python manage.py schemamigration xyz migration_name --empty
You can read more about that here:
https://docs.djangoproject.com/en/dev/ref/models/options/
Now with your custom migration also add the line to rename your table forward and backwards:
db.rename_table("old_table_name","new_table_name")
This can be enough to migrate and change the table name but if you have been using the Class Meta custom table name before then you'll have to do a bit more. So I would say as a rule, just to be safe do a search in your migration file for "old_table_name" and change any entries you find to the new table name. For example, if you were previously using the Class Meta custom table name, you will likely see:
'Meta': {'object_name': 'ModelNameYouWillChangeNext', 'db_table': "u'old_table_name'"},
So you'll need to change that old table name to the new one.
Now you can migrate with:
python manage.py migrate xyz
At this point your app should run since all you have done is change the table name and tell Django to look for the new table name.
The second step is to change your model name. The difficulty of this really depends on your app but basically you just need to change all the code that references the old model name to code that references the new model name. You also probably need to change some file names and directory names if you have used your old model name in them for organization purposes.
After you do this your app should run fine. At this point your task is pretty much accomplished and your app should run fine with a new model name and new table name. The only problem you will run into using South is the next time you create a migration using it's auto detection feature it will try to drop the old table and create a new one from scratch because it has detected your new model name. To fix this you need to create another custom migration:
python manage.py schemamigration xyz tell_south_we_changed_the_model_name_for_old_model_name --empty
The nice thing is here you do nothing since you have already changed your model name so South picks this up. Just migrate with "pass" in the migrate forwards and backwards:
python manage.py migrate xyz
Nothing is done and South now realizes it is up to date. Try:
python manage.py schemamigration xyz --auto
and you should see it detects nothing has changed