We Keep Coding

Building object models around external data

I want to integrate external data into a Django app. Let's say, for example, I want to work with GitHub issues as if they were formulated as normal models within Django. So underneath these objects, I use the GitHub API to retrieve and store data.
In particular, I also want to be able to reference the GitHub issues from models but not the other way around. I.e., I don't intend to modify or extend the external data directly.
The views would use this abstraction to fetch data, but also to follow the references from "normal objects" to properties of the external data. Simple joins would also be nice to have, but clearly there would be limitations.
Are there any examples of how to achieve this in an idiomatic way?
Ideally, this would be would also be split in a general part that describes the API in general, and a descriptive part of the classes similar to how normal ORM classes are described.

If you want to use Django Model-like interface for your Github Issues, why don't use real Django models? You can, for example, create a method fetch in your model, that will load data from the remote api and save it to your model. That way you won't need to make external requests everywhere in your code, but only when you need it. A minimal example will look like these:
import requests
from django.db import models
from .exceptions import GithubAPIError
class GithubRepo(models.Model):
api_url = models.URLField() # e.g. https://api.github.com/repos/octocat/Hello-World
class GithubIssue(models.Model):
issue_id = models.IntegerField()
repo = models.ForeignKey(GithubRepo, on_delete=models.CASCADE)
node_id = models.CharField(max_length=100)
title = models.CharField(max_length=255, null=True, blank=True)
body = models.TextField(null=True, blank=True)
"""
Other fields
"""
class Meta:
unique_together = [["issue_id", "repo"]]
#property
def url(self):
return f"{self.repo.api_url}/issues/{self.issue_id}"
def fetch_data(self):
response = requests.get(self.url)
if response.status != 200:
raise GithubAPIError("Something went wrong")
data = response.json()
# populate fields from repsonse
self.title = data['title']
self.body = data['body']
def save(
self, force_insert=False, force_update=False, using=None, update_fields=None
):
if self.pk is None: # fetch on first created
self.fetch_data()
super(GithubIssue, self).save(
force_insert, force_update, using, update_fields
)
You can also write a custom Manager for your model that will fetch data every time you call a create method - GithubIssue.objects.create()

The django way in this case would be to write a custom "db" backend.
This repo looks abandoned but still can lead you to some ideas.

I would suggest to just use normal OOP principles, Polymorphism, Association etc. to get a similar feel to real models.
But I'm not sure I would try to simulate behavior as close as I could, because the ORM is specifically designed for database interaction. I would just write my custom methods.

How to write manager class which use filter field as computed field not as a part of model fields?

I have a model Student with manager StudentManager as given below. As property gives the last date by adding college_duration in join_date. But when I execute this property computation is working well, but for StudentManager it gives an error. How to write manager class which on the fly computes some field using model fields and which is used to filter records.
The computed field is not in model fields. still, I want that as filter criteria.
class StudentManager(models.Manager):
def passed_students(self):
return self.filter(college_end_date__lt=timezone.now())
class Student(models.Model):
join_date = models.DateTimeField(auto_now_add=True)
college_duration = models.IntegerField(default=4)
objects = StudentManager()
#property
def college_end_date(self):
last_date = self.join_date + timezone.timedelta(days=self.college_duration)
return last_date
Error Django gives. when I tried to access Student.objects.passed_students()
django.core.exceptions.FieldError: Cannot resolve keyword 'college_end_date' into field. Choices are: join_date, college_duration

Q 1. How alias queries done in Django ORM?
By using the annotate(...)--(Django Doc) or alias(...) (New in Django 3.2) if you're using the value only as a filter.
Q 2. Why property not accessed in Django managers?
Because the model managers (more accurately, the QuerySet s) are wrapping things that are being done in the database. You can call the model managers as a high-level database wrapper too.
But, the property college_end_date is only defined in your model class and the database is not aware of it, and hence the error.
Q 3. How to write manager to filter records based on the field which is not in models, but can be calculated using fields present in the model?
Using annotate(...) method is the proper Django way of doing so. As a side note, a complex property logic may not be re-create with the annotate(...) method.
In your case, I would change college_duration field from IntegerField(...) to DurationField(...)--(Django Doc) since its make more sense (to me)
Later, update your manager and the properties as,
from django.db import models
from django.utils import timezone
class StudentManager(models.Manager):
<b>def passed_students(self):
default_qs = self.get_queryset()
college_end = models.ExpressionWrapper(
models.F('join_date') + models.F('college_duration'),
output_field=models.DateField()
)
return default_qs \
.annotate(college_end=college_end) \
.filter(college_end__lt=timezone.now().date())</b>
class Student(models.Model):
join_date = models.DateTimeField()
college_duration = models.DurationField()
objects = StudentManager()
#property
def college_end_date(self):
# return date by summing the datetime and timedelta objects
return <b>(self.join_date + self.college_duration).date()
Note:
DurationField(...) will work as expected in PostgreSQL and this implementation will work as-is in PSQL. You may have problems if you are using any other databases, if so, you may need to have a "database function" which operates over the datetime and duration datasets corresponding to your specific database.
Personally, I like this solution,

To quote #Willem Van Olsem's comment:
You don't. The database does not know anything about properties, etc. So it can not filter on this. You can make use of .annotate(..) to move the logic to the database side.
You can either do the message he shared, or make that a model field that auto calculates.
class StudentManager(models.Manager):
def passed_students(self):
return self.filter(college_end_date__lt=timezone.now())
class Student(models.Model):
join_date = models.DateTimeField(auto_now_add=True)
college_duration = models.IntegerField(default=4)
college_end_date = models.DateTimeField()
objects = StudentManager()
def save(self, *args, **kwargs):
# Add logic here
if not self.college_end_date:
self.college_end_date = self.join_date + timezone.timedelta(days-self.college_duration)
return super.save(*args, **kwargs)
Now you can search it in the database.
NOTE: This sort of thing is best to do from the start on data you KNOW you're going to want to filter. If you have pre-existing data, you'll need to re-save all existing instances.

Problem
You’re attempting to query on a row that doesn’t exist in the database. Also, Django ORM doesn’t recognize a property as a field to register.
Solution
The direct answer to your question would be to create annotations, which could be subsequently queried off of. However, I would reconsider your table design for Student as it introduces unnecessary complexity and maintenance overhead.
There’s much more framework/db support for start date, end date idiosyncrasy than there is start date, timedelta.
Instead of storing duration, store end_date and calculate duration in a model method. This makes more not only makes more sense as students are generally provided a start date and estimated graduation date rather than duration, but also because it’ll make queries like these much easier.
Example
Querying which students are graduating in 2020.
Students.objects.filter(end_date__year=2020)

Selecting only active records django query

In my Django project, I have an is_active boolean column in every table of my database. Every time I or the framework accesses the database, I want only the active records to show up. What is the standard way to achieve this? Certainly I don't want to check for is_active in every queries I make.

The easiest way to do this is to create a custom model manager, like this:
class OnlyActiveManager(models.Manager):
def get_queryset(self):
return super(OnlyActiveManager, self).get_queryset().filter(is_active=True)
Then, add it to your models:
class MyModel(models.Model):
objects = models.Manager()
active = OnlyActiveManager()
Next, use it like this:
foo = MyModel.active.all()
You can also use it to replace the default manager (called objects), but then you'll have to do custom queries to get all records that are in-active.

You can write a manager class for your model, A sample model manager is given below, for more you can refer Django official website
class MediaManager(models.Manager):
def get_queryset(self):
return MediaQuerySet(self.model, using=self._db)
def active(self):
return self.filter(is_active=True)
class Media(models.Model):
(..model fields..)
objects = MediaManager()
The query should be like
media = Media.objects.active()

you can do it by using django model managers.
please check django documentaion for detail django documentaion

How to check for a specific field change in a model?

I have my models.py :
class Hotel(models.Model):
name = models.CharField(max_length=20)
currency = models.ForeignKey(Currency)
class Currency(models.Mode):
code = models.CharField(max_length=3)
name = models.CharField(max_length=10)
Whenever the currency field in hotel is changing I need to be able to do something. So I have a function like :
#receiver(pre_save,sender=Hotel)
def update_something(sender,**kwargs)
obj = kwargs['instance']
old_object = Hotel.objects.get(pk=obj.pk)
'''
Now I can do anything here comparing the oldo object with the instance
'''
The thing is I don't want to make a query for this, since then the purpose of signals becomes stupid and I become a fool.
So I should be able to do something like :
updated = kwargs['update_fields']
new_currency = updated['currency']
Is their a way I can find out that change for ONLY one particular field say currency ,instead of doing a query like this. I want to get the changes related to the currency foreign key and update things before I save.
Sorry for my bad English and not being able to use too technical terms.
Thanks :)

Rather hacky solution is to save object state on initialization.
from django.forms.models import model_to_dict
class Currency(models.Mode):
code = models.CharField(max_length=3)
name = models.CharField(max_length=10)
def __init__(self):
super(Currency, self).__init__()
self.__state = model_to_dict(self)
def updated(self):
new_state = model_to_dict(self)
return dict(set(self.__state.iteritems()) - set(new_state.iteritems()))
Method updated will return difference between initial and new states.

The point of signals is better decoupling - allow other applications (that you don't necessarily know yet) to transparently hook into your application and/or avoid circular dependencies between application. There's really no point in using a signal here afaik (unless of course the signal handler is in another app and you don't want the app with the Hotel and Currency models to depend on this other app).
To answer your question: doing a query to retrieve the original state is the only way you can compare stored value and current value. Now if you're only interested in one specific field, you of course don't have to retrieve the whole model instance - just use queryset.values_list:
old_currency = Hotel.objects.filter(pk=obj.pk).values_list("currency", flat=True)[0]

Django Mongodb ListField not saving or updating

I am starting to create a webapp using Django and MongoDB. Everything is working fine when I create a model and save it into the Database. Now, I do a "Class.objects.get()" to get the object I need from my DB and I have one field called "media" which is a ListField(). I had tried doing either:
Concert.media.append(list)
or
Concert.media.extend(list)
and then
Concert.save()
This is my "Concert" object in my models.py:
class Concert(models.Model):
main_artist = models.CharField(max_length=50)
concert_id = models.AutoField(primary_key=True)
openers = ListField(EmbeddedModelField('Opener'))
concert_date = models.DateField()
slug = models.SlugField(unique=True)
media = ListField()
And when I go to see the results in does not update the object. No values where saved. If someone can help me I going to give a super cyber fist bump.

Concert is a class, not an instance. You can't save a class. You need to make an instance of the class and save that. Something like
c = Concert()
c.media.append(list)
c.save()
(btw, just as a note, list is a bad variable name because list is a type in python. Never use types as variable names (though everyone is guilty of this at one point or another, including me.))