How do I specify that a model field is implemented by a database expression instead of a table column?
I have this model:
class URLProbe(models.Model):
url = models.CharField(max_length=MAX_URL_LENGTH)
timestamp = models.DateTimeField(auto_now_add=True)
status_code = models.SmallIntegerField(null=True, blank=True) # The HTTP status code
error = models.TextField(blank=True) # stores non-HTTP errors, e.g. connection failures
def ok(self):
return self.status_code == 200
ok.boolean = True
ok.admin_order_field = Case(
When(status_code__exact=200, then=True),
default=False,
output_field=BooleanField()
)
def errmsg(self):
if self.ok():
return ''
if self.status_code is not None:
return f'Error: HTTP status {self.status_code}'
return self.error
errmsg.admin_order_field = Case(
When(status_code__exact=200, then=Value('')),
When(status_code__isnull=False,
then=Concat(Value('Error: HTTP status '), 'status_code')),
default='error',
output_field=CharField()
)
The model represents an attempt to retrieve an url, recording whether the HTTP call succeeded or what the error was.
The ok and errmsg fields here are implemented as methods, but I want to be able to use them in the Django admin as regular fields (sort/filter on them) and to sort and/or filter on them in queries. That is possible, but I need to define the query expression multiple times: in an admin_order_field, in a Django admin custom filter, and in queries where I want to use them. So that duplicates a lot of code, both for the multiple expressions and between the query expression and the python method.
What I would like to do
is define fields as query expressions, allowing any database operations to occur without further configuration. Something like:
class URLProbe(models.Model):
...
ok = ExpressionField(Case(
When(status_code__exact=200, then=True),
default=False,
output_field=BooleanField()
))
errmsg = ExpressionField(Case(
When(ok__exact=True, then=Value('')),
When(status_code__isnull=False,
then=Concat(Value('Error: HTTP status '), 'status_code')),
default='error',
output_field=CharField()
))
However I have not been able to find anything like ExpressionField or a way to do something similar. Does this exist? Can this be implemented in Django?
update: I found something that works partially. I can specify a custom manager with a default queryset that includes ok and errmsg as annotations. That allows me to use them in custom queries without duplicating the expression, but unfortunately the admin does not accept them as fields and throws SystemCheckErrors. I feel that it should work if it were not for that system check.
As far as I know, using the base Django ORM you cannot sort by properties since those operations are done at SQL level and that field is not stored in the database.
There is a project to denormalize models, that basically calculates and stores this calculated values behind the curtain for you, that would probably serve your purpose:
https://github.com/django-denorm/django-denorm
So you would use it like:
#denormalized(models.CharField, max_length=100)
def errmsg(self):
if self.ok():
return ''
if self.status_code is not None:
return f'Error: HTTP status {self.status_code}'
Related
I need to make search for looking some data from database and this search field should support some logical operations. Like AND, OR, NOT and so on.
So I found that Postgres db has something that I need. Ad Django documentation saying to us we need to use SearchQuery with parameter search_type='websearch'
And we have something like
SearchQuery("'tomato' ('red' OR 'green')", search_type='websearch') # websearch operators
When I'm tried to implement to my project I did something like that
# views.py
class DataSearch(generics.ListAPIView):
serializer_class = DataSerializer
def get_queryset(self):
queryset = Data.objects.all()
query = self.request.query_params.get('query')
if query is not None:
queryset = queryset.filter(
search_vector = SearchQuery(
query,
search_type='websearch'
)
)
return queryset
But when I'm trying to run it I get
Cannot resolve keyword 'search_vector' into field. Choices are: data, foo, foo_id
So what I should do to make it work and to make API endpoint for this type of search?
This question already has an answer here:
How can I create a django model instance with deferred fields without hitting the database?
(1 answer)
Closed 8 months ago.
I want to know if I can instanciate an empty fake model just with id of database record.
I found way to create mockup model, but I want a production-friendly solution.
Explanation of my issue :
I want to list users settings for users who choose to be displayed on public mode :
user_displayed_list = UserPublicProfile.objects.filter(
displayed = True,
).only(
'user_id',
'is_premium',
)
user_settings_list = []
for user_displayed in user_displayed_list:
# I have to send user Instance to the next method :
user_settings = self.get_user_settings(user_displayed.user)
user_settings_list.append(user_settings)
# But ’user_displayed.user’ run an new SQL query
I know I can improve my queryset as :
user_displayed_list = UserPublicProfile.objects.filter(
displayed = True,
).select_related(
'user'
).only(
'user',
'is_premium',
)
But It makes an useless join because I need only the user id field in get_user_settings():
The get_user_settings() method (it could help to understand context):
def get_user_settings(self, user)
user_settings = UserSettings.objects.get(user = user)
return user_settings
In real project, this method run more business feature
Is there a way to instanciate a User model instance with only id field filled ?
I don't want to use a custom empty class coded for this purpose. I really want an object User.
I didn't find anything for that. If it's possible, I could use it by this way :
for user_displayed in user_displayed_list:
FakeUser = User.objects.create_fake(id = user_displayed.user_id)
# I have to send user Instance to the next method :
user_settings = self.get_user_settings(FakeUser)
Without seeing the complete models, I'm assuming a bit. Assuming that UserSettings has a ForeignKey to User. Same for UserPublicProfile. Or User has ForeignKey to UserSettings. Works as well.
Assuming that, I see two solutions.
Solution #1; use the ORM to full potential
Just saw your comment about the 'legacy method, used many times'.
Django relations are very smart. They accept either the object or the ID of a ForeignKey.
You'd imagine this only works with a User. But if you pass the id, Django ORM will help you out.
def get_user_settings(self, user)
user_settings = UserSettings.objects.get(user = user)
return user_settings
So in reality, these work the same:
UserSettings.objects.get(user=1)
UserSettings.objects.get(user_id=1)
Which means this should work, without a extra query:
user_displayed_list = UserPublicProfile.objects.filter(
displayed = True,
).only(
'user_id',
'is_premium',
)
user_settings_list = []
for user_displayed in user_displayed_list:
# I have to send user Instance to the next method :
user_settings = self.get_user_settings(user_displayed.user_id) # pass the user_id instead of the object.
user_settings_list.append(user_settings)
Solution #2: chain relations
Another solution, again, still assuming quite a bit ;)
It would think you can chain the model together.
Assuming these FK exists: UserPublicProfile -> User -> UserSetting.
You could do this:
user_displayed_list = UserPublicProfile.objects.filter(
displayed = True,
).select_related(
'user', 'user__usersettings', # depends on naming of relations
).only(
'user',
'is_premium',
)
for user_displayed in user_displayed_list:
# I have to send user Instance to the next method :
user_settings = user_displayed.user.usersettings # joined, so should cause no extra queries. Depends on naming of relations.
user_settings_list.append(user_settings)
There are two models .I want to make query to extract only the app exact app related Adspaces .
models.py
class Appname(models.Model):
user=models.ForeignKey(User,related_name='appname', null=True, default=None,on_delete=models.CASCADE)
name=models.CharField(max_length=150,blank=False,null=False,help_text='Add your new App')
def __str__(self):
return self.name
def get_absolute_url(self):
return reverse("dashapp:space",kwargs={'pk':self.pk})
class Adspace(models.Model):
user=models.ForeignKey(User,related_name='adspace', null=True, default=None,on_delete=models.CASCADE)
ad_space=models.CharField(max_length=150,blank=False,null=False)
app=models.ForeignKey('Appname', related_name='appnames',default=None, on_delete=models.CASCADE)
PID_TYPE = (
('FN','FORMAT_NATIVE'),
('FNB','FORMAT_NATIVE_BANNER'),
('FI','FORMAT_INTERSTITIAL'),
('FB','FORMAT_BANNER'),
('FMR','FORMAT_MEDIUM,RECT'),
('FRV','FORMAT_REWARDED_VIDEO'),
)
format_type=models.CharField(max_length=3,choices=PID_TYPE,default='FN',blank=False, null=False)
def __str__(self):
return self.ad_space
def get_absolute_url(self):
return reverse("dashapp:create",kwargs={'pk':self.pk})
Views.py
SHowing the one where i need to the query
class spacelist(LoginRequiredMixin,ListView):
model=Adspace
template_name='adspace_list.html'
def get_queryset(self):
query_set=super().get_queryset()
return query_set.filter(user=self.request.user)
Here I need to perform One more query so that EACH APP show their own adspaces when clicked right now every app show every show adspaces.
I have the idea what to do as if i compare app_id then it'll show the exact app related adspaces, but i dont know how to write query for the same as i already have one query present.???
You could try using a Q objects: https://docs.djangoproject.com/en/2.1/topics/db/queries/#complex-lookups-with-q-objects
From what I understand you are trying to filter both on the app_id and the request user at the same time, so you could try look something like this:
from django.db.models import Q
...
def get_queryset(self):
query_set=super().get_queryset()
return query_set.filter(Q(user=self.request.user) & Q(app_id=app_id))
...
This lets you do a single filter with both your requirements at the same time (i.e. retrieve the Adspace instances for a specific user with a specific Appname).
You chain another filter at the end like this:
class spacelist(LoginRequiredMixin,ListView):
model=Adspace
template_name='adspace_list.html'
def get_queryset(self):
query_set = super().get_queryset()
query_set = query_set.filter(user=self.request.user)
app_id = [...]
return query_set.filter(app_id=app_id)
The problem left is to find out what is the app_id coming from. How do you know what is the current app? Several options here.
Option 1: From the request
It can come from the current user: self.request.user.appname.all() but that will give you multiple apps, if the user can only have one app, you should change your model Appname.user to a OneToOneField.
Otherwise, I suggest changing your related_name='appnames' to reflect the multiplicity in the reverse relationship.
Option 2: From the URL
It can come from the URL, your space list view should extract an app_id parameter from the URL where it's defined:
url(r'^(?P<app_id>[0-9]+)/spaces/$', spacelist.as_view(), name='space_list'),
And then in the spacelist view, you would get this parameter like this:
app_id = self.kwargs['app_id']
return query_set.filter(app_id=app_id)
Hope that helps
UPDATE:
Also worth noting that QuerySets are lazy, meaning the result will get evaluated as late as possible by Django. Therefore, when you call:
query_set = query_set.filter(user=self.request.user)
The Django ORM doesn't execute any DB queries yet, and you can chain more filters after that:
query_set = query_set.filter(user=self.request.user)
query_set = query_set.filter(app_id=app_id)
Which behind the scenes is extending the query that will be executed when required. But at this point, no query is actually run. To see the query that will get executed you can print out the query attribute of the QuerySet:
print(query_set.query)
Which should log something like:
SELECT "app_adspace"."user_id" ...
FROM
"app_adspace"
WHERE
"app_adspace"."user_id" = 1234 AND "app_adspace"."app_id" = 5678
I have a Django app with models accessible by both Django REST Framework and a regular form interface. The form interface has some validation checks before saving changes to the model, but not using any special Django framework, just a simple local change in the view.
I'd like to apply the same validation to forms and REST calls, so I want to move my validation into the model. I can see how to do that for simple cases using the validators field of the Field, but in one case I have a name/type/value model where the acceptable values for 'value' change depending on which type is selected. The validator doesn't get sent any information about the model that the field is in, so it doesn't have access to other fields.
How can I perform this validation, without having essentially the same code in a serializer for DRF and my POST view for the form?
I dug around codebase of drf a little bit. You can get values of all fields using following approach. Doing so, you can throw serialization error as
{'my_field':'error message} instead of {'non_field_error':'error message'}.
def validate_myfield(self, value):
data = self.get_initial() # data for all the fields
#do your validation
However, if you wish to do it for ListSerializer, i.e for serializer = serializer_class(many=True), this won't work. You will get list of empty values.
In that scenario, you could write your validations in def validate function and to avoid non_field_errors in your serialization error, you can raise ValidationError with error message as a dictionary instead of string.
def validate(self, data):
# do your validation
raise serializers.ValidationError({"your_field": "error_message"})
The validation per-field doesn't get sent any information about other fields, when it is defined like this:
def validate_myfield(self, value):
...
However, if you have a method defined like this:
def validate(self, data):
...
Then you get all the data in a dict, and you can do cross-field validation.
You can use the required package for your cross-field validation. It allows you to express your validation rules declaratively in python. You would have something like this with DRF:
class MySerializer(serializers.Serializer):
REQUIREMENTS = (
Requires("end_date", "start_date") +
Requires("end_date", R("end_date") > R("start_date")) +
Requires("end_date", R("end_date") < today.date() + one_year) +
Requires("start_date", R("start_date") < today.date() + one_year)
)
start_date = serializers.DateField(required=False, null=True, blank=True)
end_date = serializers.DateField(required=False, null=True, blank=True)
def validate(self, data):
self.REQUIREMENTS.validate(data) # handle validation error
You could put the REQUIREMENTS on your Model and have both your DRF and Django Form validate your data using it.
Here is a blog post explaining more
Racking my brain on this one. The model seems true, theoretically all of the commented permutations should work--- but the only things that can successfully retrieve the user is .filter and .all; .get doesnt work; I can deal with using either .filter or .all ---- but why isn't get working?
I'll reiterate that a direct SQL query works 100% in this case. All imports are in place and things are functioning great at a low level -- again, Filter works, All works, but get fails for some reason.
class UserModelTest(TestCase):
def test_getUserByUsername(self):
sanity = True
try:
#u = User.objects.filter(username='wadewilliams')
u = User.objects.get(username='wadewilliams')
#u = User.objects.get(pk=15773)
#u = User.objects.all()
print u
except User.DoesNotExist:
sanity = False
self.assertEqual(sanity, True)
... That test fails unless I uncomment either filter or all... both gets, no go.
And the model...
class User(models.Model):
userid = models.IntegerField(primary_key=True, db_column='userID')
username = models.CharField(max_length=135)
realname = models.CharField(max_length=150, db_column='name')
email = models.CharField(max_length=765, blank=True)
class Meta:
db_table = u'users'
def __unicode__(self):
return self.username + ' (' + self.email + ')'
The test suite creates a mock database that is blank, so no users can be found even though the exist in the production/development database.
From the docs:
Finding data from your production database when running tests?
If your code attempts to access the database when its modules are compiled, this will occur before the test database is set up, with potentially unexpected results. For example, if you have a database query in module-level code and a real database exists, production data could pollute your tests. It is a bad idea to have such import-time database queries in your code anyway - rewrite your code so that it doesn't do this.