We Keep Coding

Filter by CharField pretending it is DateField in Django ORM/mySql

I am working with a already-done mySQL Database using Django ORM and I need to filter rows by date - if it wasn't that dates are not in Date type but normal Varchar(20) stored as dd/mm/yyyy hh:mm(:ss).
With a free query I would transform the field into date and I would use > and < operators to filter the results but before doing this I was wondering whether Django ORM provides a more elegant way to do so without writing raw SQL queries.
I look forward to any suggestion.
EDIT: my raw query would look like
SELECT * FROM table WHERE STR_TO_DATE(mydate,'%d/%m/%Y %H:%i') > STR_TO_DATE('30/12/2020 00:00', '%d/%m/%Y %H:%i')
Thank you.

I will assume your model looks like this:
from django.db import models
class Event(models.Model):
mydate = models.CharField(max_length=20)
def __str__(self):
return f'Event at {self.mydate}'
You can construct a Django query expression to represent this computation. This expression consists of:
Func objects representing your STR_TO_DATE function calls.
An F object representing your field name.
A GreaterThan function to represent your > comparison.
from django.db.models import F, Func, Value
from django.db.models.lookups import GreaterThan
from .models import Event
# Create some events for this example
Event(mydate="29/12/2020 00:00").save()
Event(mydate="30/12/2020 00:00").save()
Event(mydate="31/12/2020 00:00").save()
class STR_TO_DATE(Func):
"Lets us use the STR_TO_DATE() function from SQLite directly in Python"
function = 'STR_TO_DATE'
# This Django query expression converts the mydate field
# from a string into a date, using the STR_TO_DATE function.
mydate = STR_TO_DATE(F('mydate'), Value('%d/%m/%Y %H:%i'))
# This Django query expression represents the value 30/12/2020
# as a date.
date_30_12_2020 = STR_TO_DATE(Value('30/12/2020 00:00'), Value('%d/%m/%Y %H:%i'))
# This Django query expression puts the other two together,
# creating a query like this: mydate < 30/12/2020
expr = GreaterThan(mydate, date_30_12_2020)
# Use the expression as a filter
events = Event.objects.filter(expr)
print(events)
# You can also use the annotate function to add a calculated
# column to your query...
events_with_date = Event.objects.annotate(date=mydate)
# Then, you just treat your calculated column like any other
# field in your database. This example uses a range filter
# (see https://docs.djangoproject.com/en/4.0/ref/models/querysets/#range)
events = events_with_date.filter(date__range=["2020-12-30", "2020-12-31"])
print(events)
I tested this answer with Django 4.0.1 and MySQL 8.0.

How to merge two different querysets with one common field in to one in django?

I have 2 Querysets Sales_order and Proc_order. The only common field in both is the product_id. I want to merge both these query sets to one with all fields.
sales_order has fields product_id,sales_qty,sales_price.
proc_order has fields product_id, proc_qty, proc_price. I want to merge both these to get a queryset which looks like.
combined_report which has fields product_id,sales_qty,sales_price``proc_qty, proc_price.
My final aim is to calculate the difference between the number of products.
I'm using Django 2.1

You can try this way to capture all the values.
from django.db.models import Subquery, OuterRef, FloatField
from django.db.models.functions import Cast
subquery_qs = proc_order_qs.filter(product_id=OuterRef('product_id')
combined_qs = sales_order_qs.annotate(
proc_qty = Cast(Subquery(subquery_qs.values('proc_qty')[:1]), output_field=FloatField()),
proc_price = Cast(Subquery(subquery_qs.values('proc_price')[:1]), output_field=FloatField()))
And then you can get all the values in combined_qs
combined_qs.values('product_id','sales_qty','sales_price','proc_qty', 'proc_price')

you can try to do something like this:
views.py
from itertools import chain
def yourview(request):
Sales_order = ....
Proc_order = ....
combined_report = chain(Sales_order,Proc_order)

Django make query dynamically for filter

this code works well
from django.db.models import Q
filtered = Article.objects.filter(Q(authers__id=2) | Q(issues__id=1) | Q(issues__id=3) )
However now I have list like this below, and want to make filter dynamically.
ids = [1,2,3]
for id in ids:
eachQ = Q(authers__id=isId)
#then......
how can I make query ???

Querying same field for different values(or condition) you can use __in key.
possibilities = [1,2,3]
Article.objects.filter(field__in=posibilities)
Also for dynamic queries you can pass **kwargs to filter method:
query_key = 'your_field_name__in'
Article.objects.filter(**{query_key:[1,2,3]#your search value})
You can add your multiple model field into kwargs param:
query = {'field_1':value_1,'field_2':value_2,'field_3__contains':value_3}#for example
Article.objects.filter(**query)

If you have a list of values for just one field, it`s better to use 'in' filter:
ids = [1,2,3]
articles = Article.objects.filter(authers__id__in=ids)
Otherwise, for creating 'OR' filter iteratively:
from django.db.models import Q
filters = Q()
ids = [1, 2, 3]
for pk in ids:
filters |= Q(authers__id=pk)
articles = Article.objects.filter(filters)
This can be used for dynamic filtering by multiple model fields and values.

Query and group nested documents with mongoengine

I have kind of a dual question which is keeping me from proceeding for a while already. I have read lots of articles, checked stackoverflow numerous times and read again the docs of mongoengine but I can not find the answer which works for me. I am using mongoDB to store the data of a Flask webb-app. To query the DB, I am using mongoengine. Now suppose my users model lookes like this:
Users
name: Superman
kudos:
0 0 date
1 category A
1 0 date
1 category B
name: Superman
kudos:
0 0 date
1 category A
1 0 date
1 category A
2 0 date
1 category B
The kudo's are nested documents which get created whenever a user receives a kudo. I store them as a db.ListField(date=now). This is working perfectly fine.
In a relational DB I would have a seperate kudo scheme. In mongoDB I assumend it would be the better solution to create nested documents wihtin the User collections. Otherwise you are still creating all kind of seperate scheme's which relations to others.
So here are my two main questions:
Am I correct in that my architecture is true to how mongoengine should be implemented?
How can I get a list (dict actually) of kudo's per category? So I would like to query and get Category - Count
Result should be:
kudos=[(category A, 3),(category B, 2)
If I already had something even remotely working I would provide it but I am completely stuck. Thats why I even started doubting storing the kudos in a seperate collection but I feel like I am than starting to get off track in correctly using a noSQL DB.

Assuming you have the following schema and data:
import datetime as dt
from mongoengine import *
connect(host='mongodb://localhost:27017/testdb')
class Kudo(EmbeddedDocument):
date = DateTimeField(default=dt.datetime.utcnow)
category = StringField()
class User(Document):
name = StringField(required=True)
kudos = EmbeddedDocumentListField(Kudo)
superman = User(name='superman', kudos=[Kudo(category='A')]).save()
batman = User(name='batman', kudos = [Kudo(category='A'), Kudo(category='B')]).save()
This isn't the most efficient but you can get the distribution with the following simple snippet:
import itertools
from collection import Counter
raw_kudos = User.objects.scalar('kudos')
categories_counter = Counter(k.category for k in itertools.chain.from_iterable(raw_kudos)) # raw_kudos is a list of list
print(categories_counter) # is a dict --> Counter({u'A': 1, u'B': 1})
And if you need higher performance, you'll need to use an aggregation pipeline

Django Postgresql JsonField query related dictionary keys

A part of the model that I have, which uses Django Model field, is like the following:
class SalesModel(models.Model):
some_data = models.PositiveIntegerField(db_index=True)
some_other_data = models.CharField(max_length=50)
json_data = JSONField(blank=True, null=True)
Now following is the format of the JsonData field:
[{"id": val, "contribution": "some_val", }, {"id": some_val, "contribution": "some_other_val",}, {"id": other_val, "contribution": "some_another_val"}]
i.e., the format is:
[{'id':XX, 'contribution':XX},{'id':YY, 'contribution':YY},{'id':ZZ, 'contribution':ZZ}]
Currently I can filter the Django table with the val of ID. I would now, like to know the contribution of that particular ID.
For eg, if val = 1, I would like to filter the model SalesModel which has JsonField with id = 1, and I want to show the related contribution. So, that would mean, out of the 3 possible dictionaries (as per the field construction), I would only show one dictionary (filtered by the 'ID' key of that dictionary). That would mean, if the 2nd dictionary has a matching ID, show only the 2nd contribution, if the 1st ID is matching show only the 1st contribution, and similarly for the 3rd dictionary.
Is there a way that can be done?

You could restructure your JSONField differently, by giving it a dict where the key, value pairs are id: contribution directly. This way you could use the has_key filter and KeyTransform will work, as I'm not sure it works on an array of dicts. So assuming your json_data looks like this:
{1: 'xx', 3: 'yy', 9: 'zz'}
you could query this way, based on #vanojx1 contribution:
SalesModel.filter(json_data__has_key=id)\
.annotate(contrib=KeyTransform(id, 'json_data')\
.values('contrib')
Alternatively, using raw jsonb in postgresql:
SalesModel.filter(json_data__has_key=id)\
.extra(select={'contrib': "json_data->{0}".format(id)})\
.values('contrib')

This should work DOC
SalesModel.objects.filter(json_data__id=1).values('id', 'json_data__contribution')

Yes, I guess. If I have understood it right, you will have an id to be matched or a list of ID's. So if your ID is 2:
my_id = 2
dict1 = [{"id":1, "contribution":10},{"id":2, "contribution":20},{"id":3, "contribution":30}]
for i in dict1:
if i["id"] == my_id:
print(i["contribution"])