Mongoengine: create a QuerySet from a MongoDB cursor - python

Some part of the application I'm working in is expecting a mongoengine QuerySet.
I have a MongoDB cursor with the info I need, generated by an aggregation.
Since mongoengine documentation specifies that a QuerySet is a wrapper of a MongoDB cursor, is there any way to create a QuerySet with a given cursor?
Note: There is an obvious solution, querying the database again:
queryset = Model.objects.filter(_id__in=[r['_id'] for r in cursor])
But it's rather ugly. The ideal solution would be something like calling the constructor of QuerySet, since it is a wrapper of cursor. But constructor does not accept cursor as an argument.

QuerySet is nothing but just a list. So you can convert the output you get from aggregating into a list like below and directly use that list as QuerySet.
queryset = list(cursor)

Related

Can we add add objects to django queryset like we do to python list. I have an empty django queryset, now I want to add objects to it

This is how I have initialized an Empty Queryset now I want to add objects to it.
How can I do that.
from django.db.models.query import EmptyQuerySet
EmptyQuerySet is a class to checking if a queryset is empty by .none()
A queryset represents a query you make to your database and it have to be translatable to an SQL query.
So you can't add objects to a QuerySet like to python list but you can combine queryset using set operations.
See QuerySet API Reference for more information.

Why django ORM .filter() two way binding my data?

Let's say I store my query result temporarily to a variable
temp_doc = Document.objects.filter(detail=res)
and then I want to insert some data in said model
and will be something like this
p = Document(detail=res)
p.save()
note that res are object from other model to make some FK relation.
For some reason the temp_doc will contain the new data.
Are .filter() supposed to work like that?
Because with .get() the data inside temp_doc don't change
Django Querysets are lazy, this behavior is well documented
QuerySets are lazy – the act of creating a QuerySet doesn’t involve
any database activity. You can stack filters together all day long,
and Django won’t actually run the query until the QuerySet is
evaluated.
So basically that means until you don't ask for data evaluation, database query won't be executed
In your example
temp_doc = Document.objects.filter(detail=res)
p = Document(detail=res)
p.save()
enter code here
evaluating temp_doc now would include newly created Document as database query would return it
simply constructing list would evaluate QuerySet at the start
#evaluation happens here
list(temp_doc) = Document.objects.filter(detail=res)
p = Document(detail=res)
p.save()

Pass a queryset as the argument to __in in django?

I have a list of object ID's that I am getting from a query in an model's method, then I'm using that list to delete objects from a different model:
class SomeObject(models.Model):
# [...]
def do_stuff(self, some_param):
# [...]
ids_to_delete = {item.id for item in self.items.all()}
other_object = OtherObject.objects.get_or_create(some_param=some_param)
other_object.items.filter(item_id__in=ids_to_delete).delete()
What I don't like is that this takes 2 queries (well, technically 3 for the get_or_create() but in the real code it's actually .filter(some_param=some_param).first() instead of the .get(), so I don't think there's any easy way around that).
How do I pass in an unevaluated queryset as the argument to an __in lookup?
I would like to do something like:
ids_to_delete = self.items.all().values("id")
other_object.items.filter(item_id__in=ids_to_delete).delete()
You can, pass a QuerySet to the query:
other_object.items.filter(id__in=self.items.all()).delete()
this will transform it in a subquery. But not all databases, especially MySQL ones, are good with such subqueries. Furthermore Django handles .delete() manually. It will thus make a query to fetch the primary keys of the items, and then trigger the delete logic (and also remove items that have a CASCADE dependency). So .delete() is not done as one query, but at least two queries, and often a larger amount due to ForeignKeys with an on_delete trigger.
Note however that you here remove Item objects, not "unlink" this from the other_object. For this .remove(…) [Django-doc] can be used.
I should've tried the code sample I posted, you can in fact do this. It's given as an example in the documentation, but it says "be cautious about using nested queries and understand your database server’s performance characteristics" and recommends against doing this, casting the subquery into a list:
values = Blog.objects.filter(
name__contains='Cheddar').values_list('pk', flat=True)
entries = Entry.objects.filter(blog__in=list(values))

Prefetch object not working with order_by queryset

Using Django 11 with PostgreSQL db. I have the models as shown below. I'm trying to prefetch a related queryset, using the Prefetch object and prefetch_related without assigning it to an attribute.
class Person(Model):
name = Charfield()
#property
def latest_photo(self):
return self.photos.order_by('created_at')[-1]
class Photo(Model):
person = ForeignKey(Person, related_name='photos')
created_at = models.DateTimeField(auto_now_add=True)
first_person = Person.objects.prefetch_related(Prefetch('photos', queryset=Photo.objects.order_by('created_at'))).first()
first_person.photos.order_by('created_at') # still hits the database
first_person.latest_photo # still hits the database
In the ideal case, calling person.latest_photo will not hit the database again. This will allow me to use that property safely in a list display.
However, as noted in the comments in the code, the prefetched queryset is not being used when I try to get the latest photo. Why is that?
Note: I've tried using the to_attr argument of Prefetch and that seems to work, however, it's not ideal since it means I would have to edit latest_photo to try to use the prefetched attribute.
The problem is with slicing, it creates a different query.
You can work around it like this:
...
#property
def latest_photo(self):
first_use_the_prefetch = list(self.photos.order_by('created_at'))
then_slice = first_use_the_prefetch[-1]
return then_slice
And in case you want to try, it is not possible to use slicing inside the Prefetch(query=...no slicing here...) (there is a wontfix feature request for this somewhere in Django tracker).

Django Query only one field of a model using .extra() and without using .defer() or .only()

I'm using django ORM's exact() method to query only selected fields from a set of models to save RAM. I can't use defer() or only() due to some constraints on the ORM manager I am using (it's not the default one).
The following code works without an error:
q1 = Model.custom_manager.all().extra(select={'field1':'field1'})
# I only want one field from this model
However, when I jsonify the q1 queryset, I get every single field of the model.. so extra() must not have worked, or am I doing something wrong?
print SafeString(serializers.serialize('json', q1))
>>> '{ everything!!!!!}'
To be more specific, the custom manager I am using is django-sphinx. Model.search.query(...) for example.
Thanks.
So, Im not sure if you can do exactly what you want to do. However, if you only want the values for a particular field or a few fields, you can do it with values
It likely does the full query, but the result will only have the values you want. Using your example:
q1 = Model.custom_manager.values('field1', 'field2').all()
This should return a ValuesQuerySet. Which you will not be able to use with serializers.serialize so you will have to do something like this:
from django.utils import simplejson
data = [value for value in q1]
json_dump = simplejson.dumps(data)
Another probably better solution is to just do your query like originally intended, forgetting extra and values and just use the fields kwarg in the serialize method like this:
print SafeString(serializers.serialize('json', q1, fields=('field1', 'field2')))
The downside is that none of these things actually do the same thing as Defer or Only(all the fields are returned from the database), but you get the output you desire.

Categories