We Keep Coding

Django remove duplicates from .values_list query while preserving order

I have a model say MyModel which contains a CharField type. The model has a default meta ordering which should be preserved. I am using the following query to get the list of types -
MyModel.objects.all().values_list('type', flat=True).distinct()
However, the types are getting repeated. I can do .order_by('type').distinct() but that will change the ordering which I don't want. Is there any way to get the list of types in order without manually creating a list in python? Alternative faster solutions are also welcome.
Django version - 1.11

Distinct is not matching with type because you don't specified it
use this code
MyModel.objects.all().values_list('type', flat=True).distinct("type")
instead of this code
MyModel.objects.all().values_list('type', flat=True).distinct()

You can try for this
MyModel.objects.all().values('type', flat=True).order_by('type').distinct()
it will work for you

You can do this in 2 steps:
First, get the id of the records with unique types and save them in a list:
ids = list(MyModel.objects.values_list('id', flat=True).order_by('type').distinct('type'))
Then do the filter using the ids:
MyModel.objects.values_list('type', flat=True).filter(id__in=ids)

Django - calling attribute from queryset with a string

I'm trying to loop over different query sets while not repeating myself too much and have encountered a problem using the queryset class.
This is not necessarily completely a Django-problem.
What I'm trying to do is to use my keylist, which corresponds to a django model's column names, to create a list of the data from those column names, what i want to do is something like this:
if needthisdata==1:
needdata=['column1', 'column2', 'column3']
else:
needdata=['column1', 'column4', 'column7']
entry=djangomodel.get.all().filter(identifier='id')
dictitems=[]
for n in range(0, len(needdata)):
if n==0:
dictitems=[entry.needdata[n]]
else:
dictitems.append(entry.needdata[n])
Which of course doesn't work since the queryset doesn't have a need data attribute, is there some way to call an attribute for a class with a string in this way?

A valid Django statement to obtain a single entry
First of all, there are some semantical problems here:
itentifier should probably be identifier, id, or pk;
you use .all immedately instead of first obtaining a manager (probably .objects); and
you here use a .filter(..) on the queryset to filter on an identifier, but usually this should be a .get(..), since by using a filter, zero, one or more results can be returned in an iterable.
entry = djangomodel.objects.get(id=some_id)
So now we obtain a single entry, but that of course does not resolve
obtaining the columns.
If all elements are real Django columns
In case the columns are real Django fields (so no #propertys, etc.) then we can use values_list, and perform a list(..) constructor on it:
dictitems = list(djangomodel.objects.values_list(*needdata).get(id=some_id))
If case some elements are #propertys
In case not all those fields are real Django fields, then we can use attrgetter instead:
from operator import attrgetter
dictitems = list(attrgetter(*needdata)(djangomodel.objects.get(id=some_id)))

Length of Django queryset result changes when coerced to a list

I have a reasonably complex queryset thus, which rolls up data by isoweek:
>>> MyThing.objects.all().count()
30000
>>> qs = MyThing.objects.all().order_by('date').annotate(
dw=DateWeek('date'), # uses WEEK function
dy=ExtractYear('date')
).values(
'dy','dw','group_id'
).annotate(
sum_count=Sum('count')
).values_list('dw', 'dy', 'group_id', 'sum_count')
>>> qs.count()
2000
So far so good. The problem is when I coerce this queryset into a list:
>>> len(list(qs))
30000
Why is this happening? How can I get the list of grouped values that the queryset purports to have when I count() it directly?

To solve this problem, remove the .order_by('date'). Although it is not included in the output, the database backend is still considering it at every row, causing the number of rows to inflate like that.
If you want to order the output, .order_by('dy', 'dw') after adding those annotations.
You can also add an .order_by() with no arguments to clear any ordering set previously, for instance from the Model class definition default ordering.
🦆
The reason for this behavior is explained in the django docs:
Any fields used in an order_by() call are included in the SQL SELECT
columns. This can sometimes lead to unexpected results when used in
conjunction with distinct(). If you order by fields from a related
model, those fields will be added to the selected columns and they may
make otherwise duplicate rows appear to be distinct. Since the extra
columns don’t appear in the returned results (they are only there to
support ordering), it sometimes looks like non-distinct results are
being returned.
Similarly, if you use a values() query to restrict the columns
selected, the columns used in any order_by() (or default model
ordering) will still be involved and may affect uniqueness of the
results.
The moral here is that if you are using distinct() be careful about
ordering by related models. Similarly, when using distinct() and
values() together, be careful when ordering by fields not in the
values() call.

Django returning said rows in a queryset

Using a queryset in Django (in a view) I only want to get said rows 51-100. i.e. I only want it to return these rows.
is this possible and how within .
objectQuerySet = Recipient.objects.filter(incentiveid=incentive).order_by('fullname')
I don't want to use any paging system etc this is just a one time thing?
Thank you

You can use slicing to execute a LIMIT OFFSET statement:
objectQuerySet = Recipient.objects.filter(incentiveid=incentive).order_by('fullname')[51:100]

Djapian - filtering results

I use Djapian to search for object by keywords, but I want to be able to filter results. It would be nice to use Django's QuerySet API for this, for example:
if query.strip():
results = Model.indexer.search(query).prefetch()
else:
results = Model.objects.all()
results = results.filter(somefield__lt=somevalue)
return results
But Djapian returns a ResultSet of Hit objects, not Model objects. I can of course filter the objects "by hand", in Python, but it's not realistic in case of filtering all objects (when query is empty) - I would have to retrieve the whole table from database.
Am I out of luck with using Djapian for this?

I went through its source and found that Djapian has a filter method that can be applied to its results. I have just tried the below code and it seems to be working.
My indexer is as follows:
class MarketIndexer( djapian.Indexer ):
fields = [ 'name', 'description', 'tags_string', 'state']
tags = [('state', 'state'),]
Here is how I filter results (never mind the first line that does stuff for wildcard usage):
objects = model.indexer.search(q_wc).flags(djapian.resultset.xapian.QueryParser.FLAG_WILDCARD).prefetch()
objects = objects.filter(state=1)
When executed, it now brings Markets that have their state equal to "1".

I dont know Djapian, but i am familiar with xapian. In Xapian you can filter the results with a MatchDecider.
The decision function of the match decider gets called on every document which matches the search criteria so it's not a good idea to do a database query for every document here, but you can of course access the values of the document.
For example at ubuntuusers.de we have a xapian database which contains blog posts, forum posts, planet entries, wiki entries and so on and each document in the xapian database has some additional access information stored as value. After the query, an AuthMatchDecider filters the potential documents and returns the filtered MSet which are then displayed to the user.
If the decision procedure is as simple as somefield < somevalue, you could also simply add the value of somefield to the values of the document (using the sortable_serialize function provided by xapian) and add (using OP_FILTER) an OP_VALUE_RANGE query to the original query.