We Keep Coding

Extract only the data from a Django Query Set

I am working to learn Django, and have built a test database to work with. I have a table that provides basic vendor invoice information, so, and I want to simply present a user with the total value of invoices that have been loaded to into the database. I found that the following queryset does sum the column as I'd hoped:
total_outstanding: object = Invoice.objects.aggregate(Sum('invoice_amount'))
but the result is presented on the page in the following unhelpful way:
Total $ Outstanding: {'invoice_amount__sum': Decimal('1965')}
The 1965 is the correct total for the invoices that I populated the database with, so the queryset is pulling what I want it to, but I just want to present that portion of the result to the user, without the other stuff.
Someone else asked a similar question (basically the same) here: how-to-extract-data-from-django-queryset, but the answer makes no sense to me, it is just:
k = k[0] = {'name': 'John'}
Queryset is list .
Can anyone help me with a plain-English explanation of how I can extract just the numerical result of that query for presentation to a user?

What you here get is a dictionary that maps the name of the aggregate to the corresponding value. You can use subscripting to obtain the corresponding value:
object = Invoice.objects.aggregate(
Sum('in,voice_amount')
)['invoice_amount__sum']

Queryset only get one (any) record for each foreign key

Say I have the following models:
class Manufacturer(Model):
name = models.CharField(max_length=255)
origin = models.CharField(max_length=255)
class Car(Model):
model = models.CharField(max_length=255)
manf = models.ForeignKey(Manufacturer, related_name="cars")
And then I want to run the following query:
usa_manfs = Manufacturer.objects.filter(origin='USA')
usa_manf_cars = Car.objects.filter(manf__in=usa_manfs)
However I would only like to have 1 Car per Manufacturer in the QuerySet.
I know I could go the other route and do something like usa_manfs.cars[0] however wouldn't this mean I need to do a query for each Manufacturer to get all related cars?
To clarify, I don't need any control over which instance of Car is retrieved in relation to each Manufacturer. I simply need to end up with a list of Cars wherein the Manufacturer is unique.
SOLUTION FOR NOW
After much deliberation I've decided to go for a Python solution and prevent duplicates in the loop.
The data I'm working with is likely to have relatively few duplicates (in which my example possibly doesn't hold up) on the ForeignKey field and since I'm not able to get a solution working that keeps it within Django ORM, I'll simply track the ForeignKey and exclude repeats in a loop. Since I'm looping over them anyway, I don't think I'm creating much more work overall.
e.g.:
cars = list(Car.objects.all())
final_list = []
used_before = []
for car in cars:
if not car.manf_id in used_before:
used_before.append(car.manf_id)
doStuff(car)
final_list.append(car)
If a better solution comes along I'll accept that instead.

I simply need to end up with a list of Cars wherein the
Manufacturer is unique.
How about using distinct:
cars = Car.objects.order_by('manf').distinct('manf')
Since the above is only supported on PostgreSQL, you have to work around it:
m = Manufacturer.objects.all()
cars = []
for i in m:
if Car.objects.filter(manf=i).exists():
cars.append(Car.objects.filter(manf=i).order_by('?')[1])
This will give you one random car for each manufacturer.

I am not sure how we can do it with orm. Can tell how we can achieve it through direct sql:
from django.db import connection
cursor = connection.cursor()
cursor.execute("select c.model from myapp_car c, myapp_manufacturer m where m.origin='USA' and m.id=c.manf_id group by m.id")

DJANGO Aggregate with Foreignkey

I'm trying to do a group by clause on a foreign key but am having a couple of issues.
The code;
status_applications = DevelopmentAssessment.objects.values('status').annotate(total = Count('status'))
The field status only returns an integer for the status value instead of the related object. Is there a way to have it return the related object so I can use the field names in the related table in a for loop?
Cheers,
Ben

You basically already have an answer.
status_applications is a QuerySet that contains a set of related objects (statuses) you are asking for.
status_applications[0] would be the first, [1] second and so on.
To access the count of each simply do:
for status in status_applications:
print status, status.total
If you need the whole rows of the table then simply don't use values or leave it empty:
applications = DevelopmentAssessment.objects.annotate(total = Count('status'))
or
applications = DevelopmentAssessment.objects.values().annotate(total = Count('status'))

How can I make a Django query for the first occurrence of a foreign key in a column?

Basically, I have a table with a bunch of foreign keys and I'm trying to query only the first occurrence of a particular key by the "created" field. Using the Blog/Entry example, if the Entry model has a foreign key to Blog and a foreign key to User, then how can I construct a query to select all Entries in which a particular User has written the first one for the various Blogs?
class Blog(models.Model):
...
class User(models.Model):
...
class Entry(models.Model):
blog = models.Foreignkey(Blog)
user = models.Foreignkey(User)
I assume there's some magic I'm missing to select the first entries of a blog and that I can simple filter further down to a particular user by appending:
query = Entry.objects.magicquery.filter(user=user)
But maybe there's some other more efficient way. Thanks!

query = Entry.objects.filter(user=user).order_by('id')[0]
Basically order by id (lowest to highest), and slice it to get only the first hit from the QuerySet.
I don't have a Django install available right now to test my line, so please check the documentation if somehow I have a type above:
order by
limiting querysets
By the way, interesting note on 'limiting queysets" manual section:
To retrieve a single object rather
than a list (e.g. SELECT foo FROM bar
LIMIT 1), use a simple index instead
of a slice. For example, this returns
the first Entry in the database, after
ordering entries alphabetically by
headline:
Entry.objects.order_by('headline')[0]
EDIT: Ok, this is the best I could come up so far (after yours and mine comment). It doesn't return Entry objects, but its ids as entry_id.
query = Entry.objects.values('blog').filter(user=user).annotate(Count('blog')).annotate(entry_id=Min('id'))
I'll keep looking for a better way.

Ancient question, I realise - #zalew's response is close but will likely result in the error:
ProgrammingError: SELECT DISTINCT ON expressions must match initial
ORDER BY expressions
To correct this, try aligning the ordering and distinct parts of the query:
Entry.objects.filter(user=user).distinct("blog").order_by("blog", "created")
As a bonus, in case of multiple entries being created at exactly the same time (unlikely, but you never know!), you could add determinism by including id in the order_by:
To correct this, try aligning the ordering and distinct parts of the query:
Entry.objects.filter(user=user).distinct("blog").order_by("blog", "created", "id")

Can't test it in this particular moment
Entry.objects.filter(user=user).distinct("blog").order_by("id")

In Django, how does one filter a QuerySet with dynamic field lookups?

Given a class:
from django.db import models
class Person(models.Model):
name = models.CharField(max_length=20)
Is it possible, and if so how, to have a QuerySet that filters based on dynamic arguments? For example:
# Instead of:
Person.objects.filter(name__startswith='B')
# ... and:
Person.objects.filter(name__endswith='B')
# ... is there some way, given:
filter_by = '{0}__{1}'.format('name', 'startswith')
filter_value = 'B'
# ... that you can run the equivalent of this?
Person.objects.filter(filter_by=filter_value)
# ... which will throw an exception, since `filter_by` is not
# an attribute of `Person`.

Python's argument expansion may be used to solve this problem:
kwargs = {
'{0}__{1}'.format('name', 'startswith'): 'A',
'{0}__{1}'.format('name', 'endswith'): 'Z'
}
Person.objects.filter(**kwargs)
This is a very common and useful Python idiom.

A simplified example:
In a Django survey app, I wanted an HTML select list showing registered users. But because we have 5000 registered users, I needed a way to filter that list based on query criteria (such as just people who completed a certain workshop). In order for the survey element to be re-usable, I needed for the person creating the survey question to be able to attach those criteria to that question (don't want to hard-code the query into the app).
The solution I came up with isn't 100% user friendly (requires help from a tech person to create the query) but it does solve the problem. When creating the question, the editor can enter a dictionary into a custom field, e.g.:
{'is_staff':True,'last_name__startswith':'A',}
That string is stored in the database. In the view code, it comes back in as self.question.custom_query . The value of that is a string that looks like a dictionary. We turn it back into a real dictionary with eval() and then stuff it into the queryset with **kwargs:
kwargs = eval(self.question.custom_query)
user_list = User.objects.filter(**kwargs).order_by("last_name")

Additionally to extend on previous answer that made some requests for further code elements I am adding some working code that I am using
in my code with Q. Let's say that I in my request it is possible to have or not filter on fields like:
publisher_id
date_from
date_until
Those fields can appear in query but they may also be missed.
This is how I am building filters based on those fields on an aggregated query that cannot be further filtered after the initial queryset execution:
# prepare filters to apply to queryset
filters = {}
if publisher_id:
filters['publisher_id'] = publisher_id
if date_from:
filters['metric_date__gte'] = date_from
if date_until:
filters['metric_date__lte'] = date_until
filter_q = Q(**filters)
queryset = Something.objects.filter(filter_q)...
Hope this helps since I've spent quite some time to dig this up.
Edit:
As an additional benefit, you can use lists too. For previous example, if instead of publisher_id you have a list called publisher_ids, than you could use this piece of code:
if publisher_ids:
filters['publisher_id__in'] = publisher_ids

Django.db.models.Q is exactly what you want in a Django way.

This looks much more understandable to me:
kwargs = {
'name__startswith': 'A',
'name__endswith': 'Z',
***(Add more filters here)***
}
Person.objects.filter(**kwargs)

A really complex search forms usually indicates that a simpler model is trying to dig it's way out.
How, exactly, do you expect to get the values for the column name and operation?
Where do you get the values of 'name' an 'startswith'?
filter_by = '%s__%s' % ('name', 'startswith')
A "search" form? You're going to -- what? -- pick the name from a list of names? Pick the operation from a list of operations? While open-ended, most people find this confusing and hard-to-use.
How many columns have such filters? 6? 12? 18?
A few? A complex pick-list doesn't make sense. A few fields and a few if-statements make sense.
A large number? Your model doesn't sound right. It sounds like the "field" is actually a key to a row in another table, not a column.
Specific filter buttons. Wait... That's the way the Django admin works. Specific filters are turned into buttons. And the same analysis as above applies. A few filters make sense. A large number of filters usually means a kind of first normal form violation.
A lot of similar fields often means there should have been more rows and fewer fields.