We Keep Coding

Runtime Foreign Key vs Integerfield

I have a problem. I already have two solution for my problem, but i was wondering which of those is the faster solution.
I guess that the second solution is not only more convienient- to use but also faster, but i want to be sure, so thats the reason why im asking.
My problem is i want to group multiple rows together. The group won't hold any meta data. So im only interested in runtime.
On the one hand i can use a Integer field and filter it later on when i need to get all entries that belong to the group. I guess runtime of O(n).
class SingleEntries(models.Model):
name = models.CharField(max_length=20)
group = models.IntegerField(null=True)
def find_all_group_members(id):
return SingleEntries.objects.filter(group=id)
The second solution and probably the more practicle way would be to create a foreign key to another model only using the pk there.
Then i can use the reverse relation to find all the entries that belong to the group.
class Group(models.Model):
id = models.AutoField(primary_key=True)
class SingleEntries(models.Model):
name = models.CharField(max_length=20)
group = models.ForeignKey(Group,on_delete=models.CASCADE,null=True)
def find_all_group_members(id):
return Group.objects.get(id=id).singleentries_set.all()

The first is more efficient, since this will use one query, whereas the latter will first fetch the Group, and then another one for the SingleEntries.
Indeed, if you work with:
SingleEntries.objects.filter(group=id)
this will make a simple query:
SELECT appname_singleentries.*
FROM appname_singleentries
WHERE appname_singleentries.group_id = id
It thus does not first fetch the Group into memory.
The latter will however make two queries. Indeed, it will first make a query to retrieve the Group, and then it will make a query like the one above to fetch the SingleEntries.
The two are also semantically not entirely the same: if there is no such group, then the former will return an empty QuerySet, whereas the latter will raise a Group.DoesNotExists exception.
But you can model this with:
class Group(models.Model):
pass
class SingleEntries(models.Model):
name = models.CharField(max_length=20)
group = models.ForeignKey(Group,on_delete=models.CASCADE,null=True)
def find_all_group_members(id):
return SingleEntries.objects.filter(group_id=id)
So you can use a Group model without having to retrieve the Group first.

If the groups are static in nature, that means if you don't see more groups coming to your system, you can use choices in Django.
Define choices as below
class GroupType(models.IntegerChoices):
GROUP_0 = 0, "Group 0 name"
GROUP_1 = 1, "Group 1 name"
GROUP_2 = 2, "Group 2 name"
And use it as choices field in the SingleEntries model as below
class SingleEntries(models.Model):
name = models.CharField(max_length=20)
group = models.IntegerField(choices=GroupChoices.choices, default=<set default here>)
If the groups are dynamic, meaning users can create groups whenever they want, in that case, go with your second approach of having another model for group.

django: order a QuerySet

I have a view like this:
def profile (request):
articles = Post.thing.all()
newSet = set()
def score():
for thing in articles:
Val = some calculation...
....
newSet.update([(thing,Val)])
score()
context = {
'articles': articles.order_by('id'),
'newSet':newSet,
}
return render(request,'my_profile/my_profile.html',context)
and the outcome is a Queryset which looks like this:
set([(<thing: sfd>, 1), (<thing: quality>, 0), (<thing: hello>, -1), (<thing: hey>, 4), (<thing: test>, 0)
I am now trying to order the set by the given Values so its a list which starts with the highest Value, but when i do newSet.order_by/filter/split/join
it does not work since 'set' object has no attribute join/filter/split.
Can anybody give me a hint how to sort the querySet i could not find anything helpful on my own.
I need this to work out in the view so it cannot/should not be done in the model. Thanks for any advise.

the outcome is a Queryset which looks like this
Actually this is a set (python builtin type), not a QuerySet (django's orm type).
set is an unordered collection type. To "sort" it you first have to turn it into a list - which is actually as simple as newlist = list(newset),
then you can sort the list in-place with newlist.sort(). Since you want this list to be sorted on it's items second elements, you'll need to use the key argument to tell sort on what you want to sort:
newlist.sort(key=lambda item: item[1])
or you can just change your score() function to store (score, obj) tuples instead in which case list.sort() will naturally do the RightThing (it will sort your tuples on their first item).
While we're at it: instead of calling newSet.update() with a list of a single item, you can just use newSet.add() instead, ie:
def score():
for thing in articles:
val = some calculation...
....
newset.add((thing, val))
# or if you don't want to specify a callback
# for `list.sort()`:
# newset.add((val, thing))
And finally: do you really need a set at all here ? Why not using a list right from the start ?

I think you might be slightly confused here between a set, a list and a QuerySet? A set is unordered, while a list is not. Sets don't expose the methods you listed above (filter, order_by, split, join). A QuerySet is a Django-specific class which has many useful methods.
I think it would be simpler to make newSet a list of tuples, and then to order that list by value using list.sort(key=lambda x: x[1]).
If your calculation of val is eligible for it though, I'd recommend using annotate and then doing away with newDict or newSet, and simply pass back the queryset of articles, which would be much simpler, maybe faster, and orderable by using articles.order_by('value'). If you post the calculation of val, I'll try to tell you if that's feasible.

Create an instance that represents the average of multiple instances

I have a Review Model like the one defined below (I removed a bunch of the fields in REVIEW_FIELDS). I want to find the average of a subset of the attributes and populate a ModelForm with the computed information.
REVIEW_FIELDS = ['noise']
class Review(models.Model):
notes = models.TextField(null=True, blank=True)
CHOICES = ((1, u'Quiet'), (2, u'Kinda Quiet'), (3, u'Loud'))
noise = models.models.IntegerField('Noise Level', blank-True, null=True, choices=CHOICES)
class ReviewForm(ModelForm):
class Meta:
model = Review
fields = REVIEW_FIELDS
I can easily add more fields to the model, and then add them to the REVIEW_FIELDS list. Then I can easily iterate over them in javascript, etc. In my view, I want to compute the average of a bunch of the integer fields and populate a ReviewForm with the attribute values. How can I do something like this?
stats = {}
for attr in REVIEW_FIELDS:
val = reviews.aggregate(Avg(attr)).values()[0]
if val:
stats[attr] = int(round(val,0))
r.attr = stats[attr]
r = Review()
f = ReviewForm(r)
How can I create a ReviewForm with the average values without hard-coding the attribute values? I'd like to be able to add the field, add the value to the list, and have it automatically added to the set of statistics computed.
If I'm doing something fundamentally wrong, please let me know. I'm relatively new to django, so I might be re-inventing functionality that already exists.

After looking at the docs, I pass a dictionary to the ReviewForm when instantiating it:
f = ReviewForm(stats)
It seems to work pretty well! If anyone has any suggestions on a better way to do this, I'm all ears!

django error 'too many values to unpack'

I'm learning Django by building a simple recipes app. I have a 1 table model using the 'choices' field option for recipe categories rather than using a 2nd 'categories' table and a foreign key relationship. So i created db table via syncdb and then loaded table with test data. When i go to admin and click on the 'Recipes' link in an attempt to view recipes i get the following error:
Template error
In template /var/lib/python-support/python2.6/django/contrib/admin/templates/admin/change_list.html, error at line 34
Caught an exception while rendering: too many values to unpack
If anyone can shed light on this cryptic error that would be great. Db is Sqlite. Django version is 1.0. The model is listed below:
from django.db import models
class Recipe(models.Model):
CATEGORY_CHOICES = (
(1, u'Appetizer'),
(2, u'Bread'),
(3, u'Dessert'),
(4, u'Drinks'),
(5, u'Main Course'),
(6, u'Salad'),
(7, u'Side Dish'),
(8, u'Soup'),
(9, u'Sauce/Marinade'),
(10, u'Other'),
)
name = models.CharField(max_length=255)
submitter = models.CharField(max_length=40)
date = models.DateTimeField()
category = models.SmallIntegerField(choices=CATEGORY_CHOICES)
ingredients = models.TextField()
directions = models.TextField()
comments = models.TextField(null=True, blank=True)

Edit: Updated in light of kibibu's correction.
I have encountered what I believe is this same error, producing the message:
Caught ValueError while rendering: too many values to unpack
My form class was as follows:
class CalcForm(forms.Form):
item = forms.ChoiceField(choices=(('17815', '17816')))
Note that my choices type here a tuple. Django official documentation reads as follows for the choices arg:
An iterable (e.g., a list or tuple) of 2-tuples to use as choices for
this field. This argument accepts the same formats as the choices
argument to a model field.
src: https://docs.djangoproject.com/en/1.3/ref/forms/fields/#django.forms.ChoiceField.choices
This problem was solved by my observing the documentation and using a list of tuples:
class CalcForm(forms.Form):
item = forms.ChoiceField(choices=[('17815', '17816')])
Do note that while the docs state any iterable of the correct form can be used, a tuple of 2-tuples did not work:
item = forms.ChoiceField(choices=(('17815', '17816'), ('123', '456')))
This produced the same error as before.
Lesson: bugs happen.

You should use a ChoiceField instead of SmallIntegerField

If I had to guess, it's because whatever is in the administrative template expects a list of tuples, but you've instead supplied a tuple of tuples (hence the "too many values"). Try replacing with a list instead:
CATEGORY_CHOICES = [ # Note square brackets.
(1, u'Appetizer'),
(2, u'Bread'),
(3, u'Dessert'),
(4, u'Drinks'),
(5, u'Main Course'),
(6, u'Salad'),
(7, u'Side Dish'),
(8, u'Soup'),
(9, u'Sauce/Marinade'),
(10, u'Other'),
]

Per http://code.djangoproject.com/ticket/972 , you need to move the assignment CATEGORY_CHOICES = ... outside the class statement.

I got it working. Most of the 'too many values to unpack' errors that i came across while googling were Value Error types. My error was a Template Syntax type. To load my recipe table i had imported a csv file. I was thinking maybe there was a problem somewhere in the data that sqlite allowed on import. So i deleted all data and then added 2 recipes manually via django admin form. The list of recipes loaded after that.
thanks.

I just had the same problem... my cvs file came from ms excel and the date fields gotten the wrong format at saving time. I change the format to something like '2010-05-04 13:05:46.790454' (excel gave me 5/5/2010 10:05:47) and voilaaaa no more 'too many values to unpack’

kibibu's comment to Kreychek's answer is correct. This isn't a Django issue but rather an interesting aspect of Python. To summarize:
In Python, round parentheses are used for both order of operations and tuples. So:
foo = (2+2)
will result in foo being 4, not a tuple who's first and only element is 4: 4
foo = (2+2, 3+3)
will result in foo being a two-dimensional tuple: (4,6)
To tell Python that you want to create a one-dimensional tuple instead of just denoting the order of operations, use a trailing comma:
foo = (2+2,)
will result in foo being a one-dimensional tuple who's first and only element is 4: (4,)
So for your scenario:
class CalcForm(forms.Form):
item = forms.ChoiceField(choices=(('17815', '17816'),))
would give what you want. Using a list is a great solution too (more Pythonic in my opinion), but hopefully this answer is informative since this can come up in other cases.
For example:
print("foo: %s" % (foo))
may give an error if foo is an iterable, but:
print("foo: %s" % (foo,))
or:
print("foo: %s" % [foo])
will properly convert foo to a string whether it's an iterable or not.
Documentation: http://docs.python.org/2/tutorial/datastructures.html#tuples-and-sequences

This error IMO occurs when a function returns three values, and you assign it to 2. e.g.:
def test():
a=0
b=0
c=0
....
a,b=test() <---three values returned, but only assigning to 2.
a,b,c=test() <---three values returned, assigned to 3.OK

Filter by property

Is it possible to filter a Django queryset by model property?
i have a method in my model:
#property
def myproperty(self):
[..]
and now i want to filter by this property like:
MyModel.objects.filter(myproperty=[..])
is this somehow possible?

Nope. Django filters operate at the database level, generating SQL. To filter based on Python properties, you have to load the object into Python to evaluate the property--and at that point, you've already done all the work to load it.

I might be misunderstanding your original question, but there is a filter builtin in python.
filtered = filter(myproperty, MyModel.objects)
But it's better to use a list comprehension:
filtered = [x for x in MyModel.objects if x.myproperty()]
or even better, a generator expression:
filtered = (x for x in MyModel.objects if x.myproperty())

Riffing off #TheGrimmScientist's suggested workaround, you can make these "sql properties" by defining them on the Manager or the QuerySet, and reuse/chain/compose them:
With a Manager:
class CompanyManager(models.Manager):
def with_chairs_needed(self):
return self.annotate(chairs_needed=F('num_employees') - F('num_chairs'))
class Company(models.Model):
# ...
objects = CompanyManager()
Company.objects.with_chairs_needed().filter(chairs_needed__lt=4)
With a QuerySet:
class CompanyQuerySet(models.QuerySet):
def many_employees(self, n=50):
return self.filter(num_employees__gte=n)
def needs_fewer_chairs_than(self, n=5):
return self.with_chairs_needed().filter(chairs_needed__lt=n)
def with_chairs_needed(self):
return self.annotate(chairs_needed=F('num_employees') - F('num_chairs'))
class Company(models.Model):
# ...
objects = CompanyQuerySet.as_manager()
Company.objects.needs_fewer_chairs_than(4).many_employees()
See https://docs.djangoproject.com/en/1.9/topics/db/managers/ for more.
Note that I am going off the documentation and have not tested the above.

Looks like using F() with annotations will be my solution to this.
It's not going to filter by #property, since F talks to the databse before objects are brought into python. But still putting it here as an answer since my reason for wanting filter by property was really wanting to filter objects by the result of simple arithmetic on two different fields.
so, something along the lines of:
companies = Company.objects\
.annotate(chairs_needed=F('num_employees') - F('num_chairs'))\
.filter(chairs_needed__lt=4)
rather than defining the property to be:
#property
def chairs_needed(self):
return self.num_employees - self.num_chairs
then doing a list comprehension across all objects.

I had the same problem, and I developed this simple solution:
objects = [
my_object
for my_object in MyModel.objects.all()
if my_object.myProperty == [...]
]
This is not a performatic solution, it shouldn't be done in tables that contains a large amount of data. This is great for a simple solution or for a personal small project.

PLEASE someone correct me, but I guess I have found a solution, at least for my own case.
I want to work on all those elements whose properties are exactly equal to ... whatever.
But I have several models, and this routine should work for all models. And it does:
def selectByProperties(modelType, specify):
clause = "SELECT * from %s" % modelType._meta.db_table
if len(specify) > 0:
clause += " WHERE "
for field, eqvalue in specify.items():
clause += "%s = '%s' AND " % (field, eqvalue)
clause = clause [:-5] # remove last AND
print clause
return modelType.objects.raw(clause)
With this universal subroutine, I can select all those elements which exactly equal my dictionary of 'specify' (propertyname,propertyvalue) combinations.
The first parameter takes a (models.Model),
the second a dictionary like:
{"property1" : "77" , "property2" : "12"}
And it creates an SQL statement like
SELECT * from appname_modelname WHERE property1 = '77' AND property2 = '12'
and returns a QuerySet on those elements.
This is a test function:
from myApp.models import myModel
def testSelectByProperties ():
specify = {"property1" : "77" , "property2" : "12"}
subset = selectByProperties(myModel, specify)
nameField = "property0"
## checking if that is what I expected:
for i in subset:
print i.__dict__[nameField],
for j in specify.keys():
print i.__dict__[j],
print
And? What do you think?

i know it is an old question, but for the sake of those jumping here i think it is useful to read the question below and the relative answer:
How to customize admin filter in Django 1.4

It may also be possible to use queryset annotations that duplicate the property get/set-logic, as suggested e.g. by #rattray and #thegrimmscientist, in conjunction with the property. This could yield something that works both on the Python level and on the database level.
Not sure about the drawbacks, however: see this SO question for an example.