We Keep Coding

Using '_id' in Django

I am a bit confused how Django handles '_id' property when we use ORM with some models that use foreign key.
For example:
class CartItem(models.Model):
user = models.ForeignKey('accounts.CustomUser', related_name='carts', on_delete=models.CASCADE, verbose_name='User')
product = models.ForeignKey('pizza.Product', related_name='carts', on_delete=models.CASCADE, verbose_name=_('Product'))
quantity = models.SmallIntegerField(verbose_name=_('Quantity'))
And when I use ORM with 'filter' I can easily use something like:
CartItem.objects.filter(user=1, product=1, quantity=1)
And Django kind of 'see' that I refer to 'id', but when I use exacly the same line of code, but with 'create' instead of 'filter':
CartItem.objects.create(user=1, product=1, quantity=1)
Then it throws an error saying:
Cannot assign "1": "CartItem.user" must be a "CustomUser" instance.
And to create it I need to use:
CartItem.objects.create(user_id=1, product_id=1, quantity=1)
Why is that? Is there some rule here that I don't understand?

This is the database representation of the ForeignKey [Django-doc]. A reference to model object is represented as:
Behind the scenes, Django appends "_id" to the field name to create its database column name. In the above example, the database table for the Car model will have a manufacturer_id column. (You can change this explicitly by specifying db_column) However, your code should never have to deal with the database column name, unless you write custom SQL. You’ll always deal with the field names of your model object.
So you could say that Django will construct a "twin" column, with an _id suffix. This column has the same type as the type of the primary key of the model you target, and that column will thus contain the primary key of the model object you use. Note that you can use a different field to which you target by specifying the to_field=… parameter [Django-doc].
The ForeignKey itself thus does not exists at the database, that is the logic of Django that will use the primary of that object, and store that in a column with, by default, an _id suffix.

Django conditional annotation without filtering

I have a Student model and an Entry model. Each Entry has a foreign key to a Student, a year-stamp, and two numeric values (value1 and value2). I am overriding the get_queryset() method in the StudentAdmin class, and using the Django ORM, I want to annotate a field that we'll call "specialvalue".
Students have at most one Entry for each year, but they might have none, and they might have an Entry for years in the future. The value of "specialvalue" will be equal to Entry__value1 minus Entry__value2 for the Entry for the current year. If the Student has no Entry for the current year, then specialvalue will just be equal to None, and these Students will NOT be removed from the queryset.
How can I do this? At first I tried splitting the queryset into two:
queryset_1 = queryset.filter(entry__year=THIS_YEAR).annotate(specialvalue=...)
queryset_2 = queryset.exclude(entry__year=THIS_YEAR).annotate(specialvalue=Value(None))
and then annotating them separately and merging them with the | pipe operator, but unfortunately this results in the wrong results due to a known bug in Django's ORM.
Thank you!

I think, using Django conditional expression will be applicable solution in your case. Read about Case expression for more information (https://docs.djangoproject.com/en/1.11/ref/models/conditional-expressions/#case). All you need is to check whether an entry is related to a CURRENT_YEAR or not.
Case() accepts any number of When() objects as individual arguments. Other options are provided using keyword arguments. If none of the conditions evaluate to TRUE, then the expression given with the default keyword argument is returned. If a default argument isn’t provided, None is used.
I didn't test this solution but I think it should work:
from django.db.models import Case, When, F, IntegerField
queryset = queryset.annotate(
specialvalue=Case(
When(entry__year=THIS_YEAR,
then=F('entry__value1') - F('entry__value2')
), output_field=IntegerField()
)
).values_list('specialvalue')

How do I define a SQLAlchemy function and use it as default for a table field?

The documentation has this example:
t = Table('test', meta,
Column('abc', MyType, default=func.generate_new_value(), primary_key=True)
)
Where above, when Table.insert() is used, the func.generate_new_value() expression will be pre-executed in the context of a scalar SELECT statement, and the new value will be applied to the subsequent INSERT, while at the same time being made available to the ResultProxy.inserted_primary_key attribute.
However it doesn't say how this func.generate_new_value() is defined.
I need to define a function that will select the max abc element (not a primary key in my case) from all test rows, then increment it by 1, and then pass this value to SQLAlchemy which hopefully can then INSERT it in the new row.

func.ANYTHING() is actually calling an SQL function from your database. 'ANYTHING' is the name of that function. So you will have to define your function directly on your database as an SQL function.
See also:
http://docs.sqlalchemy.org/en/latest/core/sqlelement.html?highlight=func#sqlalchemy.sql.expression.func

Filter by an object in SQLAlchemy

I have a declared model where the table stores a "raw" path identifier of an object. I then have a #hybrid_property which allows directly getting and setting the object which is identified by this field (which is not another declarative model). Is there a way to query directly on this high level?
I can do this:
session.query(Member).filter_by(program_raw=my_program.raw)
I want to be able to do this:
session.query(Member).filter_by(program=my_program)
where my_program.raw == "path/to/a/program"
Member has a field program_raw and a property program which gets the correct Program instance and sets the appropriate program_raw value. Program has a simple raw field which identifies it uniquely. I can provide more code if necessary.
The problem is that currently, SQLAlchemy simply tries to pass the program instance as a parameter to the query, instead of its raw value. This results in a Error binding parameter 0 - probably unsupported type. error.
Either, SQLAlchemy needs to know that when comparing the program, it must use Member.program_raw and match that against the raw property of the parameter. Getting it to use Member.program_raw is done simply using #program.expression but I can't figure out how to translate the Program parameter correctly (using a Comparator?), and/or
SQLAlchemy should know that when I filter by a Program instance, it should use the raw attribute.
My use-case is perhaps a bit abstract, but imagine I stored a serialized RGB value in the database and had a property with a Color class on the model. I want to filter by the Color class, and not have to deal with RGB values in my filters. The color class has no problems telling me its RGB value.

Figured it out by reading the source for relationship. The trick is to use a custom Comparator for the property, which knows how to compare two things. In my case it's as simple as:
from sqlalchemy.ext.hybrid import Comparator, hybrid_property
class ProgramComparator(Comparator):
def __eq__(self, other):
# Should check for case of `other is None`
return self.__clause_element__() == other.raw
class Member(Base):
# ...
program_raw = Column(String(80), index=True)
#hybrid_property
def program(self):
return Program(self.program_raw)
#program.comparator
def program(cls):
# program_raw becomes __clause_element__ in the Comparator.
return ProgramComparator(cls.program_raw)
#program.setter
def program(self, value):
self.program_raw = value.raw
Note: In my case, Program('abc') == Program('abc') (I've overridden __new__), so I can just return a "new" Program all the time. For other cases, the instance should probably be lazily created and stored in the Member instance.

Setting a default value in sqlalchemy

I would like to set a column default value that is based on another table in my SQLAlchemy model.
Currently I have this:
Column('version', Integer, default=1)
What I need is (roughly) this:
Column('version', Integer, default="SELECT MAX(1, MAX(old_versions)) FROM version_table")
How can I implement this in SQLAlchemy?

The documentation gives the following possibilities for default:
A scalar, Python callable, or ClauseElement representing the default
value for this column, which will be invoked upon insert if this
column is otherwise not specified in the VALUES clause of the insert.
You may look into using a simple function, or you may just be able to use a select() object.
In your case, maybe something along the lines of:
from sqlalchemy.sql import select, func
...
Column('version', Integer, default=select([func.max(1,
func.max(version_table.c.old_versions))]))

You want server_default
Column('version', Integer, server_default="SELECT MAX(1, MAX(old_versions)) FROM version_table")

If you want to use a DML statement to generate the default value, you can simply use the text method to indicate that you are passing DML. You may also need an extra set of parentheses if the engine wants to write this inside a VALUES clause , e.g.:
from sqlachemy import text
Column('version', Integer, default=text("(SELECT MAX(1, MAX(old_versions)) FROM version_table)"))
I've used this technique to use a sequence to override the server default ID generation, e.g.:
Column('version', Integer, default=text("NEXT VALUE FOR someSequence"))