We Keep Coding

Building Django Q() objects from other Q() objects, but with relation crossing context

I commonly find myself writing the same criteria in my Django application(s) more than once. I'll usually encapsulate it in a function that returns a Django Q() object, so that I can maintain the criteria in just one place.
I will do something like this in my code:
def CurrentAgentAgreementCriteria(useraccountid):
'''Returns Q that finds agent agreements that gives the useraccountid account current delegated permissions.'''
AgentAccountMatch = Q(agent__account__id=useraccountid)
StartBeforeNow = Q(start__lte=timezone.now())
EndAfterNow = Q(end__gte=timezone.now())
NoEnd = Q(end=None)
# Now put the criteria together
AgentAgreementCriteria = AgentAccountMatch & StartBeforeNow & (NoEnd | EndAfterNow)
return AgentAgreementCriteria
This makes it so that I don't have to think through the DB model more than once, and I can combine the return values from these functions to build more complex criterion. That works well so far, and has saved me time already when the DB model changes.
Something I have realized as I start to combine the criterion from these functions that is that a Q() object is inherently tied to the type of object .filter() is being called on. That is what I would expect.
I occasionally find myself wanting to use a Q() object from one of my functions to construct another Q object that is designed to filter a different, but related, model's instances.
Let's use a simple/contrived example to show what I mean. (It's simple enough that normally this would not be worth the overhead, but remember that I'm using a simple example here to illustrate what is more complicated in my app.)
Say I have a function that returns a Q() object that finds all Django users, whose username starts with an 'a':
def UsernameStartsWithAaccount():
return Q(username__startswith='a')
Say that I have a related model that is a user profile with settings including whether they want emails from us:
class UserProfile(models.Model):
account = models.OneToOneField(User, unique=True, related_name='azendalesappprofile')
emailMe = models.BooleanField(default=False)
Say I want to find all UserProfiles which have a username starting with 'a' AND want use to send them some email newsletter. I can easily write a Q() object for the latter:
wantsEmails = Q(emailMe=True)
but find myself wanting to something to do something like this for the former:
startsWithA = Q(account=UsernameStartsWithAaccount())
# And then
UserProfile.objects.filter(startsWithA & wantsEmails)
Unfortunately, that doesn't work (it generates invalid PSQL syntax when I tried it).
To put it another way, I'm looking for a syntax along the lines of Q(account=Q(id=9)) that would return the same results as Q(account__id=9).
So, a few questions arise from this:
Is there a syntax with Django Q() objects that allows you to add "context" to them to allow them to cross relational boundaries from the model you are running .filter() on?
If not, is this logically possible? (Since I can write Q(account__id=9) when I want to do something like Q(account=Q(id=9)) it seems like it would).

Maybe someone suggests something better, but I ended up passing the context manually to such functions. I don't think there is an easy solution, as you might need to call a whole chain of related tables to get to your field, like table1__table2__table3__profile__user__username, how would you guess that? User table could be linked to table2 too, but you don't need it in this case, so I think you can't avoid setting the path manually.
Also you can pass a dictionary to Q() and a list or a dictionary to filter() functions which is much easier to work with than using keyword parameters and applying &.
def UsernameStartsWithAaccount(context=''):
field = 'username__startswith'
if context:
field = context + '__' + field
return Q(**{field: 'a'})
Then if you simply need to AND your conditions you can combine them into a list and pass to filter:
UserProfile.objects.filter(*[startsWithA, wantsEmails])

What is the difference between a mongoengine.DynamicEmbeddedDocument vs mongoengine.DictField?

A mongoengine.DynamicEmbeddedDocument can be used to leverage MongoDB's flexible schema-less design. It's expandable and doesn't apply type constraints to the fields, afaik.
A mongoengine.DictField similarly allows for use of MongoDB's schema-less nature. In the documentation they simply say (w.r.t. the DictField)
This is similar to an embedded document, but the structure is not defined.
Does that mean, then, the mongoengine.fields.DictField and the mongoengine.DynamicEmbeddedDocument are completely interchangeable?
EDIT (for more information):
mongoengine.DynamicEmbeddedDocument inherits from mongoengine.EmbeddedDocument which, from the code is:
A mongoengine.Document that isn't stored in its own collection. mongoengine.EmbeddedDocuments should be used as fields on mongoengine.Documents through the mongoengine.EmbeddedDocumentField field type.
A mongoengine.fields.EmbeddedDocumentField is
An embedded document field - with a declared document_type. Only valid values are subclasses of EmbeddedDocument.
Does this mean the only thing that makes the DictField and DynamicEmbeddedDocument not totally interchangeable is that the DynamicEmbeddedDocument has to be defined through the EmbeddedDocumentField field type?

From what I’ve seen, the two are similar, but not entirely interchangeable. Each approach may have a slight advantage based on your needs. First of all, as you point out, the two approaches require differing definitions in the document, as shown below.
class ExampleDynamicEmbeddedDoc(DynamicEmbeddedDocument):
pass
class ExampleDoc(Document):
dict_approach = DictField()
dynamic_doc_approach = EmbeddedDocumentField(ExampleDynamicEmbeddedDoc, default = ExampleDynamicEmbeddedDoc())
Note: The default is not required, but the dynamic_doc_approach field will need to be set to a ExampleDynamicEmbeddedDoc object in order to save. (i.e. trying to save after setting example_doc_instance.dynamic_doc_approach = {} would throw an exception). Also, you could use the GenericEmbeddedDocumentField if you don’t want to tie the field to a specific type of EmbeddedDocument, but the field would still need to be point to an object subclassed from EmbeddedDocument in order to save.
Once set up, the two are functionally similar in that you can save data to them as needed and without restrictions:
e = ExampleDoc()
e.dict_approach["test"] = 10
e.dynamic_doc_approach.test = 10
However, the one main difference that I’ve seen is that you can query against any values added to a DictField, whereas you cannot with a DynamicEmbeddedDoc.
ExampleDoc.objects(dict_approach__test = 10) # Returns a QuerySet containing our entry.
ExampleDoc.objects(dynamic_doc_approach__test = 10) # Throws an exception.
That being said, using an EmbeddedDocument has the advantage of validating fields which you know will be present in the document. (We simply would need to add them to the ExampleDynamicEmbeddedDoc definition). Because of this, I think it is best to use a DynamicEmbeddedDocument when you have a good idea of a schema for the field and only anticipate adding fields minimally (which you will not need to query against). However, if you are not concerned about validation or anticipate adding a lot of fields which you’ll query against, go with a DictField.

Best practice for accessing distantly related Django models

Let say we have a long chain of Django models, where each references the one above through a ForeignKey field:
class One(models.Model):
# fields
class Two(models.Model):
one = models.ForeignKey(One)
...
class Ten(models.Model):
nine = models.ForeignKey(Nine)
Good! Now image, if you will, having an instance of the Ten model and wanting to grab the related One instance. This can result in long lines of attribute chaining like this:
ten_instance.nine.eight.seven.six.five.four.three.two.one
I'm wondering what the standard approach would be to this niggling issue. Do we leave it as is, being inherently descriptive and readable. Or do we aim to shorten such a line to make things more simple:
ten_instance.one
 But What's The Best Practice Here? Or is there a more simple solution?

Use Properties
My current approach would be to add a property to the Ten model, abstracting away that attribute chaining:
class Ten(models.Model):
nine = models.ForeignKey(Nine)
#property
def one(self):
return self.nine.eight.seven.six.five.four.three.two.one
I can see a downside to this tactic however, and that's the added mysticism involved. Does the Ten instance actually have a relation to the One model or not? I wouldn't be able to tell without inspecting the model myself.

You probably want to use django-mptt for sophisticated hierarchal models although it can be a bit ott. If you want a simple hierarchy then add a ForeignKey to self:
class Number(models.Model):
parent = models.ForeignKey('self', blank=True, null=True,
related_name='child')
then the query would be something like this based on a unique field, say slug:
Number.objects.get(parent__slug='one')

Python model object validation

I'm writing an interface to be used by two applications. This interface should use some DoSomethingRequest and DoSomethingResponse classes to do the communication.
Is there any library that does some model validation, for example like Django's Model?
I basically want to be able to say something like:
Object A must have a "text" property of type str(), a "number" property of type int(), an "items" property of type list(). In a DRY way.
I'm looking for something like the following, or better:
class MyEmbeddedModelClass(EmbeddedModel):
text = TextField(required = True)
class MyModel(Model):
text = TextField(required = True)
number = IntField(default = 0)
items = ListField(EmbeddedModel)
a = MyModel()
a.text = "aaaa"
a.number = 1
a.items = [
MyEmbeddedModelClass("bbbb"),
MyEmbeddedModelClass("cccc"),
MyEmbeddedModelClass("dddd")
]
a.validate()
I know I can write my own, but I'd rather use a library if available, I'm a bit new to this.

If you want to enforce interfaces, or use design-by-contract, then you probably want the zope.interface library. Despite the name, which reflects its origins in Zope, it's not actually tied to that framework at all and is quite usable outside.

I think decorators could be used for this.
check this link
Combining Descriptors with Class Decorators for Validation
For a different approach check Duck typing

Because python is dynamic, the convention is to require an object to behave like an instance of a particular class rather than enforce a specific type.
Somewhere in your code, preferably at the point where you need to access those properties, but as early as possible assert that the object has those properties and further assert that those properties are what you expect them to be.
This raises an AssertionError exception if the object o, regardless of type, if it is missing the 'someattribute' attribute:
assert(hasattr(o, 'someattribute'))
Further, if o.someattribute is not a string:
assert(isinstance(o.someattribute, basestring))

How can I do dynamic class generation in Python? (or would a series of if/elses be better)

So, I'm writing something and I've come into a roadblock on how to do it (and what is the proper way of doing things). SO, explaining the situation will help be better understand the problem, and hopefully someone will know the answer :) Here it goes:
Basically, I'm writing up some dynamic forms in Python (more specifically Django) - I am using a form factory to generate the form that I want to make. This is all fine and dandy, so far I have been defining the properties of the form in a hardcoded way, basically matching a property to a certain form (a ChoiceField, a Boolean, etc). BUT, what I would like instead of hardcoding these values is essentially create a properties dictionary dynamically, based on what information I pass it...
I basically have an array of "options", and so here are the two methods I am considering:
Have a function for my "options" model/object that will have a series of if/elses. Like:
def get_property():
if value = "artifact": #artifact being one option
return form.BooleanField(label="blah")
else if value = "environment": #environment being another type of option
return form.ChoicesField(label="blah")
etc...
Use a very polymorphic approach. In this way, I mean creating an object based on my "option" object, and will create a new object based on the option. Say maybe something like:
class Base_Property():
value = ""
def __init__(self, option):
value = form.BooleanField()
class Artifact_Property(Base_Property):
def __init__(self, option):
Base_Property.__init__(self, option)
value = form.ChoiceField(choices=some_choices_array())
If option two is the way to go, could someone explain how I can create an object dynamically based on a variable? Like, matching the name of the value (say, Artifact, to match Artifact_Property).
Thanks so much for the help! I am really interested to see what happens to be a proper way - maybe it will spark a debate :)
-Shawn

Have you considered using a dictionary? They're excellent for this sort of conditional.
def get_option(field_type):
options = {
'artifact': forms.BooleanField,
'environment': forms.Choice Field,
}
return options[field_type](label='blah')