This question gets me close to what I want to do but I am still in need of further understanding. Django. Q objects dynamicaly generate
I have a view in django that looks to see if any query params have been sent to the url. I am expecting some query objects to have multipule values.
ie. domain.com/?neighborhood=Logan Square&neighborhood=River North
I grab queryparams and put them in a list. I am now trying to iterate through that list and filter through the query params using or logic.
https://docs.djangoproject.com/en/1.11/topics/db/queries/#complex-lookups-with-q
for this I know I need to use Q objects.
the proper code for this is:
Q(neighborhood='Logan Square') | Q(neighborhood='River North')
so what I need to do is
1 adding a query Q object dynamically and then also adding the | operator dynamically for all objects in the for loop.
You don't need this at all. You can simply use __in:
MyModel.objects.filter(neighborhood__in=request.GET.getlist('neighborhood'))
If I understood your question correctly.
"You may, or may NOT get the query params."?
query = ModelName.objects.none() # setting a Q() instance (maybe skippable)
for a_item in neighbourhood_list:
query |= (Q(neighborhood = a_item))
query = ModelName.objects.filter(query)
You might have to test it, and edit this a little bit, but will do the trick you want.
Related
Sorry about the title being confusing it was hard to figure out how to word the question.
Currently I have a sqllite db with some users in it they have a first name, last name, dob, high school, and high school class. The db is connected to flask using sqlalchemy. What I'm wondering is for my search function I have 4 inputs and I want to have it so if an input isn't used then it won't be used in the search query. Say the person searches for the last name and high school I want it to search just using those parameters. I've tried doing this using a bunch of if statements but it seems messy there must be a better way. Below is the query that I use but it only works if all 4 are filled. Is there a better way than a bunch of if statements with different queries? I've looked around and haven't found anything.
userq=User.query.filter_by(first_name=fname_strip,last_name=lname_strip,hs_class=hs_class_strip).all()
You can try if/else statements like the following:
q = User.query.filter_by(first_name=first_name)
if lname_strip:
q = q.filter_by(last_name=lname_strip)
if hs_class_strip:
q= q.filter_by(hs_class=hs_class_strip)
# Execute the query
q.all()
Updated needs the q to be an assignment.
Okay so what I did was go through and make an if statement like you said but made them into different vars. Then check to see if they where none or not correct and if they were good then they queryied correctly if not then the queried for everything not null. Then changed them to be a set then did set intersection to see what was the same through all of them. Thank you for ionheart for helping me through this and providing the information this is the complete answer using his partial solution.
userf=set()
userl=set()
userc=set()
userh=set()
if fname_strip!='':
userf = User.query.filter_by(first_name=fname_strip).all()
print(userf)
else:
userf = User.query.filter(User.first_name.isnot(None))
if lname_strip!='':
userl = User.query.filter_by(last_name=lname_strip).all()
print(userl)
else:
userl = User.query.filter(User.last_name.isnot(None))
try:
int(hs_class_strip)
userc = User.query.filter_by(hs_class=hs_class_strip).all()
print(userc)
except:
userc = User.query.filter(User.hs_class.isnot(None))
if hs_strip!='':
userh = User.query.filter_by(hs=hs_strip).all()
print(userh)
else:
userh = User.query.filter(User.hs.isnot(None))
userq=[]
common=set(userf) & set(userl) & set(userc) & set(userh)
print(common)
If you pass the arguments to your search function as keyword arguments you can change the signature to accept kwargs and pass those on to the filter query
def search(**kwargs):
userq = User.query.filter_by(**kwargs).all()
This way any arguments you don't specify when calling search will not be passed onto the query, for example calling search(first_name='bob', last_name='fossil') will only add first name and surname arguments to the query
I commonly find myself writing the same criteria in my Django application(s) more than once. I'll usually encapsulate it in a function that returns a Django Q() object, so that I can maintain the criteria in just one place.
I will do something like this in my code:
def CurrentAgentAgreementCriteria(useraccountid):
'''Returns Q that finds agent agreements that gives the useraccountid account current delegated permissions.'''
AgentAccountMatch = Q(agent__account__id=useraccountid)
StartBeforeNow = Q(start__lte=timezone.now())
EndAfterNow = Q(end__gte=timezone.now())
NoEnd = Q(end=None)
# Now put the criteria together
AgentAgreementCriteria = AgentAccountMatch & StartBeforeNow & (NoEnd | EndAfterNow)
return AgentAgreementCriteria
This makes it so that I don't have to think through the DB model more than once, and I can combine the return values from these functions to build more complex criterion. That works well so far, and has saved me time already when the DB model changes.
Something I have realized as I start to combine the criterion from these functions that is that a Q() object is inherently tied to the type of object .filter() is being called on. That is what I would expect.
I occasionally find myself wanting to use a Q() object from one of my functions to construct another Q object that is designed to filter a different, but related, model's instances.
Let's use a simple/contrived example to show what I mean. (It's simple enough that normally this would not be worth the overhead, but remember that I'm using a simple example here to illustrate what is more complicated in my app.)
Say I have a function that returns a Q() object that finds all Django users, whose username starts with an 'a':
def UsernameStartsWithAaccount():
return Q(username__startswith='a')
Say that I have a related model that is a user profile with settings including whether they want emails from us:
class UserProfile(models.Model):
account = models.OneToOneField(User, unique=True, related_name='azendalesappprofile')
emailMe = models.BooleanField(default=False)
Say I want to find all UserProfiles which have a username starting with 'a' AND want use to send them some email newsletter. I can easily write a Q() object for the latter:
wantsEmails = Q(emailMe=True)
but find myself wanting to something to do something like this for the former:
startsWithA = Q(account=UsernameStartsWithAaccount())
# And then
UserProfile.objects.filter(startsWithA & wantsEmails)
Unfortunately, that doesn't work (it generates invalid PSQL syntax when I tried it).
To put it another way, I'm looking for a syntax along the lines of Q(account=Q(id=9)) that would return the same results as Q(account__id=9).
So, a few questions arise from this:
Is there a syntax with Django Q() objects that allows you to add "context" to them to allow them to cross relational boundaries from the model you are running .filter() on?
If not, is this logically possible? (Since I can write Q(account__id=9) when I want to do something like Q(account=Q(id=9)) it seems like it would).
Maybe someone suggests something better, but I ended up passing the context manually to such functions. I don't think there is an easy solution, as you might need to call a whole chain of related tables to get to your field, like table1__table2__table3__profile__user__username, how would you guess that? User table could be linked to table2 too, but you don't need it in this case, so I think you can't avoid setting the path manually.
Also you can pass a dictionary to Q() and a list or a dictionary to filter() functions which is much easier to work with than using keyword parameters and applying &.
def UsernameStartsWithAaccount(context=''):
field = 'username__startswith'
if context:
field = context + '__' + field
return Q(**{field: 'a'})
Then if you simply need to AND your conditions you can combine them into a list and pass to filter:
UserProfile.objects.filter(*[startsWithA, wantsEmails])
Is it possible to modify Django Q() objects after construction? I create a Q() object like so:
q = Q(foo=1)
is it possible to later change q to be the same as if I had constructed:
q2 = Q(foo=1, bar=2)
? There's no mention of such an interface in the Django docs that I could find.
I was looking for something like:
Q.append_clause(bar=2)
You can just make another Q() object and AND them together:
q2 = q & Q(bar=2)
You can add Q objects together, using their add method. For example:
>>> q = Q(sender=x)
>>> q.add(Q(receiver=y), Q.AND)
The second argument to add is the connector, which can also be Q.OR
EDIT: My answer is merely a different way of doing what Perrin Harkins suggested, but regarding your other concern, about different behavior of filter depending on the way you construct the query, you don't have to worry about that if you join Q objects. My example is equivalent to filter(sender=x, receiver=y), and not filter(sender=x).filter(receiver=y), because Q objects, as far as I could see in a quick test, do an immediate AND on the clauses and don't have the special behavior of filter for multi-valued relations.
In any case, nothing like looking at the SQL and making sure it really is doing the same in your specific queries.
The answers here are a little old and unsatisfactory imo. So here is my answer
This is how you deep copy:
def deep_copy(q: Q) -> Q:
new_q = Q()
# Go through the children of a query: if it's another
# query it will run this function recursively
for sub_q in q.children:
# Make sure you copy the connector in
# case of complicated queries
new_q.connector = q.connector
if isinstance(sub_q, Q):
# This will run recursively on sub queries
sub_q = get_employee_q(sub_q)
else:
pass # Do your modification here
new_q.children.append(sub_q)
return new_q
In the else condition is where your stuff (name='nathan' for example) is defined. You can change or delete that if you'd like and the Query should work fine.
I'm using django ORM's exact() method to query only selected fields from a set of models to save RAM. I can't use defer() or only() due to some constraints on the ORM manager I am using (it's not the default one).
The following code works without an error:
q1 = Model.custom_manager.all().extra(select={'field1':'field1'})
# I only want one field from this model
However, when I jsonify the q1 queryset, I get every single field of the model.. so extra() must not have worked, or am I doing something wrong?
print SafeString(serializers.serialize('json', q1))
>>> '{ everything!!!!!}'
To be more specific, the custom manager I am using is django-sphinx. Model.search.query(...) for example.
Thanks.
So, Im not sure if you can do exactly what you want to do. However, if you only want the values for a particular field or a few fields, you can do it with values
It likely does the full query, but the result will only have the values you want. Using your example:
q1 = Model.custom_manager.values('field1', 'field2').all()
This should return a ValuesQuerySet. Which you will not be able to use with serializers.serialize so you will have to do something like this:
from django.utils import simplejson
data = [value for value in q1]
json_dump = simplejson.dumps(data)
Another probably better solution is to just do your query like originally intended, forgetting extra and values and just use the fields kwarg in the serialize method like this:
print SafeString(serializers.serialize('json', q1, fields=('field1', 'field2')))
The downside is that none of these things actually do the same thing as Defer or Only(all the fields are returned from the database), but you get the output you desire.
I've been struggling for this issue for a few hours - I know there's probably a simple solution that I'm overlooking.
I have a one to many relationship with my models.
I have need to return all rows of one object with the rows for the related object.
In a sense I have this:
object
object
object_relationship.property
object_relationship.property
object
object_relationship.property
object
Now - I can run through all of these fine, but I run into an issue when I want to send these back to the html template.
I can send the object back - but how do I send the object_relationship back in the order that I have it above?
Does this make sense?
You might not need to worry too much about this, acutally... look at these models:
class Venue(base.NamedEntity, HasPerformances, HasUrl, HasLocation):
city = db.ReferenceProperty(City, collection_name='venues')
url = db.StringProperty(required=True, validator=validators.validate_url)
location = db.GeoPtProperty()
class Performance(base.Entity):
show = db.ReferenceProperty(Show, collection_name='performances', required=True)
utc_date_time = db.DateTimeProperty(required=True)
venue = db.ReferenceProperty(Venue, collection_name='performances', required=True)
In a case like this, nothing stops you from using venue.performances from either code or templates and treating it as a list. The API will automatically fire queries as needed to fetch the actual objects. The same thing goes for performance.venue.
The only problem here is performance - you've got a variant of the n+1 problem to deal with. There are workarounds, though, like this article by Nick Johnson. I'd suggest reading the API code too... it makes for interesting reading how the property get is captured and dereferenced.
My first suggestion is to denormalize the data if you are going to do many reports like that. For example, maybe you could include object.name on the object_relationship entity.
That said, you could send a list of dicts to your template, so maybe something like:
data = []
for entity in your_query:
children = [{'name': child.name} for child in entity.object_relation]
data.append({'name': object.name,
'children': children,
...
})
Then pass the data list to your template, and process it.
Please note, this will perform very badly. It will execute another query for every one of the items in your first query. Use Appstats to profile your app.