I have two entity kinds and one referring to other like:
class Entity1(db.Expando):
prop1=db.StringProperty()
prop2=db.StringProperty()
class Entity2(db.Expando):
prop3=db.ReferenceProperty(Entity1)
prop4=db.StringProperty()
Can I write a query like:
q=Entity2.all().filter("prop3.prop1 =",somevalue)
Here prop3 has a reference and will be referring to some entity of kind Entity1 and I want to know all those entities of kind Entity2 which refer to those entities of Entity1 that have prop1 as somevalue.
In my example I first fetched the key for the referenceproperty to use as a filter for the query. Then you can do the filtering based on the key in a the same amounts of queries you found keys for.
An example:
The models:
class Order(db.Model):
waiter = db.ReferenceProperty(Waiter,required=True)
date=db.DateTimeProperty(required=True)
delivered=db.BooleanProperty(default=False)
class Waiter(db.Model):
firstname=db.StringProperty(required=True)
lastname=db.StringProperty(required=True)
uuid=db.StringProperty(required=True)
The web request function:
def get(self):
waiteruuid=self.request.get("waiter")
q=Waiter.all()
q.filter('uuid =', waiteruuid)
waiterkey=q.get()
result={}
result['orders']=[]
q=Order.all()
if waiteruuid:
q.filter('waiter =', waiterkey)
orders=q.run()
Google Datastore doesn't support joins. You can fetch all the entites of Entity2 and does some manipulation to achieve what you have said.Somewhat similar to what #Mani suggested. But you can do it like
entities2 = Entity2.all()
for entity2 in entities2:
Entity1= entity.prop3.get()
if Entity1.prop1== somevalue:
#Do your processing here
Define Entity 2 as:
class Entity2(db.Expando):
entity_1_ref = db.ReferenceProperty(Entity1, collection_name = "set_of_entity_2_elements")
prop4=db.StringProperty()
This defines a collection name which can be operated from the other side of reference. (Entity1 in this case). I have taken the liberty to rename prop3 as something more appropriate.
Now you can do q = entity_1_object.set_of_entity_2_elements (a new property for all your Entity1 objects) which will give you the results you want.
For more information, read this article indepth: http://code.google.com/appengine/articles/modeling.html
Update: Sorry, I got it wrong. The above suggestion doesnt get only those elements with entity_1_object.prop1 == somevalue
You can still get it in a round about way as follows:
for obj in q:
if ( obj.prop1 == somevalue):
# Do your processing here
or
for obj in q:
if ( obj.prop1 != somevalue):
# Delete this object from the list 'q'
But obviously this is not the best way. Lets wait for a better answer!
Related
I have a model that I'm creating properties as helpers to run common queries.
One of them is a declarative select on the model (I think that's the proper vocab, but please let me know if I'm describing anything incorrectly!):
class Device:
id = Column(Integer)
children = relationship(Children)
#property
def active_children(self):
return self.children.filter(Children.status=='active').all()
This example is returning a list of children objects such as
[<Children obj>,
<Children obj>,
...]
But I also have a more complex query property that I ended up writing using the generative select method:
#property
def problem_children(self):
stmt = select(Children).\
join(Issues,
and_(Children.id == Issues.device_child_id,
Issues.log_ts.between(Children.activate_ts, Children.deactivate_ts)).\
filter(Children.status == 'active').\
filter(Issues.type == 1).\
filter(Issues.user != 'SYS').\
order_by(Children.name)
return db.session.execute(stmt).all()
This returns a list of tuples with only one Children object:
[(<Children obj>,),
(<Children obj>,),
...]
I can solve this by instead setting the last line to a list comprehension instead of using all() but that feels not-elegant. I've been looking through the documentation and I haven't found the answer yet, any insight would be greatly helpful!
#property
def problem_children(self):
stmt = ...
return [x[0] for x in db.session.execute(stmt)]
(note the above code is untested since it's adapted from my situation, sorry if there's typos or issues other than my described issue!)
My question: is there a better way to standardize my property returns so that my property API stays consistent?
I would like to be able to relate an entity of one class to another entity at the moment of the creation of both entities (one entity will have the other as it's parent and the other would have a key pointing to the other entity). It seems I am unable to obtain the key of an entity prior it gets saved to the Datastore. Is there any way to achieve the above without having to save one of the entities twice?
Below is the example:
class A(ndb.Model):
key_of_b = ndb.KeyProperty(kind='B')
class B(ndb.Model):
pass
What I am trying to do:
a = A()
b = B(parent=a.key)
a.key_of_b = b.key
a.put()
b.put()
If the key doesn't get assigned prior to the entity being saved, is there anyway I could construct it myself? Is there any way to achieve this or would the only solution be to save one of the entities twice?
You could do this with named keys but then you have to be sure you can name the two entities with unique keys:
# It is possible to construct a key for an entity that does not yet exist.
keyname_a = 'abc'
keyname_b = 'def'
key_a = ndb.Key(A, keyname_a)
key_b = ndb.Key(A, keyname_a, B, keyname_b)
a = A(id=keyname_a)
a.key_of_b = key_b
b = B(id=keyname_b, parent=key_a)
a.put()
b.put()
However, I would suggest thinking about why you would need the key_of_b property in the first place. If you only set A as the parent of B then you will always be able to navigate from from A to B and the other way around:
# If you have the A entity from somewhere and want to find B.
b = B.query(ancestor=entity_a.key).get()
# You have the B entity from somewhere and want to find A.
a = entity_b.key.parent().get()
This also gives you the opportunity to create one-to-many relationships between A and B.
I've looked through the documentation, the docs and SO questions and answers and am still struggling with understanding a small piece of this. Which should you choose and when?
This is what I've read so far (just sample):
ndb documentation
movie database structure on SO
Parent Key issues
The key class seems pretty straightforward to me. When you create an ndb entity the datastore automatically creates for you a key usually in the form of key(Kind, id) where the id is created for you .
So say you have these two models:
class Blah(ndb.Model):
last_name = ndb.StringProperty()
class Blah2(ndb.Model):
first_name = ndb.StringProperty()
blahkey = ndb.KeyProperty()
So just using the key kind and you want to make Blah1 a parent (or have several family members with the same last name)
lname = Blah(last_name = "Bonaparte")
l_key = lname.put() **OR**
l_key = lname.key.id() # spits out some long id
fname_key = l_key **OR**
fname_key = ndb.Key('Blah', lname.last_name) # which is more readable..
then:
lname = Blah2( parent=fname_key, first_name = "Napoleon")
lname.put()
lname2 = Blah2( parent=fname_key, first_name = "Lucien")
lname2.put()
So far so good (I think). Now about the KeyProperty for Blah2. Assume Blah1 is still the same.
lname3 = Blah2( first_name = "Louis", blahkey = fname_key)
lname3.put()
Is this correct ?
How to query various things
Query Last Name:
Blah.query() # all last names
Blah.query(last_name='Bonaparte') # That specific entity.
First Name:
Blah2.query()
napol = Blah2.query(first_name = "Napoleon")
bonakey = napol.key.parent().get() # returns Bonaparte's key ??
bona = bonakey.get() # I think this might be redundant
this is where I get lost. How to look for Bonaparte from first name by using either key or keyproperty. I didn't add it here and perhaps should have and that is the discussion of parents, grand parents, great grand parents since Keys keep track of ancestors/parents.
How and why would you use KeyProperty vs the inherent key class. Also imagine you had 3 sensors s1, s2, s3. Each sensor had thousands of readings but you want to keep readings associated with s1 so that you could graph say All readings for today for s1. Which would you use? KeyProperty or the key class ? I apologize if this has been answered elsewhere but I didn't see a clear example/guide about choosing which and why/how.
I think the confusion comes from using a Key. A Key is not associated with any properties inside of an entity, it is only a unique identifier to locate a single entity. It can be either a number or a string.
Fortunately, all your code looks good except for this one line:
fname_key = ndb.Key('Blah', lname.last_name) # which is more readable..
Constructing a Key takes a unique ID, which is not the same as a property. That is, it won't associate the variable lname.last_name with the property last_name. Instead, you can create your record like this:
lname = Blah(id = "Bonaparte")
lname.put()
lname_key = ndb.Key('Blah', "Bonaparte")
You are guaranteed to have only one Blah entity with that ID. In fact, if you use a string like last_name as the ID, you don't need to store it as a separate property. Think of the entity ID as an extra string property that is unique.
Next, Be careful not to assume that Blah.last_name and Blah2.first_name are unique in your queries:
lname = Blah2( parent=fname_key, first_name = "Napoleon")
lname.put()
If you do this more than once, there will be multiple entities with a first_name of Napoleon (all with the same parent key).
Continuing with your code from above:
napol = Blah2.query(first_name = "Napoleon")
bonakey = napol.key.parent().get() # returns Bonaparte's key ??
bona = bonakey.get() # I think this might be redundant
napol holds a Query, not a result. You need to call napol.fetch() to get all entities with "Napolean" (or napol.get() if you're sure there is just one entity).
bonakey is the opposite, it holds the parent entity because of the get() and not Bonaparte's key. If you left the .get() off, then bona would correctly have the parent.
Finally, your question about sensors. You may not need KeyProperty or "inherent" keys. If you have a Readings class like this:
class Readings(ndb.Model):
sensor = ndb.StringProperty()
reading = ndb.IntegerProperty()
then you can store them all in a single table without keys. (You may want to include a timestamp or other attribute.) Later, you can retrieve then with this query:
s1_readings = Readings.query(Readings.sensor == 'S1').fetch()
I'm new to NDB also, and I'm still not understanding all for now, but I think that when you create Blah2 with a parent for Napoleon, you will need the parent to query it or will not appear. For example:
napol = Blah2.query(first_name = "Napoleon")
will not get anything (and you are not using the right format for NDB), but using the parent will do:
napol = Blah2.query(ancestor=fname_key).filter(Blah2.first_name == "Napoleon").get
Don't know if this puts some light for your question.
I have a model with a reference property, eg:
class Data(db.Model):
x = db.IntegerProperty()
class Details(db.Model):
data = db.ReferenceProperty(reference_class = Data)
The data reference can be None.
I want to fetch all Details entities which have valid data, ie for which the reference property is not None.
The following works:
Details.all().filter('data !=', None).fetch(1000)
However, according to the documentation on queries, a != query will actually perform two queries, which seems unnecessary in this case. Is != optimised to only perform one query when used with None?
Alternatively, this post mentions that NULL always sorts before valid values. Therefore the following also appears to work:
Details.all().filter('data >', None).fetch(1000)
Although this would only do one query, using > instead of != makes the intent of what it is doing less obvious.
As a third option, I could add an extra field to the model:
class Details(db.Model):
data = db.ReferenceProperty(reference_class = Data)
has_data = db.BooleanProperty()
As long as I keep has_data synchronised with data, I could do:
Details.all().filter('has_data =', True).fetch(1000)
Which way would be best?
Thanks.
I would advise you to use the extra model field. This is more flexible, since it also allows you to query for Details that have no Data references. In addition, queries can only have one inequality filter, so you're better off saving this inequality filter for another property where inequality makes more sense, such as integer properties.
To make sure the flag is always updated, you can add a convenience function to Details, like so:
class Details(db.Model):
data = db.ReferenceProperty(reference_class=Data)
has_data = db.BooleanProperty(default=False)
def add_data(self, data):
""" Adds data"""
if not data: return
self.has_data = True
self.data = data
return self.put()
Is it possible to filter a Django queryset by model property?
i have a method in my model:
#property
def myproperty(self):
[..]
and now i want to filter by this property like:
MyModel.objects.filter(myproperty=[..])
is this somehow possible?
Nope. Django filters operate at the database level, generating SQL. To filter based on Python properties, you have to load the object into Python to evaluate the property--and at that point, you've already done all the work to load it.
I might be misunderstanding your original question, but there is a filter builtin in python.
filtered = filter(myproperty, MyModel.objects)
But it's better to use a list comprehension:
filtered = [x for x in MyModel.objects if x.myproperty()]
or even better, a generator expression:
filtered = (x for x in MyModel.objects if x.myproperty())
Riffing off #TheGrimmScientist's suggested workaround, you can make these "sql properties" by defining them on the Manager or the QuerySet, and reuse/chain/compose them:
With a Manager:
class CompanyManager(models.Manager):
def with_chairs_needed(self):
return self.annotate(chairs_needed=F('num_employees') - F('num_chairs'))
class Company(models.Model):
# ...
objects = CompanyManager()
Company.objects.with_chairs_needed().filter(chairs_needed__lt=4)
With a QuerySet:
class CompanyQuerySet(models.QuerySet):
def many_employees(self, n=50):
return self.filter(num_employees__gte=n)
def needs_fewer_chairs_than(self, n=5):
return self.with_chairs_needed().filter(chairs_needed__lt=n)
def with_chairs_needed(self):
return self.annotate(chairs_needed=F('num_employees') - F('num_chairs'))
class Company(models.Model):
# ...
objects = CompanyQuerySet.as_manager()
Company.objects.needs_fewer_chairs_than(4).many_employees()
See https://docs.djangoproject.com/en/1.9/topics/db/managers/ for more.
Note that I am going off the documentation and have not tested the above.
Looks like using F() with annotations will be my solution to this.
It's not going to filter by #property, since F talks to the databse before objects are brought into python. But still putting it here as an answer since my reason for wanting filter by property was really wanting to filter objects by the result of simple arithmetic on two different fields.
so, something along the lines of:
companies = Company.objects\
.annotate(chairs_needed=F('num_employees') - F('num_chairs'))\
.filter(chairs_needed__lt=4)
rather than defining the property to be:
#property
def chairs_needed(self):
return self.num_employees - self.num_chairs
then doing a list comprehension across all objects.
I had the same problem, and I developed this simple solution:
objects = [
my_object
for my_object in MyModel.objects.all()
if my_object.myProperty == [...]
]
This is not a performatic solution, it shouldn't be done in tables that contains a large amount of data. This is great for a simple solution or for a personal small project.
PLEASE someone correct me, but I guess I have found a solution, at least for my own case.
I want to work on all those elements whose properties are exactly equal to ... whatever.
But I have several models, and this routine should work for all models. And it does:
def selectByProperties(modelType, specify):
clause = "SELECT * from %s" % modelType._meta.db_table
if len(specify) > 0:
clause += " WHERE "
for field, eqvalue in specify.items():
clause += "%s = '%s' AND " % (field, eqvalue)
clause = clause [:-5] # remove last AND
print clause
return modelType.objects.raw(clause)
With this universal subroutine, I can select all those elements which exactly equal my dictionary of 'specify' (propertyname,propertyvalue) combinations.
The first parameter takes a (models.Model),
the second a dictionary like:
{"property1" : "77" , "property2" : "12"}
And it creates an SQL statement like
SELECT * from appname_modelname WHERE property1 = '77' AND property2 = '12'
and returns a QuerySet on those elements.
This is a test function:
from myApp.models import myModel
def testSelectByProperties ():
specify = {"property1" : "77" , "property2" : "12"}
subset = selectByProperties(myModel, specify)
nameField = "property0"
## checking if that is what I expected:
for i in subset:
print i.__dict__[nameField],
for j in specify.keys():
print i.__dict__[j],
print
And? What do you think?
i know it is an old question, but for the sake of those jumping here i think it is useful to read the question below and the relative answer:
How to customize admin filter in Django 1.4
It may also be possible to use queryset annotations that duplicate the property get/set-logic, as suggested e.g. by #rattray and #thegrimmscientist, in conjunction with the property. This could yield something that works both on the Python level and on the database level.
Not sure about the drawbacks, however: see this SO question for an example.