dear all
I'm upgrading one of my datastore entities from db to ndb. The order() call is a problem for me. As frontend will transfer a string such as "-created_at" or "title" for ordering. In the past, I basically put this value to query as shown below:
query.order("-created_at")
Now ndb doesn't support above syntax. Is there a recommended approach for translating order("-created_at") to order(-MyModel.created_at) ?
Thanks a lot!
Use dictionary lookup
sort_orders = {
"-created_at": -MyModel.created_at,
"some_other_order": SomeModel.some_property
}
def get_order(input):
return sort_orders.get(input)
That way you only accept valid/predefined set of possible orders.
The - unary operator does work in the dictionary.
e.g.
s~lightning-catfish> -Series.updatedDate
PropertyOrder(<updatedDate>, DESCENDING)
I would personally make the function/method a class_method of the model, and the dictionary of valid orders defined at the class level as well. That way all the definitions are in the same place.
Related
I have two models: Post, Comment (Comment has FK relation to Post).
Now I want to return all posts with theirs "response time". I get this response time in timedelta format. Can I receive it in seconds instead? I tried ExtractSecond but it is not what I'm looking for:
base_posts_queryset.annotate(
min_commnet_date=Min("comment_set__date_created"),
response_time=ExpressionWrapper(F("min_commnet_date") - F("date_created"), output_field=DurationField()),
response_time_in_sec=ExtractSecond(F("response_time"))
).filter(response_time__isnull=False).values("response_time", "response_time_in_sec")
This code returns following objects:
{'response_time': datetime.timedelta(days=11, seconds=74024, microseconds=920107), 'response_time_in_sec': 44}
What I want to achieve is basically call .seconds for each item in result queryset. I could do this in python, but mb it could be done on db level?
Sure can, but the exact mechanism may depend upon your database.
In postgres, you can use EXTRACT(epoch FROM <interval>) to get the total number of seconds.
To use this in Django, you can create a Func subclass:
class Epoch(django.db.models.expressions.Func):
template = 'EXTRACT(epoch FROM %(expressions)s)::INTEGER'
output_field = models.IntegerField()
Then you can use it directly:
base_posts.annotate(
response_time_sec=Epoch(F('min_comment_date') - F('date_created'))
)
Nice solution!
One wrinkle is that I think there is a missing 's' needed to get this to work in Django 3
class Epoch(django.db.models.expressions.Func):
template = 'EXTRACT(epoch FROM %(expressions)s)::INTEGER'
output_field = models.IntegerField()
As already answered here, it depends upon your database. As stated in Django documentation of Extract method:
Django usually uses the databases’ extract function, so you may use any lookup_name that your database supports.
So for example with PostgreSQL:
response_time_in_sec=Extract(F("response_time"), "epoch")
I've been trying to implement an 'abstract' schema class that will automatically convert values in CamelCase (serialized) to snake_case (deserialized).
class CamelCaseSchema(marshmallow.Schema):
#marshmallow.pre_load
def camel_to_snake(self, data):
return {
utils.CaseConverter.camel_to_snake(key): value for key, value in data.items()
}
#marshmallow.post_dump
def snake_to_camel(self, data):
return {
utils.CaseConverter.snake_to_camel(key): value for key, value in data.items()
}
While using something like this works nicely, it does not achieve everything applying load_from and dump_to to a field does. Namely, it fails to provide correct field names when there's an issue with deserialization. For instance, I get:
{'something_id': [u'Not a valid integer.']} instead of {'somethingId': [u'Not a valid integer.']}.
While I can post-process these emitted errors, this seems like an unnecessary coupling that I wish to avoid if I'm to make the use of schema fully transparent.
Any ideas? I tried tackling the metaclasses involved, but the complexity was a bit overwhelming and everything seemed exceptionally ugly.
You're using marshmallow 2. Marshmallow 3 is now out and I recommend using it. My answer will apply to marshmallow 3.
In marshmallow 3, load_from / dump_to have been replace by a single attribute : data_key.
You'd need to alter data_key in each field when instantiating the schema. This will happen after field instantiation but I don't think it matters.
You want to do that ASAP when the schema is instantiated to avoid inconsistency issues. The right moment to do that would be in the middle of Schema._init_fields, before the data_key attributes are checked for consistency. But duplicating this method would be a pity. Besides, due to the nature of the camel/snake case conversion the consistency checks can be applied before the conversion anyway.
And since _init_fields is private API, I'd recommend doing the modification at the end of __init__.
class CamelCaseSchema(Schema):
def __init__(self, **kwargs):
super().__init__(**kwargs)
for field_name, field in self.fields.items():
fields.data_key = utils.CaseConverter.snake_to_camel(field_name)
I didn't try that but I think it should work.
Just to know if I'm mistaked or not:
get() operations uses NDB cache, so this (Chapter is ndb.Model class):
# Get the entity
chapter_key = ndb.Key('Book', long(bookId), 'Chapter', long(chapterId))
chapter = chapter_key.get()
can use the ndb cache if is 2nd or more read of the entity.
But if I make this?
Chapter.get_by_id(long(id), parent=ndb.Key('Book', long(bookId)))
is this managed by ndb also, or this operation is a standart db operation and don't use cache?
Model.get_by_id will use the context-cache and memcache in exactly the same way as Key.get
As Greg's answer is correct, I just wanted to mention that instead of putting keys together manually, you may use urlsafe string, and pass it between functions.
Assuming you have a key:
page_key = ndb.Key('Book', long(bookId), 'Chapter', long(chapterId), 'Page', long(pageId))
Create urlsafe from it:
page_url_string = page_key.urlsafe()
To retrive the model, simply use:
page = ndb.Key(urlsafe=page_url_string).get()
Consider it if you're using models with several parentness, there should be no case when putting keys manually would be required, code gets messy really quick, as you need to pass additional variables between functions.
i will like to have two types of entities referring to each other.
but python dont know about name of second entity class in the body of first yet.
so how shall i code.
class Business(db.Model):
bus_contact_info_ = db.ReferenceProperty(reference_class=Business_Info)
class Business_Info (db.Model):
my_business_ = db.ReferenceProperty(reference_class=Business)
if you advice to use reference in only one and use the implicitly created property
(which is a query object) in other.
then i question the CPU quota penalty of using query vs directly using get() on key
Pleas advise how to write this code in python
Queries are a little slower, and so they do use a bit more resources. ReferenceProperty does not require reference_class. So you could always define Business like:
class Business(db.Model):
bus_contact_info_ = db.ReferenceProperty()
There may also be better options for your datastructure too. Check out the modelling relationships article for some ideas.
Is this a one-to-one mapping? If this is a one-to-one mapping, you may be better off denormalizing your data.
Does it ever change? If not (and it is one-to-one), perhaps you could use entity groups and structure your data so that you could just directly use the keys / key names. You might be able to do this by making BusinessInfo a child of Business, then always use 'i' as the key_name. For example:
business = Business().put()
business_info = BusinessInfo(key_name='i', parent=business).put()
# Get business_info from business:
business_info = db.get(db.Key.from_path('BusinessInfo', 'i', parent=business))
# Get business from business_info:
business = db.get(business_info.parent())
Is there a way to get the key (or id) value of a db.ReferenceProperty, without dereferencing the actual entity it points to? I have been digging around - it looks like the key is stored as the property name preceeded with an _, but I have been unable to get any code working. Examples would be much appreciated. Thanks.
EDIT: Here is what I have unsuccessfully tried:
class Comment(db.Model):
series = db.ReferenceProperty(reference_class=Series);
def series_id(self):
return self._series
And in my template:
more
The result:
more
Actually, the way that you are advocating accessing the key for a ReferenceProperty might well not exist in the future. Attributes that begin with '_' in python are generally accepted to be "protected" in that things that are closely bound and intimate with its implementation can use them, but things that are updated with the implementation must change when it changes.
However, there is a way through the public interface that you can access the key for your reference-property so that it will be safe in the future. I'll revise the above example:
class Comment(db.Model):
series = db.ReferenceProperty(reference_class=Series);
def series_id(self):
return Comment.series.get_value_for_datastore(self)
When you access properties via the class it is associated, you get the property object itself, which has a public method that can get the underlying values.
You're correct - the key is stored as the property name prefixed with '_'. You should just be able to access it directly on the model object. Can you demonstrate what you're trying? I've used this technique in the past with no problems.
Edit: Have you tried calling series_id() directly, or referencing _series in your template directly? I'm not sure whether Django automatically calls methods with no arguments if you specify them in this context. You could also try putting the #property decorator on the method.