I have an app that I want to build a "recent activity"/firehose feed of 2-3 combined types of activity such as posts, comments, and likes of posts, and something else + maybe more later. I assume this is done with a query of taking the last of the appropriate object added to the DB and combining it with the last of the other type of object and ordering the new combined list of objects by their timestamps. What is the best way to do something like this? For now, I have something like this for every time someone refreshes the page:
NewPost.objects.all().order_by('-postdate')[0:10] #takes the last 16 recently added posts
Comment.objects.all().order_by('-commentdate')[0:10] #takes equal number of comments site wide ordered by timestamp
So what is the best way to take both of these querysets and render the different Models in 1 list ordered by their timestamp? I assume the type of logic will be the same for adding additional types of objects, so I just want to know how to do it with just 2. Thanks!
I don't really like your approach since when you want to put another object on the firehose you'd need to add a third line (AnotherObject.objects.all ... etc ) to all places you need to display that firehose !
For me, the best way to do this is to create a Firehose Model with fields like: date, action (add/delete/update etc) and object (a generic Foreign Key to the object that was changed). Now, whenever you make a change to an object that you want to add to the firehose, you'd add a new instance of the FirehoseClass with the correct field values. Finally, whenever you want to display the firehose you'll just display all firehose objects.
To combine the lists, you can use create a list by using chain() from itertools, and then sort them by using sorted():
from itertools import chain
combined_lists = list(chain(new_post_list, comment_list))
sorted_combinened_list = sorted(combined_list, key=lambda instance: instance.postdate)
However, as you see, the sorting is only done by using one key. I don't know of any method to use two different keys when sorting. You could fix this by simply add a property to the Comment class, named postdate that simply returns commentdate. Or, even better, you should use the same name for creation time for all your models, e.g. created_at.
This has been answered earlier and more detailed here: How to combine 2 or more querysets in a Django view?
Related
Using Python SDK, could not find how to get all the keys from one bucket
in couchbase.
Docs reference:
http://docs.couchbase.com/sdk-api/couchbase-python-client-2.2.0/api/couchbase.html#item-api-methods
https://github.com/couchbase/couchbase-python-client/tree/master/examples
https://stackoverflow.com/questions/27040667/how-to-get-all-keys-from-couchbase
Is there a simple way to get all the keys ?
I'm a little concerned as to why you would want every single key. The number of documents can get very large, and I can't think of a good reason to want every single key.
That being said, here are a couple of ways to do it in Couchbase:
N1QL. First, create a primary index (CREATE PRIMARY INDEX ON bucketname), then select the keys: SELECT META().id FROM bucketname; In Python, you can use N1QLQuery and N1QLRequest to execute these.
Create a map/reduce view index. Literally the default map function when you create a new map/reduce view index is exactly that: function (doc, meta) { emit(meta.id, null); }. In Python, use the View class.
You don't need Python to do these things, by the way, but you can use it if you'd like. Check out the documentation for the Couchbase Python SDK for more information.
I'm a little concerned as to why you would want every single key. The number of documents can get very large, and I can't think of a good reason to want every single key.
There is a document for every customer with the key being the username for the customer. That username is only held as a one-way hash (along with the password) for authentication. It is not stored in its original form or in a form from which the original can be recovered. It's not feasible to ask the 100 million customers to provide their userids. This came from an actual customer on #seteam.
Is there a way to retrieve all objectIDs from an Algolia Index?
I know there is [*Index Name*].browse_all() which in the docs say it can retrieve 1000 objects at a time but it retrieves the entire object rather than just the objectIDs.
I can work with pagination but would rather not and do not want to pull the entire object because our indexes are not small.
Browse is the right way to go.
The good thing is that you can specify arguments while performing a browse_all and one of them can be attributesToRetrieve: [] to not retrieve any attributes. You'll therefore only get the objectID.
I have a model Entry
class Entry(db.Model):
year = db.StringProperty()
.
.
.
and for whatever reason the last name field is stored in a different model LastName:
class LastName(db.Model):
entry = db.ReferenceProperty(Entry, collection_name='last_names')
last_name = db.StringProperty()
If I query Entry and sort it by year (or any other property) using .order() how would I then sort that by the last name? I'm new to python but coming from Java I would guess there's some kind of comparator equivalent; or I'm completely wrong and there's another way to do it. I for sure cannot change my model at this point in time, though that may be the solution later. Any suggestions?
EDIT: I'm currently paginating through the results using offsets (moving to cursors soon, but I think it would be the same issue). So if I try to sort outside of the datastore I would only be sorting the current set; it's possible that the first page will be all 'B's and the second page will have 'A's, so it will only be sorted by page not by overall set. Am I screwed the way my models are currently set up?
A few issues here.
There's no way to do this sorting directly in the datastore API, either in Python or Java - as you no doubt know, the datastore is non-relational, and indirect lookups like this aren't supported.
If this was just a straight one-to-one relationship, which gave you an accessor from the Entry entity to the LastName one, you could use the standard Python sort function to sort the list:
entries.sort(key=lambda e: e.last_name.last_name)
(note that this sorts the list in place but returns None, so don't try assigning from it).
However, this won't work, because what you've actually got here is a one-to-many relationship: there are potentially many LastNames for each Entry. The definition actually recognises this: the collection_name attribute, which defines the accessor from Entry to LastName, is called last_names, ie plural.
So what you're asking doesn't really make sense: which of the potentially many LastNames do you want to sort on? You can certainly do it the other way round - given a query of LastNames, sort by entry year - but given your current structure there's not really any way of doing it.
I must say though, although I don't know the rest of your models, I suspect you have actually got that relationship the wrong way round: the ReferenceProperty should probably live on Entry pointing to LastName rather than the other way round as it is now. Then it would simply be the sort call I gave above.
I have two models in Django: groups and entries. Groups has a many-to-many field that connects it to entries. I want to select all entries that have a group (as not all do!) and be able to access their group.title field.
I've tried something along the lines of:
t = Entries.objects.select_related().exclude(group=None)
and while this returns all entries that have groups, I can't do t[0].groups to get the title. Any ideas on how this could be done?
Edit: more info
When ever I use Django's shell to inspect what is returned in t (in this example), t[0].group does not exist. The only way I can access this is via t[0].group_set.all()[0].title, which seems inefficient and like I'm doing something incorrectly.
You don't show the model code, so I can't be sure, but instead of t[0].groups, I think you want:
for g in t[0].groups.all():
print g.title
This question already has answers here:
A QuerySet by aggregate field value
(3 answers)
Closed 8 years ago.
I have a handful of records that I would like to sort based on a computed value. Got the answer over here... like so:
sorted(Profile.objects.all(), key=lambda p: p.reputation)
on a Profile class like this:
class Profile(models.Model):
...
#property
def reputation(self):
...
Unfortunately the generic view is expecting a queryset object and throws an error if I give it a list.
Is there a way to do this that returns a queryset
or...
Can I convert a list to a queryset somehow? Couldn't find anything like that in the django docs.
I am hoping not to denormalize the data, but I guess I will if I have to.
Update / Answer:
it seems that the only way to get a queryset back is if you can get all of your logic into the sql queries.
When that is not possible, (I think) you need to denormalize the data
Ok...this post is now old BUT what you could do is get all the ids of the objects in your list, then perform a model.objects.filter(pk__in=list_of_ids)
There is no point in converting a data list back to a query. A query object never holds data; it just represents a query to the database. It would have to fetch everything again if you made your list to a query, and that would be redundant and very bad performance-wise.
What you can do:
Describe how the reputation field is calculated; it's probably possible to order the data in the database somehow.
Modify the view to not require a query object. If it needs to do additional filtering etc. this should be done before any ordering, since the ordering will take less time with less entries (and less data will be fetched from the database.) So you could send the filtered query object to the sort function just before you send it to the template (which shouldn't care whether it's a query or a list.)