I have built an API with Django Rest Framework. I want to change pagination for a better user experience.
The problem:
The client makes a call to request all posts. The request looks like:
http://website.com/api/v1/posts/?page=1
This returns the first page of posts. However, new posts are always being created. So when the user requests:
http://website.com/api/v1/posts/?page=2
The posts are almost always the same as page 1 (since new data is always coming in and we are ordering by -created).
Possible Solution?
I had the idea of sending an object id along with the request so that when we grab the posts. We grab them with respect to the last query.
http://website.com/api/v1/posts/?page=2&post_id=12345
And when we paginate we filter where post_id < 12345
But this only works assuming our post_id is an integer.
Right now I'm currently only using a basic ListAPIView
class PostList(generics.ListAPIView):
"""
API endpoint that allows posts to be viewed
"""
serializer_class = serializers.PostSerializer # just a basic serializer
model = Post
Is there a better way of paginating? so that the next request for that user's session doesn't look like the first when new data is being created.
You're looking for CursorPagination, from the DRF docs:
Cursor Pagination provides the following benefits:
Provides a consistent pagination view. When used properly CursorPagination ensures that the client will never see the same item twice when paging through records, even when new items are being inserted by other clients during the pagination process.
Supports usage with very large datasets. With extremely large datasets pagination using offset-based pagination styles may become inefficient or unusable. Cursor based pagination schemes instead have fixed-time properties, and do not slow down as the dataset size increases.
You can also use -created as the ordering field as you mentioned above.
How about caching the queryset? So that the next page is served from the same query set, and not from a new one. And then you could use a parameter to get a new queryset when you want.
Something like this:
from django.core.cache import cache
class PostList(generics.ListAPIView):
def get_queryset(self):
qs_key = str(self.request.user.id) + '_key'
if 'refresh' in self.request.QUERY_PARAMS:
# get a new query set
qs = Post.objects.all()
cache.set(qs_key, qs)
return cache.get(qs_key)
So basically, only when your url will be like this:
http://website.com/api/v1/posts/?refersh=whatever
the request will return new data.
UPDATE
In order to provide each user with it's own set of posts, the cache key must contain an unique identifier (which might be the user's ID):
I also updated the code.
The downside to this approach is that for a very large number of users and a large number of posts for each user, it might not work very well.
So, here is my second idea
Use a TimeStamped model for the Post model, and filter the query set based on the created field.
I don't know much about your models and how exactly they are built, so I guess you will have to choose which solution is best for your app :)
Maybe you can add a field to every object like "created_at/updated_at". Then you can save the timestamp when the user made the request and filter out everything that came after it.
Haven't tried it myself but I guess it might work on your case
Related
I have an application created long back, now client want to expose its some of views as APIs without breaking existing functionality, so that they can directly consume APIs using REST Tools to see the reports.
Is there any easier way, I can convert my function to a REST View.
P.S - I kept code shorter here to keep question simple, but in fact, its much complex in the actual app.
eg.
URL : -
`path('/users', views.show_user_details, name='users'),`
VIEW
def show_user_details(request, user_id):
users = User.objects.all()
return render(request, "Users.html", {"users":users})
In REST Views, I want it to convert its input and output so that it can be accessible with same urls(or with little modifications), without much updating the existing views.
`path('rest/users', views.show_user_details, name='users'),` #-- I am ok to add new url like this, but without much change in existing view .
def show_user_details(request, user_id):
users = User.objects.all()
return JsonResponse({"users":users})
Due to the fact that a normal website visit is still a GET request and GET is just one of your usual REST actions, you'll probably want to prepare your own independent API endpoint. Check out django-rest-framework for that, and you might just feel at home for this task.
I have set up a Django app that queries from a PostgreSQL database. Exporting all the data within a model seems relatively straight forward but what if I wanted to export only the data from a returned queryset.
Currently I'm managing this by having a button on my page and changing the url when the user wishes to export data. The code within the view that I'm rendering is as follows.
def export_data(request,pk):
resource_class = MDPResource()
queryset = MDP.objects.filter(mdp_id=pk)
dataset = person_resource.export(queryset)
response = HttpResponse(dataset.csv, content_type='text/csv')
response['Content-Disposition'] = 'attachment; filename="mdpbasic.csv"'
return response
However, this means that I'm going to be making one query when the user initially opens the page and then a second query (in effect 2 extra) when the user wishes to export the data which I have already queried for in the previous view.
Moreover, in a section of my page I'm also using Q models for complex filtering. The above method is not sufficient for exporting data in that case (I don't want to have to reconstruct the query).
I also do not want to have to write the data to a file (if I can help it) because then that raises the issue of having to delete it using os once the user has exported their data.
Generating CSV file with Django (dynamic content)
I have already checked out this question but I'm not sure how passing a dictionary to a urlpattern works and it still seems as though the query is being made one more time anyways (still one less than my current method).
I don't know if I should be checking out apps for this particular purpose. I've looked into a couple such as django-queryset-csv and django-import-export but none of them seem to do what I've mentioned.
django: export current queryset to csv by button click in browser
I've seen this example that makes use of class-based views but I wanted to see if there was a way to do it with function-based views so I don't have to migrate all my views to class-based ones.
I would like to autogenerate a dictionary of data to POST to a django admin change form, as if the current object is not changing. Basically simulate the POST request that would happen if you do a GET on the change page from the admin, and then click Save. This should take into account which fields are editable on that particular change form, and also be able to handle InlineAdmins.
def auto_generate_changeform_data(object):
data = ???
return data
For example, in line 302 of the django admin testcases they manually create the POST data dictionary, but this is something that should be possible to autogenerate, right?
I would then use this function in a testcase where I want to test what happens when I change one particular field on that model, or even one of the inlines.
#some testcase
some_object = SomeModel.objects.get(...)
url = reverse("admin:appname_modelname_change", args=[some_object.id])
data = auto_generate_changeform_data(some_object)
data['some_field'] = 'new value' #this is the only change I want to make
response = self.client.post(url, data)
I could do all of this in a more unit-y fashion by using the methods on the ModelAdmin class instead of going through a POST request to the client, but I want a functional test not a unit test in this case. I actually do want to simulate the full POST to the url.
How can I get this data dictionary, particularly the part about inline formsets?
I've done this before by simply doing the GET request with the test client and then using BeautifulSoup to scrape out all the input and select elements.
It's not completely straightforward because of the different ways those elements represent their values, but here is some code that should work.
How can I do some processing before returning something through Tastypie for a specific user?
For example, let's say I have an app where a user has posts and can also follow other people's posts. I'd like to combine this person's posts with the posts of people they're following and return it as one array.
So let's say that in Tastypie I'd like to get the latest 20 posts from this person's timeline: I'd need to get the user, process this information and return it in JSON, but I'm not exactly sure how I'd process this and return it using Tastypie.
Any help?
Do more complex processing in get_object_list. It gets called before the dehydration process starts, i.e., before the JSON is created that gets passed back.
class PostResource(ModelResource):
class Meta:
queryset = Post.objects.all()
def get_object_list(self, request):
this_users_posts = super(PostResource, self).get_object_list(request).filter(user=User.objects.get(user=request.user))
all_the_posts_this_user_follows = super(PostResource, self).get_object_list(request).filter(follower=User.objects.get(user=request.user))
return this_users_posts | all_the_posts_this_user_follows
You need to fix those queries so that they work for your particular case. Then the trick is to combine the two different querysets you get back by concatenating them. Using | gets their full set, using & only gets their overlap. You want the full set (unless users can also follow their own posts, then you can probably call distinct() on the resulting superset).
Can anyone point to an example written in Python (django preferred) with ajax for cascading forms? Cascading Forms is basically forms whose field values change if and when another field value changes. Example Choose Country, and then States will change...
This is (mostly) front-end stuff.
As you may have noticed Django attempts to leave all the AJAX stuff up to you, so I don't think you'll find anything built in to do this.
However, using JS (which is what you'll have to do in order to do this without submitting a billion forms manually), you could easily have a django-base view your JS could communicate with:
def get_states(request, country):
# work out which states are available
#import simplesjon as sj
return sj....
Then bind your AJAX request to the onchange event of the select (I can't remember if that's right for select boxes) and populate the next field based on the return of the JSON query.
10 minute job with jquery and simplejson.
I would also suggest considering getting a mapping of all data once instead of requesting subfield values one by one. Unless the subfield choices change frequently (states/cities change?) or huge in numbers (>1000) this should offer best performance and it is less complex.
You don't even need to create a seperate view, just include a chunk of JavaScript (a JSON mapping more precisely) with your response containing the form.