Setting up Redis cache with Django - python

I have a django project on an ubuntu EC2 node, that performs a computationally intensive long running process, that typically takes over 60 seconds. I need to cache the results. Never having worked with a cache before, I am following http://michal.karzynski.pl/blog/2013/07/14/using-redis-as-django-session-store-and-cache-backend/ to use redis for this. In the article the author refers to https://docs.djangoproject.com/en/1.7/topics/cache/ which contains the following:
given a URL, try finding that page in the cache
if the page is in the cache:
return the cached page
else:
generate the page
save the generated page in the cache (for next time)
return the generated page
This is the basic logic which I will need to implement in my django view. However , although I've read through the remainder of the cache docs, I'm still unclear on how to code this logic into my django view.
My current django view contains;
from django.shortcuts import render
from django.http import HttpResponse
from django.views.decorators.csrf import csrf_exempt
#csrf_exempt
def index(request):
# long running process started on request. Resulting html page should be cached
return HttpResponse(html)
How do I code the above logic into my view?

Related

Why isn't my POST Webhook working in Django?

I use the app Salesmate and am trying to write a a client on my site that adds features through the API. For this to work I need Salesmate's Webhooks. My client is in Django.
When I send a GET request webhook it makes it into my client.
When I send a POST request webhook it never makes it into the view.
When I send a test POST request from https://reqbin.com/ it makes it into the view and performs as expected.
I've played endlessly with the JSON body and headers. There could still be something I'm missing here, or there could be a flag raised in Django invalidating the sender, or something else...
Here is Salesmate webhook request, in the app. I've played with many headers, and sent requests with and without JSON bodies.
[deleted for privacy]
Here is my django view. Super simple, but it never gets called.
from django.views.decorators.csrf import csrf_exempt
from django.http import HttpResponse
#csrf_exempt
def pab(request):
#Do something simple and trackable
return HttpResponse(keep)
I was using the Django module "Moderna" which was causing some issues, still not sure what. I started a naked django app and it started working.

Django: dynamically mapping views in urls.py

I have a viewsdirectory containing say a hundred view files. How can i make my urls.py transfer control to these view files without putting in an "intermediate handler" as described here (answered 3 years back).
Basically- is there a cleaner way for redirecting control to views in django?
The view function returns an HTML page that includes the current date and time. To display this view at a particular URL, you’ll need to create a URLconf; see URL dispatcher for instructions.
https://docs.djangoproject.com/en/1.7/topics/http/urls/
For redirecting you should use the Django redirect shortcut function
from django.shortcuts import redirect
def my_view(request):
...
return redirect('some-view-name', foo='bar')
https://docs.djangoproject.com/en/1.7/topics/http/shortcuts/#redirect

Is Django post_save triggered before/after saving instance to database?

I have a website using Django. Each post is an object called Article. I want to retrieve the post's HTML after saving it so I wrote the following post_save hook:
#receiver(models.signals.post_save, sender=Article)
def _send_article_mentions(sender, instance, **kwargs):
import requests
from django.contrib.sites.models import Site
from urlparse import urljoin
from ParallelTransport.settings import ARTICLES_URL
SITE_URL = 'http://'+Site.objects.get_current().domain
article_url = urljoin( SITE_URL, instance.get_absolute_url() )
import time
time.sleep(20)
r = requests.get(article_url)
error_file = open(ARTICLES_URL+'/'+'error.txt','w')
error_file.write('file started1\n')
m = r.status_code
error_file.write(str(m))
error_file.close()
It basically waits for 20s (added as a test) then tries to retrieve the HTML of the post using its URL, and writes the request status code to a file for debugging.
The problem is I always get status = 404 on the first save, it does work on 2nd and subsequent saves. I thought the way Django works would be, in order:
save instance to database using save(). At this point the post would get a URL
send post_save signal
But then I should be able to retrieve the HTML in post_save. Am I understanding post_save incorrectly?
Added notes:
Putting this code in save() method does not work. Nor should it. The post is added to database at the end of the save() method and so should not have any URL until save() ends.
This is on a production site, not on the development server.
I want to use the links in the HTML to send 'pingbacks' or actually webmention. But all my pingbacks are being rejected because the post does not have a URL yet. This is the bare minimum code that does not work.
Although this is a completely wrong approach(*), the problem is probably in database transactions. Current thread saves the article but within this uncommited transaction you are trying to get these data through another thread (through web server). In that case, this behaviour is fully correct. Either you need to commit before retrieving through another thread or get the HTML by another way.
(*) should be done asynchronously on the background (Celery or other more lightweight async queue app) or you can call the view directly if you want to get the HTML (depending on your view, you may have to forge the request; if too complicated, you can create a helper function that cherrypicks minimal code to render the template). If you only need to call a 3rd party API after you save something, you want to do it asynchronously. If you don't do it, the success of your "save() code" will depend on the availability of your connection or the 3rd party service and you will need to deal with transactions on place where you won't to deal with transactions ;)
Have you tried overriding the object's save method, calling super, waiting and then trying to retrieve the HTML?
Also are you using the development server? It may have issues handling the second request while the first one is still going. Maybe try it on a proper server?
I had similar problem caused probably by the same issue (Asked differently, https://plus.google.com/u/0/106729891586898564412/posts/Aoq3X1g4MvX).
I did not solve it in a proper way, but you can try playing with the database cache, or (seen it in another django database problem) close all of the database connections and requery.
Edit 2:
I have created a simple example (using Django 1.5.5) to test whether this works as intended. As far as I can tell, it does. pre_save fires before a database commit and post_save fires after.
Example detailed:
Two example models.
Article is used for triggering signals.
ArticleUrl is used to log responses from Article.get_absolute_url().
# models.py
from django.db import models
from django.core.urlresolvers import reverse
class Article(models.Model):
name = models.CharField(max_length=150)
def get_absolute_url(self):
"""
Either return the actual url or a string containing '404' if reverse
fails (Article is not saved).
"""
try:
return reverse('article:article-detail', kwargs={'pk': self.pk})
except:
return '404'
class ArticleUrl(models.Model):
article_name = models.CharField(max_length=150)
url = models.CharField(max_length=300)
def __unicode__(self):
return self.article_name + ': ' + self.url
Example views.py and urls.py were omitted as they are simple. I can add them if needed.
# views.py, url.py
Creating pre_save and post_save signals for Article:
# signals.py
from django.db.models import signals
from django.dispatch import receiver
from article.models import Article, ArticleUrl
#receiver(signals.pre_save, sender=Article)
def _log_url_pre(sender, instance, **kwargs):
article = instance
article_url = ArticleUrl(
article_name = 'pre ' + article.name,
url = article.get_absolute_url()
)
article_url.save()
#receiver(signals.post_save, sender=Article)
def _log_url_post(sender, instance, **kwargs):
article = instance
article_url = ArticleUrl(
article_name = 'post ' + article.name,
url = article.get_absolute_url()
)
article_url.save()
Importing my signals.py so Django can use it:
# __init__.py
import signals
After defining the above I went ahead and created a new Article in Django shell (python.exe manage.py shell).
>>> from article.models import *
>>> a = Article(name='abcdd')
>>> a.save()
>>> ArticleUrl.objects.all()
[<ArticleUrl: pre abcdd: 404>, <ArticleUrl: post abcdd: /article/article/8>]
The above example does seem to show that pre_save did indeed not return a url, but post_save did. Both appear to be behaving as intended.
You should check whether your code either deviates from the above example or interferes with program execution in some way.
Edit 1:
Having said that (below), according to What Happens When You Save, the post_save signal should run after a database save.
Could there be other parts of your app/site somehow interfering with this?
Original post:
According to the Django documentation, the post_save signal is sent at the end of save(), not after it.
As far as I understand, Django Signals are synchronous (in-process) so they would stall the actual save(). It won't fully complete until the signals are done.
This isn't always applicable, but have you considered a custom signal you could call after save()?

Django often returns outdated view

I'm pretty new to web development (not to programming), but I just successfully (sort of) deployed a really basic hello-world style Django app. The first time I did it, I had an issue in my HTML. Here's my whole view with the error:
from django.http import HttpResponse
import datetime
def homepage(request):
now=datetime.datetime.now()
html="<html><body><It is now %s.</body></html>" % now
return HttpResponse(html)
The extra < just after the first body tag caused the browser to display a blank page. I figured out what I did and fixed the error. I also added a title so I'd be able to track what's going on (somewhat) better. The old view became this:
from django.http import HttpResponse
import datetime
def homepage(request):
now=datetime.datetime.now()
html="<html><head><title>Hello</title></head><body>It is now %s.</body></html>" % now
return HttpResponse(html)
Now the browser displays the old view (a blank page) most of the time, just the title with a blank body sometimes, and occasionally the whole correct new view. I have no clue what's going on. I'm running nginx with flup to handle the FastCGI. Ideas?
After changing your code you need to restart your FastCGI server, not Nginx.

Optional django-sekizai for an API

I'm writing a basic API, where it extends a DetailView to display the latest object for a given model.
As part of the site, django-sekizai is used for django-cms and as a result all used templates need to have the sekizai tags, however these aren't appropriate for an API as it doesn't need CSS/Javascript but rather outputs JSON/XML/Whatever.
Ideally I'd like for sekizai's context processor to not execute for this view, when writing tests such as
class LatestTest(TestCase):
def test_head_empty(self):
c = Client()
response = c.head(reverse(LatestView.plain_view))
I get the error
TemplateSyntaxError: You must enable the 'sekizai.context_processors.sekizai' template context processor or use 'sekizai.context.SekizaiContext' to render your templates.
During the execution of the client request.

Categories