In Django, how can I return the time it took to load a page (not the date) in every page of the site, without having to write in every views.py a code similar to the following one?
start = time.time()
#model operations
loadingpagetime = time.time() - start
If using a TEMPLATE_CONTEXT_PROCESSOR is the best option.
How would I get the whole page loading time from there, instead of just getting the template loading time?
UPDATE:
As the initial question doesn't seem to be clear enough, here is an approach of what would be the Python version of what I want to do.
#!/usr/bin/env python
import cgitb; cgitb.enable()
import time
print 'Content-type: text/html\n\n'
start = time.time()
print '<html>'
print '<head>'
print '</head>'
print '<body>'
print '<div>HEADER</div>'
print '<div>'
print '<p>Welcome to my Django Webpage!</p>'
print '<p>Welcome to my Django Webpage!</p>'
print '<p>Welcome to my Django Webpage!</p>'
print '</div>'
time.sleep(3)
loadingtime = time.time() - start
print '<div>It took ',loadingtime,' seconds to load the page</div>'
print '</body>'
print '</html>'
You can create a custom middleware to log this. Here is how I create a middleware to achieve this purpose base on http://djangosnippets.org/snippets/358/ (I modified the code a bit).
Firstly, assuming your project has a name: test_project, create a file name middlewares.py, I place it in the same folder as settings.py:
from django.db import connection
from time import time
from operator import add
import re
class StatsMiddleware(object):
def process_view(self, request, view_func, view_args, view_kwargs):
'''
In your base template, put this:
<div id="stats">
<!-- STATS: Total: %(total_time).2fs Python: %(python_time).2fs DB: %(db_time).2fs Queries: %(db_queries)d ENDSTATS -->
</div>
'''
# Uncomment the following if you want to get stats on DEBUG=True only
#if not settings.DEBUG:
# return None
# get number of db queries before we do anything
n = len(connection.queries)
# time the view
start = time()
response = view_func(request, *view_args, **view_kwargs)
total_time = time() - start
# compute the db time for the queries just run
db_queries = len(connection.queries) - n
if db_queries:
db_time = reduce(add, [float(q['time'])
for q in connection.queries[n:]])
else:
db_time = 0.0
# and backout python time
python_time = total_time - db_time
stats = {
'total_time': total_time,
'python_time': python_time,
'db_time': db_time,
'db_queries': db_queries,
}
# replace the comment if found
if response and response.content:
s = response.content
regexp = re.compile(r'(?P<cmt><!--\s*STATS:(?P<fmt>.*?)ENDSTATS\s*-->)')
match = regexp.search(s)
if match:
s = (s[:match.start('cmt')] +
match.group('fmt') % stats +
s[match.end('cmt'):])
response.content = s
return response
Secondly, modify settings.py to add your middleware:
MIDDLEWARE_CLASSES = (
'django.middleware.common.CommonMiddleware',
'django.middleware.csrf.CsrfViewMiddleware',
# ... your existing middlewares ...
# your custom middleware here
'test_project.middlewares.StatsMiddleware',
)
Note: you have to add the full path to your middleware class like above, the format is:
<project_name>.<middleware_file_name>.<middleware_class_name>
A second note is I added this middleware to the end of the list because I just want to log the template load time alone. If you want to log the load time of templates + all middlewares, please put it in the beginning of MIDDLEWARE_CLASSES list (credits to #Symmitchry).
Back to the main topic, the next step is to modify your base.html or whatever pages you want to log load time, add this:
<div id="stats">
<!-- STATS: Total: %(total_time).2fs Python: %(python_time).2fs DB: %(db_time).2fs Queries: %(db_queries)d ENDSTATS -->
</div>
Note: you can name the <div id="stats"> and use CSS for that div however you want, but DON'T change the comment <!-- STATS: .... -->. If you want to change it, be sure that you test it against the regex pattern in the created middlewares.py.
Voila, enjoy the statistics.
EDIT:
For those who use CBVs (Class Based Views) a lot, you might have encountered the error ContentNotRenderedError with above solution. Have no fear, here is the fix in middlewares.py:
# replace the comment if found
if response:
try:
# detects TemplateResponse which are not yet rendered
if response.is_rendered:
rendered_content = response.content
else:
rendered_content = response.rendered_content
except AttributeError: # django < 1.5
rendered_content = response.content
if rendered_content:
s = rendered_content
regexp = re.compile(
r'(?P<cmt><!--\s*STATS:(?P<fmt>.*?)ENDSTATS\s*-->)'
)
match = regexp.search(s)
if match:
s = (s[:match.start('cmt')] +
match.group('fmt') % stats +
s[match.end('cmt'):])
response.content = s
return response
I got it working with Django 1.6.x, if you have problem with other version of Django, please ping me in comment section.
Geordi gives you an awesome breakdown of everything that happens in the request cycle. It's a middleware that generates a full call-tree to show you exactly what's going on and how long is spent in each function.
It looks like this:
I highly recommend it :)
Image credit: http://evzijst.bitbucket.org/pycon.in
I can be wrong but the first thing I remembered was document.ready and ajax...
I don't think it is the best solution, but it should be quite simple to implement not only in django but anywhere else.
When you process the request in a view you start timer, and when document.ready is triggered you make an ajax call to stop the timer.
But I should admit that #Hieu Nguyen is quite brilliant)
If you want to show page loading info to your visitors, the answer from Hieu Nguyen will be a great option. But let me also recommend you some good tools.
So if you need this data just for development purposes, take a look at django-debug-toolbar. It will attach a nice toolbar with useful statistics for every page, including time it took for DB queries, template rendering and so on.
Finally, if you need professional profiling tools for Djagno, New Relic might be a very good choice.
I've upgraded middleware.py. In few words, we add some unique text in base template, then replace it in Middleware process_response. This method looks little tricky, but we have generation time closer to true, and we have not problem with javascript. Javascript on client cannot use page headers on firs loaded page, only if create XHR object, then get page.
https://stackoverflow.com/a/35237632/2837890
Related
What I'm trying to do is have it replace all urls from an html file.
This is what I have done but I realized it also deletes everything else after it.
s = 'https://12345678.com/'
site_link = "google"
print(s[:8] + site_link)
It would return as https://google
I have made a code sample.
In this, link_template is a template for a link, and ***** represents where your site_name will go. It might look a bit confusing at first, but if you run it you'll understand.
# change this to change your URL
link_template = 'https://*****.com/'
# a site name, from your example
site_name = 'google'
# this is your completed link
site_link = site_name.join(link_template.split('*****'))
# prints the result
print(site_link)
Additionally, you can make a function for it:
def name_to_link(link_template,site_name):
return site_name.join(link_template.split('*****'))
And then you can use the function like this:
link = name_to_link('https://translate.*****.com/','google')
print(link)
I am very new to Django...
Using submit button i want to run a python file in background and display content on next page...
But my python file takes some time to take out the result so in between i wanted to put a loading html page in between....
I have written some code which correctlty runs the python file but i am not able to incorporate the loading page in between...
Take a look at my function in views.py
def submit(request):
info = request.POST['info']
print('value is ', info)
filename = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
result = run(['python', filename, info], stdout= PIPE )
return render_to_response("loading.html")
run(['python', filename, info], stdout= PIPE )
return render(request, 'python_output.html', context)
ACTUAL RESULT:
return render_to_response("loading.html")
works but then the control does not shifts to run command...
I want to run the loading html page and run the python file in background and when python file run finishes it should move to python _output.html page where output is displayed...
Expected:
Loading page should work and after that control should shift to run command and then it should go to python_output.html page.../
The return statement will terminate the execution of the function so anything below it will never be reached.
You can use Javascript to show the loading icon and then use JQuery to run a GET request in the background where you call a custom view from Django that will output the result of the command. When data is received you can then remove the icon and process the data as you want.
Basic Example :
Django
------
url(r'^command/', views.command, name='command'),
def command(request):
info = request.POST['info']
filename = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"
result = run(['python', filename, info], stdout= PIPE
return result
Javascript
----------
<img id="loading-icon" src="loading.gif">
$.get("/command", function(text)
{
$("#loading-icon").remove();
process(text);
});
You need to understand the basic flow in Django
You can only add one return in your view. after the execution of the first return it goes to the response middleware so all other returns below are ignored.The loading can be done in javascript in frontend
What you want involves a bit more work than you might have expected:
Install a background task framework like celery (and a queue like Redis or RabbitMQ to store tasks) that will fetch tasks from the queue and process them.
Your initial view needs to start a background task.
You want to keep track of the task_id for this task, either by returning it in your response to the user or saving it in the session of the user. Your view responds with the HTML page saying "please have some patience..." and javascript to handle what follows (see below).
Then you need a view that can check the status of the task (based on either the task_id passed in the query or saved in the current session). It responds with JSON, either by saying "status = processing..." or "status = done" with the result of the task.
In the HTML page, you need javascript that queries that view at regular intervals, until the status is "done" and then processes the result of the task to update the HTML of the page.
Search for "django celery tutorial" and you'll find plenty of examples.
Try to use an ajax request when you load your first page (loading.html) to run the python file in background and when it is done, display the result via output.html.
Using JQuery, in the template file, you must call a function like this :
<script>
var url_file_to_run = "{% url "your_app:file_to_run_adress" 0 %}";
var url = "{% url "your_app:python_output" 0 %}";
$.ajax({
url: url_file_to_run,
}
}).done(function(data) {
$( location ).attr("href", url);
});
</script>
I hope i understand your problem.
I am launching a 2nd version of a project, and I want the users to be able to upgrade to the new format. It's not going to be a forced upgrade, so essentially, users will say "Make my page a V2 page". The URLs for their pages will stay the same.
I'd like to be able in the view to say:
def v1_page(request, page_id):
page = get_object_or_404(Page, id=page_id)
if page.upgraded:
# use the v2 view instead (which ends with a return render_to_response('v2_base_page.html', variables))
v2_page(request, page)
else:
# load this page
Or am i going about it the wrong way? I dont want a massive if/else statement.
Function should return result of v2_page call:
def v1_page(request, page_id):
page = get_object_or_404(Page, id=page_id)
if page.upgraded:
return v2_page(request, page)
else:
# load this page
The following django middleware function is used to identify active links in a django response object. If a link is active, its marked with a css class and the href attribute gets replaced by javascript:void(null);. Using this function, the last two lines before the return statement are so slow that i cant use it, further, no css, js and images are rendered. However, if i put these two calls into the for loop, everything is fine and fast. But, i dont want these two calls executed for each active link on page, instead i want them executed only once, and that doesnt work, i really cant see why and what the for loop has to do with it. Its no BeautifulSoup issue, because its the same with re.sub('\s+','',response.content) or the replace function. As far as i have investigated this, i can tell you that the very last line before the return statement is the slow one, as long as its not executed inside the for loop. I'm really excited about a possible explanation.
import re
from django_projects.projects.my_project.settings import SITE_NAME
from BeautifulSoup import BeautifulSoup
class PostRender():
def process_response(self, request, response):
link_pattern=re.compile('<a.*href="(http://%s)*%s".*>' % (SITE_NAME,request.path),re.IGNORECASE)
klass_pattern=re.compile('class="[^"]*"',re.IGNORECASE)
href_pattern=re.compile('href="(http://%s)*%s(\?.*)*"' % (SITE_NAME,request.path),re.IGNORECASE)
#find all active links
search=re.finditer(link_pattern ,response.content)
for res in search:
result=res.group()
klassname='class="active"'
if 'class' in result:
klass=re.search(klass_pattern,result).group().split('=')[1]
if len(klass) != 0:
klassname='class="%s %s"' % (klass[1:-1],'active')
link=re.sub(href_pattern,'href="javascript:void(null);"',re.sub(klass_pattern,klassname,result))
response.content=re.sub(result,link,response.content)
soup=BeautifulSoup(response.content)
response.content=soup.prettify()
return response
So, I'm trying to make a simple call using jQuery .getJSON to my local web server using python/django to serve up its requests. The address being used is:
http://localhost:8000/api/0.1/tonight-mobile.json?callback=jsonp1290277462296
I'm trying to write a simple web view that can access this url and return a JSON packet as the result (worried about actual element values/layout later).
Here's my simple attempt at just alerting/returning the data:
$.getJSON("http://localhost:8000/api/0.1/tonight-mobile.json&callback=?",
function(json){
alert(json);
<!--$.each(json.items, function(i,item){
});-->
});
I am able to access this URL directly, either at http://localhost:8000/api/0.1/tonight-mobile.json or http://localhost:8000/api/0.1/tonight-mobile.json&callback=jsonp1290277462296 and get back a valid JSON packet... So I'm assuming it's in my noob javascript:)
My views.py function that is generating this response looks as follows:
def tonight_mobile(request):
callback = request.GET.get('callback=?', '')
def with_rank(rank, place):
return (rank > 0)
place_data = dict(
Places = [make_mobile_place_dict(request, p) for p in Place.objects.all()]
)
xml_bytes = json.dumps(place_data)
xml_bytes = callback + '(' + xml_bytes + ');'
return HttpResponse(xml_bytes, mimetype="application/json")
With corresponding urls.py configuration:
(r'^tonight-mobile.json','iphone_api.views.tonight_mobile'),
I am still somewhat confused on how to use callbacks, so maybe that is where my issue lies. Note I am able to call directly a 'blah.json' file that is giving me a response, but not through a wired URL. Could someone assist me with some direction?
First, callback = request.GET.get('callback=?', '') won't get you the value of callback.
callback = request.GET.get( 'callback', None )
Works much better.
To debug this kind of thing. You might want to include print statements in your Django view function so you can see what's going on. For example: print repr(request.GET) is a helpful thing to put in a view function so that you can see the GET dictionary.