I have a website that delivers a list of questions to users. This set of questions is the same for all users. I have these questions stored in a text file. The content of this file will not change while the server is running.
I cannot use a static page to display these questions because I have some logic to decide when to show which question.
I would like to cache these questions in memory (instead of reading the file off the hard drive every time a user connects). I am using Django 1.7.
I did read about caching from Django's website, I think the suggested methods (like memcached and database) are too heavy for my need. What would be a good solution for my situation? use a global variable?
Thanks!
There are so many caching back-ends you can use as listed in https://docs.djangoproject.com/en/1.7/topics/cache/
I haven't tried the file system or local memory caching myself, I always needed memcached, but looks like they're available, and the rest is a piece of cake!
from django.core import cache
cache_key = 'questions'
questions = cache.cache.get(cache_key) # to get
if questions:
# use questions you fetched from cache
else:
questions = { 'question1': 'How are you?'} #Something serializable that has your questions
cache.cache.set(cache_key, questions)
Related
I'm making a website that allows users to submit a single post to multiple social media platforms, and track how those posts perform across those platforms.
Currently I use an authenticated instance of PRAW to poll Reddit every so often and fetch all the comments, using something like this:
r = get_authorized_praw_instance(my_user_id)
link = r.submission(id = database.get_link())
link.comments.replace_more(limit = None) # This line places a LOT of requests
for comment in link.comments.list():
store_the_comment(comment)
This is quite expensive though, and I'd rather not brute-force my way through updating my local cache.
Is there another way to do this? Perhaps I could poll a certain endpoint and request all new/edited comments from after a certain timestamp?
Basically I have a program which scraps some data from a website, I need to either print it out to a django template or to REST API without using a database. How do I do this without a database?
Your best bet is to
a.) Perform the scraping in views themselves, and pass the info in a context dict to the template
or
b.) Write to a file and have your view pull info from the file.
Django can be run without a database, but it depends on what applications you enable. Some of the default functionality (auth, sites, contenttypes) requires a database. So you'd need to disable those. If you need to use them, you're SOL.
Other functionality (like sessions) usually uses a database, but you can configure it to use a cache or file or something else.
I've taken two approaches in the past:
1) Disable the database completely and disable the applications that require the database:
DATABASES = {}
2) Use a dummy sqlite database just so it works out of box with the default apps without too much tweaking, but don't really use it for anything. I find this method faster and good for setting up quick testing/prototyping.
And to actually get the data from the scraper into your view, you can take a number of approaches. Store the data in a cache, or just write it directly to your context variables, etc.
I have a method to load all ideas from database. There are few comments on each idea.
I have store users id who commented on ideas, in respective table.
I have a web service which contains all the data related to user id
When i load a page, it took time to fetch information for all users using web service.
I want to use databse caching to store that web service response and use that later on.
How to achieve this, to reduce page load timing?
when should i store that web service response in cache table?
The Django documentation on the cache framework is pretty easy to follow and will show you exactly how to set up a database cache for pages and other things in your views, including for how long you'd like the cache to exist, TIMEOUT, as well as other arguments a cache can take.
Another way to speed up accessing your DB information is to take advantage of CONN_MAX_AGE for your database in your settings.py, who's time can be dependent on how often the database needs to be accessed or how much traffic the site gets (as an example). This is basically letting your DB connection know how long to stay open, and can be found in the settings documentation.
You can store information in a cache when something on the site occurs (such as a new comment) or when that particular request is made for a particular page. It can be entirely up to the needs of your project.
In my multiple choice Quiz show project on google app engine multiple users can use the webapp simultaneously once they are login. But due to some reason they are interfering with each others instances.
Scenario example: Suppose user A wants to use the quiz show for 10 questions and at the same time user B wants to run the quiz show for 10 questions on another machine. But since they are using the application at the same, they are only getting 5 questions each and their result getting messed up.
Does anybody know how to avoid it ? I am not using any session or cookies till now. Is that a solution or something else?
Thanks
#views.py
def display(request):
skipped_questions=[]
question_number=[]
user_answer_list=[]
answer_list=[]
all_questions=[]
if request.method=='POST':
initial_value=1
id_list=[]
result=Questions.objects.all()
for i in result:
id_value=i.id
id_list.append(id_value)
data=request.POST.copy()
total_question=data['number_of_question']
mytime=data['time']
seconds=59
minutes=int(mytime)-1
already_questions=Random_model.objects.all().delete()
already_answers=User_answer.objects.all().delete()
random_questions_list=random.sample(id_list,int(total_question))
for i in random_questions_list:
random_model=Random_model()
random_model.list_id=i
random_model.initial_value=int(initial_value)
random_model.save()
initial_value+=1
question_list=1
a=Random_model.objects.get(initial_value=question_list)
new_question=Questions.objects.get(id=a.list_id)
template_value={ 'output': new_question,'minutes':minutes,'seconds':seconds,'question_list':question_list }
return render_to_response("quiz.html",template_value)
Followup-#Adam:Hi,I have removed global variables and again the program is working fine when I am working alone on my laptop. But when I am asking my colleague to try from his end,we both are getting same questions and interfering in each others sessions due to which end output getting messed up. I started using gae-sessions and able to use request.session but how should I use gae-sessions in this scenario.
Let me know if I am not clear.
Without some concrete details about what kind of data your application stores to make one session different from any other, it is impossible to give you anything really useful, but one approach would be to store it in memcache keyed off of the user's user_id.
Completely hypothetical for-example code:
def get_session_data():
from google.appengine.api import users
found_session = None
user = users.get_current_user()
if user:
from google.appengine.api import memcache
users_session = memcache.get(user.user_id())
return found_session
def save_session_data(session_object):
from google.appengine.api import users
from google.appengine.api import memcache
memcache.set(users.get_current_user().user_id(), serialized_object)
Now, before you go cutting and pasting, there are a lot of caveats to this approach, and it is meant only as a suggested starting point. Memcache is not guaranteed to hold items in memory, and there are plenty of other competing implementations that would be more reliable in some respects.
Fundamentally, I'd suggest using cookies to store the session data, but AppEngine doesn't have native support for cookies, so you'd have to go find an implementation of them and include it in your code. There are a number of fine implementations that are available on Google Code.
Here are some libraries to pick from that provide cookie support. There are even more.
gae-utilities
gae-sessions
app-engine-oil
FOLLOWUP, based on the sample code that you just added:
I don't want to put too fine of a point on it, but what you're doing just ain't gonna work.
Using global variables is generally a bad idea, but it is specifically an unworkable idea in a piece of code that is going to be called by many different users in an overlapping-fashion. The best advice that I can give you is to take all of the painful global variables (which are really specific to a particular user), and store them in a dictionary that is specific to a particular user. The pickling/unpickling code that I posted above is a workable approach, but seriously, until you get rid of those globals, your code isn't going to work.
as a personal project I am trying to write a wiki with the help of django. I'm a beginner when it comes to web development. I am at the (early) point where I need to decide how to store the wiki sites. I have three approaches in mind and would like to know your suggestion.
Flat files
I considered a flat file approach with a version control system like git or mercurial. Firstly, I would have some example wikis to look at like http://hatta.sheep.art.pl/. Secondly, the vcs would probably deal with editing conflicts and keeping the edit history, so I would not have to reinvent the wheel. And thirdly, I could probably easily clone the wiki repository, so I (or for that matter others) can have an offline copy of the wiki.
On the other hand, as far as I know, I can not use django models with flat files. Then, if I wanted to add fields to a wiki site, like a category, I would need to somehow keep a reference to that flat file in order to associate the fields in the database with the flat file. Besides, I don't know if it is a good idea to have all the wiki sites in one repository. I imagine it is more natural to have kind of like a repository per wiki site resp. file. Last but not least, I'm not sure, but I think using flat files would limit my deploying capabilities because web hosts maybe don't allow creating files (I'm thinking, for example, of Google App Engine)
Storing in a database
By storing the wiki sites in the database I can utilize django models and associate arbitrary fields with the wiki site. I probably would also have an easier life deploying the wiki. But I would not get vcs features like history and conflict resolving per se. I searched for django-extensions to help me and I found django-reversion. However, I do not fully understand if it fit my needs. Does it track model changes like for example if I change the django model file, or does it track the content of the models (which would fit my need). Plus, I do not see if django reversion would help me with edit conflicts.
Storing a vcs repository in a database field
This would be my ideal solution. It would combine the advantages of both previous approaches without the disadvantages. That is; I would have vcs features but I would save the wiki sites in a database. The problem is: I have no idea how feasible that is. I just imagine saving a wiki site/source together with a git/mercurial repository in a database field. Yet, I somehow doubt database fields work like that.
So, I'm open for any other approaches but this is what I came up with. Also, if you're interested, you can find the crappy early test I'm working on here http://github.com/eugenkiss/instantwiki-test
In none of your choices have you considered whether you wish to be able to search your wiki. If this is a consideration, having the 'live' copy of each page in a database with full text search would be hugely beneficial. For this reason, I would personally go with storing the pages in a database every time - otherwise you'll have to create your own index somewhere.
As far as version logging goes, you only need store the live copy in an indexable format. You could automatically create a history item within your 'page' model when an changed page is written back to the database. You can cut down on the storage overhead of earlier page revisions by compressing the data, should this become necessary.
If you're expecting a massive amount of change logging, you might want to read this answer here:
How does one store history of edits effectively?
Creating a wiki is fun and rewarding, but there are a lot of prebuilt wiki software packages already. I suggest Wikipedia's List of wiki software. In particular, MoinMoin and Trac are good. Finally, John Sutherland has made a wiki using Django.