Best Way to Store Data from Large Web Crawl [closed] - python

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 11 years ago.
I am using a python web crawler for various social networking sites and am trying to determine the best way to store large amounts of data (mostly xml/text data) that I screen scrape. Could you suggest any databases that would be appropriate and easily accessible. Something that works well with python would be nice. Additionally, I would want to go back and parse the data at a later date.

Related

Using pytables and pandas with Django [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 9 years ago.
Does anyone know of a good way to use pytables with the Django as a datastore (mainly reading data)? I am hoping to store large amounts of log data using pytables and then access it in my django app (via pandas - the pandas/pytables interplay is described by one of the comments).
I believe that the Django has an MVC architecture where the models wrap an RDS. Is it possible for to wrap a PyTable instead of an RDS in the model? Or would I need to do this in the controller (which I am guessing would break the MVC)?
Thanks in advance for your help.

Profiling a Python Web Application [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I wrote a web app with Pyramid in python, but I'm struggling to find a suitable profiler. Any suggestions? Thanks.
If you're using pyramid 1.2a1 or better you might try the pyramid_debugtoolbar. linesman also seems like a pretty good choice.

Loggly performance [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I'm getting ready to deploy an app to Heroku and I'm considering adding Loggly as a plugin. I love the idea of being able to aggregate and search my logs more easily, but I'm a bit concerned about what kind of performance hit I might take because of it. Any experiences (good or bad) out there with Loggly?
Just as a note: I'm using Python with Django 1.3
Just for future reference for people... I tried loggly out and was generally very pleased with them. While I never did any formal benchmarking tests (with our without them). Any performance issues were not noticeable to me.

What alternatives to pinax exists? [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I am planning on transferring my django site to pinax so I dont have to deal with the user registration and what not. Before I make this move, I was wondering what the alternative options are so I dont have to make this transfer twice.
So what alternatives to pinax exists? and which one is your favorite?
I don't think there's any alternative, luckily it's active again lately.

What's the best Django app/forum that is super simple and can use the existing Django authentication system? [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I want to wrap it inside a template. That's very important.
There a several forum app exists for django.
Look here django forums
And specially look at DjangoBB

Categories