How to make a product search engine in Python - python

I have a database of possibly infinitely many products.
Every single product has a Title and tags representing it.
I am trying to make a system where somebody can search for the product using the Title primarily, but helping to sort out the Top 100 products using tags.
Some people have suggested Django (Taggit) or just forming an API.
I am not sure how relevant that is and how needed it is for something simple like the task mentioned above.
I am also looking for something efficient.

Related

The best way to work with larger data queries?

After one month of learning and toying around with python and doing tons and tons of exercieses and samples I am still not really able to answer one important question to myself. If I generate data, for example with loading them from a xml or from scraping. How do I work with them? Right now I put each and every entrie directly into a sqlite db.
For exmaple:
I read a news feed, I put id, title, description, link, tags into one row in my sqlite db. I do some categorizing according to keywords within the the title or description. Each news entry goes into one row. After that I can read them row by row or at once.
I can filter them, sort them... But somehow I feel like there is a better way. Put them all with a dictionary into one big list and somehow work with this list. And then after I worked thru them sorted them or got even more infos with that data. Put them into the sqlite db. But none of my books or tutorial talk about this topic!? I try to make an example is a news article more important if a certain keyword comes up multiple times or in with an other keyword. Compare the news from that page with the news from another page with same keywords.
I guess u will be laughing and say. He is talking about ..... how can he not know that. But I am new to all of that. Sooorrry.
Thank you for your help guiding me in the right direction.

Referencing if/then statements with a model?

I have a site for my students where I have pulled in a bunch of information from different websites they use for instruction. I want to be able to award 'badges' if they meet certain criteria, like reading 50 books, or spending 100 minutes practicing math, etc. That data is stored as objects in different models on my site.
I can't figure out how to cleanly make different badge objects with unique requirements for unlocking each badge without being incredibly repetitive.
My basic model:
class Sticker(models.Model):
name = models.CharField(max_length=255,)
slug = models.CharField(max_length=255,)
image = models.ImageField(upload_to='stickers')
#requirements = ?
I'd like to be able to easily add badges with just a line of code like this as the requirement:
if books_read >= 50:
return True
Part of the complexity is that there are a lot of different types of requirements with different models for all the different "Stickers" they can earn.
I tried Django Badgify but ran into a couple issues. My students aren't actual Django Users, and I don't know how to separate that part out, as the script is far more complex than I'm currently able to modify with my skillset.
What's the best way to work in a bit of code like this? Is there a way I can run code that's entered into a TextField? Thanks in advance.
If you are trying to make a badge per achievement then you could try this... (everything I give is an example)
Check the perquisites
If they match then proceed to 4
If not match then check next achievement and go to 1
Give them the badge!
You need to make that same script for every achievement.
Im not entirely sure about the modules functions you are using but I would imagine that you need to check each user account separately per achievement.

Querying a model based on user input in Django

I am new to Django and I am trying to build a basic search/filter feature; for example, a basic version of the refine/filter part on amazon while searching for products. (I am using Sqlite3 in development)
I think I could implement a filter in which you could click part of a form and it would return a page with the database items that match the query, however, I am not sure on how I could do this if the search contained more than one part to the query, for example if the search was to find a book that was published before 2009 and costs more than £4.99, I am unsure on how to do this.
I am looking to build a checkbox type of filter rather than a search like google.
This sort of filter/search
All help is appreciated, Thank You.
https://django-filter.readthedocs.io
This is what you're looking for.

How to get all Wikipedia articles from category and sub categories using Python? [duplicate]

I want to get all the articles names under a category and its sub-categories.
Options I'm aware of:
Using the Wikipedia API. Does it have such an option??
d/l the dump. Which format would be better for my usage?
There is also an option to search in Wikipedia something like incategory:"music", but I didn't see an option to view that in XML.
Please share your thoughts
The following resource will help you to download all pages from the category and all its subcategories:
http://en.wikipedia.org/wiki/Wikipedia:CatScan
There is also an API available here:
https://www.mediawiki.org/wiki/API:Categorymembers
You can do this through the following two API methods:
For articles pages for this category
YOUR_URL/api.php?action=query&format=json&list=categorymembers&cmtitle=Category:Music
For get subcategories:
YOUR_URL/api.php?action=query&format=json&list=categorymembers&cmtype=subcat&cmtitle=Category:Music
You can get more info on Mediawiki API
Note that Wikipedia's categorization system is not a tree, or even an acyclic graph. It is quite possible that by continually following subcategory links you will eventually wind up back where you started.
If you are going to be making many such queries, you would be best served by downloading a database dump. If this will be an infrequent thing and will only be dealing with small categories, you could probably get away with making repeated queries to list=categorymembers.
incategory:"music" does not appear to do subcategory searching.

Performing multiple searches of a term in Twitter

I have little working knowledge of python. I know that there is something called a Twitter search API, but I'm not really sure what I'm doing. I know what I need to do:
I need point data for a class. I thought I would just pull up a map of the world in a GIS application, select cities that have x population or larger, then export those selections to a new table. That table would have a key and city name.
next i randomly select 100 of those cities. Then I perform a search of a certain term (in this case, Gaddafi) for each of those 100 cities. All I need to know is how many posts there were on a certain day (or over a few days depending on amount of tweets there were).
I just have a feeling there is something that already exsists that does this, and I'm having a hard time finding it. I've dowloaded and installed python-twitter but have no idea how to get this search done. Anyone know where I can find or how I can make this tool? Any suggestions would really help. Thanks!
A tweet itself comes with a geo tag. But it is a new feature and majority tweets do not have it. So it is not possible to search for all tweets containing "Gaddafi" from a city given the city name.
What you could do is the reverse, you search for "Gaddafi" first (regardless of geo location), using search api. Then, for each tweet, find the location of the poster (either thru the RESTful api or use some sort of web scraping).
so basically you can classify the tweets collected according to the location of the poster.
I think only tweepy have access to both twitter search API as well as RESTful API.

Categories