What kind of search should be used - python

I'm working on a online store using Django, the question is simple, for a simple model named Product with "name" and "description" fields, should I try a full text search using PostgreSQL or a simple query using "icontains" field lookup?

The simplest way to use full text search is to search a single term against a single column in the database.
example: Product.objects.filter(description_text__search='lorem')
Searching against a single field is great but rather limiting. To query against both fields, use a SearchVector
Same way you can use SearchQuery too.

Related

django icontains to search postgres database

I currently have a Django application and postgres database. I want to have a search bar that allows users to enter in a value and it will search some of the fields of the model to search for matching values. I want this to work even for values of "". I currently have:
MyModel.objects.filter(myfield__icontains=search_query).order_by(...)
How would I make this so that it can search multiple fields of the model at the same time. What is the most efficient way to do so? Is "icontains" okay for this?
Any help would be greatly appreciated!
Doing this through regular filter queries and icontains is not advisable as it becomes inefficient pretty quickly - you certainly don't want to be doing that on multiple large text fields.
However, PostgreSQL comes with a full text search engine which is designed for exactly this purpose. Django provides support for this.
You can define a SearchVector to perform full text search on multiple fields at once, e.g., :
from django.contrib.postgres.search import SearchVector
MyModel.objects.annotate(
search=SearchVector('field_1') + SearchVector('field_2'),
).filter(search='search_query')
The documentation I've linked to provides a lot of additional information on how to perform ranking etc. on search results.
Another alternative is to use a search engine like Elasticsearch - whether this is necessary depends on how many objects you have and what kind of filtering and ranking you need to do on results.
You can use Q to search multiple fields, for example:
fields that you want to search:
field0
field1
field2
Django search code:
from django.db.models import Q
search_result = MyModel.objects.filter(
Q(field0_icontains=search_query) |
Q(field1_icontains=search_query) |
Q(field2_icontains=search_query)
).order_by(...)

django filter query / wildcard symbol in database

I have a django model with a field path. This field could contain a wildcard at the end, for example like this: /products/*. Is there a single query solution that allows me to .filter(path__specialmatch="/products/product-1/xy/")?
My solution for now would be stripping segment after segment of the lookup path, and then check with an .filter(path=remaining_segments + "*").
Is this possible with a single query instead?
It is not possible.
Just for reference: As this had to be used for a django redirects app, I've come across django-robust-redirects that does it with a dynamic urls file (see middleware.py, utils.py). Though not answering my question, I'll let this stay for reference.

Escaping special characters in django-haystack query

I'm trying to use Solr specific syntax with some of my django-haytack queries. For example I'd like to search: "state:Georgia", but haystack sends it to Solr as "state\:Georgia", breaking the query. A Raw query can be used but it seems to need to know the field and query beforehand like so: sqs = SearchQuerySet().filter(author=Raw('state:Georgia')), but I'm not always sure beforehand what the exact field should be. It could be state, collector, material, category or a number of others. Does anyone know a way around this or how I can access the user's actual query?
Or perhaps I'm going down the wrong path and a custom Clean method would be in order?
Thanks

haystack solr search ALL fields

I have a solr search engine set up with multiple fields and I want to be able to search ALL fields.
I can do a .filter(content='string') but this only searches whatever fields are in the document=True
EDIT
Also, some of the non document=True fields have different filters/tokenisers applied so im guessing that would not work with adding them into a single field...
Maybe you can make a second field with 'use_template' and a template displaying ALL fields.
I never tried to do this, but this sound a good way to do it to me.
EDIT since OP comment:
Then my best bet is to eaither sublass SearchQueryset to add a method or to create a function that will loop and all fields in your SearchIndex and do something like:
qs = SearchQuerySet().filter(content=query)
for field in fieldlist:
qs = qs.filter_or(**{'field':query})
I have no idea if this works at all but that's worth trying.
#neolaser: I think what you want can be achieved by using DisMax search. It allows searching through multiple fields and specify the boost value for each of them. For more details:
http://wiki.apache.org/solr/SolrRelevancyFAQ
http://wiki.apache.org/solr/DisMaxQParserPlugin
You can search all the fields buy including them all into your filtering query parameter or by naming them in the query string (e.g. if you need to search for "keyword" search for "((field_1:keyword) OR (field_2:keyword) OR (field_3: keyword))" instead).
However, it is usually better to have a dedicated field concatenating all the others you need to search and search this single field. You can set up a copyfield in your schema to have that content generated automatically when your document is indexed.

Django Sphinx Text Search

I am trying out Sphinx search in my Django project. All setup done & it works but need some clarification from someone who has actually used this setup.
In my Sphinx search while indexing, I have used 'name' as the field in my MySQL to be searchable & all other fields in sql_query to be as attributes (according to Sphinx lingo).
So when I search from my Model instance in Django, I get the search results alright but it does not have the 'name' field in the search results. I get all the other attributes.
However, I get the 'id' of the search term. Technically, I could get the 'name' by again querying MySQL but I want to avoid this. Is there anything I am not doing here?
Here's a shot in the dark -
Try to get the name of your index in sphinx.conf same as the table_name you are trying to index. This is a quirk which is missed by lot of people.

Categories