Django rank search in Postgres not matching - python

I have a Tutorial model with name 'building'.
Here is search.
tutorial_search = Tutorial.objects.annotate(
rank=SearchRank(SearchVector('name'), query)
).filter(rank__gte=0.0001).order_by('-rank')
This query finds my model
query = 'bui:*'
But this one doesnt
query = 'buildi:*'
I cant figure out what is causing it. Is it english accent?
Seems like simple search.
EDIT:
I tried pure SQL queries in postgres, and I get the same result.
This works:
SELECT to_tsvector('building') ## to_tsquery('(build:*)');
Returns True
But this one:
SELECT to_tsvector('building') ## to_tsquery('(buildi:*)');
Returns False
Why?
Thanks!

If I understood you asked for the motivations of the behavior of your queries.
You can test the lexems generated from your words in your queries as below:
SELECT ts_lexize('english_stem', 'building');
ts_lexize
-----------
{build}
SELECT ts_lexize('english_stem', 'build');
ts_lexize
-----------
{build}
SELECT ts_lexize('english_stem', 'buildi');
ts_lexize
-----------
{buildi}
As you can see the lexems generated by 'building' and 'build' are the same ('build') and 'buildi' generate a different 'buildi'.
This is because the first two work and the third does not.
More info in the Testing and Debugging Text Search documentation.

Related

Commenting in Python: Is -- legitimate?

I have inherited a set of Python code, and I think I know the answer to this, but wish to be sure.
It looks like both #s and --s are used interchangeably throughout the code to signify comments but any searches I am doing do not yield information about using the --.
I used to do the --s within Teradata a long time ago. Am I missing something?
Adding in some additional information too: I am running the code within Spyder (Python3.6). Perhaps that will shed more light on what is going on.
Below is an example of the -- seeming to work:
qry = """
select s2019.sk2019,
s2018.sk2018
from (select distinct h1.key sk2019,
e.Territory Territory2019 from cdw.fact_header
left join cdw.dim_event e on h1.event = e.event
where e.sy in (2019)
and e.program_name = 'xyz'
and a.Country = 'USA'
-- and h1.code = 'DC'
and h1.key > 0
One line comments are done using the # while multiline comments are done using the triple-quote
""" Text here
and here"""
the -- comment is used in sqlite with python. Do you have that comment inside a query that is also written inside a string/triple-quote comment? Like:
""" SELECT * FROM TABLE
--WHERE N = X"""
UPDATE 1:
Here you can see and example of comment inside the query done in a python code
query = """ SELECT * FROM TABLE
--WHERE N = X"""
By putting -- inside the query, before WHERE I've made everything that follows ignored. So when the query is executed I will only execute the first line of the query.

Django Q object query filtering with multiple conditions failing

I am trying to apply multiple conditions to my filter. The model looks like this
class modelChat(models.Model):
source = models.ForeignKey(modelEmployer,related_name = 'rsource',on_delete=models.CASCADE,null=True,default=None,blank=True)
job = models.ForeignKey(modelJob,on_delete=models.CASCADE,null=True,default=None,blank=True)
destination = models.ForeignKey(modelEmployer,related_name = 'rdestination',on_delete=models.CASCADE,null=True,default=None,blank=True)
Initially I am trying to obtain an instance of chat that involves 2 parties based on a job. At one point source can be a destination and sometimes the destination can be the source. but the job remains the same.
This is what my query looks like
querySet = modelChat.objects.filter(
(Q(source=modelEmployerSourceInstance) | Q(destination=modelEmployerSourceInstance))
&
(Q(destination=modelEmployerDestinationInstance) | Q(destination=modelEmployerDestinationInstance))
&
Q(job_id=job_id)
)
The job id is correct and I know there is only one item in the DB. However this query alway returns back an empty item. Any suggestions why this is wrong and how I can fix it ?
I can't say for sure if that's the problem since you forgot to show what you really have in your DB but here:
(Q(destination=modelEmployerDestinationInstance) | Q(destination=modelEmployerDestinationInstance))
I assume you want:
(Q(source=modelEmployerDestinationInstance) | Q(destination=modelEmployerDestinationInstance))
instead...
Note that the logical would be much more obvious with shorter names, ie source and destination instead of modelEmployerSourceInstance modelEmployerDestinationInstance:
q = (
(Q(source=source) | Q(destination=source))
& (Q(source=destination | Q(destination=destination))
& Q(job_id=job_id)
)
querySet = modelChat.objects.filter(q)
Meaningful names are a good thing, but they have to be short and distinct enough. With "modelEmployerXXXInstance", you have four words to parse, and with the only distinctive (hence relevant) part of the name being in third position, your brain tends to skip over this part. The "model", "Employer" and "Instance" parts are actually just noise.

Pymongo how to use full text search

I am looking to implement full text search in my python application using pymongo. I have been looking at this question but for some reason I am unable to implement this in my project as I am getting an error no such cmd: text. Can anyone direct me on what I am doing wrong?
Here is my code:
db = client.test
collection = db.videos
def search_for_videos(self, search_text)
self.db.command("text", "videos",
search=search_text,
limit=10)
The collection I am trying to search is called videos however I am not sure if I am putting this in the correct parameter, and I also am not sure if I need the line project={"name": 1, "_id": 0}.
The documentation here I believe is using the mongo shell to execute commands, however I wish to perform this action in my code.
I have looked at using the db.videos.find() function, but cannot seem to implement it correctly either.
How to I use PyMongo Full Text Search from my Python Code?
First be sure that you have a text index created on the field as mentioned here or you can just do it with pymongo too :
collection.create_index([('your field', 'text')])
Using pymongo you can do this to search:
collection.find({"$text": {"$search": your search}})
your function should look like this:
def search_for_videos(search_text):
collection.find({"$text": {"$search": search_text}}).limit(10)
I hope this helps you.
First create a text index based on the field you want to do the search on.
from pymongo import TEXT
db = MongoClient('localhost',port = 27017).DBNAME
db.collection.create_index([('FIELD_NAME',TEXT)],default_language ="english")
once you create the text index use the following query to search text. Depending on the size of your database, it might take long to create the text index.
db.collection.find({"$text": {"$search": your search}})

Django/Postgres - No function matches the given name and argument types

I'm trying to create a search system in my Django and Postgresql project but I keep running into an error when I try to make a query.
Whenever I try these commands in the shell:
vector = SearchVector('title','tags')
query = SearchQuery('book') | SearchQuery('harry')
My_Library.objects.annotate(similarity=TrigramSimilarity(vector,test),).filter(similarity__gt=0.3).order_by('-similarity')
I get the error:
"No function matches the given name and argument types. You might need to add explicit type casts."
I've been testing other options for a while, and the only way I can successfully pass a search query without an error is by using two strings in the place of query and vector.
My_Library.objects.annotate(similarity=TrigramSimilarity('title','my search query'),).filter(similarity__gt=0.3).order_by('-similarity')
This will successfully pass my search with no error.
Why am I getting this error, and how can I fix it?
I've been basing my code off of this Full Text Search documentation
TrigramSimilarity takes 2 strings as arguments
You're trying to pass it a SearchVector and a SearchQuery.
that won't work
If you want to search by multiple tags, you probably need to aggregate multiple of the similarity queries with a | and then sort on similarity, something like:
from django.db.models import Q
My_Library.objects.annotate(
Q(similarity=TrigramSimilarity('title','my search query'),)) |
Q(similarity=TrigramSimilarity('title','my search query'),))
).filter(similarity__gt=0.3).order_by('-similarity')
More details on Q
https://docs.djangoproject.com/en/1.11/ref/models/querysets/#q-objects

obtain current status of Wikipedia articles?

I am using Python and MySQL to query mediawiki database to get the current status of articles (i.e. whether the article is FA, GA, GAN etc.) but have been unable to do so.
I know current status is stored in the old_text field of the text table. I was trying to something like:
loc = select (locate('currentstatus', old_text))
query = ('select substring(old_text, '%s', 20) from wikidb where page_id = 1234' % (loc))
but unfortunately loc gives the first occurrence of currentstatus and not the last which is not very 'current' since the newest/latest status is on the bottom.
I am not sure how to fix it or if I am using the right approach.
For Wikipedia, it would be more to the point to examine the categories the article is in. Or if processing raw wikitext, look for the corresponding template:
Featured articles (FA) are in [[category:Featured articles]] and use {{featured article}}, which references [[template:featured article]]
Good articles (GA) are in [[category:Good articles]] and use {{good article}}, which references [[template:good article]]
Both those categories are hidden, so you would have to enable the preference for displaying hidden categories, or traverse the category contents to see if the article is there.
Other article classes (A, B, C, FL, Start, Stub, List, undefined) are assessed on the corresponding talk page using one or more WikiProject templates. There is no standard.

Categories