search = {
"from": str(start),
"size": str(size),
"query": {
"bool": {
"must": {
"multi_match": {
"query":query,
"fields":["name","description","tags","comments","created","creator","transaction","wallet"],
"operator":"or"}
},
"filter": { "term": { "channel": channel } } } } }
This is the python dict object. It gets the following error:
elasticsearch.BadRequestError: BadRequestError(400, 'parsing_exception', '[bool] malformed query, expected [END_OBJECT] but found [FIELD_NAME]')
I'm not seeing it. Please help. Start, size, query, and channel are all variables.
I have looked at a lot of example elasticsearch queries. Nothing I've tried has gotten passed syntax errors. I've also tried simple_search_string and a simple multi_match. I always need start and size, and always need to filter on channel.
So the issue is some of those fields are arrays and need [] inside them. Specifically must and filter. Adding appropriate braces solved the issue. Here's the new format:
search = {
"from": start,
"query": {
"bool": {
"must": [
{ "multi_match": {
"query": query,
"fields": ["name","description","tags","comments","created","creator","transaction","wallet"]
} },
{ "match": {
"channel": channel
} }
]
}
}
}
Notice I've also dropped using the filter and just added another match term. I'm using size in the search call as one of its parameters.
Related
I am not able to understand the implementation of the elastic search query along with the synonym table. With a general query, I don't have any search problems but incorporating synonyms as become an issue to me.
es.search(index='data_inex', body={
"query": {
"match": {"inex": "tren"}
},
"settings": {
"filter": {
"synonym": {
"type": "synonym",
"lenient": true,
"synonyms": [ "foo, baz", "tren, hut" ]
}
}
}
}
)
Also, is it possible to use a file instead of this array?
Check the documentation: Click Here
You can configure synonyms file as well:
PUT /test_index
{
"settings": {
"index": {
"analysis": {
"analyzer": {
"synonym": {
"tokenizer": "whitespace",
"filter": [ "synonym" ]
}
},
"filter": {
"synonym": {
"type": "synonym",
"synonyms_path": "analysis/synonym.txt" // <======== location of synonym file
}
}
}
}
}
}
Please note:
Changes in the synonyms file will not reflect in the documents indexed before the change. Re-indexing is required for the same.
You cannot change the mapping (including the analyzer) of an existing field. What you need to do if you want to change the mapping of existing documents is reindex those documents to another index with the updated mapping.
Search query doesn't support "settings".
I'm curious about the best approach to count the instances of a particular field, across all documents, in a given ElasticSearch index.
For example, if I've got the following documents in index goober:
{
'_id':'foo',
'field1':'a value',
'field2':'a value'
},
{
'_id':'bar',
'field1':'a value',
'field2':'a value'
},
{
'_id':'baz',
'field1':'a value',
'field3':'a value'
}
I'd like to know something like the following:
{
'index':'goober',
'field_counts':
'field1':3,
'field2':2,
'field3':1
}
Is this doable with a single query? or multiple? For what it's worth, I'm using python elasticsearch and elasticsearch-dsl clients.
I've successfully issued a GET request to /goober and retrieved the mappings, and am learning how to submit requests for aggregations for each field, but I'm interested in learning how many times a particular field appears across all documents.
Coming from using Solr, still getting my bearings with ES. Thanks in advance for any suggestions.
The below will return you the count of docs with "field2":
POST /INDEX/_search
{
"size": 0,
"query": {
"bool": {
"filter": {
"exists": {
"field": "field2"
}
}
}
}
}
And here is an example using multiple aggregates (will return each agg in a bucket with a count), using field exist counts:
POST /INDEX/_search
{
"size": 0,
"aggs": {
"field_has1": {
"filter": {
"exists": {
"field": "field1"
}
}
},
"field_has2": {
"filter": {
"exists": {
"field": "field2"
}
}
}
}
}
The behavior within each agg on the second example will mimic the behavior of the first query. In many cases, you can take a regular search query and nest those lookups within aggregate buckets.
Quick time-saver based on existing answer:
interesting_fields = ['field1', 'field2']
body = {
'size': 0,
'aggs': {f'has_{field_name}': {
"filter": {
"exists": {
"field": f'export.{field_name}'
}
}
} for field_name in interesting_fields},
}
print(requests.post('http://localhost:9200/INDEX/_search', json=body).json())
I am using Python to query Elasticsearch with a custom query. Let's look at a very simple example that will search for a given term in the field 'name' and another one in the 'surname' field of the document:
from elasticsearch import Elasticsearch
import json
# read query from external JSON
with open('query.json') as data_file:
read_query= json.load(data_file)
# search with elastic search and show hits
es = Elasticsearch()
# set query through body parameter
res = es.search(index="test", doc_type="articles", body=read_query)
print("%d documents found" % res['hits']['total'])
for doc in res['hits']['hits']:
print("%s) %s" % (doc['_id'], doc['_source']['content']))
'query.json'
{
"query": {
"bool": {
"should": [
{
"match": {
"name": {
"query": "Star",
"boost": 2
}
}
},
{
"match": {
"surname": "Fox"
}
}
]
}
}
}
Now, I am expecting the input of search words from the user, the first word that is typed in is used for the field 'name' and the second one for 'surname'. Let's imagine I will replace the {$name} and {$surname} with the two words that have been typed in by the user using python:
'query.json'
{
"query": {
"bool": {
"should": [
{
"match": {
"name": {
"query": "{$name}",
"boost": 2
}
}
},
{
"match": {
"surname": "{$surname}"
}
}
]
}
}
}
Now the problem arises when the user doesn't input the surname but only the name, so I end up with the following query:
'query.json'
{
"query": {
"bool": {
"should": [
{
"match": {
"name": {
"query": "Star",
"boost": 2
}
}
},
{
"match": {
"surname": ""
}
}
]
}
}
}
The field "surname" is now empty and elasticsearch will look for hits where "surname" is an empty string, which is not what I want. I want to ignore the surname field if the input term is empty. Is there any mechanism in elasticsearch to set a part of query to be ignored if the given term is empty?
{
"query": {
"bool": {
"should": [
{
"match": {
"name": {
"query": "Star",
"boost": 2
}
}
},
{
"match": {
"surname": "",
"ignore_if_empty" <--- this would be really cool
}
}
]
}
}
}
Maybe there is any other way of generating query strings? I can't seem to find anything about query generation in Elasticsearch. How do you guys do it? Any input is welcome!
Python DSL seems to be the proper way of doing it https://github.com/elastic/elasticsearch-dsl-py/
i am trying to write a query where it searches in elastic that a particular field is null.this query us executed in python using Python Elasticsearch Client.
query:
{
"_source": ["name"],
"query": {
"nested": {
"path": "experience",
"query": {
"match": {
"experience.resignation_date": {
"query": None
}
}
}
}
}
}
since its python i have used None in the query part but it throwing me this error.
elasticsearch.exceptions.RequestError: TransportError(400, 'parsing_exception', '[match] unknown token [VALUE_NULL] after [query]')
The missing query is deprecated, you're looking for bool/must_not + exists
{
"_source": [
"name"
],
"query": {
"nested": {
"path": "experience",
"query": {
"bool": {
"must_not": {
"exists": {
"field": "experience.resignation_date"
}
}
}
}
}
}
}
With this expression you're not querying for null, you're saying use null as the query.
Query must always be one of the query types that ElasticSearch has defined, such as for example the "match" query.
The kind of syntax you wanted to write here was
query: {
match: {
"experience.resignation_date": None
}
}
Where you are asserting that the value matches "None"
However, there is a better specific query type for matching documents with empty field values called "missing".
It will match null fields as well as documents which lack the property all together. Whether this is more appropriate for you depends on your needs.
EDIT: As a subsequent answer points out "missing" is actually deprecated now. The equivalent is to negate the "exists" query instead.
query: {
bool: {
must_not: {
exists: "experience.resignation_date"
}
}
}
I am recently using elasticsearch in a website. The scenario is, I have to search a string on afield. So, if the field is named as title then my search query was,
"query" :{"match": {"title": my_query_string}}.
But now I need to add another field in it. Let say, category. So i need to find the matches of my string which are in category :some_category and which have title : my_query_string I tried with multi_match. But it does not give me the result i am looking for. I am looking into query filter now. But is there way of adding two fields in such criteria in my match query?
GET indice/_search
{
"query": {
"bool": {
"should": [
{
"match": {
"title": "title"
}
},
{
"match": {
"category": "category"
}
}
]
}
}
}
Replace should with must if desired.
Ok, so I think that what you need is something like this:
"query": {
"filtered": {
"query": {
"match": {
"title": YOUR_QUERY_STRING,
}
},
"filter": {
"term": {
"category": YOUR_CATEGORY
}
}
}
}
If your category field is analyzed, then you will need to use match instead of term in the filter.
"query": {
"filtered": {
"query": {
"bool": {
"should": [
{"match": {"title": "bold title"},
{"match": {"body": "nice body"}}
]
}
},
"filter": {
"term": {
"category": "xxx"
}
}
}
}