Parse ElasticSearch time format - python

I want to know what is the time format of 2021-02-11T14:05:22.123123 to put in query like
query =
'{
"sort": [
{
"date": {
"order": "desc"
}
}
],
"query": {
"bool": {
"must": [
{
"range": {
"date": {
"gte": "2021-02-11T14:05:22.123123",
"format": "WHAT ???????"
}
}
}
]
}
}
}'
What should I need to write into "format"

You need to use this below format of date, in order to parse 2021-02-11T14:05:22.123123
{
"mappings": {
"properties": {
"date": {
"type": "date",
"format": "yyyy-MM-dd'T'HH:mm:ss.SSSSSS"
}
}
}
}

Related

Why i'm getting null value instead of aggregated response?

I'm trying to perform min aggregation using nested aggregation in elasticsearch but still getting null values..
GET /my_index/_search
{
"query": {
"match": {
"FirstName": "Cheryl"
}
},
"aggs": {
"art": {
"nested": {
"path": "art"
},
"aggs": {
"min_price": {
"min": {
"field": "art.Income"
}
}
}
}
}
}
Mappings :
{
"mappings": {
"properties": {
"art": {
"type": "nested",
"properties": {
"FirstName": {
"type": "text"
},
"Price": {
"type": "integer"
}
}
}
}
}
}

Unhashable type 'dict' when trying to send an Elasticsearch

I keep on getting the following error in Python
Exception has occurred: TypeError unhashable type: 'dict'
on line 92
"should": [],
"must_not": []
This is the query string
res = es.search(
scroll = '2m',
index = "logstash-*",
body = {
{
"aggs": {
"2": {
"terms": {
"field": "src_ip.keyword",
"size": 50,
"order": {
"1": "desc"
}
},
"aggs": {
"1": {
"cardinality": {
"field": "src_ip.keyword"
}
}
}
}
},
"size": 0,
"_source": {
"excludes": []
},
"stored_fields": [
"*"
],
"script_fields": {},
"docvalue_fields": [
{
"field": "#timestamp",
"format": "date_time"
},
{
"field": "flow.start",
"format": "date_time"
},
{
"field": "timestamp",
"format": "date_time"
},
{
"field": "tls.notafter",
"format": "date_time"
},
{
"field": "tls.notbefore",
"format": "date_time"
}
],
"query": {
"bool": {
"must": [
{
"range": {
"#timestamp": {
"gte": 1555777931992,
"lte": 1558369931992,
"format": "epoch_millis"
}
}
}
],
"filter": [
{
"match_all": {}
}
],
"should": [],
"must_not": []
}
}
}
})
the value of body is a set ({ } without key-value is a set literal, e.g., {1,2} is a set). Inside this set you have a dictionary.
Items in a set have to be hashable, and dictionary isn't.
As the comment from #Carcigenicate says, it seems like a typo of having {{ }} instead of { } for the value of body.
Elasticsearch documentation shows that body should be a dictionary.
More about sets from python docs

How can I generate an elasticsearch query from a boolean expression, using Python?

I'd like to "translate" a string like:
A AND (C OR B) AND NOT D
into an Elasticsearch query like:
{
"query": {
"bool": {
"must": {
"term": {
"text": "A"
}
},
"must_not": {
"term": {
"text": "D"
}
},
"should": [
{
"term": {
"text": "B"
}
},
{
"term": {
"text": "C"
}
}
],
"minimum_should_match": 1,
"boost": 1
}
}
}
does exists some library which I can use ?
any help appreciated
Thanks!
ok according to:
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html
I can do query like:
{
"query": {
"query_string" : {
"default_field" : "text",
"query" : (this AND (submitted OR flowers) AND NOT blight"
}
}
}
which works great.

Elasticsearch's format error

I wrote
def user(lat, lon, distance, start_time, end_time):
param = {
"query": {
"filter": {
"geo_distance": {
"distance": distance,
"distance_type": "plane",
"location": {
"lat": lat,
"lon": lon
}
}
},
"query": {
"bool": {
"must": [
{"match": {"start_time": start_time}},
{"match": {"end_time": end_time}}
]
}
}
}
}
num = 0
results = get_data().query(param)
But TransportError(400, 'parsing_exception', 'no [query] registered for [filter]') error happens.I think the way of writing format of Elasticsearch is wrong.I used as reference,https://www.elastic.co/guide/en/elasticsearch/reference/5.2/query-dsl-geo-distance-query.html .But I cannot find the wrong point.How should I fix this?What is wrong in my code?
Your query needs to be like this:
param = {
"query": {
"bool": {
"filter": {
"geo_distance": {
"distance": distance,
"distance_type": "plane",
"location": {
"lat": lat,
"lon": lon
}
}
},
"must": [
{
"match": {
"start_time": start_time
}
},
{
"match": {
"end_time": end_time
}
}
]
}
}
}

ElasticSearch error: [function_score] malformed query, expected [END_OBJECT] but found [FIELD_NAME]

The following JSON structure gives me an error when doing a query:
{
"query": {
"function_score": {
"query": {
"bool": {
"should": [
{
"multi_match": {
"query": "BRCA1",
"fuzziness": "AUTO",
"fields": [
"Long_Name",
"Short_Name",
"Uniprot_ID^10",
"Genes^2",
"Diseases^2",
"Function",
"Domains"
]
}
},
{
"term": {
"Is_Reviewed": true
}
},
{
"term": {
"Has_Function": true
}
}
]
}
}
},
"field_value_factor": {
"field": "Number_Of_Structures"
}
},
"size": 100
}
The error is:
[function_score] malformed query, expected [END_OBJECT] but found [FIELD_NAME]
The bool query on its own works perfectly, but as soon as I use function_score, it stops working. I have tried to follow this example: https://www.elastic.co/guide/en/elasticsearch/guide/master/boosting-by-popularity.html
Any ideas as to what I am doing wrong would be much appreciated!
You must put field_value_factor one level higher, inside function_score:
{
"query": {
"function_score": {
"query": {
"bool": {
"should": [
{
"multi_match": {
"query": "BRCA1",
"fuzziness": "AUTO",
"fields": [
"Long_Name",
"Short_Name",
"Uniprot_ID^10",
"Genes^2",
"Diseases^2",
"Function",
"Domains"
]
}
},
{
"term": {
"Is_Reviewed": true
}
},
{
"term": {
"Has_Function": true
}
}
]
}
},
"field_value_factor": {
"field": "Number_Of_Structures"
}
}
},
"size": 100
}

Categories