After updating Python package elasticsearch from 7.6.0 to 8.1.0, I started to receive an error at this line of code:
count = es.count(index=my_index, body={'query': query['query']} )["count"]
receive following error message:
DeprecationWarning: The 'body' parameter is deprecated and will be
removed in a future version. Instead use individual parameters.
count = es.count(index=ums_index, body={'query': query['query']}
)["count"]
I don't understand how to use the above-mentioned "individual parameters".
Here is my query:
query = {
"bool": {
"must":
[
{"exists" : { "field" : 'device'}},
{"exists" : { "field" : 'app_version'}},
{"exists" : { "field" : 'updatecheck'}},
{"exists" : { "field" : 'updatecheck_status'}},
{"term" : { "updatecheck_status" : 'ok'}},
{"term" : { "updatecheck" : 1}},
{
"range": {
"#timestamp": {
"gte": from_date,
"lte": to_date,
"format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd"
}
}
}
],
"must_not":
[
{"term" : { "device" : ""}},
{"term" : { "updatecheck" : ""}},
{"term" : { "updatecheck_status" : ""}},
{
"terms" : {
"app_version" : ['2.2.1.1', '2.2.1.2', '2.2.1.3', '2.2.1.4', '2.2.1.5',
'2.2.1.6', '2.2.1.7', '2.1.2.9', '2.1.3.2', '0.0.0.0', '']
}
}
]
}
}
In the official documentation, I can't find any chance to find examples of how to pass my query in new versions of Elasticsearch.
Possibly someone has a solution for this case other than reverting to previous versions of Elasticsearch?
According to the documentation, this is now to be done as follows:
# ✅ New usage:
es.search(query={...})
# ❌ Deprecated usage:
es.search(body={"query": {...}})
So the queries are done directly in the same line of code without "body", substituting the api you need to use, in your case "count" for "search".
You can try the following:
# ✅ New usage:
es.count(query={...})
# ❌ Deprecated usage:
es.count(body={"query": {...}})
enter code here
You can find out more by clicking on the following link:
https://github.com/elastic/elasticsearch-py/issues/1698
For example, if the query would be:
GET index-00001/_count
{
"query" : {
"match_all": {
}
}
}
Python client would be the next:
my_index = "index-00001"
query = {
"match_all": {
}
}
hits = en.count(index=my_index, query=query)
or
hits = en.count(index=my_index, query={"match_all": {}})
Using Elasticsearch 8.4.1, I got the same warning when creating indices via Python client.
I had to this this way instead:
settings = {
"number_of_shards": 2,
"number_of_replicas": 1
}
mappings = {
"dynamic": "true",
"numeric_detection": "true",
"_source": {
"enabled": "true"
},
"properties": {
"p_text": {
"type": "text"
},
"p_vector": {
"type": "dense_vector",
"dims": 768
},
}
}
es.indices.create(index=index_name, settings=settings, mappings=mappings)
Hope this helps.
Related
{
"_id" : ObjectId("63920f965d15e98e3d7c450c"),
"first_name" : "mymy",
"last_activity" : 1669278303.4341061,
"username" : null,
"dates" : {
"29.11.2022" : {
},
"30.11.2022" : {
}
},
"user_id" : "1085116517"
}
How can I find all documents with 29.11.2022 contained in date? I tried many things but in all of them it detects the dot letter as something else.
Use $getField in $expr.
db.collection.find({
$expr: {
$eq: [
{},
{
"$getField": {
"field": "29.11.2022",
"input": "$dates"
}
}
]
}
})
Mongo Playground
I have tried this query:
body = {
"query": {
"bool": {
"must_not": [{
"match": {
"script": "doc['updated_time'].value == doc['created_time'].value"
}
}]
}
}
}
And my indexed document is:
"hits" : [
{
"_index" : "cam_canvas_update",
"_type" : "_doc",
"_id" : "101",
"_score" : 1.0,
"_source" : {
"created_time" : "2021-08-11T13:44:13.282406282Z",
"updated_time" : "2021-08-11T13:44:13.285397500Z",
"engagement" : "Ford",
"tag_set_2" : "Renew",
"tag_set_3" : "Disputed",
"instance_numbers" : 1,
"canvas_name" : "First",
"recordid" : "ford1",
"pf" : "C6000",
"tag_set_1" : "Sally",
"ldos_date" : "7/7/2018",
"architecture" : "webex"
}
]
I want to compare created_time and updated time of all documents
and as output need only updated documents.
Want to write csv only with that updated documents in elasticsearch.
You need to use filter and script in your query like below:
{
"query": {
"bool": {
"filter": [{
"script": {
"script": "doc['updated_time'].value != doc['created_time'].value"
}
}]
}
}
}
If you don't want milliseconds to be compared, you can use this script instead of previous version:
{
"query": {
"bool": {
"filter": [
{
"script": {
"script": {
"inline": "doc['updated_time'].value.getMillis()/1000 != doc['created_time'].value.getMillis()/1000",
"lang": "painless"
}
}
}
]
}
}
}
Please let me know if you have any problem with this query.
I have a documents in collection and I want to find document and update elements of list.
Here is sample data:
{
{
"_id" : ObjectId("5edd3faaf6c9d938e0bfd966"),
"id" : 1,
"status" : "XXX",
"number" : [
{
"code" : "AAA"
},
{
"code" : "CVB"
},
{
"code" : "AAA"
},
{
"code" : "BBB"
}
]
},
{
"_id" : ObjectId("asseffsfpo2dedefwef"),
"id" : 2,
"status" : "TUY",
"number" : [
{
"code" : "PPP"
},
{
"code" : "SSD"
},
{
"code" : "HDD"
},
{
"code" : "IOO"
}
]
}
}
I planed to find where "id":1 and value of number.code in ["AAA", "BBB"], change number.code to "DDD". I did it with following code:
db.test.update(
{
id: 1,
"number.code": {$in: ["AAA", "BBB"]}
},
{
$set: {"number.$[elem].code": "VVV"}
},
{ "arrayFilters": [{ "elem.code": {$in: ["AAA", "BBB"]} }], "multi": true, "upsert": false
}
)
It works in mongodb shell, but in python (with pymongo) it doesn't with the following error:
raise TypeError("%s must be True or False" % (option,))
TypeError: upsert must be True or False
Please help me. What can I do?
pymongo just has syntax that's a tad different. it would look like this:
db.test.update_many(
{
"id": 1,
"number.code": {"$in": ["AAA", "BBB"]}
},
{
"$set": {"number.$[elem].code": "VVV"}
},
array_filters=[{"elem.code": {"$in": ["AAA", "BBB"]}}],
upsert=False
)
multi flag not needed with update_many.
upsert is False by default hence also redundant.
You can find pymongo's docs here.
I'm trying to make the following aggregate in my Python Project:
pipeline = [
{
'$group': {
'date': { '$max': "$date" },
'_id': {
'interface': "$interface",
'message': "$message",
'server': "$server"
},
'record_count': {
'$sum': '1'
}
}
}
]
errors = EntryError.objects.aggregate(pipeline)
But when the aggregate function is executed, it gives me the following error:
pymongo.errors.OperationFailure: Each element of the 'pipeline' array must be an object
But the same pipeline code works on Robo3T and when using mongo shell.
What am I doing wrong?
I figured out what I was doing wrong.
The code that solved everything it's this one:
pipeline = {
"$group": {
"date": {"$max": "$date"},
"_id": {
"interface": "$interface",
"message": "$message",
"server": "$server"
},
"record_count": {
"$sum": 1
}
}
}
errors = EntryError.objects.filter(
date__gte=start_date,
date__lte=end_date
).aggregate(pipeline)
"pipeline" as dict instead of list.
I am trying to build an index using the python API, with the following code (In particular I am trying to configure an analyzer):
doc = {
"settings": {
"analysis": {
"analyzer": {
"folding": {
"tokenizer": "standard",
"filter": [ "lowercase", "asciifolding" ]
}
}
}
}
}
res = es.indices.create(index='index_db',body=doc)
But when I try to feed the database with some example data: 'My œsophagus caused a débâcle' (the same example of the website) I don't obtain : 'my, oesophagus, caused, a, debacle' but again: 'my, œsophagus caused, a, débâcle'. I think the problem is in the creation of the index. Do I use the correct syntax?
After several attempt I found the solution. It was a syntax problem.
The correct answer is:
doc = {
"index" : {
"analysis" : {
"analyzer" : {
"default" : {
"tokenizer" : "standard",
"filter" : ["standard", "asciifolding"]
}
}
}
}
}
es.indices.create(index='forensic_db',body=doc)