I have tried this query:
body = {
"query": {
"bool": {
"must_not": [{
"match": {
"script": "doc['updated_time'].value == doc['created_time'].value"
}
}]
}
}
}
And my indexed document is:
"hits" : [
{
"_index" : "cam_canvas_update",
"_type" : "_doc",
"_id" : "101",
"_score" : 1.0,
"_source" : {
"created_time" : "2021-08-11T13:44:13.282406282Z",
"updated_time" : "2021-08-11T13:44:13.285397500Z",
"engagement" : "Ford",
"tag_set_2" : "Renew",
"tag_set_3" : "Disputed",
"instance_numbers" : 1,
"canvas_name" : "First",
"recordid" : "ford1",
"pf" : "C6000",
"tag_set_1" : "Sally",
"ldos_date" : "7/7/2018",
"architecture" : "webex"
}
]
I want to compare created_time and updated time of all documents
and as output need only updated documents.
Want to write csv only with that updated documents in elasticsearch.
You need to use filter and script in your query like below:
{
"query": {
"bool": {
"filter": [{
"script": {
"script": "doc['updated_time'].value != doc['created_time'].value"
}
}]
}
}
}
If you don't want milliseconds to be compared, you can use this script instead of previous version:
{
"query": {
"bool": {
"filter": [
{
"script": {
"script": {
"inline": "doc['updated_time'].value.getMillis()/1000 != doc['created_time'].value.getMillis()/1000",
"lang": "painless"
}
}
}
]
}
}
}
Please let me know if you have any problem with this query.
Related
generate unique id in nested document - Pymongo
my database looks like this...
{
"_id":"5ea661d6213894a6082af6d1",
"blog_id":"blog_one",
"comments": [
{
"user_id":"1",
"comment":"comment for blog one this is good"
},
{
"user_id":"2",
"comment":"other for blog one"
},
]
}
I want to add unique id in each and every comment,
I want it to output like this,
{
"_id":"5ea661d6213894a6082af6d1",
"blog_id":"blog_one",
"comments": [
{
"id" : "something" (auto generate unique),
"user_id":"1",
"comment":"comment for blog one this is good"
},
{
"id" : "something" (auto generate unique),
"user_id":"2",
"comment":"other for blog one"
},
]
}
I'm using PyMongo, is there a way to update this kind of document?
it's possible or not?
This update will add an unique id value to each of the comments array with nested documents. The id value is calculated based upon the present time as milliseconds. This value is incremented for each array element to get the new id value for the nested documents of the array.
The code runs with MongoDB version 4.2 and PyMongo 3.10.
pipeline = [
{
"$set": {
"comments": {
"$map": {
"input": { "$range": [ 0, { "$size": "$comments" } ] },
"in": {
"$mergeObjects": [
{ "id": { "$add": [ { "$toLong" : datetime.datetime.now() }, "$$this" ] } },
{ "$arrayElemAt": [ "$comments", "$$this" ] }
]
}
}
}
}
}
]
collection.update_one( { }, pipeline )
The updated document:
{
"_id" : "5ea661d6213894a6082af6d1",
"blog_id" : "blog_one",
"comments" : [
{
"id" : NumberLong("1588179349566"),
"user_id" : "1",
"comment" : "comment for blog one this is good"
},
{
"id" : NumberLong("1588179349567"),
"user_id" : "2",
"comment" : "other for blog one"
}
]
}
[ EDIT ADD ]
The following works from mongo shell. It adds unique id for the comments array's nested documents - unique across the documents.
db.collection.aggregate( [
{
"$unwind": "$comments" },
{
"$group": {
"_id": null,
"count": { "$sum": 1 },
"docs": { "$push": "$$ROOT" },
"now": { $first: "$$NOW" }
}
},
{
"$addFields": {
"docs": {
"$map": {
"input": { "$range": [ 0, "$count" ] },
"in": {
"$mergeObjects": [
{ "comments_id": { "$add": [ { "$toLong" : "$now" }, "$$this" ] } },
{ "$arrayElemAt": [ "$docs", "$$this" ] }
]
}
}
}
}
},
{
"$unwind": "$docs"
},
{
"$addFields": {
"docs.comments.comments_id": "$docs.comments_id"
}
},
{
"$replaceRoot": { "newRoot": "$docs" }
},
{
"$group": {
"_id": { "_id": "$_id", "blog_id": "$blog_id" },
"comments": { "$push": "$comments" }
}
},
{
$project: {
"_id": 0,
"_id": "$_id._id",
"blog_id": "$_id.blog_id",
"comments": 1
}
}
] ).forEach(doc => db.blogs.updateOne( { _id: doc._id }, { $set: { comments: doc.comments } } ) )
You can use ObjectId constructor to create the ids and place them in your nested documents.
I want to search text inside fields.
I tried to fix my problem from this documentation
One of my index contains items which structure is the following:
{
url: "https://exampleurl.com"
username: "some_username"
}
Here is my querys:
"query": {
"multi_match": {
"query": keyword,
"type": "phrase",
"fields": [ "username", "url" ]
}
}
Also bool query:
"query": {
"bool": {
"must": {
"multi_match": {
"query": keyword,
"type": "phrase",
"fields": [ "username", "url" ]
}
},
}
}
"query": {
"bool": {
"must": [{
"match": {
"username": keyword,
}
}, {
"match": {
"url": keyword
}
}]
}
}
But result is a empty array
please try the below query.
Create Index
PUT test
{
"settings" : {
"number_of_shards" : 1
},
"mappings" : {
"properties" : {
"url" : { "type" : "text" },
"username" : { "type" : "text" }
}
}
}
Insert Document
PUT test/_doc/1
{
"url" : "https://exampleurl.com",
"username" : "Arjun Das"
}
Search
GET test/_search
{
"query": {
"multi_match": {
"query": "http",
"type": "best_fields",
"fields": [ "username", "url" ],
"fuzziness":"2"
}
}
}
"search_id" : "7f2d683165",
"uploaded_time" : "2019-05-10 15:25:35.373",
"processing_end_time" : "2019-05-10 15:25:38.115",
"batches" : {
"5cd598617026837753891a2b" : {
"is_reviewed" : false,
"batch_name" : "T--45"
}
}
How can I write correct query for if i want("is_reviewed" : true)?
I have try this:
query={"query": { "nested" : {"path" : "batches","query" : {"bool" : {"should" : [ { "match" : {"batches.is_reviewed" : true} }]}}}}}
res=es.search(index="cool",body={"query":{"match":{"pass":"true"}}})
I want output only "pass:true".
You can either try this:
{
"query": {
"query_string": {
"query": "batches.\\*.is_reviewed:true"
}
}
}
or this if batches is nested:
{
"query": {
"nested": {
"path": "batches",
"query": {
"bool": {
"should": [
{
"query_string": {
"query": "batches.\\*.is_reviewed:true"
}
}
]
}
}
}
}
}
I have a elastic search index collection like below,
"_index":"test",
"_type":"abc",
"_source":{
"file_name":"xyz.ex"
"metadata":{
"format":".ex"
"profile":[
{"date_value" : "2018-05-30T00:00:00",
"key_id" : "1",
"type" : "date",
"value" : [ "30-05-2018" ]
},
{
"key_id" : "2",
"type" : "freetext",
"value" : [ "New york" ]
}
}
Now I need to search for document by matching key_id to its value. (key_id is some field whose value is stored in "value")
Ex. For key_id='1'field, if it's value = "30-05-2018" it should match the above document.
I tried mapping this as a nested object, But I am not able to write query to search with 2 or more key_id matching its respective value.
This is how I would do it. You need to AND together via bool/filter (or bool/must) two nested queries for each of the condition pair, since you want to match two different nested elements from the same parent document.
{
"query": {
"bool": {
"filter": [
{
"nested": {
"path": "metadata.profile",
"query": {
"bool": {
"filter": [
{
"term": {
"metadata.profile.f1": "a"
}
},
{
"term": {
"metadata.profile.f2": true
}
}
]
}
}
}
},
{
"nested": {
"path": "metadata.profile",
"query": {
"bool": {
"filter": [
{
"term": {
"metadata.profile.f1": "b"
}
},
{
"term": {
"metadata.profile.f2": false
}
}
]
}
}
}
}
]
}
}
}
I'd like to "translate" a string like:
A AND (C OR B) AND NOT D
into an Elasticsearch query like:
{
"query": {
"bool": {
"must": {
"term": {
"text": "A"
}
},
"must_not": {
"term": {
"text": "D"
}
},
"should": [
{
"term": {
"text": "B"
}
},
{
"term": {
"text": "C"
}
}
],
"minimum_should_match": 1,
"boost": 1
}
}
}
does exists some library which I can use ?
any help appreciated
Thanks!
ok according to:
https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-query-string-query.html
I can do query like:
{
"query": {
"query_string" : {
"default_field" : "text",
"query" : (this AND (submitted OR flowers) AND NOT blight"
}
}
}
which works great.