I am newbie in Python. I have some difficulties generating a nested JSON using for loop in python. For generating a nested JSON, I got the length of dictionary on runtime and based on the dictionary length I want to generate nested JSON. eg. I got the length of dictionary is 4. The dictionary length may vary. Here is my data_dict dictionary:
data_dict = {"PHOTO_1" : {"key1" : "PHOTO_2", "key2" : "PHOTO_3", "key3" : "PHOTO_4"}, "PHOTO_2" : {"key1" : "PHOTO_1", "key2" : "PHOTO_3"},"PHOTO_3" : {"key1" : "PHOTO_2"},"PHOTO_4" : {"key1" : "PHOTO_1", "key2" : "PHOTO_2", "key3" : "PHOTO_3"}}
Expected result :
{
"Requests": [
{
"photo": {
"photoId": {
"id": "PHOTO_1"
},
"connections": {
"target": {
"id": "PHOTO_2"
}
}
},
"updateData": "connections"
},
{
"photo": {
"photoId": {
"id": "PHOTO_1"
},
"connections": {
"target": {
"id": "PHOTO_3"
}
}
},
"updateData": "connections"
},
{
"photo": {
"photoId": {
"id": "PHOTO_1"
},
"connections": {
"target": {
"id": "PHOTO_4"
}
}
},
"updateData": "connections"
},
{
"photo": {
"photoId": {
"id": "PHOTO_2"
},
"connections": {
"target": {
"id": "PHOTO_1"
},
}
},
"updateData": "connections"
},
{
"photo": {
"photoId": {
"id": "PHOTO_2"
},
"connections": {
"target": {
"id": "PHOTO_3"
},
}
},
"updateData": "connections"
},
{
"photo": {
"photoId": {
"id": "PHOTO_3"
},
"connections": {
"target": {
"id": "PHOTO_2"
},
}
},
"updateData": "connections"
},
{
"photo": {
"photoId": {
"id": "PHOTO_4"
},
"connections": {
"target": {
"id": "PHOTO_1"
},
}
},
"updateData": "connections"
},
{
"photo": {
"photoId": {
"id": "PHOTO_4"
},
"connections": {
"target": {
"id": "PHOTO_2"
},
}
},
"updateData": "connections"
},
{
"photo": {
"photoId": {
"id": "PHOTO_4"
},
"connections": {
"target": {
"id": "PHOTO_3"
},
}
},
"updateData": "connections"
}
]
}
Please help. I'm not getting how to solve this query? Please don't mark it duplicate. I have already checked all the answers and my JSON query is totally different.
The solution using itertools.permutations() function:
import itertools, json
data_dict = {"first_photo" : "PHOTO_1", "second_photo" : "PHOTO_2", "Thrid" : "PHOTO_3"}
result = {"Requests":[]}
for pair in sorted(itertools.permutations(data_dict.values(), 2)):
result["Requests"].append({"photo":{"photoId":{"id": pair[0]},
"connections":{"target":{"id": pair[1]}}},"updateData": "connections"})
print(json.dumps(result, indent=4))
The additional approach for the new input dict:
data_dict = {"PHOTO_1" : {"key1" : "PHOTO_2", "key2" : "PHOTO_3", "key3" : "PHOTO_4"}, "PHOTO_2" : {"key1" : "PHOTO_1", "key2" : "PHOTO_3"},"PHOTO_3" : {"key1" : "PHOTO_2"},"PHOTO_4" : {"key1" : "PHOTO_1", "key2" : "PHOTO_2", "key3" : "PHOTO_3"}}
result = {"Requests":[]}
for k,d in sorted(data_dict.items()):
for v in sorted(d.values()):
result["Requests"].append({"photo":{"photoId":{"id": k},
"connections":{"target":{"id": v}}},"updateData": "connections"})
print(json.dumps(result, indent=4))
Related
I have a json file name input which as follows
{
"abc": {
"dbc": {
"type": "string",
"metadata": {
"description": "Name of the namespace"
}
},
"fgh": {
"type": "string",
"metadata": {
"description": "Name of the Topic"
}
}
},
"resources": [
{
"sku": {
"name": "[parameters('sku')]"
},
"properties": {},
"resources": [
{
"resources": [
{
"resources": [
{
"properties": {
"filterType": "SqlFilter",
"sqlFilter": {
"sqlExpression": "HAI"
}
}
}
]
}
]
}
]
}
]
}
I want "sqlExpression": "HAI" value to be replaced with BYE as below
"sqlExpression": "BYE"
I want python code to do it, I tried the below code but not working
input['resources'][0]['resources'][0]['resources'][0]['resources'][0][properties][0][sqlFilter][0][sqlExpression][0]='BYE'
inp = {
"abc": {
"dbc": {
"type": "string",
"metadata": {
"description": "Name of the namespace"
}
},
"fgh": {
"type": "string",
"metadata": {
"description": "Name of the Topic"
}
}
},
"resources": [
{
"sku": {
"name": "[parameters('sku')]"
},
"properties": {},
"resources": [
{
"resources": [
{
"resources": [
{
"properties": {
"filterType": "SqlFilter",
"sqlFilter": {
"sqlExpression": "HAI"
}
}
}
]
}
]
}
]
}
]
}
inp['resources'][0]['resources'][0]['resources'][0]['resources'][0]['properties']['sqlFilter']['sqlExpression']='BYE'
print(inp)
Result
{'abc': {'dbc': ...truncated... {'sqlExpression': 'BYE'}}}]}]}]}]}
I have a MongoDB document structure like following:
Structure
{
"stores": [
{
"items": [
{
"feedback": [],
"item_category": "101",
"item_id": "10"
},
{
"feedback": [],
"item_category": "101",
"item_id": "11"
}
]
},
{
"items": [
{
"feedback": [],
"item_category": "101",
"item_id": "10"
},
{
"feedback": ["A feedback"],
"item_category": "101",
"item_id": "11"
},
{
"feedback": [],
"item_category": "101",
"item_id": "12"
},
{
"feedback": [],
"item_category": "102",
"item_id": "13"
},
{
"feedback": [],
"item_category": "102",
"item_id": "14"
}
],
"store_id": 500
}
]
}
This is a single document in a collection. Some field are deleted to produce minimal representation of the data.
What I want is to get items only if the feedback field in the items array is not empty. The expected result is:
Expected result
{
"stores": [
{
"items": [
{
"feedback": ["A feedback"],
"item_category": "101",
"item_id": "11"
}
],
"store_id": 500
}
]
}
This is what I tried based on examples in this, which I think pretty same situation, but it didn't work. What's wrong with my query, isn't it the same situation in zipcode search example in the link? It returns everything like in the first JSON code, Structure:
What I tried
query = {
'date': {'$gte': since, '$lte': until},
'stores.items': {"$elemMatch": {"feedback": {"$ne": []}}}
}
Thanks.
Please try this :
db.yourCollectionName.aggregate([
{ $match: { 'date': { '$gte': since, '$lte': until }, 'stores.items': { "$elemMatch": { "feedback": { "$ne": [] } } } } },
{ $unwind: '$stores' },
{ $match: { 'stores.items': { "$elemMatch": { "feedback": { "$ne": [] } } } } },
{ $unwind: '$stores.items' },
{ $match: { 'stores.items.feedback': { "$ne": [] } } },
{ $group: { _id: { _id: '$_id', store_id: '$stores.store_id' }, items: { $push: '$stores.items' } } },
{ $project: { _id: '$_id._id', store_id: '$_id.store_id', items: 1 } },
{ $group: { _id: '$_id', stores: { $push: '$$ROOT' } } },
{ $project: { 'stores._id': 0 } }
])
We've all these stages as you need to operate on an array of arrays, this query is written assuming you're dealing with a large set of data, Since you're filtering on dates just in case if your documents size is way less after first $match then you can avoid following $match stage which is in between two $unwind's.
Ref 's :
$match,
$unwind,
$project,
$group
This aggregate query gets the needed result (using the provided sample document and run from the mongo shell):
db.stores.aggregate( [
{ $unwind: "$stores" },
{ $unwind: "$stores.items" },
{ $addFields: { feedbackExists: { $gt: [ { $size: "$stores.items.feedback" }, 0 ] } } },
{ $match: { feedbackExists: true } },
{ $project: { _id: 0, feedbackExists: 0 } }
] )
Following is the kibana JSON of a single row,
{
"_index": "questionanswers",
"_type": "doc",
"_id": "3",
"_version": 1,
"_score": 0,
"_source": {
"question": {
"id": 3,
"text": "Your first salary",
"answer_type": "FL",
"question_type": "BQ"
},
"candidate": {
"id": 13
},
"job": {
"id": 6
},
"id": 3,
"status": "AN",
"answered_on": "2019-07-12T09:26:01+00:00",
"answer": "12222222"
},
"fields": {
"answered_on": [
"2019-07-12T09:26:01.000Z"
]
}
}
I have an sql query like,
Select * from questionanswers where question.id = 3 and answer between 1250 and 1253666
I have converted this to elasticsearch query as follows,
{
"size": 1000,
"query": {
"bool": {
"must": [
{
"term": {
"question.id":3
}
},
{
"range": {
"answer": {
"from": 1250,
"to": 1253666999,
"include_lower": true,
"include_upper": true,
"boost": 1
}
}
}
],
"adjust_pure_negative": true,
"boost": 1
}
}
}
Here answer is declared as String , But i holds Date,FLoat and String values.
"question": {
"id": 3,
"text": "Your first salary",
"answer_type": "FL",
"question_type": "BQ"
},
Here answer_type tells which type of answer it is expecting.
When I try to run this query I am not getting desired results. I am getting an empty response on this hit.
But actually, there is a row that satisfies this query.
How my elasticsearch query should be so that I can filter with
question.id = 3 , question.answer_type = "FL" and answer between 1250 and 1253666```
see your document again. The answer is a string value, and you are treating as a number in your query. So it does not work obviously.
change the mapping for this field to number.
Here is the document I indexed in a test index and ran your query again and it works
Indexing the document ( see the field answer)
POST /so-index4/_doc/1
{
"question": {
"id": 3,
"text": "Your first salary",
"answer_type": "FL",
"question_type": "BQ"
},
"candidate": {
"id": 13
},
"job": {
"id": 6
},
"id": 3,
"status": "AN",
"answered_on": "2019-07-12T09:26:01+00:00",
"answer": 12222222,
"fields": {
"answered_on": [
"2019-07-12T09:26:01.000Z"
]
}
}
and the query (same query that you provided above)
GET /so-index4/_search
{
"size": 1000,
"query": {
"bool": {
"must": [
{
"term": {
"question.id":3
}
},
{
"range": {
"answer": {
"from": 1250,
"to": 1253666999,
"include_lower": true,
"include_upper": true,
"boost": 1
}
}
}
],
"adjust_pure_negative": true,
"boost": 1
}
}
}
the result
{
"took" : 5,
"timed_out" : false,
"_shards" : {
"total" : 5,
"successful" : 5,
"skipped" : 0,
"failed" : 0
},
"hits" : {
"total" : 1,
"max_score" : 2.0,
"hits" : [
{
"_index" : "so-index4",
"_type" : "_doc",
"_id" : "1",
"_score" : 2.0,
"_source" : {
"question" : {
"id" : 3,
"text" : "Your first salary",
"answer_type" : "FL",
"question_type" : "BQ"
},
"candidate" : {
"id" : 13
},
"job" : {
"id" : 6
},
"id" : 3,
"status" : "AN",
"answered_on" : "2019-07-12T09:26:01+00:00",
"answer" : 12222222,
"fields" : {
"answered_on" : [
"2019-07-12T09:26:01.000Z"
]
}
}
}
]
}
}
I keep on getting the following error in Python
Exception has occurred: TypeError unhashable type: 'dict'
on line 92
"should": [],
"must_not": []
This is the query string
res = es.search(
scroll = '2m',
index = "logstash-*",
body = {
{
"aggs": {
"2": {
"terms": {
"field": "src_ip.keyword",
"size": 50,
"order": {
"1": "desc"
}
},
"aggs": {
"1": {
"cardinality": {
"field": "src_ip.keyword"
}
}
}
}
},
"size": 0,
"_source": {
"excludes": []
},
"stored_fields": [
"*"
],
"script_fields": {},
"docvalue_fields": [
{
"field": "#timestamp",
"format": "date_time"
},
{
"field": "flow.start",
"format": "date_time"
},
{
"field": "timestamp",
"format": "date_time"
},
{
"field": "tls.notafter",
"format": "date_time"
},
{
"field": "tls.notbefore",
"format": "date_time"
}
],
"query": {
"bool": {
"must": [
{
"range": {
"#timestamp": {
"gte": 1555777931992,
"lte": 1558369931992,
"format": "epoch_millis"
}
}
}
],
"filter": [
{
"match_all": {}
}
],
"should": [],
"must_not": []
}
}
}
})
the value of body is a set ({ } without key-value is a set literal, e.g., {1,2} is a set). Inside this set you have a dictionary.
Items in a set have to be hashable, and dictionary isn't.
As the comment from #Carcigenicate says, it seems like a typo of having {{ }} instead of { } for the value of body.
Elasticsearch documentation shows that body should be a dictionary.
More about sets from python docs
I have a dictionary which is contain multiple keys and values and the values also contain the key, value pair. I am not getting how to create dynamic json using this dictionary in python. Here's the dictionary:
image_dict = {"IMAGE_1":{"img0":"IMAGE_2","img1":"IMAGE_3","img2":"IMAGE_4"},"IMAGE_2":{"img0":"IMAGE_1", "img1" : "IMAGE_3"},"IMAGE_3":{"img0":"IMAGE_1", "img1":"IMAGE_2"},"IMAGE_4":{"img0":"IMAGE_1"}}
My expected result like this :
{
"data": [
{
"image": {
"imageId": {
"id": "IMAGE_1"
},
"link": {
"target": {
"id": "IMAGE_2"
},
"target": {
"id": "IMAGE_3"
},
"target": {
"id": "IMAGE_4"
}
}
},
"updateData": "link"
},
{
"image": {
"imageId": {
"id": "IMAGE_2"
},
"link": {
"target": {
"id": "IMAGE_1"
},
"target": {
"id": "IMAGE_3"
}
}
},
"updateData": "link"
},
{
"image": {
"imageId": {
"id": "IMAGE_3"
},
"link": {
"target": {
"id": "IMAGE_1"
},
"target": {
"id": "IMAGE_2"
}
}
},
"updateData": "link"
} ,
{
"image": {
"imageId": {
"id": "IMAGE_4"
},
"link": {
"target": {
"id": "IMAGE_1"
}
}
},
"updateData": "link"
}
]
}
I tried to solve it but I didn't get expected result.
result = {"data":[]}
for k,v in sorted(image_dict.items()):
for a in sorted(v.values()):
result["data"].append({"image":{"imageId":{"id": k},
"link":{"target":{"id": a}}},"updateData": "link"})
print(json.dumps(result, indent=4))
In Python dictionaries you can't have 2 values with the same key. So you can't have multiple targets all called "target". So you can index them. Also I don't know what this question has to do with dynamic objects but here's the code I got working:
import re
dict_res = {}
ind = 0
for image in image_dict:
lin_ind = 0
sub_dict = {'image' + str(ind): {'imageId': {image}, 'link': {}}}
for sub in image_dict[image].values():
sub_dict['image' + str(ind)]['link'].update({'target' + str(lin_ind): {'id': sub}})
lin_ind += 1
dict_res.update(sub_dict)
ind += 1
dict_res = re.sub('target\d', 'target', re.sub('image\d', 'image', str(dict_res)))
print dict_res