I have a query in mongo db, tried lots of solution but still not found it working. Any help will be appreciated.
How to find all keys named "channel" in document?
db.clients.find({"_id": 69})
{
"_id" : 69,
"configs" : {
"GOOGLE" : {
"drid" : "1246ABCD",
"adproviders" : {
"adult" : [
{
"type" : "landing",
"adprovider" : "abc123",
"channel" : "abc456"
},
{
"type" : "search",
"adprovider" : "xyz123",
"channel" : "xyz456"
}
],
"nonadult" : [
{
"type" : "landing",
"adprovider" : "pqr123",
"channel" : "pqr456"
},
{
"type" : "search",
"adprovider" : "lmn123",
"channel" : "lmn456"
}
]
}
},
"channel" : "ABC786",
"_cls" : "ClientGoogleDoc"
}
}
Trying to find keys with name channel
db.clients.find({"_id": 69, "channel": true})
Expecting:
{"channels": ["abc456", "xyz456", "ABC786", "xyz456", "pqr456", "lmn456", ...]}
As far as I know, you'd have to use python to recursively traverse the dictionary yourself in order to build the list that you want above:
channels = []
def traverse(my_dict):
for key, value in my_dict.items():
if isinstance(value, dict):
traverse(value)
else:
if key == "channel":
channels.append(value)
traverse({"a":{"channel":"abc123"}, "channel":"xyzzz"})
print(channels)
output:
['abc123', 'xyzzz']
However, using a thing called projections you can get sort of close to what you want (but not really, since you have to specify all of the channels manually):
db.clients.find({"_id": 69}, {"configs.channel":1})
returns:
{ "_id" : ObjectId("69"), "configs" : { "channel" : "ABC786" } }
If you want to get really fancy, you could write a generator function to generate all the keys in a given dictionary, no matter how deep:
my_dict = { "a": {
"channel":"abc123",
"key2": "jjj",
"subdict": {"deep_key": 5, "channel": "nested"}
},
"channel":"xyzzz"}
def getAllKeys(my_dict):
for key, value in my_dict.items():
yield key, value
if isinstance(value, dict):
for key, value in getAllKeys(value):
yield key, value
for key, value in getAllKeys(my_dict):
if key == "channel":
print value
output:
nested
abc123
xyzzz
You can use the $project mongodb operator to get only the value for the specific key. Check the documentation at http://docs.mongodb.org/manual/reference/operator/aggregation/project/
Related
I am trying to load a JSON file to parse the contents nested in the root object. Currently I have the JSON file open and loaded as such:
with open(outputFile.name) as f:
data = json.load(f)
For the sake of the question here is an example of what the contents of the JSON file are like:
{
"rootObject" :
{
"person" :
{
"address" : "some place ave. 123",
"age" : 47,
"name" : "Joe"
},
"kids" :
[
{
"age" : 20,
"name" : "Joey",
"studySubject":"math"
},
{
"age" : 16,
"name" : "Josephine",
"studySubject":"chemistry"
}
],
"parents" :
{
"father" : "Joseph",
"mother" : "Joette"
}
How do I access the nested objects in "rootObject", such as "person", "kids" and its contents, and "parents"?
Below code using recursive function can extract values using specific key in a nested dictionary or 'lists of dictionaries':
data = {
"rootObject" :
{
"person" :
{
"address" : "some place ave. 123",
"age" : 47,
"name" : "Joe"
},
"kids" :
[
{
"age" : 20,
"name" : "Joey",
"studySubject":"math"
},
{
"age" : 16,
"name" : "Josephine",
"studySubject":"chemistry"
}
],
"parents" :
{
"father" : "Joseph",
"mother" : "Joette"
}
}}
def get_vals(nested, key):
result = []
if isinstance(nested, list) and nested != []: #non-empty list
for lis in nested:
result.extend(get_vals(lis, key))
elif isinstance(nested, dict) and nested != {}: #non-empty dict
for val in nested.values():
if isinstance(val, (list, dict)): #(list or dict) in dict
result.extend(get_vals(val, key))
if key in nested.keys(): #key found in dict
result.append(nested[key])
return result
get_vals(data, 'person')
Output
[{'address': 'some place ave. 123', 'age': 47, 'name': 'Joe'}]
The code for loading the JSON object should look like this:
from json import loads, load
with open("file.json") as file:
var = loads(load(file))
# loads() transforms the string in a python dict object
I have below sample data in JSON format :
project_cost_details is my database result set after querying.
{
"1": {
"amount": 0,
"breakdown": [
{
"amount": 169857,
"id": 4,
"name": "SampleData",
"parent_id": "1"
}
],
"id": 1,
"name": "ABC PR"
}
}
Here is full json : https://jsoneditoronline.org/?id=2ce7ab19af6f420397b07b939674f49c
Expected output :https://jsoneditoronline.org/?id=56a47e6f8e424fe8ac58c5e0732168d7
I have this sample JSON which i created using loops in code. But i am stuck at how to convert this to expected JSON format. I am getting sequential changes, need to convert to tree like or nested JSON format.
Trying in Python :
project_cost = {}
for cost in project_cost_details:
if cost.get('Parent_Cost_Type_ID'):
project_id = str(cost.get('Project_ID'))
parent_cost_type_id = str(cost.get('Parent_Cost_Type_ID'))
if project_id not in project_cost:
project_cost[project_id] = {}
if "breakdown" not in project_cost[project_id]:
project_cost[project_id]["breakdown"] = []
if 'amount' not in project_cost[project_id]:
project_cost[project_id]['amount'] = 0
project_cost[project_id]['name'] = cost.get('Title')
project_cost[project_id]['id'] = cost.get('Project_ID')
if parent_cost_type_id == cost.get('Cost_Type_ID'):
project_cost[project_id]['amount'] += int(cost.get('Amount'))
#if parent_cost_type_id is None:
project_cost[project_id]["breakdown"].append(
{
'amount': int(cost.get('Amount')),
'name': cost.get('Name'),
'parent_id': parent_cost_type_id,
'id' : cost.get('Cost_Type_ID')
}
)
from this i am getting sample JSON. It will be good if get in this code only desired format.
Also tried this solution mention here : https://adiyatmubarak.wordpress.com/2015/10/05/group-list-of-dictionary-data-by-particular-key-in-python/
I got approach to convert sample JSON to expected JSON :
data = [
{ "name" : "ABC", "parent":"DEF", },
{ "name" : "DEF", "parent":"null" },
{ "name" : "new_name", "parent":"ABC" },
{ "name" : "new_name2", "parent":"ABC" },
{ "name" : "Foo", "parent":"DEF"},
{ "name" : "Bar", "parent":"null"},
{ "name" : "Chandani", "parent":"new_name", "relation": "rel", "depth": 3 },
{ "name" : "Chandani333", "parent":"new_name", "relation": "rel", "depth": 3 }
]
result = {x.get("name"):x for x in data}
#print(result)
tree = [];
for a in data:
#print(a)
if a.get("parent") in result:
parent = result[a.get("parent")]
else:
parent = ""
if parent:
if "children" not in parent:
parent["children"] = []
parent["children"].append(a)
else:
tree.append(a)
Reference help : http://jsfiddle.net/9FqKS/ this is a JavaScript solution i converted to Python
It seems that you want to get a list of values from a dictionary.
result = [value for key, value in project_cost_details.items()]
I must be really slow because I spent a whole day googling and trying to write Python code to simply list the "code" values only so my output will be Service1, Service2, Service2. I have extracted json values before from complex json or dict structure. But now I must have hit a mental block.
This is my json structure.
myjson='''
{
"formatVersion" : "ABC",
"publicationDate" : "2017-10-06",
"offers" : {
"Service1" : {
"code" : "Service1",
"version" : "1a1a1a1a",
"index" : "1c1c1c1c1c1c1"
},
"Service2" : {
"code" : "Service2",
"version" : "2a2a2a2a2",
"index" : "2c2c2c2c2c2"
},
"Service3" : {
"code" : "Service4",
"version" : "3a3a3a3a3a",
"index" : "3c3c3c3c3c3"
}
}
}
'''
#convert above string to json
somejson = json.loads(myjson)
print(somejson["offers"]) # I tried so many variations to no avail.
Or, if you want the "code" stuffs :
>>> [s['code'] for s in somejson['offers'].values()]
['Service1', 'Service2', 'Service4']
somejson["offers"] is a dictionary. It seems you want to print its keys.
In Python 2:
print(somejson["offers"].keys())
In Python 3:
print([x for x in somejson["offers"].keys()])
In Python 3 you must use the list comprehension because in Python 3 keys() is a 'view', not a list.
This should probably do the trick , if you are not certain about the number of Services in the json.
import json
myjson='''
{
"formatVersion" : "ABC",
"publicationDate" : "2017-10-06",
"offers" : {
"Service1" : {
"code" : "Service1",
"version" : "1a1a1a1a",
"index" : "1c1c1c1c1c1c1"
},
"Service2" : {
"code" : "Service2",
"version" : "2a2a2a2a2",
"index" : "2c2c2c2c2c2"
},
"Service3" : {
"code" : "Service4",
"version" : "3a3a3a3a3a",
"index" : "3c3c3c3c3c3"
}
}
}
'''
#convert above string to json
somejson = json.loads(myjson)
#Without knowing the Services:
offers = somejson["offers"]
keys = offers.keys()
for service in keys:
print(somejson["offers"][service]["code"])
There is what i have in source:
"VirtualizationType": "hvm",
"Tags": [
{
"Value": "Test",
"Key": "Name"
}
]
and here is a bit of my code:
for item in result:
temp = {"Tags": item["Value"], "VirtualizationType": item["VirtualizationType"]}
output.append(temp)
I can easily export VirtualizationType but couldn`t do it with Tags
An example of how to create a dictionary in Python, from docs:
tel = {'jack': 4098, 'sape': 4139}
In this example jack, snape are the keys and 4098, 4139 are the values.
Values can also be strings and a list of dictionary like in your case,
your code can modified into this:
dictionary = {"Virtualization Type" : "hvm", "Tags" : [{"Value" : "Test", "Key" : "Name"}] } #Runs in the python interpreter with no errors
I've a dict as follows
{
"key1" : "value1",
"key2" : "value2",
"key3" : "value3",
"key4" : {
"key5" : "value5"
}
}
If the dict has key1==value1, I'll append the dict into a list.
Suppose key1==value1 is not present in the first key value pair, whereas it is inside nested dict as follows:
{
"key2" : "value2",
"key3" : "value3",
"key4" : {
"key5" : "value5",
"key1" : "value1",
"key6" : {
"key7" : "value7",
"key1" : "value1"
}
},
"key8" : {
"key9" : "value9",
"key10" : {
"key11" : "value11",
"key12" : "value12",
"key1" : "value1"
}
}
}
In the above dict, I've to check first whether there is key1=value1. If not, I've to traverse the nested dict and if it found in the nested dict, I've to append that dict to the list. If the nested dict is also a nested dict but key1=value1 is find in the first key value pair, then no need to check the inner dict(Eg key4 has key1=value1 in the in the first key value pair. Hence no need to check the inner one eventhough key6 has key1=value1).
So finally, I'll have the list as follows.
[
{
"key5" : "value5",
"key1" : "value1",
"key6" : {
"key7" : "value7",
"key1" : "value1"
}
},
{
"key11" : "value11",
"key12" : "value12",
"key1" : "value1"
}
]
How to achieve this?
Note: The depth of the dict may vary
if a dict contains key1 and value1 we will add it to the list and finish.
if not, we will got into all the values in the dict that are dict and do the same logic as well
l = []
def append_dict(d):
if d.get("key1") == "value1":
l.append(d)
return
for k,v in d.items():
if isinstance(v, dict):
append_dict(v)
append_dict(d)
print l
an iterative solution will be adding to queue the dict we would like to check:
from Queue import Queue
q = Queue()
l = []
q.put(d)
while not q.empty():
d = q.get()
if d.get("key1") == "value1":
l.append(d)
continue
for k,v in d.items():
if isinstance(v, dict):
q.put(v)
print l
As #shashank noted, usinq a stack instead of a queue will also work
it is BFS vs DFS for searching in the dictionary