I have a collection in MongoDB of nested objects (basically a tree structure). I am trying to access certain "id" and "user_id" values in "children".
The collection looks like this:
Image of a tree object in MongoDB
When I query "children" I get, e.g., this as output:
[{'children': [{'children': [{'children': [{'id': 737992252891537408, 'user_id': 3240396143}], 'id': 737991958161940480, 'user_id': 3240396143}], 'id': 737986305481682944, 'user_id': 56377143}], 'id': 737979183599652864, 'user_id': 3240396143}], 'id': 737978059291234304, 'user_id': 3240396143}]}
How do I efficiently access all the "id"'s with the 'user_id' = 56377143? I cannot seem to get all of them when it is nested too deep.
I tried using a for loop like this but it does not output all the 'id's which match the 'user_id's
val= "children"
lst_rt= []
lst_ids = []
def get_value(mydict, keys):
if type(mydict) == dict:
print(mydict[0]['user_id'], mydict[0]['id'], "TEST")
return get_value(mydict[keys], val)
if type(mydict) == list and keys in mydict[0] and mydict[0]['user_id'] == 56377143 :
print(mydict[0]['id'],mydict[0]['user_id'], 'COND')
return get_value(mydict[0][keys], val)
elif mydict[0]['user_id'] == 56377143 and mydict[0]['id'] != None:
print(mydict[0]['id'], mydict[0]['user_id'])
lst_rt.append(int(mydict[0]['id']))
return mydict[0]['id']
for x in root_tweets:
print(get_value(x['children'], val))
Related
I have a problem. I have a dict my_Dict. This is somewhat nested. However, I would like to 'clean up' the dict my_Dict, by this I mean that I would like to separate all nested ones and also generate a unique ID so that I can later find the corresponding object again.
For example, I have detail: {...}, this nested, should later map an independent dict my_Detail_Dict and in addition, detail should receive a unique ID within my_Dict. Unfortunately, my list that I give out is empty. How can I remove my slaughtered keys and give them an ID?
my_Dict = {
'_key': '1',
'group': 'test',
'data': {},
'type': '',
'code': '007',
'conType': '1',
'flag': None,
'createdAt': '2021',
'currency': 'EUR',
'detail': {
'selector': {
'number': '12312',
'isTrue': True,
'requirements': [{
'type': 'customer',
'requirement': '1'}]
}
}
}
def nested_dict(my_Dict):
my_new_dict_list = []
for key in my_Dict.keys():
#print(f"Looking for {key}")
if isinstance(my_Dict[key], dict):
print(f"{key} is nested")
# Add id to nested stuff
my_Dict[key]["__id"] = 1
my_nested_Dict = my_Dict[key]
# Delete all nested from the key
del my_Dict[key]
# Add id to key, but not the nested stuff
my_Dict[key] = 1
my_new_dict_list.append(my_Dict[key])
my_new_dict_list.append(my_Dict)
return my_new_dict_list
nested_dict(my_Dict)
[OUT] []
# What I want
[my_Dict, my_Details_Dict, my_Data_Dict]
What I have
{'_key': '1',
'group': 'test',
'data': {},
'type': '',
'code': '007',
'conType': '1',
'flag': None,
'createdAt': '2021',
'currency': 'EUR',
'detail': {'selector': {'number': '12312',
'isTrue': True,
'requirements': [{'type': 'customer', 'requirement': '1'}]}}}
What I want
my_Dict = {'_key': '1',
'group': 'test',
'data': 18,
'type': '',
'code': '007',
'conType': '1',
'flag': None,
'createdAt': '2021',
'currency': 'EUR',
'detail': 22}
my_Data_Dict = {'__id': 18}
my_Detail_Dict = {'selector': {'number': '12312',
'isTrue': True,
'requirements': [{'type': 'customer', 'requirement': '1'}]}, '__id': 22}
The following code snippet will solve what you are trying to do:
my_Dict = {
'_key': '1',
'group': 'test',
'data': {},
'type': '',
'code': '007',
'conType': '1',
'flag': None,
'createdAt': '2021',
'currency': 'EUR',
'detail': {
'selector': {
'number': '12312',
'isTrue': True,
'requirements': [{
'type': 'customer',
'requirement': '1'}]
}
}
}
def nested_dict(my_Dict):
# Initializing a dictionary that will store all the nested dictionaries
my_new_dict = {}
idx = 0
for key in my_Dict.keys():
# Checking which keys are nested i.e are dictionaries
if isinstance(my_Dict[key], dict):
# Generating ID
idx += 1
# Adding generated ID as another key
my_Dict[key]["__id"] = idx
# Adding nested key with the ID to the new dictionary
my_new_dict[key] = my_Dict[key]
# Replacing nested key value with the generated ID
my_Dict[key] = idx
# Returning new dictionary containing all nested dictionaries with ID
return my_new_dict
result = nested_dict(my_Dict)
print(my_Dict)
# Iterating through dictionary to get all nested dictionaries
for item in result.items():
print(item)
If I understand you correctly, you wish to automatically make each nested dictionary it's own variable, and remove it from the main dictionary.
Finding the nested dictionaries and removing them from the main dictionary is not so difficult. However, automatically assigning them to a variable is not recommended for various reasons. Instead, what I would do is store all these dictionaries in a list, and then assign them manually to a variable.
# Prepare a list to store data in
inidividual_dicts = []
id_index = 1
for key in my_Dict.keys():
# For each key, we get the current value
value = my_Dict[key]
# Determine if the current value is a dictionary. If so, then it's a nested dict
if isinstance(value, dict):
print(key + " is a nested dict")
# Get the nested dictionary, and replace it with the ID
dict_value = my_Dict[key]
my_Dict[key] = id_index
# Add the id to previously nested dictionary
dict_value['__id'] = id_index
id_index = id_index + 1 # increase for next nested dic
inidividual_dicts.append(dict_value) # store it as a new dictionary
# Manually write out variables names, and assign the nested dictionaries to it.
[my_Details_Dict, my_Data_Dict] = inidividual_dicts
I have a list of dictionaries, themselves with nested lists of dictionaries. All of the nest levels have a similar structure, thankfully. I desire to sort these nested lists of dictionaries. I grasp the technique to sort a list of dictionaries by value. I'm struggling with the recursion that will sort the inner lists.
def reorder(l, sort_by):
# I have been trying to add a recursion here
# so that the function calls itself for each
# nested group of "children". So far, fail
return sorted(l, key=lambda k: k[sort_by])
l = [
{ 'name': 'steve',
'children': [
{ 'name': 'sam',
'children': [
{'name': 'sally'},
{'name': 'sabrina'}
]
},
{'name': 'sydney'},
{'name': 'sal'}
]
},
{ 'name': 'fred',
'children': [
{'name': 'fritz'},
{'name': 'frank'}
]
}
]
print(reorder(l, 'name'))
def reorder(l, sort_by):
l = sorted(l, key=lambda x: x[sort_by])
for item in l:
if "children" in item:
item["children"] = reorder(item["children"], sort_by)
return l
Since you state "I grasp the technique to sort a list of dictionaries by value" I will post some code for recursively gathering data from another SO post I made, and leave it to you to implement your sorting technique. The code:
myjson = {
'transportation': 'car',
'address': {
'driveway': 'yes',
'home_address': {
'state': 'TX',
'city': 'Houston'}
},
'work_address': {
'state': 'TX',
'city': 'Sugarland',
'location': 'office-tower',
'salary': 30000}
}
def get_keys(some_dictionary, parent=None):
for key, value in some_dictionary.items():
if '{}.{}'.format(parent, key) not in my_list:
my_list.append('{}.{}'.format(parent, key))
if isinstance(value, dict):
get_keys(value, parent='{}.{}'.format(parent, key))
else:
pass
my_list = []
get_keys(myjson, parent='myjson')
print(my_list)
Is intended to retrieve all keys recursively from the json file. It outputs:
['myjson.address',
'myjson.address.home_address',
'myjson.address.home_address.state',
'myjson.address.home_address.city',
'myjson.address.driveway',
'myjson.transportation',
'myjson.work_address',
'myjson.work_address.state',
'myjson.work_address.salary',
'myjson.work_address.location',
'myjson.work_address.city']
The main thing to note is that if isinstance(value, dict): results in get_keys() being called again, hence the recursive capabilities of it (but only for nested dictionaries in this case).
I have a list of dictionaries in python and I would like to override old value with duplicate value. Please let me know how can I do.
{'message': [{'name': 'raghav', 'id': 10}, {'name': 'raghav', 'id': 11}]}
Output should be:
{'message': [ {'name': 'raghav', 'id': 11}]}
I don't know what you mean by "override old value with duplicate value". If you mean just picking the second dict from the list, you could:
print({k: [v[1]] for (k, v) in data.items()})
If the idea is to update the "name" with a newer value of "id" as you move along the list, then maybe:
def merge_records(data):
records = data['message']
users = {}
for record in records:
name = record['name']
id_ = record['id']
users[name] = id_
new_records = []
for name, id_ in users.items():
new_records.append({'name': name, 'id': id_})
return {'message': new_records}
But, if you have any control over how the data is represented, you might reconsider. You probably want a different data structure.
Here you go:
d = {'message': [{'name': 'raghav', 'id': 10}, {'name': 'raghav', 'id': 11}]}
#loop over outer dictionary
for key, value in d.items():
d[key] = [dict([t for k in value for t in k.items()])]
print(d)
Edit:
As per your requirement:
d = {'message': [ {'name': 'raghav', 'id': 11}, {'name': 'krish', 'id': 20}, {'name': 'anu', 'id': 30}]}
for key, value in d.items():
print [dict((k1,v1)) for k1,v1 in dict([tuple(i.items()) for i in value for val in i.items()]).items()]
I have an array of dicts retrieved from a web API. Each dict has a name, description, 'parent', and children key. The children key has an array of dicts as it value. For the sake of clarity, here is a dummy example:
[
{'name': 'top_parent', 'description': None, 'parent': None,
'children': [{'name': 'child_one'},
{'name': 'child_two'}]},
{'name': 'child_one', 'description': None, 'parent': 'top_parent',
'children': []},
{'name': 'child_two', 'description': None, 'parent': 'top_parent',
'children': [{'name': 'grand_child'}]},
{'name': 'grand_child', 'description': None, 'parent': 'child_two',
'children': []}
]
Every item in in the array. An item could be the top-most parent, and thus not exist in any of the children arrays. An item could be both a child and a parent. Or an item could only be a child (have no children of its own).
So, in a tree structure, you'd have something like this:
top_parent
child_one
child_two
grand_child
In this contrived and simplified example top_parent is a parent but not a child; child_one is a child but not a parent; child_two is a parent and a child; and grand_child is a child but not a parent. This covers every possible state.
What I want is to be able to iterate over the array of dicts 1 time and generate a nested dict that properly represents the tree structure (however, it 1 time is impossible, the most efficient way possible). So, in this example, I would get a dict that looked like this:
{
'top_parent': {
'child_one': {},
'child_two': {
'grand_child': {}
}
}
}
Strictly speaking, it is not necessary to have item's without children to not be keys, but that is preferable.
Fourth edit, showing three versions, cleaned up a bit. First version works top-down and returns None, as you requested, but essentially loops through the top level array 3 times. The next version only loops through it once, but returns empty dicts instead of None.
The final version works bottom up and is very clean. It can return empty dicts with a single loop, or None with additional looping:
from collections import defaultdict
my_array = [
{'name': 'top_parent', 'description': None, 'parent': None,
'children': [{'name': 'child_one'},
{'name': 'child_two'}]},
{'name': 'child_one', 'description': None, 'parent': 'top_parent',
'children': []},
{'name': 'child_two', 'description': None, 'parent': 'top_parent',
'children': [{'name': 'grand_child'}]},
{'name': 'grand_child', 'description': None, 'parent': 'child_two',
'children': []}
]
def build_nest_None(my_array):
childmap = [(d['name'], set(x['name'] for x in d['children']) or None)
for d in my_array]
all_dicts = dict((name, kids and {}) for (name, kids) in childmap)
results = all_dicts.copy()
for (name, kids) in ((x, y) for x, y in childmap if y is not None):
all_dicts[name].update((kid, results.pop(kid)) for kid in kids)
return results
def build_nest_empty(my_array):
all_children = set()
all_dicts = defaultdict(dict)
for d in my_array:
children = set(x['name'] for x in d['children'])
all_dicts[d['name']].update((x, all_dicts[x]) for x in children)
all_children.update(children)
top_name, = set(all_dicts) - all_children
return {top_name: all_dicts[top_name]}
def build_bottom_up(my_array, use_None=False):
all_dicts = defaultdict(dict)
for d in my_array:
name = d['name']
all_dicts[d['parent']][name] = all_dicts[name]
if use_None:
for d in all_dicts.values():
for x, y in d.items():
if not y:
d[x] = None
return all_dicts[None]
print(build_nest_None(my_array))
print(build_nest_empty(my_array))
print(build_bottom_up(my_array, True))
print(build_bottom_up(my_array))
Results in:
{'top_parent': {'child_one': None, 'child_two': {'grand_child': None}}}
{'top_parent': {'child_one': {}, 'child_two': {'grand_child': {}}}}
{'top_parent': {'child_one': None, 'child_two': {'grand_child': None}}}
{'top_parent': {'child_one': {}, 'child_two': {'grand_child': {}}}}
You can keep a lazy mapping from names to nodes and then rebuild the hierarchy by processing just the parent link (I'm assuming data is correct, so if A is marked as the parent of B iff B is listed among the children of A).
nmap = {}
for n in nodes:
name = n["name"]
parent = n["parent"]
try:
# Was this node built before?
me = nmap[name]
except KeyError:
# No... create it now
if n["children"]:
nmap[name] = me = {}
else:
me = None
if parent:
try:
nmap[parent][name] = me
except KeyError:
# My parent will follow later
nmap[parent] = {name: me}
else:
root = me
The children property of the input is used only to know if the element should be stored as a None in its parent (because has no children) or if it should be a dictionary because it will have children at the end of the rebuild process. Storing nodes without children as empty dictionaries would simplify the code a bit by avoiding the need of this special case.
Using collections.defaultdict the code can also be simplified for the creation of new nodes
import collections
nmap = collections.defaultdict(dict)
for n in nodes:
name = n["name"]
parent = n["parent"]
me = nmap[name]
if parent:
nmap[parent][name] = me
else:
root = me
This algorithm is O(N) assuming constant-time dictionary access and makes only one pass on the input and requires O(N) space for the name->node map (the space requirement is O(Nc) for the original nochildren->None version where Nc is the number of nodes with children).
My stab at it:
persons = [\
{'name': 'top_parent', 'description': None, 'parent': None,\
'children': [{'name': 'child_one'},\
{'name': 'child_two'}]},\
{'name': 'grand_child', 'description': None, 'parent': 'child_two',\
'children': []},\
{'name': 'child_two', 'description': None, 'parent': 'top_parent',\
'children': [{'name': 'grand_child'}]},\
{'name': 'child_one', 'description': None, 'parent': 'top_parent',\
'children': []},\
]
def findParent(name,parent,tree,found = False):
if tree == {}:
return False
if parent in tree:
tree[parent][name] = {}
return True
else:
for p in tree:
found = findParent(name,parent,tree[p],False) or found
return found
tree = {}
outOfOrder = []
for person in persons:
if person['parent'] == None:
tree[person['name']] = {}
else:
if not findParent(person['name'],person['parent'],tree):
outOfOrder.append(person)
for person in outOfOrder:
if not findParent(person['name'],person['parent'],tree):
print 'parent of ' + person['name'] + ' not found
print tree
results in:
{'top_parent': {'child_two': {'grand_child': {}}, 'child_one': {}}}
It also picks up any children whose parent has not been added yet, and then reconciles this at the end.
I have an error with this line. I am working with a dictionary from a file with an import. This is the dictionary:
users = [{'id':1010,'name':"Administrator",'type':1},{'id':1011,'name':"Administrator2",'type':1}]
And the method with which the work is as follows:
def addData(dict, entry):
new = {}
x = 0
for i in dict.keys():
new[i] = entry(x)
x += 1
dict.append(new)
Where "dict" would be "users", but the error is that the dictionary does not recognize me as such. Can anyone tell me, I have wrong in the dictionary?
That's not a dicionary, it's a list of dictionaries!
EDIT: And to make this a little more answer-ish:
users = [{'id':1010,'name':"Administrator",'type':1},{'id':1011,'name':"Administrator2",'type':1}]
newusers = dict()
for ud in users:
newusers[ud.pop('id')] = ud
print newusers
#{1010: {'type': 1, 'name': 'Administrator'}, 1011: {'type': 1, 'name': 'Administrator2'}}
newusers[1012] = {'name': 'John', 'type': 2}
print newusers
#{1010: {'type': 1, 'name': 'Administrator'}, 1011: {'type': 1, 'name': 'Administrator2'}, 1012: {'type': 2, 'name': 'John'}}
Which is essentially the same as dawgs answer, but with a simplified approach on generating the new dictionary
Perhaps you are looking to do something along these lines:
users = [{'id':1010,'name':"Administrator",'type':1},{'id':1011,'name':"Administrator2",'type':1}]
new_dict={}
for di in users:
new_dict[di['id']]={}
for k in di.keys():
if k =='id': continue
new_dict[di['id']][k]=di[k]
print new_dict
# {1010: {'type': 1, 'name': 'Administrator'}, 1011: {'type': 1, 'name': 'Administrator2'}}
Then you can do:
>>> new_dict[1010]
{'type': 1, 'name': 'Administrator'}
Essentially, this is turning a list of anonymous dicts into a dict of dicts that are keys from the key 'id'