We Keep Coding

how to sort a Dictionary in list? [duplicate]

This question already has answers here:
How do I sort a list of dictionaries by a value of the dictionary?
(20 answers)
Closed 2 months ago.
I had Question in python
Imagine a list with dictionaries in it
how can we sort it by a value in dictionary ?
Imagine this list :
lst = [
{
"a" : 3,
"b" : 2
},
{
"a" : 1,
"b" : 4
},
{
"a" : 2,
"b" : 3
}
]
how can we sort this list by value of "a" in each dictionary (python)
i mean i want this list at the end :
lst = [
{
"a" : 1,
"b" : 4
},
{
"a" : 2,
"b" : 3
},
{
"a" : 3,
"b" : 2
}
]

You could provide a lambda key to sorted:
>>> lst = [
... {
... "a" : 3,
... "b" : 2
... },
... {
... "a" : 1,
... "b" : 4
... },
... {
... "a" : 2,
... "b" : 3
... }
... ]
>>> sorted(lst, key=lambda d: d["a"])
[{'a': 1, 'b': 4}, {'a': 2, 'b': 3}, {'a': 3, 'b': 2}]

One approach, use the key argument with itemgetter:
from operator import itemgetter
lst = [{"a": 3, "b": 2}, {"a": 1, "b": 4}, {"a": 2, "b": 3}]
res = sorted(lst, key=itemgetter("a"))
print(res)
Output
[{'a': 1, 'b': 4}, {'a': 2, 'b': 3}, {'a': 3, 'b': 2}]
From the documentation on itemgetter:
Return a callable object that fetches item from its operand using the
operand’s getitem() method. If multiple items are specified,
returns a tuple of lookup values. For example:
After f = itemgetter(2), the call f(r) returns r[2].
After g =
itemgetter(2, 5, 3), the call g(r) returns (r[2], r[5], r[3]).

adding value in nested dictionaries in python

I am trying to add value to inside dictionary.
aDict = { "id" :
{"name": None },
"id2" :
{"foo": None},
...
}
for k, v in aDict.items():
temp = [1,2,3,4]
aDict[k][v] = temp
then I got error, TypeError: unhashable type: 'dict'
How can I add value in nested dictionary?
---edit ---
My expected output is,
aDict = { "id" :
{"name": [1,2,3,4] },
"id2" :
{"foo": [1,2,3,4] },
...
}

When you do aDict[k], you already got the value which is dict and then you assign the temp to the specific key of the dict.
aDict = { 'id' :
{'name': None },
'id2':
{'foo':None}
}
for k, v in aDict.items():
temp = [1,2,3,4]
for keys in v.keys():
aDict[k][keys] = temp
Output
{'id': {'name': [1, 2, 3, 4]}, 'id2': {'foo': [1, 2, 3, 4]}}

For any arbitrary dictionary of dictionaries (no matter how deep it is), this works:
def go_deeper(aDict):
for k, v in aDict.items():
if v is None:
aDict[k] = temp
else:
go_deeper(v)
return aDict
Usage:
>>> temp = [1,2,3,4]
>>> go_deeper(aDict)
For example, for input:
aDict = { 'id' :
{'name': None },
"id2" :
{"foo":
{'bar': None }
}
}
the above code returns:
{'id': {'name': [1, 2, 3, 4]}, 'id2': {'foo': {'bar': [1, 2, 3, 4]}}}

Try this :
temp = [1,2,3,4]
for k in aDict:
for j in aDict[k]:
aDict[k][j]=temp
OUTPUT :
{'id': {'name': [1, 2, 3, 4]}, 'id2': {'foo': [1, 2, 3, 4]}}

You can get all keys by using d.keys() then add temp to this dictionary.
aDict = { "id" :
{"name": None },
"id2" :
{"foo": None},
"id3" :
{"bar": None, "boo": None},
}
temp = [1, 2, 3, 4]
for k, v in aDict.items():
for newKey in v.keys():
v[newKey] = temp
Result:
{'id': {'name': [1, 2, 3, 4]},
'id2': {'foo': [1, 2, 3, 4]},
'id3': {'bar': [1, 2, 3, 4], 'boo': [1, 2, 3, 4]}}

I would avoid the use of nested loops. And hence to get your desired output, I would do the following:
aDict = { "id" :
{"name": None },
"id2" :
{"foo": None}
}
for value in aDict.values():
temp = [1,2,3,4]
# obtain the key of inner dictionary
# this line could be written even better, if you're concerned about memory!
key = list(value.keys())[0]
# update the inner value
value[key] = temp
When I run this it gives your desired output
user#Kareems-MBP:Desktop$ python aDict.py
{'id': {'name': [1, 2, 3, 4]}, 'id2': {'foo': [1, 2, 3, 4]}}
Lastly, the TypeError you were getting TypeError: unhashable type: 'dict' is because you're trying to reference an item in the dictionary using a dictionary. Items in a dictionary can be referenced only using their keys. E.g. if we have the following dictionary:
myDict = {
'firstname': 'John',
'lastname': 'White'
}
and we want to reference the first item, we can do that only using myDict['firstname'], we cannot do that even with indices myDict[0]. You can imagine that you were doing something like this myDict[{'firstname': 'John'}].
I hope this has been helpful!

Pandas json_normalize on recursively nested json

I have a json file with a deeply nested recursive structure:
{"children": [
"val" = x
"data" = y
"children": [{
"val" = x
"data" = y
"children": [{
....
"val" = x
"data" = y
"children": [{
"val" = x
"data" = y
"children": [{
....
Using pandas json_normalize as follows:
json_normalize(data = self.data["children"], record_path="children")
gives dataframe where the first level is flattened but the deepers levels remain json strings within the dataframe.
How can i flatten my dataframe such that the entire json tree is unpacked and flattened?

Providing your json is well formatted and has the same structure at all levels you can extract all the data by passing a List of keywords to json_normalize from each level.
json = {'children': [{
'val': 1,
'data': 2,
'children': [{
'val': 3,
'data' : 4,
'children': [{'val' : 4,
'data' : 5}],
}],
},{
'val' : 6,
'data' : 7,
'children': [{
'val' : 8,
'data' : 9,
'children': [{'val' : 10,
'data' : 11}],
}]
}]}
for i in range(1,3):
print( json_normalize(data = json,record_path=['children']*i) )
This gives the following output, which you can use recursively add into a single DataFrame if you wish.
children data val
0 [{'val': 3, 'data': 4, 'children': [{'val': 4,... 2 1
1 [{'val': 8, 'data': 9, 'children': [{'val': 10... 7 6
children data val
0 [{'val': 4, 'data': 5}] 4 3
1 [{'val': 10, 'data': 11}] 9 8
data val
0 5 4
1 11 10

Create a dict from two lists of dicts with matching value

I have two lists of dictionaries, dict1 and dict2.
dict1 = [
{
'id': 0,
'name': 'James'
}, {
'id': 1,
'name': 'Bob'
}
]
dict2 = [
{
'id': 0,
'name': 'James'
}, {
'id': 1,
'name': 'James'
}, {
'id': 2,
'name': 'Bob'
}
]
And i want to create a dict like that:
result = {'James': [0, 1], 'Bob': [2]}
With the names from dict1 as keys and as value the list of "id" fields having the same name.
What's a clean way to do that in Python?

IIUC, I believe last line should be Bob and not James. That way, using pandas
import pandas as pd
>>> df1 = pd.DataFrame(dict1)
>>> df2 = pd.DataFrame(dict2)
>>> df2.groupby('name').agg(list).to_dict()['id']
{'Bob': [2], 'James': [0, 1]}
To filter only names that are in dict1,
>>> df2 = df2[df2['name'].isin(df1['name'])]
and group and agg after that

You can put the names in dict1 in a set first, so that when you iterate over dict2, you can check if the current name is in the set before adding it to the resulting dict of lists:
names = {d['name'] for d in dict1}
result = {}
for d in dict2:
if d['name'] in names:
result.setdefault(d['name'], []).append(d['id'])
result becomes:
{'James': [0, 1], 'Bob': [2]}

You can also do it in pure Python as following:
dict1 = [
{
'id': 0,
'name': 'James'
}, {
'id': 1,
'name': 'Bob'
}
]
dict2 = [
{
'id': 0,
'name': 'James'
}, {
'id': 1,
'name': 'James'
}, {
'id': 2,
'name': 'Bob'
}
]
names = [elem['name'] for elem in dict1]
result = dict((name,[]) for name in names)
for elem in dict2:
result[elem['name']].append(elem['id'])
print(result)
Output:
{'James': [0, 1], 'Bob': [2]}

IIUC, here's a defaultdict solution:
>>> from collections import defaultdict
>>>
>>> result = defaultdict(list)
>>> names = {d['name'] for d in dict1}
>>>
>>> for d in dict2:
...: name = d['name']
...: if name in names:
...: result[name].append(d['id'])
...:
>>> result
defaultdict(list, {'Bob': [2], 'James': [0, 1]})

Merging dictionaries with same keys

I wrote some python code to pull data from SQL server and I'm currently trying to merge the data. I tried to pull the data into a Dataframe and then work with it, but wasn't able to do that.
The current form that I set the data up as is like this :
[ { a : { 1 : ( x,y,z,...) }},
{ a : { 2 : ( x,y,z,...) }},
{ a : { 3 : ( x,y,z,...) }} ]
This is where I want to get to
[ { a : { 1 : ( x,y,z,...) , 2 : (x,y,...) , 3 : (x,y,z,...) } ]

Use a nested dictionary structure via collections.defaultdict.
Note that in this implementation duplicate inner keys are not permitted; for example, you cannot have two dictionaries with outer key 'a' and inner key 1. In this case, the last will take precedence.
from collections import defaultdict
lst = [ { 'a' : { 1 : ( 3, 4, 5 ) }},
{ 'a' : { 2 : ( 6, 7, 8 ) }},
{ 'a' : { 3 : ( 1, 2, 3 ) }},
{ 'c' : { 4 : ( 5, 9, 8 ) }},
{ 'b' : { 1 : ( 6, 6, 8 ) }},
{ 'c' : { 3 : ( 2, 5, 7 ) }}]
d = defaultdict(dict)
for item in lst:
key = next(iter(item))
d[key].update(item[key])
# defaultdict(dict,
# {'a': {1: (3, 4, 5), 2: (6, 7, 8), 3: (1, 2, 3)},
# 'b': {1: (6, 6, 8)},
# 'c': {3: (2, 5, 7), 4: (5, 9, 8)}})

How's this?
data = [{'a': {1: 4}, 'b': {7: 8}},
{'a': {2: 5}, 'b': {9: 10}},
{'a': {3: 6}}]
all_keys = set().union(*data)
result = {}
for key in all_keys:
result[key] = {}
for d in data:
if key in d:
result[key].update(d[key])
print(result) # {'b': {7: 8, 9: 10}, 'a': {1: 4, 2: 5, 3: 6}}

You can make use of the reduce function and dict.update to transform the data. Assuming that 'a' is your only key, you can do this:
a = [
{'a': {1: (1, 2, 3)}},
{'a': {2: (4, 5, 6)}},
{'a': {3: (7, 8, 9)}}
]
def update(d, c):
d['a'].update(c['a'])
return d
print reduce(update, a, {'a':{}}) #Prints {'a': {1: (1, 2, 3), 2: (4, 5, 6), 3: (7, 8, 9)}}