I am trying to take two lists, each of which are lists of dictionaries with the same keys, and output versions of each list that only contain dictionaries that share common values for one of the keys. For example:
#before:
json1 = [{'id':1, 'name':'john', 'age': 3}, {'id':2, 'name':'jack', 'age':5}]
json2 = [{'id':3, 'name':'john', 'age': 5}, {'id':1, 'name':'jill', 'age':3}]
#Do some operation that merges based on the key 'id'
json1 = [{'id':1, 'name':'john', 'age': 3}]
json2 = [{'id':1, 'name':'jill', 'age':3}]
So, merging the lists of dicts based on id would output what I wrote above. Merging based on another key, say 'name', would only keep the first dict of each list.
Does anyone know a good way to do this?
EDIT
Sorry about the list names, I guess to be extremely accurate I'll call them json1 and json2
I thing your merging function could be something like that
def merge(key, l1, l2):
k1 = { d[key] for d in l1 }
k2 = { d[key] for d in l2 }
keys = k1.intersection(k2)
f1 = [ d for d in l1 if d[key] in keys ]
f2 = [ d for d in l2 if d[key] in keys ]
return f1, f2
That is :
take values of the key used for merging (in your examples 'id' or 'name')
in sets to avoid duplicates
find common values in the 2 sets
keep only dicts from the initial lists where the key take one of the common values
If you take merge('id', json1, json2) you get a 2-tuple of your resulting json1 and json2
Assuming I understand you, I'd do this in two passes: first find the common values, and then build the new lists:
>>> j1 = [{'id':1, 'name':'john', 'age': 3}, {'id':2, 'name':'jack', 'age':5}]
>>> j2 = [{'id':3, 'name':'john', 'age': 5}, {'id':1, 'name':'jill', 'age':3}]
>>> jj = (j1, j2)
>>> common = set.intersection(*({d['id'] for d in j} for j in jj))
>>> common
set([1])
>>> jjnew = [[d for d in j if d['id'] in common] for j in jj]
>>> jjnew
[[{'age': 3, 'id': 1, 'name': 'john'}], [{'age': 3, 'id': 1, 'name': 'jill'}]]
And similarly for name:
>>> common = set.intersection(*({d['name'] for d in j} for j in jj))
>>> jjnew = [[d for d in j if d['name'] in common] for j in jj]
>>> jjnew
[[{'age': 3, 'id': 1, 'name': 'john'}], [{'age': 5, 'id': 3, 'name': 'john'}]]
Related
I am trying to create a dictionary with keys from x and values from a list of tuples 'users'. I need output in two formats: dict of lists & list of dict.
#input
users = [(2, "jhon", "jhon#company.com"),
(3, "mike", "mike#company.com"),
(4, "sam", "sam#company.com")]
x = ("id", "name", "email")
#expected output:
dict_of_lists = {'id': [2, 3, 4], 'name': ['jhon', 'mike', 'sam'], 'email': ['jhon#company.com', 'mike#company.com', 'sam#company.com']}
list_of_dicts = [{'id': 2, 'name': 'jhon', 'email': 'jhon#company.com'}, {'id': 3, 'name': 'mike', 'email': 'mike#company.com'}, {'id': 4, 'name': 'sam', 'email': 'same#company.com'}]
My code to generate dict of lists:
Dict = {}
def Add(key, value):
Dict[key] = value
ListId=[]
ListName=[]
ListEmail=[]
for i in range(len(users)):
ListId.append(users[i][0])
ListName.append(users[i][1])
ListEmail.append(users[i][2])
Add(x[0],ListId)
Add(x[1],ListName)
Add(x[2],ListEmail)
print(Dict)
My code to generate list of dict:
res = []
for i in range(len(users)):
Dict = {x[0] : users[i][0] , x[1] : users[i][1] , x[2]: users[i][2]}
res.append(Dict)
print(res)
I am looking for any other efficient solution to this question. Also, I have hardcoded index value from tuple x & list of tuple 'users' to access tuple value. How can I make code dynamic such that when a new key is added to x & value to users, it gets added to my output?
I have checked the web to find an answer but couldn't find anything similar to my question.
This is my first StackOverflow question. In case you have any suggestions for my question to write or ask in a better way then do let me know.
For the first one:
dict_of_lists = dict(zip(x, map(list, zip(*users))))
# or if tuples are fine
# dict_of_lists = dict(zip(x, zip(*users)))
output:
{'id': [2, 3, 4],
'name': ['jhon', 'mike', 'sam'],
'email': ['jhon#company.com', 'mike#company.com', 'sam#company.com']}
For the second:
list_of_dicts = [dict(zip(x,l)) for l in users]
output:
[{'id': 2, 'name': 'jhon', 'email': 'jhon#company.com'},
{'id': 3, 'name': 'mike', 'email': 'mike#company.com'},
{'id': 4, 'name': 'sam', 'email': 'sam#company.com'}]
Use list comprehension and dict comprehension:
list_of_dicts = [dict(zip(x,u)) for u in users]
dict_of_lists = {k: [u[i] for u in users] for i, k in enumerate(x)}
I have a list of dictionaries with multiple KVP each
list_dict = [{'id': 1, 'name': 'sana'}, {'id': 2, 'name': 'art'}, {'id': 3, 'name': 'tiara'}]
I want to transform this into this format:
final_dict = {1: 'sana', 2: 'art', 3: 'tiara'}
I've been trying dict comprehensions but it does not work. Here's the best that I could do:
{k:v for d in list_dict for k, v in d.items()}
You don't need d.items(), you can just access the id and name properties of each dict.
{d['id']: d['name'] for d in list_dict}
for each element of the list you want the d["id"] to be the key and d["name"] to be the value, so the dictionary comprehension would look like this:
{d["id"]: d["name"] for d in list_dict}
You can try
final_dict={}
for dico in list_dict:
final_dict[dico['id']] = dico['name']
There's probably a few different ways you can do this. Here's a nice simple way of doing it using a pandas dataframe:
import pandas as pd
df = pd.DataFrame(list_dict) # <- convert to dataframe
df = df.set_index('id') # <- set the index to the field you want as the key
final_dict = df['name'].to_dict() # <- convert the series 'name' to a dict
print(final_dict)
{1: 'sana', 2: 'art', 3: 'tiara'}
I have a dictionary which is converted from a dataframe as below :
a = d.to_json(orient='index')
Dictionary :
{"0":{"yr":2017,"PKID":"58306, 57011","Subject":"ABC","ID":"T001"},"1":{"yr":2018,"PKID":"1234,54321","Subject":"XYZ","ID":"T002"}}
What I need is it be in a list, so essentially a list of dictionary.
So i just add a [] because that is the format to be used in the rest of the code.
input_dict = [a]
input_dict :
['
{"0":{"yr":2017,"PKID":"58306, 57011","Subject":"ABC","ID":"T001"},"1":{"yr":2018,"PKID":"1234,54321","Subject":"XYZ","ID":"T002"}}
']
I need to get the single quotes removed just after the [ and just before the ]. Also, have the PKID values in form of list.
How can this be achieved ?
Expected Output :
[ {"yr":2017,"PKID":[58306, 57011],"Subject":"ABC","ID":"T001"},"1":{"yr":2018,"PKID":[1234,54321],"Subject":"XYZ","ID":"T002"} ]
NOTE : The PKID column has multiple integer values which have to come as a lift of integers. a string is not acceptable.
so we need like "PKID":[58306, 57011] and not "PKID":"[58306, 57011]"
pandas.DataFrame.to_json returns a string (JSON string), not a dictionary. Try to_dict instead:
>>> df
col1 col2
0 1 3
1 2 4
>>> [df.to_dict(orient='index')]
[{0: {'col1': 1, 'col2': 3}, 1: {'col1': 2, 'col2': 4}}]
>>> df.to_dict(orient='records')
[{'col1': 1, 'col2': 3}, {'col1': 2, 'col2': 4}]
Here is one way:
from collections import OrderedDict
d = {"0":{"yr":2017,"PKID":"58306, 57011","Subject":"ABC","ID":"T001"},"1":{"yr":2018,"PKID":"1234,54321","Subject":"XYZ","ID":"T002"}}
list(OrderedDict(sorted(d.items())).values())
# [{'ID': 'T001', 'PKID': '58306, 57011', 'Subject': 'ABC', 'yr': 2017},
# {'ID': 'T002', 'PKID': '1234,54321', 'Subject': 'XYZ', 'yr': 2018}]
Note the ordered dictionary is ordered by text string keys, as supplied. You may wish to convert these to integers first before any processing via d = {int(k): v for k, v in d.items()}.
You are converting your dictionary to json which is a string. Then you wrap your resulting string a list. So, naturally, the result is a string inside of a list.
Try instead: [d] where d is your raw dictionary (not converted json
You can use a list comprehension
Ex:
d = {"0":{"yr":2017,"PKID":"58306, 57011","Subject":"ABC","ID":"T001"},"1":{"yr":2018,"PKID":"1234,54321","Subject":"XYZ","ID":"T002"}}
print [{k: v} for k, v in d.items()]
Output:
[{'1': {'PKID': '1234,54321', 'yr': 2018, 'ID': 'T002', 'Subject': 'XYZ'}}, {'0': {'PKID': '58306, 57011', 'yr': 2017, 'ID': 'T001', 'Subject': 'ABC'}}]
What about something like this:
from operator import itemgetter
d = {"0":{"yr":2017,"PKID":"58306, 57011","Subject":"ABC","ID":"T001"},"1":
{"yr":2018,"PKID":"1234,54321","Subject":"XYZ","ID":"T002"}}
sorted_d = sorted(d.items(), key=lambda x: int(x[0]))
print(list(map(itemgetter(1), sorted_d)))
Which Outputs:
[{'yr': 2017, 'PKID': '58306, 57011', 'Subject': 'ABC', 'ID': 'T001'},
{'yr': 2018, 'PKID': '1234,54321', 'Subject': 'XYZ', 'ID': 'T002'}]
So I have two list of dictionaries which look like the following:
A = [{'id':'xyz', 'count': 3},{'id':'zzz', 'count': 1}]
B = [{'id':'xyz', 'count': 4}]
I want the final output to be like:
C = [{'id':'xyz', 'count': 7},{'id':'zzz', 'count': 1}]
So in case that if the value of the first key is the same in both lists, we add up the values for the second key. Any idea how I might achieve this? So far I have tried:
for elem in A:
if elem['id'] in B.elementsKey['id']:
# TO DO
I am stock where I am trying to update the value for the other element accordingly.
NOTE: Each id is unique.
Thanks
from collections import Counter
C_counts = Counter()
for l in (A, B):
C_counts.update({x['id']: x['count'] for x in l})
C = [{'id':k, 'count':c} for (k, c) in C_counts.items()]
If you really don't want to import collections.Counter you can do:
_C = {}
for l in A, B:
for d in l:
_C[d['id']] = d['count'] + _C.get(d['id'], 0)
C = [{'id':k, 'count':c} for (k, c) in _C.items()]
Here's one way to do it using itertools.groupby:
from itertools import groupby
from operator import itemgetter
f = itemgetter('id')
lst = [{'id': k, 'count': sum(d['count'] for d in g)}
for k, g in groupby(sorted(A+B, key=f), f)]
print(lst)
# [{'id': 'xyz', 'count': 7}, {'id': 'zzz', 'count': 1}]
i have a python list of dictionary as shown below:
mylist = [{'id':1,'value':4},{'id':1,'value':6},{'id':2,'value':6},{'id':3,'value':9},{'id':3,'value':56},{'id':3,'value':67},]
i am trying to create a new list of dictionaries like this by doing some operations on the above shown list of dictionaries
newlist = [{'id':1,'value':[4,6]},{'id':2,'value':[6]},{'id':3,'value':[9,56,67]}]
Does anyone know a good way to do this?
If list items are sorted by id, you can use itertools.groupby:
>>> mylist = [{'id':1,'value':4},{'id':1,'value':6},{'id':2,'value':6},{'id':3,'value':9},{'id':3,'value':56},{'id':3,'v alue':67},]
>>> import itertools
>>> [{'id': key, 'value': [x['value'] for x in grp]}
... for key, grp in itertools.groupby(mylist, key=lambda d: d['id'])]
[{'id': 1, 'value': [4, 6]},
{'id': 2, 'value': [6]},
{'id': 3, 'value': [9, 56, 67]}]
You can construct the entire the list of dictionaries as a single dictionary with multiple values, using defaultdict, like this
from collections import defaultdict
d = defaultdict(list)
for item in mylist:
d[item['id']].append(item['value'])
And then using list comprehension, you can reconstruct the required list of dictionaries like this
print[{'id': key, 'value': d[key]} for key in d]
# [{'id':1, 'value':[4, 6]}, {'id':2, 'value':[6]}, {'id':3, 'value':[9,56,67]}]
You could also use dict comprehension:
newlist = {key: [entries[key] for entries in diclist] for key, value in diclist[0].items()}