Related
I've got two lists:
lst1 = [{"name": "Hanna", "age":3},
{"name": "Kris", "age": 18},
{"name":"Dom", "age": 15},
{"name":"Tom", "age": 5}]
and the second one contains a few of above key name values under different key:
lst2 = [{"username": "Kris", "Town": "Big City"},
{"username":"Dom", "Town": "NYC"}]
I would like to merge them with result:
lst = [{"name": "Hanna", "age":3},
{"name": "Kris", "age": 18, "Town": "Big City"},
{"name":"Dom", "age": 15, "Town": "NYC"},
{"name":"Tom", "age":"5"}]
The easiest way is to go one by one (for each element from lst1, check whether it exists in lst2), but for big lists, this is quite ineffective (my lists have a few hundred elements each). What is the most effective way to achieve this?
To avoid iterating over another list again and again, you can build a name index first.
lst1 = [{"name": "Hanna", "age":3},
{"name": "Kris", "age": 18},
{"name":"Dom", "age": 15},
{"name":"Tom", "age": 5}]
lst2 = [{"username": "Kris", "Town": "Big City"},
{"username":"Dom", "Town": "NYC"}]
name_index = { dic['username'] : idx for idx, dic in enumerate(lst2) if dic.get('username') }
for dic in lst1:
name = dic.get('name')
if name in name_index:
dic.update(lst2[name_index[name]]) # update in-place to further save time
dic.pop('username')
print(lst1)
One way to do this a lot more efficient than by lists is to create an intermediate dictionary from lst1 with name as key, so that you're searching a dictionary not a list.
d1 = {elem['name']: {k:v for k,v in elem.items()} for elem in lst1}
for elem in lst2:
d1[elem['username']].update( {k:v for k,v in elem.items() if k != 'username'} )
lst = list(d1.values())
Output:
[{'name': 'Hanna', 'age': 3}, {'name': 'Kris', 'age': 18, 'Town': 'Big City'}, {'name': 'Dom', 'age': 15, 'Town': 'NYC'}, {'name': 'Tom', 'age': 5}]
edited to only have one intermediate dict
Use zip function to pair both lists. We need to order both lists using some criteria, in this case, you must use the username and name keys for the lists because those values will be your condition to perform the updating action, for the above reason is used the sorted function with key param. It is important to sort them out to get the match.
Finally your list lst2 has a little extra procedure, I expanded it taking into account the length of lst1, that is what I do using lst2 * abs(len(lst1) - len(lst2). Theoretically, you are iterating once over an iterable zip object, therefore I consider this could be a good solution for your requirements.
for d1, d2 in zip(sorted(lst1, key=lambda d1: d1['name']),
sorted(lst2 * abs(len(lst1) - len(lst2)), key=lambda d2: d2['username'])):
if d1['name'] == d2['username']:
d1.update(d2)
# Just we delete the username
del d1['username']
print(lst1)
Output:
[{'name': 'Hanna', 'age': 3}, {'name': 'Kris', 'age': 18, 'Town': 'Big City'}, {'name': 'Dom', 'age': 15, 'Town': 'NYC'}, {'name': 'Tom', 'age': 5}]
I have multiple lists of data, for example: age, name, gender, etc. All of them in order, meaning that the x record of every list belongs to the same person.
What I'm trying to create is a list of dictionaries from these lists in the best pythonic way. I was able to create it using one of the lists, but not sure how to scale it from there.
What I currently have:
ages = [20, 21, 30]
names = ["Jhon", "Daniel", "Rob"]
list_of_dicts = [{"age": value} for value in ages]
It returns:
[{'age': 20}, {'age': 21}, {'age': 30}]
What I want:
[{'age': 20, 'name': 'Jhon'}, {'age': 21, 'name': 'Daniel'}, {'age': 30, 'name': 'Rob'}]
You need to zip:
ages = [20, 21, 30]
names = ["Jhon", "Daniel", "Rob"]
list_of_dicts = [{"age": value, 'name': name}
for value, name in zip(ages, names)]
You can take this one step further and use a double zip (useful if you have many more keys):
keys = ['ages', 'names']
lists = [ages, names]
list_of_dicts = [dict(zip(keys, x)) for x in zip(*lists)]
output:
[{'age': 20, 'name': 'Jhon'},
{'age': 21, 'name': 'Daniel'},
{'age': 30, 'name': 'Rob'}]
Less obvious code than #mozway's, but has imho one advantage - it relies only on a single definition of a mapping dictionary so if you need to add/remove keys you have to change only one k:v pair.
ages = [20, 21, 30]
names = ["Jhon", "Daniel", "Rob"]
d = {
"name" : names,
"age" : ages
}
list_of_dicts = [dict(zip(d,t)) for t in zip(*d.values())]
print(list_of_dicts)
The input dictionary of dictionaries are dict1 and dict2.
dict1 = {company1:[{'age':27,'weight':200,'name':'john'},{'age':23,'weight':180,'name':'peter'}],
company2:[{'age':30,'weight':190,'name':'sam'},{'age':32,'weight':210,'name':'clove'},{'age':21,'weight':170,'name':'steve'}],
company3:[{'age':36,'weight':175,'name':'shaun'},{'age':40,'weight':205,'name':'dany'},{'age':25,'weight':160,'name':'mark'}]
company4:[{'age':36,'weight':155,'name':'lina'},{'age':40,'weight':215,'name':'sammy'},{'age':25,'weight':190,'name':'matt'}]
}
dict2 = {company2:[{'age':30},{'age':45},{'age':52}],
company4:[{'age':43},{'age':67},{'age':22},{'age':34},{'age':42}]
}
I am trying to write a logic where I can check inner key ('age') of each compay key in dict2 exist in same company key dict1, even if one value of inner key 'age' matches with inner key ('age') in dict1 of same company key, then save it to a third dictionary. Please check the below example
Example:
company2:[{'age':30}]
matches with
company2:[{'age':30,'weight':190,'name':'sam'}, ...]
Also I want to save the key:values of dict1 which doesn't appered in dict2 to the dict3, As we can see in the below example company1 key does not apper in dict2.
Example:
company1:[{'age':27,'weight':200,'name':'john'},{'age':23,'weight':180,'name':'peter'}]
and
company3:[{'age':36,'weight':175,'name':'shaun'},{'age':40,'weight':205,'name':'dany'},{'age':25,'weight':160,'name':'mark'}]
Expected Output:
dict3 = {company1:[{'age':27,'weight':200,'name':'john'},{'age':23,'weight':180,'name':'peter'}],
company2:[{'age':30,'weight':190,'name':'sam'},{'age':32,'weight':210,'name':'clove'},{'age':21,'weight':170,'name':'steve'}]
company3:[{'age':36,'weight':175,'name':'shaun'},{'age':40,'weight':205,'name':'dany'},{'age':25,'weight':160,'name':'mark'}]}
pardon my explanation!
This solution might be better done using some other method more succinctly. However, it accomplishes the desired result.
from pprint import pprint
dict3 = dict()
dict1 = {'company1':[{'age':27,'weight':200,'name':'john'},{'age':23,'weight':180,'name':'peter'}],
'company2':[{'age':30,'weight':190,'name':'sam'},{'age':32,'weight':210,'name':'clove'},{'age':21,'weight':170,'name':'steve'}],
'company3':[{'age':36,'weight':175,'name':'shaun'},{'age':40,'weight':205,'name':'dany'},{'age':25,'weight':160,'name':'mark'}],
'company4':[{'age':36,'weight':155,'name':'lina'},{'age':40,'weight':215,'name':'sammy'},{'age':25,'weight':190,'name':'matt'}]
}
dict2 = {'company2':[{'age':30},{'age':45},{'age':52}],
'company4':[{'age':43},{'age':67},{'age':22},{'age':34},{'age':42}]
}
for company, array in dict1.items():
if company not in dict2:
dict3[company] = array
else:
# all the ages for this company in dict1
ages = set(map(lambda x: x['age'], array))
for dictref in dict2[company]:
if dictref['age'] in ages:
dict3[company] = array
break
pprint(dict3)
Output was
{'company1': [{'age': 27, 'name': 'john', 'weight': 200},
{'age': 23, 'name': 'peter', 'weight': 180}],
'company2': [{'age': 30, 'name': 'sam', 'weight': 190},
{'age': 32, 'name': 'clove', 'weight': 210},
{'age': 21, 'name': 'steve', 'weight': 170}],
'company3': [{'age': 36, 'name': 'shaun', 'weight': 175},
{'age': 40, 'name': 'dany', 'weight': 205},
{'age': 25, 'name': 'mark', 'weight': 160}]}
I have a situation where I need to print the frequencies of data appearing in one of the columns of my dataframe.
Suppose my column is status, then performing
df['status'].value_counts().to_dict()
outputs
{
"Deleted": 56,
"New": 25,
"Draft": 24,
"Assigned": 11,
"Job Complete": 10,
"Active": 8,
"Requested": 3,
"Cancelled": 3,
"Footage Provided": 1
}
I want to format the output as:
{
{status: "Deleted", value: 56},
{status: "New", value: 25},
{status: "Draft", value: 24},
...
}
I'm new to pandas. Please help.
You can use the below list comprehension:
print([{'status': k, 'value': v} for k, v in df['status'].value_counts().to_dict().items()])
Output's gonna be expected.
You can just reformat the pandas output dictionary to your desired list format. Iterate over the dictionary and append the keys and values as a dictionary object to you list:
d1 = df['status'].value_counts().to_dict()
l = []
for k, v in d1.items():
l.append({'status': k, 'value': v})
print(l)
Output
[{'status': 'Deleted', 'value': 56},
{'status': 'New', 'value': 25},
{'status': 'Draft', 'value': 24},
...
]
Hi I have a dictionary like below
{
'namelist': [{'name':"John",'age':23,'country':'USA'},
{'name':"Mary",'age':12,'country':'Italy'},
{'name':"Susan",'age':32,'country':'UK'}],
'classteacher':'Jon Smith'
}
I would like to know is it possible to change it to
{
'namelist': [{'name_1':"John",'age_1':23,'country_1':'USA'},
{'name_2':"Mary",'age_2':12,'country_3':'Italy'},
{'name_3':"Susan",'age_3':32,'country_3':'UK'}],
'classteacher':'Jon Smith'
}
By adding _1, _2 .... on every last position of every key
Is it possible? Thank you for your help
You can add the new values in the initial list with changing the key and removing the initial values yourdict[j+'_'+str(num)] = yourdict.pop(j)
keys() returns all the keys of a dict (name, age, country in your case)
a = {'namelist': [{'name':"John",'age':23,'country':'USA'},
{'name':"Mary",'age':12,'country':'Italy'},
{'name':"Susan",'age':32,'country':'UK'}]}
num = 1
for i in a['namelist']:
for j in list(i.keys()):
i[j+'_'+str(num)] = i.pop(j)
num += 1
print(a)
# {'namelist': [
# {'name_1': 'John', 'country_1': 'USA', 'age_1': 23},
# {'name_2': 'Mary', 'country_2': 'Italy', 'age_2': 12},
# {'name_3': 'Susan', 'age_3': 32, 'country_3': 'UK'}]}
Here is my one-line style solution, which works even if you have many keys other than 'namelist':
d = {'namelist': [{'name':"John",'age':23,'country':'USA'},
{'name':"Mary",'age':12,'country':'Italy'},
{'name':"Susan",'age':32,'country':'UK'}],
'classteacher':'Jon Smith'
}
d = {k:[{f'{k2}_{nb}':v2 for k2,v2 in i.items()} for nb,i in enumerate(v,1)] if isinstance(v,list) else v for k,v in d.items()}
print(d)
# {'namelist': [{'name_1': 'John', 'age_1': 23, 'country_1': 'USA'},
# {'name_2': 'Mary', 'age_2': 12, 'country_2': 'Italy'},
# {'name_3': 'Susan', 'age_3': 32, 'country_3': 'UK'}]},
# 'classteacher': 'Jon Smith'
# }
However as Aran-Fey said, this is not really readable and very difficult to maintain. So I also suggest you the solution with nested for loops:
d1 = {'namelist': [{'name':"John",'age':23,'country':'USA'},
{'name':"Mary",'age':12,'country':'Italy'},
{'name':"Susan",'age':32,'country':'UK'}],
'classteacher':'Jon Smith'}
for k1,v1 in d1.items():
if isinstance(v1,list):
for nb,d2 in enumerate(v1,1):
for k2 in list(d2):
d2[f'{k2}_{nb}'] = d2.pop(k2)
print(d1)
# {'namelist': [{'name_1': 'John', 'age_1': 23, 'country_1': 'USA'},
# {'name_2': 'Mary', 'age_2': 12, 'country_2': 'Italy'},
# {'name_3': 'Susan', 'age_3': 32, 'country_3': 'UK'}]},
# 'classteacher': 'Jon Smith'
# }
Using enumerate
Ex:
d = {'namelist': [{'name':"John",'age':23,'country':'USA'},
{'name':"Mary",'age':12,'country':'Italy'},
{'name':"Susan",'age':32,'country':'UK'}]}
d["namelist"] = [{k+"_"+str(i): v for k,v in value.items()} for i , value in enumerate(d["namelist"], 1)]
print(d)
Output:
{'namelist': [{'age_1': 23, 'country_1': 'USA', 'name_1': 'John'},
{'age_2': 12, 'country_2': 'Italy', 'name_2': 'Mary'},
{'age_3': 32, 'country_3': 'UK', 'name_3': 'Susan'}]}
You will have to create new key, with value correspond to old key. You can achieve this easily in one line using dict.pop
I will assume you want to add index of the row into field name. For other fields or modified them in other ways, you can do similarly.
for index, row in enumerate(a['namelist']):
row['name_%d' % index] = row.pop('name')
Output:
{'namelist': [{'age': 23, 'country': 'USA', 'name_0': 'John'},
{'age': 12, 'country': 'Italy', 'name_1': 'Mary'},
{'age': 32, 'country': 'UK', 'name_2': 'Susan'}]}
You can use dict and list comprehensions:
d = {'namelist': [{'name': "John", 'age': 23, 'country': 'USA'},
{'name': "Mary", 'age': 12, 'country': 'Italy'},
{'name': "Susan", 'age': 32, 'country': 'UK'}]}
d = {k: [{'_'.join((n, str(i))): v for n, v in s.items()} for i, s in enumerate(l, 1)] for k, l in d.items()}
d would become:
{'namelist': [{'name_1': 'John', 'age_1': 23, 'country_1': 'USA'}, {'name_2': 'Mary', 'age_2': 12, 'country_2': 'Italy'}, {'name_3': 'Susan', 'age_3': 32, 'country_3': 'UK'}]}
Use dictionary comprehension:
mydictionary['namelist'] = [dict((key + '_' + str(i), value) for key,value in mydictionary['namelist'][i-1].items()) for i in [1, 2, 3]]
for i, dct in enumerate(inp['namelist'], 1):
for key, value in list(dct.items()): # take a copy since we are modifying the dct
del dct[key] # delete old pair
dct[key+'_'+str(i)] = value # new key format
This would be in place. You are not using extra memory. Iterating over each value inside the dict and then deleting the old key-value pair and adding the it with a change in the key name.