Related
I have a list of dictionaries. I want to loop through this list of dictionary and for each specific name (an attribute inside each dictionary), I want to create a dictionary where the key is the name and the value of this key is a list which dynamically appends to the list in accordance with a specific condition.
For example, I have
d = [{'Name': 'John', 'id': 10},
{'Name': 'Mark', 'id': 21},
{'Name': 'Matthew', 'id': 30},
{'Name': 'Luke', 'id': 11},
{'Name': 'John', 'id': 20}]
I then built a list with only the names using names=[i['Name'] for i in dic1] so I have a list of names. Notice John will appear twice in this list (at the beginning and end). Then, I want to create a for-loop (for name in names), which creates a dictionary 'ID' that for its value is a list which appends this id field as it goes along.
So in the end I'm looking for this ID dictionary to have:
John: [10,20]
Mark: [21]
Matthew: [30]
Luke: [11]
Notice that John has a list length of two because his name appears twice in the list of dictionaries.
But I can't figure out a way to dynamically append these values to a list inside the for-loop. I tried:
ID={[]} #I also tried with just {}
for name in names:
ID[names].append([i['id'] for i in dic1 if i['Name'] == name])
Please let me know how one can accomplish this. Thanks.
Don't loop over the list of names and go searching for every one in the list; that's very inefficient, since you're scanning the whole list all over again for every name. Just loop over the original list once and update the ID dict as you go. Also, if you build the ID dict first, then you can get the list of names from it and avoid another list traversal:
names = ID.keys()
The easiest solution for ID itself is a dictionary with a default value of the empty list; that way ID[name].append will work for names that aren't in the dict yet, instead of blowing up with a KeyError.
from collections import defaultdict
ID = defaultdict(list)
for item in d:
ID[item['Name']].append(item['id'])
You can treat a defaultdict like a normal dict for almost every purpose, but if you need to, you can turn it into a plain dict by calling dict on it:
plain_id = dict(ID)
The Thonnu has a solution using get and list concatenation which works without defaultdict. Here's another take on a no-import solution:
ID = {}
for item in d:
name, number = item['Name'], item['id']
if name in ID:
ID[name].append(number)
else:
ID[name] = [ number ]
Using collections.defaultdict:
from collections import defaultdict
out = defaultdict(list)
for item in dic1:
out[item['Name']].append(item['id'])
print(dict(out))
Or, without any imports:
out = {}
for item in dic1:
out[item['Name']] = out.get(item['Name'], []) + [item['id']]
print(out)
Or, with a list comprehension:
out = {}
[out.update({item['Name']: out.get(item['Name'], []) + [item['id']]}) for item in dic1]
print(out)
Output:
{'John': [10, 20], 'Mark': [21], 'Matthew': [30], 'Luke': [11]}
dic1 = [{'Name': 'John', 'id': 10}, {'Name': 'Mark', 'id': 21}, {'Name': 'Matthew', 'id': 30}, {'Name': 'Luke', 'id': 11}, {'Name': 'John', 'id': 20}]
id_dict = {}
for dic in dic1:
key = dic['Name']
if key in id_dict:
id_dict[key].append(dic['id'])
else:
id_dict[key] = [dic['id']]
print(id_dict) # {'John': [10, 20], 'Mark': [21], 'Matthew': [30], 'Luke': [11]}
You can use defaultdict for this to initiate a dictionary with a default value. In this case the default value will be empty list.
from collections import defaultdict
d=defaultdict(list)
for item in dic1:
d[item['Name']].append(item['id'])
Output
{'John': [10, 20], 'Mark': [21], 'Matthew': [30], 'Luke': [11]} # by converting (not required) into pure dict dict(d)
You can do in a easy version
dic1=[{'Name': 'John', 'id':10}, {'Name': 'Mark', 'id':21},{'Name': 'Matthew', 'id':30}, {'Name': 'Luke', 'id':11}, {'Name': 'John', 'id':20}]
names=[i['Name'] for i in dic1]
ID = {}
for i, name in enumerate(names):
if name in ID:
ID[name].append(dic1[i]['id'])
else:
ID[name] = [dic1[i]['id']]
print(ID)
I have a nested python dictionary in the following format
{"exist":{"name":["a","b"],"country":["US"],"team":["SP","FR"]}, "not_exist":{}}
I want to append the following dictionary to the above one:
{"exist":{"age":[23,43,45],"sports":["football","rugby"]}, "not_exist":{"title":["Mr","Ms"]}}
So that the resulting dictionary looks like
{"exist":{"name":["a","b"],"country":["US"],"team":["SP","FR"],"age":[23,43,45],"sports":["football","rugby"]},
"not_exist":{"title":["Mr","Ms"]}}
I tried with update method, but its not working.
You can also use | operator (for python 3.9+) between dicts:
dct1 = {"exist":{"name":["a","b"],"country":["US"],"team":["SP","FR"]}, "not_exist":{}}
dct2 = {"exist":{"age":[23,43,45],"sports":["football","rugby"]}, "not_exist":{"title":["Mr","Ms"]}}
output = {k: v | dct2[k] for k, v in dct1.items()}
print(output)
# {'exist': {'name': ['a', 'b'], 'country': ['US'], 'team': ['SP', 'FR'], 'age': [23, 43, 45], 'sports': ['football', 'rugby']},
# 'not_exist': {'title': ['Mr', 'Ms']}}
For python with version lower than 3.9, you can use:
for k, v in dct2.items():
dct1[k].update(v)
print(dct1)
Note that this modifies your original dct1.
You can traverse the second dictionary and append all those values to the first one as follows:
d = {"exist":{"name":["a","b"],"country":["US"],"team":["SP","FR"]}, "not_exist":{}}
d1 = {"exist":{"age":[23,43,45],"sports":["football","rugby"]}, "not_exist":{"title":["Mr","Ms"]}}
for i in list(d1["exist"].keys()):
d["exist"][i] = d1["exist"][i]
print(d1)
Output:
{'exist': {'age': [23, 43, 45], 'sports': ['football', 'rugby']}, 'not_exist': {'title': ['Mr', 'Ms']}}
dict1 = {"exist":{"name":["a","b"],"country":["US"],"team":["SP","FR"]}, "not_exist":{}}
dict2 = {"exist":{"age":[23,43,45],"sports":["football","rugby"]}, "not_exist":{"title":["Mr","Ms"]}}
dict1["exist"].update(dict2["exist"])
dict1["not_exist"].update(dict2["not_exist"])
print(dict1)
This simple code snippet did the trick for me?
Output:
{'exist': {'name': ['a', 'b'], 'country': ['US'], 'team': ['SP', 'FR'], 'age': [23, 43, 45], 'sports': ['football', 'rugby']}, 'not_exist': {'title': ['Mr', 'Ms']}}
I have multiple lists of data, for example: age, name, gender, etc. All of them in order, meaning that the x record of every list belongs to the same person.
What I'm trying to create is a list of dictionaries from these lists in the best pythonic way. I was able to create it using one of the lists, but not sure how to scale it from there.
What I currently have:
ages = [20, 21, 30]
names = ["Jhon", "Daniel", "Rob"]
list_of_dicts = [{"age": value} for value in ages]
It returns:
[{'age': 20}, {'age': 21}, {'age': 30}]
What I want:
[{'age': 20, 'name': 'Jhon'}, {'age': 21, 'name': 'Daniel'}, {'age': 30, 'name': 'Rob'}]
You need to zip:
ages = [20, 21, 30]
names = ["Jhon", "Daniel", "Rob"]
list_of_dicts = [{"age": value, 'name': name}
for value, name in zip(ages, names)]
You can take this one step further and use a double zip (useful if you have many more keys):
keys = ['ages', 'names']
lists = [ages, names]
list_of_dicts = [dict(zip(keys, x)) for x in zip(*lists)]
output:
[{'age': 20, 'name': 'Jhon'},
{'age': 21, 'name': 'Daniel'},
{'age': 30, 'name': 'Rob'}]
Less obvious code than #mozway's, but has imho one advantage - it relies only on a single definition of a mapping dictionary so if you need to add/remove keys you have to change only one k:v pair.
ages = [20, 21, 30]
names = ["Jhon", "Daniel", "Rob"]
d = {
"name" : names,
"age" : ages
}
list_of_dicts = [dict(zip(d,t)) for t in zip(*d.values())]
print(list_of_dicts)
name=[]
age=[]
address=[]
...
for line in pg:
for key,value in line.items():
if key == 'name':
name.append(value)
elif key == 'age':
age.append(value)
elif key == 'address':
address.append(value)
.
.
.
Is it possible to use list comprehension for above code because I need to separate lots of value in the dict? I will use the lists to write to a text file.
Source Data:
a = [{'name': 'paul', 'age': '26.', 'address': 'AU', 'gender': 'male'},
{'name': 'mei', 'age': '26.', 'address': 'NY', 'gender': 'female'},
{'name': 'smith', 'age': '16.', 'address': 'NY', 'gender': 'male'},
{'name': 'raj', 'age': '13.', 'address': 'IND', 'gender': 'male'}]
I don't think list comprehension will be a wise choice because you have multiple lists.
Instead of making multiple lists and appending to them the value if the key matches you can use defaultdict to simplify your code.
from collections import defaultdict
result = defaultdict(list)
for line in pg:
for key, value in line.items():
result[key].append(value)
You can get the name list by using result.get('name')
['paul', 'mei', 'smith', 'raj']
This probably won't work the way you want: Your'e trying to assign the three different lists, so you would need three different comprehensions. If your dict is large, this would roughly triple your execution time.
Something straightforward, such as
name = [value for for key,value in line.items() if key == "name"]
seems to be what you'd want ... three times.
You can proceed as :
pg=[{"name":"name1","age":"age1","address":"address1"},{"name":"name2","age":"age2","address":"address2"}]
name=[v for line in pg for k,v in line.items() if k=="name"]
age=[v for line in pg for k,v in line.items() if k=="age"]
address=[v for line in pg for k,v in line.items() if k=="address"]
In continuation with Vishal's answer, please dont use defaultdict. Using defaultdict is a very bad practice when you want to catch keyerrors. Please use setdefault.
results = dict()
for line in pg:
for key, value in line.items():
result.setdefault(key, []).append(value)
Output
{
'name': ['paul', 'mei', 'smith', 'raj'],
'age': [26, 26, 26, 13],
...
}
However, note that if all dicts in pg dont have the same keys, you will lose the relation/correspondence between the items in the dict
Here is a really simple solution if you want to use pandas:
import pandas as pd
df = pd.DataFrame(a)
name = df['name'].tolist()
age = df['age'].tolist()
address = df['address'].tolist()
print(name)
print(age)
print(address)
Output:
['paul', 'mei', 'smith', 'raj']
['26.', '26.', '16.', '13.']
['AU', 'NY', 'NY', 'IND']
Additionally, if your end result is a text file, you can skip the list creation and write the DataFrame (or parts thereof) directly to a CSV with something as simple as:
df.to_csv('/path/to/output.csv')
Hi I have a dictionary like below
{
'namelist': [{'name':"John",'age':23,'country':'USA'},
{'name':"Mary",'age':12,'country':'Italy'},
{'name':"Susan",'age':32,'country':'UK'}],
'classteacher':'Jon Smith'
}
I would like to know is it possible to change it to
{
'namelist': [{'name_1':"John",'age_1':23,'country_1':'USA'},
{'name_2':"Mary",'age_2':12,'country_3':'Italy'},
{'name_3':"Susan",'age_3':32,'country_3':'UK'}],
'classteacher':'Jon Smith'
}
By adding _1, _2 .... on every last position of every key
Is it possible? Thank you for your help
You can add the new values in the initial list with changing the key and removing the initial values yourdict[j+'_'+str(num)] = yourdict.pop(j)
keys() returns all the keys of a dict (name, age, country in your case)
a = {'namelist': [{'name':"John",'age':23,'country':'USA'},
{'name':"Mary",'age':12,'country':'Italy'},
{'name':"Susan",'age':32,'country':'UK'}]}
num = 1
for i in a['namelist']:
for j in list(i.keys()):
i[j+'_'+str(num)] = i.pop(j)
num += 1
print(a)
# {'namelist': [
# {'name_1': 'John', 'country_1': 'USA', 'age_1': 23},
# {'name_2': 'Mary', 'country_2': 'Italy', 'age_2': 12},
# {'name_3': 'Susan', 'age_3': 32, 'country_3': 'UK'}]}
Here is my one-line style solution, which works even if you have many keys other than 'namelist':
d = {'namelist': [{'name':"John",'age':23,'country':'USA'},
{'name':"Mary",'age':12,'country':'Italy'},
{'name':"Susan",'age':32,'country':'UK'}],
'classteacher':'Jon Smith'
}
d = {k:[{f'{k2}_{nb}':v2 for k2,v2 in i.items()} for nb,i in enumerate(v,1)] if isinstance(v,list) else v for k,v in d.items()}
print(d)
# {'namelist': [{'name_1': 'John', 'age_1': 23, 'country_1': 'USA'},
# {'name_2': 'Mary', 'age_2': 12, 'country_2': 'Italy'},
# {'name_3': 'Susan', 'age_3': 32, 'country_3': 'UK'}]},
# 'classteacher': 'Jon Smith'
# }
However as Aran-Fey said, this is not really readable and very difficult to maintain. So I also suggest you the solution with nested for loops:
d1 = {'namelist': [{'name':"John",'age':23,'country':'USA'},
{'name':"Mary",'age':12,'country':'Italy'},
{'name':"Susan",'age':32,'country':'UK'}],
'classteacher':'Jon Smith'}
for k1,v1 in d1.items():
if isinstance(v1,list):
for nb,d2 in enumerate(v1,1):
for k2 in list(d2):
d2[f'{k2}_{nb}'] = d2.pop(k2)
print(d1)
# {'namelist': [{'name_1': 'John', 'age_1': 23, 'country_1': 'USA'},
# {'name_2': 'Mary', 'age_2': 12, 'country_2': 'Italy'},
# {'name_3': 'Susan', 'age_3': 32, 'country_3': 'UK'}]},
# 'classteacher': 'Jon Smith'
# }
Using enumerate
Ex:
d = {'namelist': [{'name':"John",'age':23,'country':'USA'},
{'name':"Mary",'age':12,'country':'Italy'},
{'name':"Susan",'age':32,'country':'UK'}]}
d["namelist"] = [{k+"_"+str(i): v for k,v in value.items()} for i , value in enumerate(d["namelist"], 1)]
print(d)
Output:
{'namelist': [{'age_1': 23, 'country_1': 'USA', 'name_1': 'John'},
{'age_2': 12, 'country_2': 'Italy', 'name_2': 'Mary'},
{'age_3': 32, 'country_3': 'UK', 'name_3': 'Susan'}]}
You will have to create new key, with value correspond to old key. You can achieve this easily in one line using dict.pop
I will assume you want to add index of the row into field name. For other fields or modified them in other ways, you can do similarly.
for index, row in enumerate(a['namelist']):
row['name_%d' % index] = row.pop('name')
Output:
{'namelist': [{'age': 23, 'country': 'USA', 'name_0': 'John'},
{'age': 12, 'country': 'Italy', 'name_1': 'Mary'},
{'age': 32, 'country': 'UK', 'name_2': 'Susan'}]}
You can use dict and list comprehensions:
d = {'namelist': [{'name': "John", 'age': 23, 'country': 'USA'},
{'name': "Mary", 'age': 12, 'country': 'Italy'},
{'name': "Susan", 'age': 32, 'country': 'UK'}]}
d = {k: [{'_'.join((n, str(i))): v for n, v in s.items()} for i, s in enumerate(l, 1)] for k, l in d.items()}
d would become:
{'namelist': [{'name_1': 'John', 'age_1': 23, 'country_1': 'USA'}, {'name_2': 'Mary', 'age_2': 12, 'country_2': 'Italy'}, {'name_3': 'Susan', 'age_3': 32, 'country_3': 'UK'}]}
Use dictionary comprehension:
mydictionary['namelist'] = [dict((key + '_' + str(i), value) for key,value in mydictionary['namelist'][i-1].items()) for i in [1, 2, 3]]
for i, dct in enumerate(inp['namelist'], 1):
for key, value in list(dct.items()): # take a copy since we are modifying the dct
del dct[key] # delete old pair
dct[key+'_'+str(i)] = value # new key format
This would be in place. You are not using extra memory. Iterating over each value inside the dict and then deleting the old key-value pair and adding the it with a change in the key name.