We Keep Coding

Convert Pandas DataFrame to dictionairy

I have a simple DataFrame:
Name Format
0 cntry int
1 dweight str
2 pspwght str
3 pweight str
4 nwspol str
I want a dictionairy as such:
{
"cntry":"int",
"dweight":"str",
"pspwght":"str",
"pweight":"str",
"nwspol":"str"
}
Where dict["cntry"] would return int or dict["dweight"] would return str.
How could I do this?

How about this:
import pandas as pd
df = pd.DataFrame({'col_1': ['A', 'B', 'C', 'D'], 'col_2': [1, 1, 2, 3], 'col_3': ['Bla', 'Foo', 'Sup', 'Asdf']})
res_dict = dict(zip(df['col_1'], df['col_3']))
Contents of res_dict:
{'A': 'Bla', 'B': 'Foo', 'C': 'Sup', 'D': 'Asdf'}

You're looking for DataFrame.to_dict()
From the documentation:
>>> df = pd.DataFrame({'col1': [1, 2],
... 'col2': [0.5, 0.75]},
... index=['row1', 'row2'])
>>> df
col1 col2
row1 1 0.50
row2 2 0.75
>>> df.to_dict()
{'col1': {'row1': 1, 'row2': 2}, 'col2': {'row1': 0.5, 'row2': 0.75}}
You can always invert an internal dictionary if it's not mapped how you'd like it to be:
inv_dict = {v: k for k, v in original_dict['Name'].items()}

I think you want is:
df.set_index('Name').to_dict()['Format']
Since you want to use the values in the Name column as the keys to your dict.
Note that you might want to do:
df.set_index('Name').astype(str).to_dict()['Format']
if you want the values of the dictionary to be strings.

Getting key values from list outside dictionary

I am trying to calculate a “score” for each key in a dictionary. The values for the key values are in a different list. Simplified example:
I have:
Key_values = ['a': 1, 'b': 2, 'c': 3, 'd': 4]
My_dict = {'player1': ['a', 'd', 'c'], 'player2': ['b', 'a', 'd']}
I want:
Scores = ['player1': 8, 'player2': 7]

You can create it using a dict comprehension:
Key_values = {'a': 1, 'b': 2, 'c': 3, 'd': 4}
My_dict = {'player1': ['a', 'd', 'c'], 'player2': ['b', 'a', 'd']}
scores = {player: sum(Key_values[mark] for mark in marks) for player, marks in My_dict.items()}
print(scores)
# {'player1': 8, 'player2': 7}

Try this:
>>> Key_values = {"a" : 1, "b" : 2, "c": 3, "d" : 4}
>>> My_dict = {"player1":["a", "d", "c"], "player2":["b", "a", "d"]}
>>> Scores= {k: sum(Key_values.get(v_el, 0) for v_el in v) for k,v in My_dict.items()}
>>> Scores
{'player1': 8, 'player2': 7}

try this:
score = {}
key_values = {'a': 1, 'b': 2, 'c': 3, 'd': 4}
my_dict = {'player1': ['a', 'c', 'd'], 'player2': ['b', 'a', 'd']}
scr = 0
for i in my_dict.keys(): # to get all keys from my_dict
for j in my_dict[i]: # iterate the value list for key.
scr += key_values[j]
score[i] = scr
scr = 0
print(score)

Try this: (Updated the syntax in question. key-value pairs are enclosed within curley braces.)
Key_values = {‘a’ : 1, ‘b’ : 2, ‘c’: 3, ‘d’ : 4}
My_dict = {‘player1’=[‘a’, ‘d’, ‘c’], ‘player2’=[‘b’, ‘a’, ‘d’]}
Scores = dict()
for key, value in My_dict.items():
total = 0
for val in value:
total += Key_values[val]
Scores[key] = total
print(Scores)
# {‘player1’ : 8, ‘player2: 7}

You can do it with appropriate dict methods and map, should be the fastest among the ones already posted.
Key_values = {'a': 1, 'b': 2, 'c': 3, 'd': 4}
My_dict = {'player1': ['a', 'd', 'c'], 'player2': ['b', 'a', 'd']}
new_dict = {key:sum(map(Key_values.get,My_dict[key])) for key in My_dict}
print(new_dict)
Output:
{'player1': 8, 'player2': 7}

How do you combine lists of multiple dictionaries in Python?

I'd like to merge a list of dictionaries with lists as values. Given
arr[0] = {'number':[1,2,3,4], 'alphabet':['a','b','c']}
arr[1] = {'number':[3,4], 'alphabet':['d','e']}
arr[2] = {'number':[6,7], 'alphabet':['e','f']}
the result I want would be
merge_arr = {'number':[1,2,3,4,3,4,6,7,], 'alphabet':['a','b','c','d','e','e','f']}
could you recommend any compact code?

If you know these are the only keys in the dict, you can hard code it. If it isn't so simple, show a complicated example.
from pprint import pprint
arr = [
{
'number':[1,2,3,4],
'alphabet':['a','b','c']
},
{
'number':[3,4],
'alphabet':['d','e']
},
{
'number':[6,7],
'alphabet':['e','f']
}
]
merged_arr = {
'number': [],
'alphabet': []
}
for d in arr:
merged_arr['number'].extend(d['number'])
merged_arr['alphabet'].extend(d['alphabet'])
pprint(merged_arr)
Output:
{'alphabet': ['a', 'b', 'c', 'd', 'e', 'e', 'f'],
'number': [1, 2, 3, 4, 3, 4, 6, 7]}

arr = [{'number':[1,2,3,4], 'alphabet':['a','b','c']},{'number':[3,4], 'alphabet':['d','e']},{'number':[6,7], 'alphabet':['e','f']}]
dict = {}
for k in arr[0].keys():
dict[k] = sum([dict[k] for dict in arr], [])
print (dict)
output:
{'number': [1, 2, 3, 4, 3, 4, 6, 7], 'alphabet': ['a', 'b', 'c', 'd', 'e', 'e', 'f']}

Here is code that uses defaultdict to more easily collect the items. You could leave the result as a defaultdict but this version converts that to a regular dictionary. This code will work with any keys, and the keys in the various dictionaries can differ, as long as the values are lists. Therefore this answer is more general than the other answers given so far.
from collections import defaultdict
arr = [{'number':[1,2,3,4], 'alphabet':['a','b','c']},
{'number':[3,4], 'alphabet':['d','e']},
{'number':[6,7], 'alphabet':['e','f']},
]
merge_arr_default = defaultdict(list)
for adict in arr:
for key, value in adict.items():
merge_arr_default[key].extend(value)
merge_arr = dict(merge_arr_default)
print(merge_arr)
The printed result is
{'number': [1, 2, 3, 4, 3, 4, 6, 7], 'alphabet': ['a', 'b', 'c', 'd', 'e', 'e', 'f']}

EDIT: As noted by #pault, the solution below is of quadratic complexity, and therefore not recommended for large lists. There are more optimal ways to go around it.
However if you’re looking for compactness and relative simplicity, keep reading.
If you want a more functional form, this two-liner will do:
arr = [{'number':[1,2,3,4], 'alphabet':['a','b','c']},{'number':[3,4], 'alphabet':['d','e']},{'number':[6,7], 'alphabet':['e','f']}]
keys = ['number', 'alphabet']
merge_arr = {key: reduce(list.__add__, [dict[key] for dict in arr]) for key in keys}
print arr
Outputs:
{'alphabet': ['a', 'b', 'c', 'd', 'e', 'e', 'f'], 'number': [1, 2, 3, 4, 3, 4, 6, 7]}
This won't merge recursively.
If you want it to work with arbitrary keys, not present in each dict, use:
keys = {k for k in dict.keys() for dict in arr}
merge_arr = {key: reduce(list.__add__, [dict.get(key, []) for dict in arr]) for key in keys}

How to sum all the values that belong to the same key?

I'm pulling data from the database and assuming i have something like this:
Product Name Quantity
a 3
a 5
b 2
c 7
I want to sum the Quantity based on Product name, so this is what i want:
product = {'a':8, 'b':2, 'c':7 }
Here's what I'm trying to do after fetching the data from the database:
for row in result:
product[row['product_name']] += row['quantity']
but this will give me: 'a'=5 only, not 8.

Option 1: pandas
This is one way, assuming you begin with a pandas dataframe df. This solution has O(n log n) complexity.
product = df.groupby('Product Name')['Quantity'].sum().to_dict()
# {'a': 8, 'b': 2, 'c': 7}
The idea is you can perform a groupby operation, which produces a series indexed by "Product Name". Then use the to_dict() method to convert to a dictionary.
Option 2: collections.Counter
If you begin with a list or iterator of results, and wish to use a for loop, you can use collections.Counter for O(n) complexity.
from collections import Counter
result = [['a', 3],
['a', 5],
['b', 2],
['c', 7]]
product = Counter()
for row in result:
product[row[0]] += row[1]
print(product)
# Counter({'a': 8, 'c': 7, 'b': 2})
Option 3: itertools.groupby
You can also use a dictionary comprehension with itertools.groupby. This requires sorting beforehand.
from itertools import groupby
res = {i: sum(list(zip(*j))[1]) for i, j in groupby(sorted(result), key=lambda x: x[0])}
# {'a': 8, 'b': 2, 'c': 7}

If you insist on using loops, you can do this:
# fake data to make the script runnable
result = [
{'product_name': 'a', 'quantity': 3},
{'product_name': 'a', 'quantity': 5},
{'product_name': 'b', 'quantity': 2},
{'product_name': 'c', 'quantity': 7}
]
# solution with defaultdict and loops
from collections import defaultdict
d = defaultdict(int)
for row in result:
d[row['product_name']] += row['quantity']
print(dict(d))
The output:
{'a': 8, 'b': 2, 'c': 7}

Since you mention pandas
df.set_index('ProductName').Quantity.sum(level=0).to_dict()
Out[20]: {'a': 8, 'b': 2, 'c': 7}

Use tuple to store the result.
Edit:
Not clear if the data mentioned is really a dataframe.
If yes then li = [tuple(x) for x in df.to_records(index=False)]
li = [('a', 3), ('a', 5), ('b', 2), ('c', 7)]
d = dict()
for key, val in li:
val_old = 0
if key in d:
val_old = d[key]
d[key] = val + val_old
print(d)
Output
{'a': 8, 'b': 2, 'c': 7}

Unpack dictionary that contains a list of dictionaries and insert in columns

With the data below, I'm trying to unfold a dictionary that contains a list of dictionaries, and then group each key with the corresponding values of the other dictionaries together. For example:
result = {
'themes' : [{
'a' : 'orange',
'b' : 6,
'c' : 'neutral',
'd' : 6,
'e' : 0.24
}, {
'a' : 'banana',
'b' : 6,
'c' : 'neutral',
'd' : 6,
'e' : 0.16
}, {
'a' : 'phone',
'b' : 5,
'c' : 'neutral',
'd' : 5,
'e' : 0.02
}
]
}
...should become something along these lines:
themes={'a' : ['orange','banana', 'phone']}
count={'b' : [6,6,5]}
s_score={'c' : [neutral, neutral, neutral]}
...and so on.
I've looked here, here, and here among other places, but couldn't find something close enough to what I want to do. This came pretty close, but it's checking for at least one or more common values, whereas mine should group common keys. I know I can separate the outer key from the values like this:
>>>(k, v), = result.items()
>>>k
>>>'themes'
>>>v
>>>[{
'a' : 'orange',
'b :6,
'c' : 'neutral',
'd' : 6,
'e' : 0.24
}, {
'a' : 'banana',
'b' : 6,
'c' : 'neutral',
'd' : 6,
'e' : 0.16
}, {
'a' : 'phone',
'b' : 5,
'c' : 'neutral',
'd' : 5,
'e' : 0.02
}
]
but how do I get the v list of dictionaries to the way I described? Do I have to convert them to sets first?
To make my intention clear, my ultimate goal is iterate through the list of values of the keys that I want to keep, so I can enter them into their respective columns in my fairly basic flask-sqlalchemy SQLite database. So in the end I'll be able to query and get them displayed as html:
+-----------------+----------+----------+-------+
| a | b | c | d |
+-----------------+----------+----------+-------+
| orange | 2.4 | neutral | 6 |
| banana | 1.6 | neutral | 6 |
+-----------------+----------+----------+-------+

dict1 = {}
for eachKey in list(set(",".join(each.keys()) for each in result["themes"]))[0].split(","):
dict1[eachKey] = [each[eachKey] for each in result["themes"]]
print dict1
It will reduce your result to following dictionary-
{'a': ['orange', 'banana', 'phone'], 'c': ['neutral', 'neutral', 'neutral'], 'b': ['6', 6, 5], 'e': [0.24, 0.16, 0.02], 'd': [6, 6, 5]}

Try this using defaultdict
from collections import defaultdict
d = defaultdict(list)
for i,j in result.iteritems():
for k in j:
for l,m in k.iteritems():
d[l].append(m)
>>>d
defaultdict(<type 'list'>, {'a': ['orange', 'banana', 'phone'], 'c': ['neutral', 'neutral', 'neutral'], 'b': ['6', 6, 5], 'e': [0.24, 0.16, 0.02], 'd': [6, 6, 5]})
Now you can parse it by
themes = {'a':d['a']}
>>>themes
{'a': ['orange', 'banana', 'phone']}
And so on.Hope this helps

You can keep themes, count and score in one dictionary -- final_dict.
In code:
>>> lst = result['themes']
>>> final_dict = {}
>>> for d in lst:
... for (k, v) in d.items():
... final_dict.setdefault(k, []).append(v)
>>> print final_dict
{'a': ['orange', 'banana', 'phone'], 'c': ['neutral', 'neutral', 'neutral'], 'b': [6, 6, 5], 'e': [0.24, 0.16, 0.02], 'd': [6, 6, 5]}

You should first flatMap all your values in the list of tuples ([('a', 'orange'), ('c', 'neutral'), ('b', '6')..]) and then groupBy first element. I would do it this way:
import itertools
pairs = itertools.chain.from_iterable([d.items() for d in result["themes"]])
result = {}
for key, elem in pairs:
result.setdefault(key, []).append(elem)
print result