We Keep Coding

Python dictionary comprehension - unhashable type: dict [duplicate]

This question already has answers here:
How do I merge a list of dicts into a single dict?
(11 answers)
Closed 1 year ago.
I need to perform below operations:
iterate over a list of dictionary [{},{}]
call a transform function which transforms each dictionary, which returns a dictionary.
Here key and values are not static, but dataframe name and dataframe value. So dictionary may have one ore more key-value pair.
which I would need to store in a final dictionary
Expected : expected data would be a dictionary:
{"key1":"val1", "key2":"val2", "key3":"val3"} # ... in actual, the key would be dataframe name, value would be dataframe value
Simplified Use case:
dictname = [{"key1":"val1","key2":"val2"},{"key3":"value3"}] # input list of dictionary
def transform(each):
return each # to oversimplify, this would be a dictionary with one or more keys with transformations.
final = {transform(each) for each in dictname}
final
went over other related threads on the issue, could not figure out as how to handle the specific case. Could anyone please guide?e

There are several things wrong in your code.
The dict comprehension is as follows: {key: value, for key, value in something that outputs two things}. In your case transform_each outputs a dict. So fixing this we obtain:
dictname = {"key1":"val1","key2":"val2"} # input dictionary
def transform(each):
return {each: "new_value"}
final = {key: transform(each) for key, each in dictname.items()}
final # {'key1': {'val1': 'new_value'}, 'key2': {'val2': 'new_value'}}
This is not what you want. You need to change only the value of the dict. That is the second thing wrong: Your function must output a value, not a dict. Otherwise, as seen, you got a dict of dicts. You can fix it as follows:
dictname = {"key1":"val1","key2":"val2"} # input dictionary
def transform(each):
return "new_value"
final = {key: transform(each) for key, each in dictname.items()}
final # {'key1': 'new_value', 'key2': 'new_value'}

Define transform as
def transform(each):
return <the new value>
and then use
result = {k: transform(k) for k in dictname}
If you cannot update transform use
from collections import ChainMap
result = dict(ChainMap(*(transform(k) for k in dictname)))

This is an updated answer for the updated question. I think the following should do what you want:
dictname = [{"key1":"val1", "key2":"val2"}, {"key3":"val3"}]
def transform(each):
return each
final = {k:v for d in dictname for k, v in transform(d).items()}
The transform function takes one of the dictionaries as an argument, and returns a dictionary. The returned dictionaries from all the calls to transform are combined into the top-level dictionary final.
The example above defined final to be:
{'key1': 'val1', 'key2': 'val2', 'key3': 'val3'}
In this example, transform merely returns the dict that was passed to it, but it could return any dict you want.

For each value in 1 dictionary take keys and for those keys in 2 dictionary print difference in values for all specific pairs of keys

It's my first time here so sorry if I'm not conforming to some unspoken rule of the community
So i have a dictionary mydict
{'ENST00000309502': ['ADORA1'],
'ENST00000337894': ['ADORA1'],
'ENST00000618295': ['ADORA1'],
'ENST00000380573': ['ADRA1A'],
'ENST00000519229': ['ADRA1A'],
'ENST00000337474': ['AVPR2'],
'ENST00000358927': ['AVPR2'],
'ENST00000370049': ['AVPR2'],
'ENST00000216629': ['BDKRB1'],
'ENST00000611804': ['BDKRB1']...}
And for each specific value, I need to take the keys which are keys in my second dictionary mydict2
{'ENST00000216629':['3.61','3.45','3.65'...]
'ENST00000255380':['3.1','3.05','3.15'...]
'ENST00000304421':['3.61','3.15','3.65'...]...}
And i need to find difference in values of mydict2 for each of the key for the same value in mydict
and print result like this
mydict[value] mydict2[key1] mydict2[key2] difference(list of values)
ADORA1 ENST00000309502 ENST00000337894 [3.61,3.25,3.14]
I only need one combination of difference so only 1-2 not 2-1
for some values in mydict there are more than two keys.
Sorry for the long post I'm new in programming and I know I need for if loop, but im not sure how to write it.
Thanks in advance for whoever helps.

In order to loop through all possible values in mydict, we need a set of values in mydict.
Assuming that each value (e.g. ['ADORA1']) is a list with one element, we can flatten the list and create the set of values using
set_of_values = {value[0] for value in mydict.values()}
Then, we need the list of mydict keys corresponding to each value.
We can use list comprehension and create a new dictionary mapping each mydict value to the list of mydict keys.
reversed_dict = {v:[key for (key, value) in mydict.items() if value[0] == v] for v in set_of_values}
The rest is straightforward. For each key in reversed_dict, we can print the corresponding mydict keys and mydict2 values.
for (key, values) in reversed_dict.items():
print(f'{key} {values[0]} {values[1]} {mydict2[values[0]]} {mydict2[values[1]]}')
This will print
ADORA1 ENST00000309502 ENST00000337894 [3.61,3.25,3.14, ...] ['3.1','3.05','3.15'...]
for example.
It is unclear what a difference of a list is. If you need set difference, you can use python set using, for example,
set1 = set(mydict2[values[0]])
set2 = set(mydict2[values[1]])
print(set1 - set2)

I'm assuming you are looking for something like this:
# Loop over the "values" in mydict1 (i.e. the string in the value)
for value in {value[0] for value in mydict1.values()}:
# Find the keys in mydict1 that correspond to the selected value
keys = tuple(key for key in mydict1 if mydict1[key] == [value])
# Loop over pairs of distinct keys
num_of_keys = len(keys)
for key1, key2 in {(keys[i], keys[j])
for i in range(num_of_keys)
for j in range(i+1, num_of_keys)}:
# Check if both keys are keys in mydict2
if (key1 in mydict2) and (key2 in mydict2):
# Calculate the difference: The elements of the values (lists)
# of mydict2 are strings, not numbers: What do you mean by
# 'difference'?
difference = [(float(mydict2[key1]) - float(mydict2[key2][i]))
for i in range(len(mydict2[key1]))]
print(value, key1, key2, difference)
Some things are a bit odd:
Why are the values of mydict1 lists that contain only one string if you're actually interested in the string (it's not a problem, just a bit odd)?
What is the "difference"? The elements in the values of mydict2 (lists) are strings, not numbers?
EDIT: Okay, for that kind of "difference" I'd try:
....
# Check if both keys are keys in mydict2
if (key1 in mydict2) and (key2 in mydict2):
# Select all values from mydict2[key1] that are not in
# mydict2[key2]
difference = set(mydict2[key1]).difference(set(mydict2[key2]))
print(value, key1, key2, difference)
Or use a list comprehension:
....
# Check if both keys are keys in mydict2
if (key1 in mydict2) and (key2 in mydict2):
# Select all values from mydict2[key1] that are not in
# mydict2[key2]
difference = [value
for value in mydict2[key1]
if value not in mydict2[key2]]
print(value, key1, key2, difference)

How can I extract a value from a list of dictionaries?

I have a dictionary:
dict = {
"Bolig":[
{
"Internal_id":27698,
"Title":"Thorshammer 95 7500 Holstebro",
"Area":87.0,
"Rent":6295.0,
"Room":4
}
],
"Contact":[
{
"Name":"John Doe",
"Email":"doe#gmail.com",
"Phone":"33751010212"
}
]
}
I want to extract the value of one of the keys : For example the value of the Internal_id key gives me the value 27698.
I tried to combine the dictionaries to a single dict by:
new_dict = {key: value for r in dict for key,value in r.iteritems()}
and extract but I got an error:
AttributeError: 'str' object has no attribute 'iteritems'
Is there a way I could achieve this?

for r in dict will iterate over the keys of the dictionary: "Bolig" and "Contact". So, r will be a string, and strings don't have the iteritems method (nor do dictionaries in Python 3 - you should be using items instead).
You should iterate over the stitched together values of dict:
import itertools
stitched_values = itertools.chain.from_iterable(dict.values())
result = {key: value for r in stitched_values for key, value in r.items()}
Or, if each value of the dictionary is guaranteed to be a list of one element:
result = {key: value for r in dict.values() for key, value in r[0].items()}

We need to target each layer one by one, the first is a dictionary so we use the key to target the nested dictionaries and in this case that is "Bolig" or "Contact". Then we use [ ] brackets to access the list and pass in an index. Then finally we target the attribute inside the innermost dictionary.
Now that we know how to access each dictionary and list you can understand how to automate this using loops.
x = dict['Bolig'][0]['Internal_id']
print(x) # 27698

In the dictionary comprehension
{key: value for r in dict for key,value in r.iteritems()}
r is the keys of dict, which in your case are strings. You can change this to
{key: value for r in dict.values() for key,value in r[0].items()}

Filtering dictionary based on first key in a tuple key (key1, key2), where key1

I have a dictionary pair_counts['key1, key2':'valueA'] and a dictionary item_counts['key3':'valueB']. I need to create a new dictionary reduced_pair_counts['key1, key2', 'valueA] where key1 is in item_counts['key3']
I'm new to python, so I'm just not sure how to access the first key in a tuple key.
I need a new dictionary where the key/svalues are the same as pair_counts, but only if the first half of the tuple-key is somewhere in item_counts[key]

First be careful about notations.
pair_counts = {(key1, key2):valueA,...}
item_counts = {key3:valueB, ...}
you want:
new_dict = {key:value for key, value in pair_counts.items() if key[0] in item_counts}

Multiple keys per value

Is it possible to assign multiple keys per value in a Python dictionary. One possible solution is to assign value to each key:
dict = {'k1':'v1', 'k2':'v1', 'k3':'v1', 'k4':'v2'}
but this is not memory efficient since my data file is > 2 GB. Otherwise you could make a dictionary of dictionary keys:
key_dic = {'k1':'k1', 'k2':'k1', 'k3':'k1', 'k4':'k4'}
dict = {'k1':'v1', 'k4':'v2'}
main_key = key_dict['k2']
value = dict[main_key]
This is also very time and effort consuming because I have to go through whole dictionary/file twice. Is there any other easy and inbuilt Python solution?
Note: my dictionary values are not simple string (as in the question 'v1', 'v2') rather complex objects (contains different other dictionary/list etc. and not possible to pickle them)
Note: the question seems similar as How can I use both a key and an index for the same dictionary value?
But I am not looking for ordered/indexed dictionary and I am looking for other efficient solutions (if any) other then the two mentioned in this question.

What type are the values?
dict = {'k1':MyClass(1), 'k2':MyClass(1)}
will give duplicate value objects, but
v1 = MyClass(1)
dict = {'k1':v1, 'k2':v1}
results in both keys referring to the same actual object.
In the original question, your values are strings: even though you're declaring the same string twice, I think they'll be interned to the same object in that case
NB. if you're not sure whether you've ended up with duplicates, you can find out like so:
if dict['k1'] is dict['k2']:
print("good: k1 and k2 refer to the same instance")
else:
print("bad: k1 and k2 refer to different instances")
(is check thanks to J.F.Sebastian, replacing id())

Check out this - it's an implementation of exactly what you're asking: multi_key_dict(ionary)
https://pypi.python.org/pypi/multi_key_dict
(sources at https://github.com/formiaczek/python_data_structures/tree/master/multi_key_dict)
(on Unix platforms it possibly comes as a package and you can try to install it with something like:
sudo apt-get install python-multi-key-dict
for Debian, or an equivalent for your distribution)
You can use different types for keys but also keys of the same type. Also you can iterate over items using key types of your choice, e.g.:
m = multi_key_dict()
m['aa', 12] = 12
m['bb', 1] = 'cc and 1'
m['cc', 13] = 'something else'
print m['aa'] # will print '12'
print m[12] # will also print '12'
# but also:
for key, value in m.iteritems(int):
print key, ':', value
# will print:1
# 1 : cc and 1
# 12 : 12
# 13 : something else
# and iterating by string keys:
for key, value in m.iteritems(str):
print key, ':', value
# will print:
# aa : 12
# cc : something else
# bb : cc and 1
m[12] = 20 # now update the value
print m[12] # will print '20' (updated value)
print m['aa'] # will also print '20' (it maps to the same element)
There is no limit to number of keys, so code like:
m['a', 3, 5, 'bb', 33] = 'something'
is valid, and either of keys can be used to refer to so-created value (either to read / write or delete it).
Edit: From version 2.0 it should also work with python3.

Using python 2.7/3 you can combine a tuple, value pair with dictionary comprehension.
keys_values = ( (('k1','k2'), 0), (('k3','k4','k5'), 1) )
d = { key : value for keys, value in keys_values for key in keys }
You can also update the dictionary similarly.
keys_values = ( (('k1',), int), (('k3','k4','k6'), int) )
d.update({ key : value for keys, value in keys_values for key in keys })
I don't think this really gets to the heart of your question but in light of the title, I think this belongs here.

The most straightforward way to do this is to construct your dictionary using the dict.fromkeys() method. It takes a sequence of keys and a value as inputs and then assigns the value to each key.
Your code would be:
dict = dict.fromkeys(['k1', 'k2', 'k3'], 'v1')
dict.update(dict.fromkeys(['k4'], 'v2'))
And the output is:
print(dict)
{'k1': 'v1', 'k2': 'v1', 'k3': 'v1', 'k4': 'v2'}

You can build an auxiliary dictionary of objects that were already created from the parsed data. The key would be the parsed data, the value would be your constructed object -- say the string value should be converted to some specific object. This way you can control when to construct the new object:
existing = {} # auxiliary dictionary for making the duplicates shared
result = {}
for k, v in parsed_data_generator():
obj = existing.setdefault(v, MyClass(v)) # could be made more efficient
result[k] = obj
Then all the result dictionary duplicate value objects will be represented by a single object of the MyClass class. After building the result, the existing auxiliary dictionary can be deleted.
Here the dict.setdefault() may be elegant and brief. But you should test later whether the more talkative solution is not more efficient -- see below. The reason is that MyClass(v) is always created (in the above example) and then thrown away if its duplicate exists:
existing = {} # auxiliary dictionary for making the duplicates shared
result = {}
for k, v in parsed_data_generator():
if v in existing:
obj = existing[v]
else:
obj = MyClass(v)
existing[v] = obj
result[k] = obj
This technique can be used also when v is not converted to anything special. For example, if v is a string, both key and value in the auxiliary dictionary will be of the same value. However, the existence of the dictionary ensures that the object will be shared (which is not always ensured by Python).

I was able to achieve similar functionality using pandas MultiIndex, although in my case the values are scalars:
>>> import numpy
>>> import pandas
>>> keys = [numpy.array(['a', 'b', 'c']), numpy.array([1, 2, 3])]
>>> df = pandas.DataFrame(['val1', 'val2', 'val3'], index=keys)
>>> df.index.names = ['str', 'int']
>>> df.xs('b', axis=0, level='str')
0
int
2 val2
>>> df.xs(3, axis=0, level='int')
0
str
c val3

I'm surprised no one has mentioned using Tuples with dictionaries. This works just fine:
my_dictionary = {}
my_dictionary[('k1', 'k2', 'k3')] = 'v1'
my_dictionary[('k4')] = 'v2'