We Keep Coding

Python dictionary comprehension - unhashable type: dict [duplicate]

This question already has answers here:
How do I merge a list of dicts into a single dict?
(11 answers)
Closed 1 year ago.
I need to perform below operations:
iterate over a list of dictionary [{},{}]
call a transform function which transforms each dictionary, which returns a dictionary.
Here key and values are not static, but dataframe name and dataframe value. So dictionary may have one ore more key-value pair.
which I would need to store in a final dictionary
Expected : expected data would be a dictionary:
{"key1":"val1", "key2":"val2", "key3":"val3"} # ... in actual, the key would be dataframe name, value would be dataframe value
Simplified Use case:
dictname = [{"key1":"val1","key2":"val2"},{"key3":"value3"}] # input list of dictionary
def transform(each):
return each # to oversimplify, this would be a dictionary with one or more keys with transformations.
final = {transform(each) for each in dictname}
final
went over other related threads on the issue, could not figure out as how to handle the specific case. Could anyone please guide?e

There are several things wrong in your code.
The dict comprehension is as follows: {key: value, for key, value in something that outputs two things}. In your case transform_each outputs a dict. So fixing this we obtain:
dictname = {"key1":"val1","key2":"val2"} # input dictionary
def transform(each):
return {each: "new_value"}
final = {key: transform(each) for key, each in dictname.items()}
final # {'key1': {'val1': 'new_value'}, 'key2': {'val2': 'new_value'}}
This is not what you want. You need to change only the value of the dict. That is the second thing wrong: Your function must output a value, not a dict. Otherwise, as seen, you got a dict of dicts. You can fix it as follows:
dictname = {"key1":"val1","key2":"val2"} # input dictionary
def transform(each):
return "new_value"
final = {key: transform(each) for key, each in dictname.items()}
final # {'key1': 'new_value', 'key2': 'new_value'}

Define transform as
def transform(each):
return <the new value>
and then use
result = {k: transform(k) for k in dictname}
If you cannot update transform use
from collections import ChainMap
result = dict(ChainMap(*(transform(k) for k in dictname)))

This is an updated answer for the updated question. I think the following should do what you want:
dictname = [{"key1":"val1", "key2":"val2"}, {"key3":"val3"}]
def transform(each):
return each
final = {k:v for d in dictname for k, v in transform(d).items()}
The transform function takes one of the dictionaries as an argument, and returns a dictionary. The returned dictionaries from all the calls to transform are combined into the top-level dictionary final.
The example above defined final to be:
{'key1': 'val1', 'key2': 'val2', 'key3': 'val3'}
In this example, transform merely returns the dict that was passed to it, but it could return any dict you want.

Python: add to a dictionary in a list

I have the following dictionary (It is for creating json),
temp = {'logs':[]}
I want to append dictionaries, but i only got 1 key:val at a time.
what I tried:
temp['logs'].append({key:val})
This does as expected and appends the dict to the array.
But now I want to add a key/val pair to this dictionary, how can I do this?
I've tried using append/extend but that just adds a new dictionary to the list.

But now I want to add a key/val pair to this dictionary
You can index the list and update that dictionary:
temp['logs'][0].update({'new_key': 'new_value'})

You can use this command to change your dict values :
>>> temp['logs'][0]={'no':'val'}
>>> temp
{'logs': [{'no': 'val'}]}
And this one to add values :
>>> temp['logs'][0].update({'yes':'val'})
>>> temp
{'logs': [{'key': 'val', 'yes': 'val'}]}

There must be unique "key" every time you append it. (If it is for json)
Also making "=" will update your old dictionary
What I have done when I was stuck once is
user = {}
name,password,id1 = [],[],[]
user1=session.query(User).all()
for i in user1:
name=i.name
password=i.password
id1=i.id
user.update({ id1:{
"name" : name,
"password" : password,
}
})
check this link might be helpful to you
How to convert List of JSON frames to JSON frame

Note that adding a dictionary (or any object) to a list only stores a reference, not a copy.
You can therefor do this:
>>> temp = {'logs': []}
>>> log_entry = {'key1': 'val1'}
>>> temp['logs'].append(log_entry)
>>> temp
{'logs': [{'key1': 'val1'}]}
>>> log_entry['key2'] = 'val2'
>>> temp
{'logs': [{'key2': 'val2', 'key1': 'val1'}]}
However, you might be able to circumvent to whole issue by using dict comprehension (only in Python >=2.7)
>>> temp = {'logs': [{key: value for key, value in my_generator}]

Try this example:
temp = {
'logs':[]
}
[temp['logs'].append(log) for log in errors['logs']]
Your log data would be list with multiple dictionary

Looping through dict elements in python [duplicate]

This question already has answers here:
python: read json and loop dictionary
(2 answers)
Closed 6 years ago.
I have a piece of json which was converted to a dict using the json function.
From this:
{
"imageIds": [
{
"imageTag": "1.2",
"imageDigest": "sha256:8b67b1691b29e27a5ccbd6fea5c97c951a025ccd45b26d4c24567ca3c4c0f13b"
},
{
"imageTag": "1.0",
"imageDigest": "sha256:aa52a12bd6e516659452af5b9ed0fad8659f9e0cea6a986c6bfe02af388df189"
}
]
}
To this:
>>> print data
{u'imageIds': [{u'imageTag': u'1.2', u'imageDigest': u'sha256:8b67b1691b29e27a5ccbd6fea5c97c951a025ccd45b26d4c24567ca3c4c0f13b'}, {u'imageTag': u'1.0', u'imageDigest': u'sha256:aa52a12bd6e516659452af5b9ed0fad8659f9e0cea6a986c6bfe02af388df189'}]}
In this example the number of keys (imageIds) is fixed but there could be any amount of imageTags under imageIds.
What I'm trying to do is loop through the 'imageTag' elements to read the tag number and perform an operation. If i wanted to loop through the key it seems straightforward with something simple like:
for key in data:
print key, 'corresponds to', data[key]
However I'm uncertain on how I loop through the items under the key.
What I want to achieve is to print out:
1.2
1.0

Iterate over inner dict the same way you do for the outer one:
for key, value in data.iteritems():
#now value can be a dictionary as well
#for innerkey, innervalues in value[0].iteritems():
# print innerkey, innervalues
#in order to only print the elements that have imageTag as the key, simply do:
print value[0]['imageTag']

How can I get Python to automatically create missing key/value pairs in a dictionary? [duplicate]

This question already has answers here:
Is there a standard class for an infinitely nested defaultdict?
(6 answers)
Closed 9 years ago.
I'm creating a dictionary structure that is several levels deep. I'm trying to do something like the following:
dict = {}
dict['a']['b'] = True
At the moment the above fails because key 'a' does not exist. At the moment I have to check at every level of nesting and manually insert an empty dictionary. Is there some type of syntactic sugar to be able to do something like the above can produce:
{'a': {'b': True}}
Without having to create an empty dictionary at each level of nesting?

As others have said, use defaultdict. This is the idiom I prefer for arbitrarily-deep nesting of dictionaries:
def nested_dict():
return collections.defaultdict(nested_dict)
d = nested_dict()
d[1][2][3] = 'Hello, dictionary!'
print(d[1][2][3]) # Prints Hello, dictionary!
This also makes checking whether an element exists a little nicer, too, since you may no longer need to use get:
if not d[2][3][4][5]:
print('That element is empty!')
This has been edited to use a def rather than a lambda for pep8 compliance. The original lambda form looked like this below, which has the drawback of being called <lambda> everywhere instead of getting a proper function name.
>>> nested_dict = lambda: collections.defaultdict(nested_dict)
>>> d = nested_dict()
>>> d[1][2][3]
defaultdict(<function <lambda> at 0x037E7540>, {})

Use defaultdict.
Python: defaultdict of defaultdict?

Or you can do this, since dict() function can handle **kwargs:
http://docs.python.org/2/library/functions.html#func-dict
print dict(a=dict(b=True))
# {'a': {'b' : True}}

If the depth of your data structure is fixed (that is, you know in advance that you need mydict[a][b][c] but not mydict[a][b][c][d]), you can build a nested defaultdict structure using lambda expressions to create the inner structures:
two_level = defaultdict(dict)
three_level = defaultdict(lambda: defaultdict(dict))
four_level = defaultdict(lamda: defaultdict(lambda: defaultdict(dict)))

check dictionary for doubled keys [duplicate]

This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
How to raise error if duplicates keys in dictionary
I was recently generating huge dictionaries with hundreds of thousands of keys (such that noticing a bug by looking at them wasn't feasible). They were syntactically correct, yet there was a bug somewhere. It boiled down to "duplicate keys":
{'a':1, ..., 'a':2}
this code compiles fine and I could not figure out why a key has value of 2 as I expected 1. The problem is obvious now.
The question is how I can prevent that in the future. I think this is impossible within python. I used
grep "'.*'[ ]*:" myfile.py | sort | uniq -c | grep -v 1
which is not bulletproof. Any other ideas (within python, this grep is just to illustrate what I'd tried)?
EDIT: I don't want duplicate keys, just need to spot that this occurs and edit data manually

A dict cannot contain double keys. So all you need to do is execute the code and then dump the repr() of the dict.
Another option is creating the dict items as (key, value) tuples. By storing them in a list you can easily create a dict from them and then check if the len()s of the dict/list differ.

If you need to have multiple values per key you can store the values in a list using defaultdict.
>>> from collections import defaultdict
>>> data_dict = defaultdict(list)
>>> data_dict['key'].append('value')
>>> data_dict
defaultdict(<type 'list'>, {'key': ['value']})
>>> data_dict['key'].append('second_value')
>>> data_dict
defaultdict(<type 'list'>, {'key': ['value', 'second_value']})

Are you generating a Python file containing a giant dictionary? Something like:
print "{"
for lines in file:
key, _, value = lines.partition(" ")
print " '%s': '%s',"
print "}"
If so, there's not much you can do to prevent this, as you cannot easily override the construction of the builtin dict.
Instead I'd suggest you validate the data while constructing the dictionary string. You could also generate different syntax:
dict(a = '1', a = '2')
..which will generate a SyntaxError if the key is duplicated. However, these are not exactly equivalent, as dictionary keys are a lot more flexible than keyword-args (e.g {123: '...'} is valid, butdict(123 = '...')` is an error)
You could generate a function call like:
uniq_dict([('a', '...'), ('a', '...')])
Then include the function definition:
def uniq_dict(values):
thedict = {}
for k, v in values:
if k in thedict:
raise ValueError("Duplicate key %s" % k)
thedict[k] = v
return thedict

You don't say or show exactly how you're generating the dictionary display you have where the duplicate keys are appearing. But that is where the problem lies.
Instead of using something like {'a':1, ..., 'a':2} to construct the dictionary, I suggest that you use this form: dict([['a', 1], ..., ['a', 2]]) which will create one from a supplied list of [key, value] pairs. This approach will allow you to check the list of pairs for duplicates before passing it to dict() to do the actual construction of the dictionary.
Here's an example of one way to check the list of pairs for duplicates:
sample = [['a', 1], ['b', 2], ['c', 3], ['a', 2]]
def validate(pairs):
# check for duplicate key names and raise an exception if any are found
dups = []
seen = set()
for key_name,val in pairs:
if key_name in seen:
dups.append(key_name)
else:
seen.add(key_name)
if dups:
raise ValueError('Duplicate key names encountered: %r' % sorted(dups))
else:
return pairs
my_dict = dict(validate(sample))