We Keep Coding

Deleting multiple key value pairs from dictionary in python

I generated a python dictionary for all the duplicate images in a folder. The python dictonary now contains values in the following format:
{
"image_1.jpg": ['image_xyz.jpg', 'image_abc.jpg'],
"image_xyz.jpg": ["image_1.jpg", "image_abc.jpg"],
"image_abc.jpg": ["image_xyz.jpg","image_1.jpg"],
"image_2.jpg": ["image_3.jpg"],
"image_3.jpg": ["image_2.jpg"],
"image_5.jpg": []
}
Each key, value pair thus appears atleast twice in the list. Empty list for keys are present which have no duplicates.
Is there a way to delete all the duplicate key value pairs present? so that the dictionary looks like the following:
{
"image_1.jpg": ['image_xyz.jpg', 'image_abc.jpg'],
"image_2.jpg": ["image_3.jpg"],
"image_5.jpg": []
}
I tried using list to first store all the values from the key value pair and then deleting them from the dictionary but it empties the whole dictionary.

source = {
"image_1.jpg": ['image_xyz.jpg', 'image_abc.jpg'],
"image_xyz.jpg": ["image_1.jpg", "image_abc.jpg"],
"image_abc.jpg": ["image_xyz.jpg","image_1.jpg"],
"image_2.jpg": ["image_3.jpg"],
"image_3.jpg": ["image_2.jpg"],
"image_5.jpg": []
}
dest = dict()
for k,v in source.items():
ok = True
for k1,v1 in dest.items():
if k in v1: ok = False
if ok: dest[k] = v
print(dest) # New filtered dict

I usually do this method when getting rid of duplicates in a list:
First put all the values in a matrix / 2 dimensional list including the key, so the first 3 values would be like this:
{
"image_1.jpg": ['image_xyz.jpg', 'image_abc.jpg'],
"image_xyz.jpg": ["image_1.jpg", "image_abc.jpg"],
"image_abc.jpg": ["image_xyz.jpg","image_1.jpg"],
}
would turn into:
List=[
["image_1.jpg","image_xyz.jpg","image_abc.jpg"],
["image_xyz.jpg","image_1.jpg","image_abc"],
["image_abc.jpg","image_xyz.jpg","image_1.jpg"]
]
make sure the keys are all in the 0th position so that you can save them.
keys=[x[0] for x in List]
then sort the list:
sorted_list=[sorted(x) for x in List]
then simply compare them using an if statement in a nested for loop and if a list is equal to another then delete it:
for i in sorted_list:
for j,k in enumerate(sorted_list):
if i==k:
del sorted_list[j] # deleting any equal lists
now that the duplicates are gone and you have the keys convert the list back into a dictionary if preferred
Overall code if needed:
List=[
["image_1.jpg","image_xyz.jpg","image_abc.jpg"],
["image_xyz.jpg","image_1.jpg","image_abc"],
["image_abc.jpg","image_xyz.jpg","image_1.jpg"]
]
keys=[x[0] for x in List]
sorted_list=[sorted(x) for x in List]
for i in sorted_list:
for j,k in enumerate(sorted_list):
if i==k:
del sorted_list[j] # deleting any equal lists

iteratively appending N items to list gives last item N times instead

I am accessing a list of dictionary items list_of_dict = [{'ka':'1a', 'kb':'1b', 'kc':'1c'},{'ka':'2a'},{'ka':'3a', 'kb':'3b', 'kc':'3c'}], and trying to conditionally append each entry to another list article_list using a function add_entries.
My desired output
article_list = [{x:1a, y:1b, z:1c}, {x:2a}, {x:3a, y:3b, z:3c}]
My code:
def add_entries(list_of_dict):
keys = ['x','y','z']
#defining a temporary dictionary here
my_dict = dict.fromkeys(keys,0)
entry_keys = ['ka','kb','kc']
my_list = []
for item in list_of_dict:
# conditionally append the entries into the temporary dictionary maintaining desired key names
my_dict.update({a: item[b] for a,b in zip(keys, entry_keys) if b in item})
my_list.append(my_dict)
return my_list
if __name__ == "__main__":
list_of_dict = [{'ka':'1a', 'kb':'1b', 'kc':'1c'},{'ka':'2a'},{'ka':'3a', 'kb':'3b', 'kc':'3c'}]
article_list = []
returned_list = add_entries(list_of_dict)
article_list.extend(returned_list)
My output
article_list = [{x:3a, y:3b, z:3c}, {x:3a, y:3b, z:3c}, {x:3a, y:3b, z:3c}]

Whats wrong
my_list.append(my_dict) appends a reference to the my_dict object to my_list. Therefore, at the end of the for loop in your example, my_list is a list of references to the exact same dictionary in memory.
You can see this for yourself using the function id. id, at least in CPython, basically gives you the memory address of an object. If you do
article_list.extend(returned_list)
print([id(d) for d in article_list])
You'll get a list of identical memory addresses. On my machine I get
[139920570625792, 139920570625792, 139920570625792]
The consequence is that updating the dictionary affects 'all of the dictionaries' in your list. (quotes because really there are not multiple dictionaries in your list, just many times the exact same one). So in the end, only the last update operation is visible to you.
A good discussion on references and objects in Python can be found in this answer https://stackoverflow.com/a/30340596/8791491
The fix
Moving the definition of my_dict inside the for loop, means that you get a new, separate dictionary for each element of my_list. Now, the update operation won't affect the other dictionaries in the list, and my_list is a list of references to several different dictionaries.
def add_entries(list_of_dict):
keys = ['x','y','z']
entry_keys = ['ka','kb','kc']
my_list = []
for item in list_of_dict:
#defining a new dictionary here
my_dict = dict.fromkeys(keys,0)
# conditionally append the entries into the temporary dictionary maintaining desired key names
my_dict.update({a: item[b] for a,b in zip(keys, entry_keys) if b in item})
my_list.append(my_dict)
return my_list

You can use this.
keys=['x','y','z']
res=[{k1:d[k]} for d in list_of_dict for k,k1 in zip(d,keys)]
output
[{'x': '1a'},
{'y': '1b'},
{'z': '1c'},
{'x': '2a'},
{'x': '3a'},
{'y': '3b'},
{'z': '3c'}]

Try this:
list_new_d=[]
for d in list_of_dict:
new_d={}
for k,v in d.items():
if k == 'ka':
new_d['x'] = v
if k == 'kb':
new_d['y'] = v
if k == 'kc':
new_d['z'] = v
list_new_d.append(new_d)

Use A Single Line To Loop Two Dictionaries To Create List

I have looked at several threads similar but unable to get a workable solution. I am looping (2) dictionaries trying to create a single list from the values of one based on the keys of another. I have it done with for loops but looking to use a single line if possible. My code with for loops
for k, v in dict1.items():
for value in dict2[k]:
temp.append(value)
On the first loop thru the temp list would be and is from above code:
[16,18,20,22,24,26]
I then use min to get the min value of the list. Now I want to condense the for loop to a one liner. I have put together
temp=[dict2.values() for k in dict1.keys() if k in dict2.keys()]
When executed, instead of temp being a single list for the k that exist in the dict1, I get a list of list for all the values from all dict2.
[[16,18,20,22,24,26], [12,16,18,20,22,24], [16,18,22,26,30,32]]
It seems to be ignoring the if statement. I know my dict1 has only 1 key in this situation and I know the 1 key exist in the dict2. Is my one liner wrong?
Input Values for dictionaries:
dict1={'table1':[16,18,20,22,24,26]}
dict2={'table1':[16,18,20,22,24,26],'table2': [12,16,18,20,22,24], 'table3': [16,18,22,26,30,32]}

You can iterate through one dictionary checking for matching keys and create a list of lists. Use chain.from_iterable to flatten list and call min():
from itertools import chain
dict1 = {'table1': [16,18,20,22,24,26]}
dict2 = {'table1': [16,18,20,22,24,26], 'table2': [12,16,18,20,22,24], 'table3': [16,18,22,26,30,32]}
temp = [dict2[k] for k in dict1 if k in dict2]
print(min(chain.from_iterable(temp)))
# 16

The reason why your list comprehension does not work:
It looks like dict2 has 3 key-value pairs, and the values are [16,18,20,22,24,26], [12,16,18,20,22,24]and [16,18,22,26,30,32]. What you're doing in your list comprehension translates to
for k in dict1.keys():
if k in dict2.keys():
temp.append(dict2.values())
So if dict1has, let's say, 3 keys, this for loop will repeat 3 times. Because, as you said in a comment above, only one key is shared between dict1and dict2, the if statement only is True once, so all items of dict2.values() will be appended to temponce. What you want to do, if i got that right, is to append all items INSIDE one of the values of dict2, namely the one assigned to the one key that the two dicts share. Your idea was pretty close, you just have to add one little thing. As a one liner, it would look like this:
temp = [x for x in dict2[k] for k in dict1.keys() if k in dict2.keys()]
or, differently:
temp = [dict2[k] for k in set(dict1.keys()).intersection(set(dict2.keys()))]

You can use the operator itemgetter():
from operator import itemgetter
from itertools import chain
dict1 = {'table1': [16,18,20,22,24,26], 'table2': [12,16,18,20,22,24]}
dict2 = {'table1': [16,18,20,22,24,26], 'table2': [12,16,18,20,22,24], 'table3': [16,18,22,26,30,32]}
common_keys = set(dict1).intersection(dict2)
sublists = itemgetter(*common_keys)(dict2)
if len(common_keys) == 1:
max_val = max(sublists)
else:
max_val = max(chain.from_iterable(sublists))
print(max_val)
# 26

combining a list of dictionaries with another dictionary

I have a list with a set amount of dictionaries inside which I have to compare to one other dictionary.
They have the following form (there is no specific form or pattern for keys and values, these are randomly chosen examples):
list1 = [
{'X1': 'Q587', 'X2': 'Q67G7', ...},
{'AB1': 'P5K7', 'CB2': 'P678', ...},
{'B1': 'P6H78', 'C2': 'BAA5', ...}]
dict1 = {
'X1': set([B00001,B00020,B00010]),
'AB1': set([B00001,B00007,B00003]),
'C2': set([B00001,B00002,B00003]), ...
}
What I want to have now is a new dictionary which has as keys: the values of the dictionaries in list1. and as values the values of dict1. And this only when the keys intersect in compared dictionaries.
I have done this in the following way:
nDicts = len(list1)
resultDict = {}
for key in range(0,nDicts):
for x in list1[key].keys():
if x in dict1.keys():
resultDict.update{list1[key][x]:dict1[x]}
print resultDict
The desired output should be of the form:
resulDict = {
'Q587': set([B00001,B00020,B00010]),
'P5K7': set([B00001,B00007,B00003]),
'BAA5': set([B00001,B00002,B00003]), ...
}
This works but since the amount of data is so high this takes forever.
Is there a better way to do this?
EDIT: I have changed the input values a little, the only ones that matter are the keys which intersect between the dictionaries within list1 and those within dict1.

The keys method in Python 2.x makes a list with a copy of all of the keys, and you're doing this not only for each dict in list1 (probably not a big deal, but it's hard to know for sure without knowing your data), but also doing it for dict1 over and over again.
On top of that, doing an in test on a list takes a long time, because it has to check each value in the list until it finds a match, but doing an in test on a dictionary is nearly instant, because it just has to look up the hash value.
Both keys are actually completely unnecessary—iterating a dict gives you the keys in order (an unspecified order, but the same is true for calling keys()), and in-checking a dict searches the same keys you'd get with keys(). So, just removing them does the same thing, but simpler, faster, and with less memory used. So:
for key in range(0,nDicts):
for x in list1[key]:
if x in dict1:
resultDict={list1[key][x]:dict1[x]}
print resultDict
There are also ways you can simplify this that probably won't help performance that much, but are still worth doing.
You can iterate directly over list1 instead of building a huge list of all the indices and iterating that.
for list1_dict in list1:
for x in list1_dict:
if x in dict1:
resultDict = {list_dict[x]: dict1[x]}
print resultDict
And you can get the keys and values in a single step:
for list1_dict in list1:
for k, v in list1_dict.iteritems():
if k in dict1:
resultDict = {v: dict1[k]}
print resultDict
Also, if you expect most of the values to be found, it will take about twice as long to first check for the value and then look it up as it would to just try to look it up and handle failure. (This is not true if most of the values will not be found, however.) So:
for list1_dict in list1:
for k, v in list1_dict.iteritems():
try:
resultDict = {v: dict1[k]}
print resultDict
except KeyError:
pass

You can simplify and optimize your operation with set intersections; as of Python 2.7 dictionaries can represent keys as sets using the dict.viewkeys() method, or dict.keys() in Python 3:
resultDict = {}
for d in list1:
for sharedkey in d.viewkeys() & dict1:
resultDict[d[sharedkey]] = dict1[sharedkey]
This can be turned into a dict comprehension even:
resultDict = {d[sharedkey]: dict1[sharedkey]
for d in list1 for sharedkey in d.viewkeys() & dict1}
I am assuming here you wanted one resulting dictionary, not a new dictionary per shared key.
Demo on your sample input:
>>> list1 = [
... {'X1': 'AAA1', 'X2': 'BAA5'},
... {'AB1': 'AAA1', 'CB2': 'BAA5'},
... {'B1': 'AAA1', 'C2': 'BAA5'},
... ]
>>> dict1 = {
... 'X1': set(['B00001', 'B00002', 'B00003']),
... 'AB1': set(['B00001', 'B00002', 'B00003']),
... }
>>> {d[sharedkey]: dict1[sharedkey]
... for d in list1 for sharedkey in d.viewkeys() & dict1}
{'AAA1': set(['B00001', 'B00002', 'B00003'])}
Note that both X1 and AB1 are shared with dictionaries in list1, but in both cases, the resulting key is AAA1. Only one of these wins (the last match), but since both values in dict1 are exactly the same anyway that doesn't make any odds in this case.
If you wanted separate dictionaries per dictionary in list1, simply move the for d in list1: loop out:
for d in list1:
resultDict = {d[sharedkey]: dict1[sharedkey] for sharedkey in d.viewkeys() & dict1}
if resultDict: # can be empty
print resultDict
If you really wanted one dictionary per shared key, move another loop out:
for d in list1:
for sharedkey in d.viewkeys() & dict1:
resultDict = {d[sharedkey]: dict1[sharedkey]}
print resultDict

#!/usr/bin/env python
list1 = [
{'X1': 'AAA1', 'X2': 'BAA5'},
{'AB1': 'AAA1', 'CB2': 'BAA5'},
{'B1': 'AAA1', 'C2': 'BAA5'}
]
dict1 = {
'X1': set(['B00001','B00002','B00003']),
'AB1': set(['B00001','B00002','B00003'])
}
g = ( k.iteritems() for k in list1)
ite = ((a,b) for i in g for a,b in i if dict1.has_key(a))
d = dict(ite)
print d

Adding a random key from a dictionary to a list in python 3.3

I'm struggling to figure out what's wrong with my code. I am trying to randomly select a key from several dictionaries, then add that key to the beginning of a list. For example:
import random
list1 = ["list"]
dict1 = {"key1" : "def1", "key2" : "def2", "key3" : "def3"}
dict2 = {"key4" : "def4", "key5" : "def5", "key6" : "def6"}
DICTIONARIES = [dict1, dict2]
value = random.choice(DICTIONARIES)
key = random.choice(list(value.keys()))
list1[:0] = key
print (list1)
What I want, is a print result of [key5, list]. What I get is ['k', 'e', 'y', '5', list]
Any ideas? is there a better way to search multiple dictionaries for a random key that will produce the desired results?
Thanks.

I suppose that item variable is the same as list1. If yes, try this:
list1[:0] = [key]
Or, alternatively you may use the list.insert function instead of slice assignment:
list1.insert(0, key)
Your version was working like the following:
Before assignment: list1 = ['list'], key = 'key5'
Left side of assignment refers to element before 'list' in list1
Right side refers to value of key, which is "key5".
"key5" is a sequence of k, e, y and 5.
So, by list1[:0] = key we concatenate "key5" sequence and list1.
But in list1[:0] = [key] we concatenate [key] sequence (that has only one element that equals to "key5") and list1. And that's what we actually want.

Something like this:
import random
list1 = ["list"]
dict1 = {"key1" : "def1", "key2" : "def2", "key3" : "def3"}
dict2 = {"key4" : "def4", "key5" : "def5", "key6" : "def6"}
all_keys = list(dict1.keys() | dict2.keys()) #creates a list containing all keys
key = random.choice(all_keys)
list1.insert(0,key)
print (list1) #prints ['key1', 'list']

In Python string objects are iterable, so, this line
list1[:0] = key
and this
list1[:0] = list(key)
are equals, and means replace all elements from 0th position to 0th position in list1 with elements from key. You should use insert function, or make a list with key element and assign it.