We Keep Coding

Convert string to dictionary with list of values

What is the best way to convert a string to dictionary with value of dictionary as a list
for example
str = "abc=1,abc=2,abc=3,xyz=5,xyz=6"
i need the output as:
d = {"abc":["1","2","3"],"xyz":["5","6"]}
I'm very new to python.
my code:
d = {k: [v] for k, v in map(lambda item: item.split('='), s.split(","))}

Here is the solution with dict.setdefault method.
>>> help({}.setdefault)
Help on built-in function setdefault:
setdefault(key, default=None, /) method of builtins.dict instance
Insert key with a value of default if key is not in the dictionary.
Return the value for key if key is in the dictionary, else default.
>>> your_str = "abc=1,abc=2,abc=3,xyz=5,xyz=6"
>>>
>>> result = {}
>>>
>>> for pair in your_str.split(","):
... name, val = pair.split("=")
... result.setdefault(name, []).append(val)
>>> result
{'abc': ['1', '2', '3'], 'xyz': ['5', '6']}
You could also use defaultdict with default factory as list
>>> from collections import defaultdict
>>>
>>> your_str = "abc=1,abc=2,abc=3,xyz=5,xyz=6"
>>>
>>> result = defaultdict(list)
>>> for pair in str.split(","):
... name, val = pair.split("=")
... result[name].append(val)
...
>>> dict(result)
{'abc': ['1', '2', '3'], 'xyz': ['5', '6']}

The reason the code you have tried already isn't giving you the desired result is the fact that you are overwriting the value assigned to each key as you iterate over the list. What you need to do is append to the value already assigned to the key - except if the key doesn't exist, in which case you need to initialise that key.
This would be one way to go:
s1 = "abc=1,abc=2,abc=3,xyz=5,xyz=6"
list1 = [(each.split('=')) for each in s1.split(',')]
d = {}
for key, val in list1:
if key in d.keys():
d[key].append(val)
else:
d[key] = [val]
print (d)
#result: {'abc': ['1', '2', '3'], 'xyz': ['5', '6']}
You could simplify this and eliminate the if-else by using defaultdict, like so:
from collections import defaultdict
d = defaultdict(lambda: [])
s1 = "abc=1,abc=2,abc=3,xyz=5,xyz=6"
list1 = [(each.split('=')) for each in s1.split(',')]
for key, val in list1:
d[key].append(val)
print (d)
#result: {'abc': ['1', '2', '3'], 'xyz': ['5', '6']}

# initialize a dictionary
d = {}
# split the string (my_str) according to "," in order to get pairs such as 'abc/1' and 'xyz/5' in a list
for elt in my_str.split(",") :
# for each string of the list, split according to '/' to get the pairs ['abc', 1]
# complete the dictionary
if elt.split('/')[0] not in d.keys():
d[elt.split('/')[0]] = [elt.split('/')[1]]
else :
d[elt.split('/')[0]].append(elt.split('/')[1])

Is there a better way to iterate through a dict while comparing against a list of values?

Consider a fixed list of keys and a dictionary of random key/values. I have to iterate through the dictionary and if a key is not in the list (keys that may or may not exist in the dictionary), then I add it to a new dictionary.
d = {'key1': '1', 'keyA': 'A', 'key2': '2', 'keyB': 'B', ...}
new_d = {}
for k, v in d.items():
if k not in ['key1', 'keyB', 'key_doesnotexist']:
new_d[k] = v
To optimize this, I thought about iterating through the list of keys first and popping anything that matches the list to get rid of the inner loop so something like this:
d = {'key1': '1', 'keyA': 'A', 'key2': '2', 'keyB': 'B', ...}
new_d = {}
for x in ['key1', 'keyB', 'key_doesnotexist']:
d.pop(x, default=None)
for k, v in d.items():
new_d[k] = v
Just wondering if there are any faster methods I should be aware of.

This might work faster:
d = {'key1': '1', 'keyA': 'A', 'key2': '2', 'keyB': 'B'}
new_d = {k: d[k] for k in d.keys() - ['key1', 'keyB', 'key_doesnotexist']}
Prints:
>>> new_d
{'keyA': 'A', 'key2': '2'}

Use a dict comprehension to select the items you want:
new_d = {k: v for k, v in d.items()
if k not in ['key1', 'keyB', 'key_doesnotexist']}
Or simply copy the dict and delete the unwanted entries:
import copy
new_d = copy.deepcopy(d)
for k in ['key1', 'keyB', 'key_doesnotexist']:
new_d.del(k)

Update dictionary key(s) by drop starts with value from key in Python

I have a dictionary dict:
dict = {'drop_key1': '10001', 'drop_key2':'10002'}
The key(s) in dict startswith drop_, i would like to update dict by dropping drop_ value from key(s):
dict = {'key1': '10001', 'key2':'10002'}
What is the best approach to do it?

something like
d1 = {'drop_key1': '10001', 'drop_key2':'10002'}
d2 = {k[5:]:v for k,v in d1.items()}
print(d2)
output
{'key1': '10001', 'key2': '10002'}

One approach is, for each key value in the dictionary, you can replace the part of the string with the new string value. For instance:
d = {k.replace('drop_', ''): v for k, v in d.items() if k.strip().startswith('drop_')}
or you can define a function, and get the index of the searched string ("drop_"). If the search string index is 0, then remove it. For instance:
def change_key(key, search):
start_idx = key.find(search)
if start_idx == 0:
key = key.replace(search, "")
return key
d = {change_key(k, search="drop_"): v for k, v in d.items()}
Result:
{'key1': '10001', 'key2': '10002'}
Note that if you use a method, then you can guarantee to remove the search string if it is at the beginning of the string. For instance:
d = {' drop_key1': '10001', 'drop_key2': '10002'}
d = {change_key(k, search="drop_"): v for k, v in d.items()}
Result:
{' drop_key1': '10001', 'key2': '10002'}

Iterating through nested dict in a particular order with python3

I have a nested dict which has lists as well as dict in it as given below.
m = {'abc':
{'bcd': [
{'cde':'100','def':'200','efg':'300'},
{'cde':'3000','def':'500','efg':'4000'}
],
'ghi':
{'mnc': [
{'xyz':'8827382','mnx':'e838','wyew':'2232'}
]
}
}
}
My requirement is to match mnx key and if the value is 'e838' then get the value of the other keys in that particular dict. So from the above example I may require the value of xyz key.
For this, I have created a recursive looping function as given below which is working. However my question is whether there is a better / easy way to do it. Also what can be done in the same code if I need to get all the values with key mnx. Thanks.
Note: I am converting an XML into dict with the help of jxmlease lib.
def iterate_dict(dict1,key1,val1):
for key, value in dict1.items():
if key == key1 and value == val1:
return dict1
if isinstance(value,list):
for item1 in value:
if isinstance(item1,dict):
for k,v in item1.items():
if k == key1 and v == val1:
return item1
if isinstance(value,dict):
for key,var in value.items():
if key == key1 and var == val1:
return value
else:
return iterate_dict(value,key1,val1)

You can sort of "flatten" the dict into a list of dicts and then query as necessary:
def flatten_dict(d):
flattened = []
current = {}
for k, v in d.items():
if isinstance(v, dict):
flattened.extend(flatten_dict(v))
elif isinstance(v, list):
flattened.extend(sum((flatten_dict(v_d) for v_d in v), []))
else:
current[k] = v
if len(current) > 0:
flattened = [current] + flattened
return flattened
def values_in_flattened(flattened, key):
return list(filter(None, (d.get(key, None) for d in flattened))) or None
m = {'abc': {'bcd':[{'cde':'100','def':'200','efg':'300'},{'cde':'3000','def':'500','efg':'4000'}], 'ghi':{'mnc':[{'xyz':'8827382','mnx':'e838','wyew':'2232'}]}}}
mf = flatten_dict(m)
efg_vals = values_in_flattened(mf, 'efg')
print(mf)
print(efg_vals)
>>>
[{'xyz': '8827382', 'mnx': 'e838', 'wyew': '2232'}, {'def': '200', 'efg': '300', 'cde': '100'}, {'def': '500', 'efg': '4000', 'cde': '3000'}]
['300', '4000']

m['abc']['bcd'] + m['abc']['ghi']['mnc']
out:
[{'cde': '100', 'def': '200', 'efg': '300'},
{'cde': '3000', 'def': '500', 'efg': '4000'},
{'mnx': 'e838', 'wyew': '2232', 'xyz': '8827382'}]
you should build a list of dict to iterate, rather than use raw data.

This code does the searching using recursive generators, so it will yield all the solutions as it finds them.
When iterate_dict finds a dict with the desired (key, value) pair it calls filter_dict, which creates a new dict to contain the output. This new dict contains the items of the dict passed to filter_dict except that it filters out the desired (key, value) pair, it also filters out any lists or dicts that that dict may contain. However, iterate_dict will recursively process those lists or dicts looking for further matches. If you don't want iterate_dict to look for further matches, it's easy to modify the code so that it doesn't do that; please see below.
If you want to search for dicts that contain the desired key and don't care about the value associated with that key you can pass None as the val argument, or just omit that arg.
I've modified your data slightly so we can test the recursive search for further matches in a dict that contains a match.
def filter_dict(d, key):
return {k: v for k, v in d.items()
if k != key and not isinstance(v, (dict, list))}
def iterate_dict(d, key, val=None):
if key in d and (val is None or d[key] == val):
yield filter_dict(d, key)
yield from iterate_list(d.values(), key, val)
def iterate_list(seq, key, val):
for v in seq:
if isinstance(v, list):
yield from iterate_list(v, key, val)
elif isinstance(v, dict):
yield from iterate_dict(v, key, val)
# test
data = {
'abc': {
'bcd': [
{'cde':'100', 'def':'200', 'efg':'300'},
{'cde':'3000', 'def':'500', 'efg':'4000'},
{'abc': '1', 'mnx': '2', 'ijk': '3',
'zzz': {'xyzz':'44', 'mnx':'e838', 'yew':'55'}
},
],
'ghi': {
'mnc': [
{'xyz':'8827382', 'mnx':'e838', 'wyew':'2232'}
]
}
}
}
for d in iterate_dict(data, 'mnx', 'e838'):
print(d)
output
{'yew': '55', 'xyzz': '44'}
{'xyz': '8827382', 'wyew': '2232'}
Here's a search that looks for all dicts containing the 'mnx' key:
for d in iterate_dict(data, 'mnx'):
print(d)
output
{'ijk': '3', 'abc': '1'}
{'xyzz': '44', 'yew': '55'}
{'wyew': '2232', 'xyz': '8827382'}
If you don't want each dict to be recursively searched for further matches once a match has been found in it, just change iterate_dict to:
def iterate_dict(d, key, val=None):
if key in d and (val is None or d[key] == val):
yield filter_dict(d, key)
else:
yield from iterate_list(d.values(), key, val)

remove the keys which has blank values

how to remove keys in a dict which do not have any value. I have a dict as :
d = {'CB': '', 'CA': [-7.5269999504089355, -2.2330000400543213, 6.748000144958496], 'C': [-8.081000328063965, -3.619999885559082, 6.406000137329102], 'N': [-6.626999855041504, -2.318000078201294, 7.9029998779296875], 'H':''}
I want to remove keys which values are blank. I need output as :
d = {'CA': [-7.5269999504089355, -2.2330000400543213, 6.748000144958496], 'C': [-8.081000328063965, -3.619999885559082, 6.406000137329102], 'N': [-6.626999855041504, -2.318000078201294, 7.9029998779296875]}
how to do this?

Use a dict-comprehension:
>>> {k:v for k, v in d.items() if v != ''}
{'N': [-6.626999855041504, -2.318000078201294, 7.9029998779296875], 'CA': [-7.5269999504089355, -2.2330000400543213, 6.748000144958496], 'C': [-8.081000328063965, -3.619999885559082, 6.406000137329102]}
If by empty you meant any falsy value then use just if v.
If you want to modify the original dict itself(this will affect all the references to the dict object):
for k, v in d.items():
if not v: del d[k]

>> {key:value for key, value in d.items() if value}

If you really need to do this in one-line:
for k in [k for k,v in d.iteritems() if not v]: del d[k]
This is ugly. It makes a list of all of the keys in d that have a "blank" value and then iterates over this list removing the keys from d (to avoid the unsafe removal of items from dictionary during iteration).
I'd suggest that three lines would be more readable:
blanks = [k for k,v in d.iteritems() if not v]
for k in blanks:
del d[k]
If making a fresh dict is acceptable, use a dict comprehension:
d2 = {k:v for k,v in d.iteritems() if not v}