I am trying to convert the below to a dict comprehension
my_dict = {'a': None, 'b': None, 'c': ['1', '2', '3']}
new_dict = {}
for k, v in my_dict.items():
if not v:
new_dict[k] = None
else:
for item in v:
new_dict[f'{k}{item}'] = None
I'm trying to translate my dict to
new_dict = {'a': None, 'b': None, 'c1': None, 'c2': None, 'c3': None}
I'm fairly comfortable with basic list and dict comprehensions but struggling with this one, currently looking something like this but clearly I am a bit off on the syntax:
{k: None if not v else f'{k}{item}': None for item in v for k, v in my_dict.items()}
This abomination will do:
{fk: None
for k, v in my_dict.items()
for fk in ([k] if v is None else (k + fv for fv in v))}
If the value is None, you just want the key.
If the value is not None, you want a list of each value concatenated with the key.
Homogenise that to always returning a list, either of one key or multiple:
[k] if v is None else [k + fv for fv in v]
Then you're looking at a "simple" nested comprehension:
{k: None for k in [['a'], ['b'], ['c1', 'c2', 'c3']] for fk in k}
You can do this with the help of itertools.chain.from_iterable:
>>> {
y: None
for y in itertools.chain.from_iterable(
k if not v else (f"{k}{x}" for x in v) for k, v in my_dict.items()
)
}
{'a': None, 'b': None, 'c1': None, 'c2': None, 'c3': None}
I was proud of my answer:
new_dict = dict(
sum(
[
[(k, None)] if not value else [(f"{k}{v}", None) for v in value]
for k, value in my_dict.items()
],
[],
)
)
... until I saw deceze's answer, which I still need to digest.
Related
Consider a fixed list of keys and a dictionary of random key/values. I have to iterate through the dictionary and if a key is not in the list (keys that may or may not exist in the dictionary), then I add it to a new dictionary.
d = {'key1': '1', 'keyA': 'A', 'key2': '2', 'keyB': 'B', ...}
new_d = {}
for k, v in d.items():
if k not in ['key1', 'keyB', 'key_doesnotexist']:
new_d[k] = v
To optimize this, I thought about iterating through the list of keys first and popping anything that matches the list to get rid of the inner loop so something like this:
d = {'key1': '1', 'keyA': 'A', 'key2': '2', 'keyB': 'B', ...}
new_d = {}
for x in ['key1', 'keyB', 'key_doesnotexist']:
d.pop(x, default=None)
for k, v in d.items():
new_d[k] = v
Just wondering if there are any faster methods I should be aware of.
This might work faster:
d = {'key1': '1', 'keyA': 'A', 'key2': '2', 'keyB': 'B'}
new_d = {k: d[k] for k in d.keys() - ['key1', 'keyB', 'key_doesnotexist']}
Prints:
>>> new_d
{'keyA': 'A', 'key2': '2'}
Use a dict comprehension to select the items you want:
new_d = {k: v for k, v in d.items()
if k not in ['key1', 'keyB', 'key_doesnotexist']}
Or simply copy the dict and delete the unwanted entries:
import copy
new_d = copy.deepcopy(d)
for k in ['key1', 'keyB', 'key_doesnotexist']:
new_d.del(k)
I have a dictionary dict:
dict = {'drop_key1': '10001', 'drop_key2':'10002'}
The key(s) in dict startswith drop_, i would like to update dict by dropping drop_ value from key(s):
dict = {'key1': '10001', 'key2':'10002'}
What is the best approach to do it?
something like
d1 = {'drop_key1': '10001', 'drop_key2':'10002'}
d2 = {k[5:]:v for k,v in d1.items()}
print(d2)
output
{'key1': '10001', 'key2': '10002'}
One approach is, for each key value in the dictionary, you can replace the part of the string with the new string value. For instance:
d = {k.replace('drop_', ''): v for k, v in d.items() if k.strip().startswith('drop_')}
or you can define a function, and get the index of the searched string ("drop_"). If the search string index is 0, then remove it. For instance:
def change_key(key, search):
start_idx = key.find(search)
if start_idx == 0:
key = key.replace(search, "")
return key
d = {change_key(k, search="drop_"): v for k, v in d.items()}
Result:
{'key1': '10001', 'key2': '10002'}
Note that if you use a method, then you can guarantee to remove the search string if it is at the beginning of the string. For instance:
d = {' drop_key1': '10001', 'drop_key2': '10002'}
d = {change_key(k, search="drop_"): v for k, v in d.items()}
Result:
{' drop_key1': '10001', 'key2': '10002'}
I would like to know how if there exists any python function to merge two dictionary and combine all values that have a common key.
I have found function to append two dict, to merge two dict but not to combine its values.
Example:
D1 = [{k1: v01}, {k3: v03}, {k4: v04},}],
D2 = [{k1: v11}, {k2: v12}, {k4: v14},}],
this should be the expected result:
D3 = [
{k1: [v01, v11]},
{k2: [ v12]},
{K3: [v03 ]},
{k4: [v04, v14]},
]
There is no built-in function for this but you can use a defaultdict for this:
from collections import defaultdict
d = defaultdict(list)
for other in [d1, d1]:
for k, v in other.items():
d[k].append(v)
A solution, without importing anything:
# First initialize data, done correctly here.
D1 = [{'k1': 'v01'}, {'k3': 'v03'}, {'k4': 'v04'}]
D2 = [{'k1': 'v11'}, {'k2': 'v12'}, {'k4': 'v14'}]
# Get all unique keys
keys = {k for d in [*D1, *D2] for k in d}
# Initialize an empty dict
D3 = {x:[] for x in keys}
# sort to maintain order
D3 = dict(sorted(D3.items()))
#Iterate and extend
for x in [*D1, *D2]:
for k,v in x.items():
D3[k].append(v)
# NOTE: I do not recommend you convert a dictionary into a list of records.
# Nonetheless, here is how it would be done.
# To convert to a list
D3_list = [{k:v} for k,v in D3.items()]
print(D3_list)
# [{'k1': ['v01', 'v11']},
# {'k2': ['v12']},
# {'k3': ['v03']},
# {'k4': ['v04', 'v14']}]
If you meant to use actual dicts, instead of lists of dicts, this is easier.
D1 = dict(k1=1, k3=3, k4=4)
D2 = dict(k1=11, k2=12, k4=14)
There isn't a simple built-in function to do this, but the setdefault method is close.
It tries to get the given key, but creates it if it doesn't exist.
D3 = {}
for k, v in D1.items() | D2.items():
D3.setdefault(k, set()).add(v)
And the result.
{'k4': {4, 14}, 'k1': {1, 11}, 'k3': {3}, 'k2': {12}}
This all assumes the order doesn't matter, just combining sets.
A more generic way to merge dicts together may look like this.
(To answer a similar SO question)
def merge(combiner, dicts):
new_dict = {}
for d in dicts:
for k, v in d.items():
if k in new_dict:
new_dict[k] = combiner(new_dict[k], v)
else:
new_dict[k] = v
return new_dict
x = {'a': 'A', 'b': 'B'}
y = {'b': 'B', 'c': 'C'}
z = {'a': 'A', 'd': 'D'}
merge_dicts(combiner= lambda x, y: f'{x} AND {y}', dicts=(x,y,z))
# {'a': 'A AND A', 'b': 'B AND B', 'c': 'C', 'd': 'D'}
how to remove keys in a dict which do not have any value. I have a dict as :
d = {'CB': '', 'CA': [-7.5269999504089355, -2.2330000400543213, 6.748000144958496], 'C': [-8.081000328063965, -3.619999885559082, 6.406000137329102], 'N': [-6.626999855041504, -2.318000078201294, 7.9029998779296875], 'H':''}
I want to remove keys which values are blank. I need output as :
d = {'CA': [-7.5269999504089355, -2.2330000400543213, 6.748000144958496], 'C': [-8.081000328063965, -3.619999885559082, 6.406000137329102], 'N': [-6.626999855041504, -2.318000078201294, 7.9029998779296875]}
how to do this?
Use a dict-comprehension:
>>> {k:v for k, v in d.items() if v != ''}
{'N': [-6.626999855041504, -2.318000078201294, 7.9029998779296875], 'CA': [-7.5269999504089355, -2.2330000400543213, 6.748000144958496], 'C': [-8.081000328063965, -3.619999885559082, 6.406000137329102]}
If by empty you meant any falsy value then use just if v.
If you want to modify the original dict itself(this will affect all the references to the dict object):
for k, v in d.items():
if not v: del d[k]
>> {key:value for key, value in d.items() if value}
If you really need to do this in one-line:
for k in [k for k,v in d.iteritems() if not v]: del d[k]
This is ugly. It makes a list of all of the keys in d that have a "blank" value and then iterates over this list removing the keys from d (to avoid the unsafe removal of items from dictionary during iteration).
I'd suggest that three lines would be more readable:
blanks = [k for k,v in d.iteritems() if not v]
for k in blanks:
del d[k]
If making a fresh dict is acceptable, use a dict comprehension:
d2 = {k:v for k,v in d.iteritems() if not v}
Something like:
for (a,b) in kwargs.iteritems():
if not b : del kwargs[a]
This code raise exception because changing of dictionary when iterating.
I discover only non pretty solution with another dictionary:
res ={}
res.update((a,b) for a,b in kwargs.iteritems() if b is not None)
Thanks
Another way to write it is
res = dict((k,v) for k,v in kwargs.iteritems() if v is not None)
In Python3, this becomes
res = {k:v for k,v in kwargs.items() if v is not None}
You can also use filter:
d = dict(a = 1, b = None, c = 3)
filtered = dict(filter(lambda item: item[1] is not None, d.items()))
print(filtered)
{'a': 1, 'c': 3}
d = {'a': None, 'b': 'myname', 'c': 122}
print dict(filter(lambda x:x[1], d.items()))
{'b': 'myname', 'c': 122}
I like the variation of your second method:
res = dict((a, b) for (a, b) in kwargs.iteritems() if b is not None)
it's Pythonic and I don't think that ugly. A variation of your first is:
for (a, b) in list(kwargs.iteritems()):
if b is None:
del kwargs[a]
If you need to handle nested dicts, then you can leverage a simple recursive approach:
# Python 2
from collections import Mapping
def filter_none(d):
if isinstance(d, Mapping):
return dict((k, filter_none(v)) for k, v, in d.iteritems() if v is not None)
else:
return d
# Python 3
from collections.abc import Mapping
def filter_none(d):
if isinstance(d, Mapping):
return {k: filter_none(v) for k, v in d.items() if v is not None}
else:
return d
To anybody who may interests, here's another way to get rid of None value. Instead of deleting the key, I change the value of None with a placeholder for the same key.
One use case is applying with Spark RDD.map onto null valued JSON.
def filter_null(data, placeholder="[spark]nonexists"):
# Replace all `None` in the dict to the value of `placeholder`
return dict((k, filter_null(v, placeholder) if isinstance(v, dict) else v if v
is not None else placeholder) for k, v in data.iteritems())
Sample output:
>>> filter_null({'a':None,'b':"nul", "c": {'a':None,'b':"nul"}})
{'a': '[spark]nonexists', 'c': {'a': '[spark]nonexists', 'b': 'nul'}, 'b': 'nul'}
For python3, change the iteritems() to items().
The recursive approach to also filter nested lists of dicts in the dictionary:
def filter_none(d):
if isinstance(d, dict):
return {k: filter_none(v) for k, v in d.items() if v is not None}
elif isinstance(d, list):
return [filter_none(v) for v in d]
else:
return d
Sample output:
data = {'a': 'b', 'c': None, 'd':{'e': 'f', 'h': None, 'i':[{'j': 'k', 'l': None}]}}
print(filter_none(data))
>>> {'a': 'b', 'd': {'e': 'f', 'i': [{'j': 'k'}]}}