I need to create a data payload for a restful web api. The template for the dict contains some optional keys. The usual way is to create a dict with all the required keys and then check and add optional keys one by one:
def create_payload(key1, key2, ..., key10=None,key11=None):
data = {
'key1' : key1,
'key2' : key2,
...
'nested' : {}
}
if key10:
data[ 'nested' ][ 'key10' ] = key10
if key11:
data[ 'key11' ] = key11
return data
Is there an alternative way to start with the data template and then automatically delete optional keys that are None?
def create_payload(key1, key2, ..., key10=None,key11=None):
data = {
'key1' : key1,
'key2' : key2,
...
'nested' : {
'key10' : key10
},
'key11' : key11
}
# delete keys that are None
return data
UPDATE
Some of you suggested doing dictionary comprehension to remove None keys:
{k: v for k, v in metadata.items() if v is not None}
In my attempt to create a simple example, I didn't account for the possibility that some of the keys may be nested. (I am updating the example above.) Is there a dictionary comprehension that can exclude None values inside nested keys?
Yes, you can use for example dictionary comprehension:
return {k: v for k, v in data.items() if v is not None}
or for a nested structure:
def create_payload(key1, key2, key10=None,key11=None):
data = {
# ...
}
def strip_nones(dc):
return {
k: strip_nones(v) if isinstance(v, dict) else v
for k, v in dc.items() if v is not None
}
return strip_nones(data)
We create thus a new dictionary that only retains the non-None values.
But you can probably reduce the amount of logic to:
def create_payload(**kwargs):
return {k: v for k, v in kwargs.items() if v is not None}
Here you can call the create_payload function will all kinds of parameters, and it will return a dictionary with all non-None values. If you want to include those as well, it is even simply:
def create_payload(**kwargs):
return kwargs
Related
I have a dictionary of key-value pairs, where the value is itself a dictionary. I would like to change the names of key values in that nested dictionary based on a predefined conversion.
I am using two lists to match up the values of the nested keys that I am trying to conver (Old_item1 should become New_item1):
comparison_list = ['Old_item1', 'Old_item2']
new_prods = ['New_item1', 'New_item2']
old_dict = {
'Company1':
{
'Old_item1':
{
'key1': val,
'key2': val
},
'Old_item2':
{
'key1': val,
'key2': val
}
}
}
I tried this:
new_dict = {}
for i in comparison_list:
for j in new_prods:
new_dict['Company1'][j] = test['Company1'][i]
I get a KeyError: KeyError: 'Company1'
The desired output for each item I add to each list is:
new_dict = {
'Company1':
{
'New_item1':
{
'key1': val # old item key, val
'key2': val # old item key, val
}
}
}
You can make a dictionary of mappings from the old to the new items, and then use it to create a new sub-dictionary for each company. This then needs to be wrapped inside an outer loop over companies (although here there is only one).
For example:
comparison_list = ['Old_item1', 'Old_item2']
new_prods = ['New_item1', 'New_item2']
old_dict = {'Company1':
{'Old_item1':
{'key1': 2,
'key2': 3},
'Old_item2':
{'key1': 4,
'key2': 5}}}
key_mappings = dict(zip(comparison_list, new_prods))
new_dict = {k: {key_mappings[k1]: v1 for k1, v1 in v.items()}
for k, v in old_dict.items()}
print(new_dict)
gives:
{'Company1': {'New_item1': {'key1': 2, 'key2': 3}, 'New_item2': {'key1': 4, 'key2': 5}}}
Here is the mappings dictionary key_mappings which we used:
{'Old_item1': 'New_item1', 'Old_item2': 'New_item2'}
The easiest way to go about this is to re-create your dictionary with a comprehension:
new_dict = {company: update_item(v) for company, v in old_dict.items()}
Then parse out update_item to it's own function. You could do this inline but it makes it difficult to understand.
conversion_lookup = {
'Old_item1': 'NewItem1',
'Old_item2': 'NewItem2',
}
def update_item(item: dict) -> dict:
return { conversion_lookup.get(k, k): v for k, v in item.items() }
new_dict = {company: update_item(v) for company, v in old_dict.items()}
For the conversion here I'm using a dictionary describing the conversion. If you need to construct this automatedly:
comparison_list = ['Old_item1', 'Old_item2']
new_prods = ['New_item1', 'New_item2']
conversion_lookup = { v: new_prods[idx] for idx, v in enumerate(comparison_list) }
The reason I like a dictionary is that you can use some_dict.get(a_value, a_default_value). Then, if your value isn't in your conversion dictionary you can fall back to the original value. That's what I'm doing here:
conversion_lookup.get(k, k)
The second k is the original item's value, which is a good thing to use if your conversion list doesn't include what you want.
One approach is to do it step by step. First create the dicts with the new names and old keys, then remove the old keys, and finally add in the new names.
# Keep the values from the old keys
values = [old_dict['Company1'][old_name] for old_name in comparison_list]
# Remove old names, and add in the
for new_key, value, old in zip(new_prods, values, comparison_list):
old_dict['Company1'].pop(old)
old_dict['Company1'][new_key] = value
I have a dictionary that looks like this:
d = {key1 : {(key2,key3) : value}, ...}
so it is a dictionary of dictionaries and in the inside dict the keys are tuples.
I would like to get a triple nested dict:
{key1 : {key2 : {key3 : value}, ...}
I know how to do it with 2 loops and a condition:
new_d = {}
for key1, inside_dict in d.items():
new_d[key1] = {}
for (key2,key3), value in inside_dict.items():
if key2 in new_d[key1].keys():
new_d[key1][key2][key3] = value
else:
new_d[key1][key2] = {key3 : value}
Edit: key2 values are not guaranteed to be unique. This is why I added the condition
It feels very unpythonic to me.
Is there a faster and/or shorter way to do this?
You could use the common trick for nesting dicts arbitrarily, using collections.defaultdict:
from collections import defaultdict
tree = lambda: defaultdict(tree)
new_d = tree()
for k1, dct in d.items():
for (k2, k3), val in dct.items():
new_d[k1][k2][k3] = val
If I understand the problem correctly, for this case you can wrap all the looping up in a dict comprehension. This assumes that your data is unique:
data = {"key1": {("key2", "key3"): "val"}}
{k: {keys[0]: {keys[1]: val}} for k,v in data.items() for keys, val in v.items()}
I'm trying to compare two large dictionaries that describe the contents of product catalogs. Each dictionary consists of a unique, coded key and a list of terms for each key.
dict1 = {
"SKU001": ["Plumbing", "Pumps"],
"SKU002": ["Motors"],
"SKU003": ["Snow", "Blowers"],
"SKU004": ["Pnuematic", "Hose", "Pumps"],
...
}
dict2 = {
"FAS001": ["Pnuematic", "Pumps"],
"GRA001": ["Lawn", "Mowers"],
"FAS002": ["Servo", "Motors"],
"FAS003": ["Hose"],
"GRA002": ["Snow", "Shovels"],
"GRA003": ["Water", "Pumps"]
...
}
I want to create a new dictionary that borrows the keys from dict1 and whose values are a list of keys from dict2 where at least one of their term values match. The ideal end result may resemble this:
match_dict = {
"SKU001": ["FAS001", "GRA003"],
"SKU002": ["FAS002"],
"SKU003": ["GRA002"],
"SKU004": ["FAS001", "FAS003", "GRA003],
...
}
I'm having issues creating this output though. Is it possible to create a list of keys and assign it as a value to another key? I've made a few attempts using nested loops like below, but the output isn't as desired and I'm unsure if it's even working properly. Any help is appreciated!
matches = {}
for key, values in dict1.items():
for value in values:
if value in dict2.values():
matches[key] = value
print(matches)
This is one possible implementation:
dict1 = {
"SKU001": ["Plumbing", "Pumps"],
"SKU002": ["Motors"],
"SKU003": ["Snow", "Blowers"],
"SKU004": ["Pnuematic", "Hose", "Pumps"],
}
dict2 = {
"FAS001": ["Pnuematic", "Pumps"],
"GRA001": ["Lawn", "Mowers"],
"FAS002": ["Servo", "Motors"],
"FAS003": ["Hose"],
"GRA002": ["Snow", "Shovels"],
"GRA003": ["Water", "Pumps"]
}
match_dict_test = {
"SKU001": ["FAS001", "GRA003"],
"SKU002": ["FAS002"],
"SKU003": ["GRA002"],
"SKU004": ["FAS001", "FAS003", "GRA003"],
}
# Find keys for each item in dict2
dict2_reverse = {}
for k, v in dict2.items():
for item in v:
dict2_reverse.setdefault(item, []).append(k)
# Build dict of matches
match_dict = {}
for k, v in dict1.items():
# Keys in dict2 associated to each item
keys2 = (dict2_reverse.get(item, []) for item in v)
# Save sorted list of keys from dict2 without repetitions
match_dict[k] = sorted(set(k2i for k2 in keys2 for k2i in k2))
# Check result
print(match_dict == match_dict_test)
# True
Assuming that dict1 and dict2 can have duplicate value entries, you would need to build an intermediate multi-map dictionary and also handle uniqueness of the expanded value list for each SKU:
mapDict = dict()
for prod,attributes in dict2.items():
for attribute in attributes:
mapDict.setdefault(attribute,[]).append(prod)
matchDict = dict()
for sku,attributes in dict1.items():
for attribute in attributes:
matchDict.setdefault(sku,set()).update(mapDict.get(attribute,[]))
matchDict = { sku:sorted(prods) for sku,prods in matchDict.items() }
print(matchDict)
{'SKU001': ['FAS001', 'GRA003'], 'SKU002': ['FAS002'], 'SKU003': ['GRA002'], 'SKU004': ['FAS001', 'FAS003', 'GRA003']}
I'm trying to iterate through a dictionary that looks like:
d = {
"list_one": [
"hello",
"two",
"three"
],
"list_two": [
"morning",
"rain"
]
}
I'm using the function:
def combine_words(d):
for k, v in d.items():
a = {k: ("|".join(v))}
return a
When I run this with print, my output is just one key, value pair. I'm not sure what is happening here. My ideal out put would be:
{
'list_one': 'hello|two|three',
'list_two': 'morning|rain'
}
def combine_words(d):
for k, v in d.items():
a = {k: ("|".join(v))}
return a
This constantly reassigns the dictionary to a, and isn't combining the results
def combine_words(d):
a = {}
for k, v in d.items():
a[k] = ("|".join(v))
return a
Would keep adding new entries to the dictionary
a gets replaced by a new dict each time through your loop. You want a dict comprehension.
def combine_words(d):
return {k: "|".join(v) for k, v in d.items()}
The problem is that you change the value of a during every iteration of your for loop because you reassign a. You can modify your code to get around this:
for k, v in d.items():
a[k] = "|".join(v)
Alternatively, you can use a dict comprehension:
return {k: "|".join(v) for k, v in d.items()}
Ok, I'm stuck, need some help from here on...
If I've got a main dictionary like this:
data = [ {"key1": "value1", "key2": "value2", "key1": "value3"},
{"key1": "value4", "key2": "value5", "key1": "value6"},
{"key1": "value1", "key2": "value8", "key1": "value9"} ]
Now, I need to go through that dictionary already to format some of the data, ie:
for datadict in data:
for key, value in datadict.items():
...filter the data...
Now, how would I in that same loop somehow (if possible... if not, suggest alternatives please) check for values of certain keys, and if those values match my presets then I would add that whole list to another dictionary, thus effectively creating smaller dictionaries as I go along out of this main dictionary based on certain keys and values?
So, let's say I want to create a sub-dictionary with all the lists in which key1 has value of "value1", which for the above list would give me something like this:
subdata = [ {"key1": "value1", "key2": "value2", "key1": "value3"},
{"key1": "value1", "key2": "value8", "key1": "value9"} ]
Here is a not so pretty way of doing it. The result is a generator, but if you really want a list you can surround it with a call to list(). Mostly it doesn't matter.
The predicate is a function which decides for each key/value pair if a dictionary in the list is going to cut it. The default one accepts all. If no k/v-pair in the dictionary matches it is rejected.
def filter_data(data, predicate=lambda k, v: True):
for d in data:
for k, v in d.items():
if predicate(k, v):
yield d
test_data = [{"key1":"value1", "key2":"value2"}, {"key1":"blabla"}, {"key1":"value1", "eh":"uh"}]
list(filter_data(test_data, lambda k, v: k == "key1" and v == "value1"))
# [{'key2': 'value2', 'key1': 'value1'}, {'key1': 'value1', 'eh': 'uh'}]
Net of the issues already pointed out in other comments and answers (multiple identical keys can't be in a dict, etc etc), here's how I'd do it:
def select_sublist(list_of_dicts, **kwargs):
return [d for d in list_of_dicts
if all(d.get(k)==kwargs[k] for k in kwargs)]
subdata = select_sublist(data, key1='value1')
The answer is too simple, so I guess we are missing some information. Anyway:
result = []
for datadict in data:
for key, value in datadict.items():
thefiltering()
if datadict.get('matchkey') == 'matchvalue':
result.append(datadict)
Also, you "main dictionary" is not a dictionary but a list. Just wanted to clear that up.
It's an old question, but for some reason there is no one-liner syntax answer:
{ k: v for k, v in <SOURCE_DICTIONARY>.iteritems() if <CONDITION> }
For example:
src_dict = { 1: 'a', 2: 'b', 3: 'c', 4: 'd' }
predicate = lambda k, v: k % 2 == 0
filtered_dict = { k: v for k, v in src_dict.iteritems() if predicate(k, v) }
print "Source dictionary:", src_dict
print "Filtered dictionary:", filtered_dict
Will produce the following output:
Source dictionary: {1: 'a', 2: 'b', 3: 'c', 4: 'd'}
Filtered dictionary: {2: 'b', 4: 'd'}
Inspired by the answer of Skurmedal, I split this into a recursive scheme to work with a database of nested dictionaries. In this case, a "record" is the subdictionary at the trunk. The predicate defines which records we are after -- those that match some (key,value) pair where these pairs may be deeply nested.
def filter_dict(the_dict, predicate=lambda k, v: True):
for k, v in the_dict.iteritems():
if isinstance(v, dict) and _filter_dict_sub(predicate, v):
yield k, v
def _filter_dict_sub(predicate, the_dict):
for k, v in the_dict.iteritems():
if isinstance(v, dict) and filter_dict_sub(predicate, v):
return True
if predicate(k, v):
return True
return False
Since this is a generator, you may need to wrap with dict(filter_dict(the_dict)) to obtain a filtered dictionary.