update dictionary in nested loop - python

I am having trouble updating a dictionary. i am just extracting certain fields within each key - but the output is not as expected. data, expected output and code are below. Thanks for looking, i appreciate any comments
categories = {'categories_a1' : [{'group': '13GH9', 'number': '1'},{'group': '17KPO', 'number': '73'}, {'group': '26BN11', 'number': '2'}, {'group': '813W', 'number': '99'}],
'categories_a2' : [{'group': '99ITY', 'number': '12'},{'group': 'JH871', 'number': '15'}, {'group': 'OLH83', 'number': '99'}, {'group': '44RTQ', 'number': '1'}]}
xpected= {'categories_a1' : [{'13GH9': '1'},{'17KPO':'73'}, {'26BN11':'2'}, {'813W': '99'}],
'categories_a2' : [{'99ITY':'12'},{'JH871': '15'}, {'OLH83': '99'}, {'44RTQ':'1'}]}
out={}
for k in categories.keys():
for i in categories[k]:
x = {k: v for k, v in zip([i['group']], [i['number']])}
out[k] = x
out.update(out)

Let's first clean up some general weirdness:
out.update(out)
This line does effectively nothing and should be omitted.
x = {k: v for k, v in zip([i['group']], [i['number']])}
This makes little sense; we create lists with one element each and iterate over them in parallel. We could just as easily just use those values directly: x = {i['group']: i['number']}.
After swapping that in, let's consider the part that causes the actual problem:
for i in categories[k]:
x = {i['group']: i['number']}
out[k] = x
The problem here is that you want out[k] to constitute a list of all of the modified dictionaries, but x is repeatedly being assigned one of those dictionaries, and the result then becomes out[k]. What you presumably intended to do is repeatedly append those dictionaries to a new empty list:
x = []
for i in categories[k]:
x.append({i['group']: i['number']})
out[k] = x
However, it's clear that you're already familiar and comfortable with comprehensions, and this is an ideal place to use one:
out[k] = [{i['group']: i['number']} for i in categories[k]]
And, of course, we can extend this technique to the overall loop:
out = {
k: [{i['group']: i['number']} for i in v]
for k, v in categories.items()
}
Please carefully study the structure of this code and make sure you understand the technique. We have a source dictionary that we want to transform to create our output, and the rule is: the key remains unchanged, the value (which is a list) undergoes its own transformation. So we start by writing the skeleton for a dict comprehension, using .items() to give us key-value pairs:
out = {
k: # we need to fill in something to do with `v` here
for k, v in categories.items()
}
Then we figure out what we're doing with the value: each element of the list is a dictionary; the way that we process the list is iterative (each element of the input list tells us an element to use in the output list), but the processing of those elements is not (we look at exactly two hard-coded values from that dict, and make a dict from them). Given an element i of the list, the corresponding dict that we want has exactly one key-value pair, which we can compute as {i['group']: i['number']}. So we wrap a list comprehension around that: [{i['group']: i['number']} for i in v]; and we insert that into the dict comprehension skeleton, giving us the final result.

One approach:
for key, value in categories.items():
categories[key] = [{ d["group"] : d["number"] } for d in value]
print(categories)
Output
{'categories_a1': [{'13GH9': '1'}, {'17KPO': '73'}, {'26BN11': '2'}, {'813W': '99'}], 'categories_a2': [{'99ITY': '12'}, {'JH871': '15'}, {'OLH83': '99'}, {'44RTQ': '1'}]}

Related

How to find and append dictionary values which are present in a list

I have a list which has unique sorted values
arr = ['Adam', 'Ben', 'Chris', 'Dean', 'Flower']
I have a dictionary which has values as such
dict = {
'abc': {'Dean': 1, 'Adam':0, 'Chris':1},
'def': {'Flower':0, 'Ben':1, 'Dean':0}
}
From looking at values from arr I need to have each item and if the value isn't present in subsequent smaller dict that should be assigned a value -1
Result
dict = {
'abc': {'Adam':0, 'Ben':-1, 'Chris':1, 'Dean': 1, 'Flower':-1},
'def': {'Adam':-1, 'Ben':1, 'Chris':-1, 'Dean': 0, 'Flower':0}
}
how can I achieve this using list and dict comprehensions in python
dd = {
key: {k: value.get(k, -1) for k in arr}
for key, value in dd.items()
}
{k: value.get(k, -1) for k in arr} will make sure that your keys are in the same order as you defined in the arr list.
A side note on the order of keys in dictionary.
Dictionaries preserve insertion order. Note that updating a key does
not affect the order. Keys added after deletion are inserted at the
end.
Changed in version 3.7: Dictionary order is guaranteed to be insertion
order. This behavior was an implementation detail of CPython from 3.6.
Please do not make a variable called dict, rename it to dct or something since dict it is a reserved python internal.
As for your question: just iterate through your dct and add the missing keys using setdefault:
arr = ['Adam', 'Ben', 'Chris', 'Dean', 'Flower']
dct = {
'abc': {'Dean': 1, 'Adam':0, 'Chris':1},
'def': {'Flower':0, 'Ben':1, 'Dean':0}
}
def add_dict_keys(dct, arr):
for key in arr:
dct.setdefault(key, -1)
return dct
for k, v in dct.items():
add_dict_keys(v, arr)
print(dct) # has updated values

customize dictionary key and values

I have a question about change dictionary format.
The dictionary is :
{'index': 'cfs_nucleus_bespoke_88260', 'host': 'iaasn00018224.svr.us.jpmchase.net', 'source': '/logs/tomcat7inst0/localhost_tomcat7inst0_access_log2018-11-02.txt', '_time': '2018-11-02 19:46:50.000 EDT', 'count': '1'}
I want to ask is there a way for me to change the format like below:
{"column1":{'index': 'cfs_nucleus_', 'host': 'iaasn00018224.net'}, "column2":{'source': '/logs/tomcat7inst0/localhost_tomcat7inst0_access_log2018-11-02.txt'}, "column3":{'_time': '2018-11-02, 'count': '1'}}
You can do the following:
dict1 = {'index': 'cfs_nucleus_bespoke_88260', 'host': 'iaasn00018224.svr.us.jpmchase.net', 'source': '/logs/tomcat7inst0/localhost_tomcat7inst0_access_log2018-11-02.txt', '_time': '2018-11-02 19:46:50.000 EDT', 'count': '1'}
d1_items = list(dict1.items())
col_width = 2
dict2 = {f'column{col_num // col_width + 1}': {k: v for k, v in d1_items[col_num:col_num + col_width]} for col_num in range(0, len(dict1), col_width)}
Try it online!
There are a few moving parts that interact to create this solution:
Dict comprehensions
Python has a neat trick where it allows you to embed for in loops in iterable and dict declarations to efficiently cycle and modify a set of elements. Here, the outer iterator is range(0, len(dict1), col_width): this goes through a sequence of integers starting from 0, and progressively increases by col_width until it is greater than or equal to the size of the list, which functions to choose the start index of each col_width-sized dict segment.
Tuple unpacking
dict1.items() is convenient because it returns a dict view of 2-tuples of each dictionary key and its value. Later, we utilize tuple unpacking k: v for k, v in d1_items[ ... ], where a tuple of variables is flattened into two variables that can then easily form a key-value pair of the currrent dictionary comprehension. (This is only in newer versions of Python.)
List slicing
d1_items[col_num:col_num + col_width] is basically a way of getting a sublist. The syntax in relatively straightforward: starting from position col_num, get a sublist up to and excluding the element col_num + col_width (i.e. a sublist of size col_width).
Formatted string literals
Preceding a string with f makes it a formatted string literal. Anything within { } is interpreted as literal Python syntax (with the exception of ', {, and }, which may vary behavior by context.) Here, in f'column{col_num // col_width + 1}', it allows us to label each column with a bit of division and a +1 offset to start counting from 1 instead of 0. (This is new to Python 3.6)

filter list of dictionaries based on a particular value of a key in that dictionary

I have a list of dictionaries as follows
dict = {2308:[{'name':'john'},{'age':'24'},{'employed':'yes'}],3452:[{'name':'sam'},{'age':'45'},{'employed':'yes'}],1234:[{'name':'victor'},{'age':'72'},{'employed':'no'}]}
I want to filter out the above dictionary to new dictionary named new_dict whose age >30.
I tried the following. As I new to programming, could not get the logic.
new_dict =[var for var in dict if dict['age']>30]
But I know it is list of dictionaries, so is there any way I can get the new list dictionaries with age >30
You can use the following dict comprehension (assuming you store the dictionary in variable d rather than dict, which would shadow the built-in dict class):
{k: v for k, v in d.items() if any(int(s.get('age', 0)) > 30 for s in v)}
This returns:
{3452: [{'name': 'sam'}, {'age': '45'}, {'employed': 'yes'}], 1234: [{'name': 'victor'}, {'age': '72'}, {'employed': 'no'}]}
Solution
people = {
2308:[
{'name':'john'},{'age':'24'},{'employed':'yes'}
],
3452:[
{'name':'sam'},{'age':'45'},{'employed':'yes'}
],
1234:[
{'name':'victor'},{'age':'72'},{'employed':'no'}
]
}
people_over_30 = {}
for k, v in people.items():
for i in range(len(v)):
if int(v[i].get('age', 0)) > 30:
people_over_30[k] = [v]
print(people_over_30)
Output
(xenial)vash#localhost:~/python$ python3.7 quote.py
{3452: [[{'name': 'sam'}, {'age': '45'}, {'employed': 'yes'}]], 1234: [[{'name': 'victor'}, {'age': '72'}, {'employed': 'no'}]]}
Comments
Unless this is your desired structure for this particular code, I would suggest reformatting your dictionary to look like this
people = {
2308:
{'name':'john','age':'24','employed':'yes'}
,
3452:
{'name':'sam','age':'45','employed':'yes'}
,
1234:
{'name':'victor','age':'72','employed':'no'}
}
And then your work would be much simpler and able to handle with this instead
for k, v in people.items():
if int(v.get('age', 0)) > 30:
people_over_30[k] = [v]

Iterating JSON structure of list object and finding next keys

I have a Json list and I want to print all the keys from a given key till the end of dictionary. But the code I wrote is very complex. How to do it with less complexity ?
I'm using Python 3
dictionary = [{"a": "1"}, {"b": "2"}, {"c": "3"}, {"d": "4"}]
try:
for token in dictionary:
if "b" in list(token.keys())[0]:
new_dict = dictionary[len(list(token.keys())[0]):]
for i in new_dict:
print(new_dict[len(list(i.keys())[0]):])
break
else:
print("Inception")
except Exception as error:
print(str(error))
DESIRED
Input: b
Output: c, d
My Output:
Inception
[{'c': '3'}, {'d': '4'}]
[{'c': '3'}, {'d': '4'}]
[{'c': '3'}, {'d': '4'}]
Use itertools.dropwhile() to skip all dictionaries that don't have a 'b' key:
from itertools import dropwhile
filtered = dropwhile(lambda t: 'b' not in t, dictionary)
next(filtered) # skip the dictionary with `b` in it.
for token in filtered:
print(token)
This prints all dictionaries after the first. If you only need to print their keys, do so explicitly:
filtered = dropwhile(lambda t: 'b' not in t, dictionary)
next(filtered) # skip the dictionary with `b` in it.
for token in filtered:
print(*token, sep='\n')
This prints the keys on separate lines; if there is just one key, then that's all that'll be printed for each token dictionary.)
As a side note: you really do not want to use list(dict.keys())[0]. Before Python 3.6, dictionaries have no set order (instead being subject to insertion and deletion history and the current random hash seed), so if you have more than one key what value you'll get is a gamble. And all you want to do is see if a key is present, so use a key in dictobject membership test.
To get the first key out of each dictionary, I'd use next(iter(dictobject)), avoiding creating a list:
first_key = next(iter(dictobject))
I'd only use this if I had single-key dictionaries. I'd also avoid using dictionaries for such a scenario; perhaps you really wanted to use an ordered dictionary instead (in Python < 3.6, use collections.OrderedDict(), otherwise use the regular dict type).
This is one way. The idea is to find the index of the dictionary with desired key. Then filter your list accordingly.
keys = [next(iter(x)) for x in dictionary]
res = keys[keys.index('b')+1:]
# ['c', 'd']
You can re-write your code as follows for your desired result.
found = False
for data in dictionary:
if found:
print(list(data.keys())[0])
if 'b' in data.keys():
found = True

Filtering a dictionary value from a list of dictionary using lambda and filter in python

I have a list of dictionaries as below:
l = [{'abc': 'def'}, {'ghi': 'jul'}, {'Name': 'my-name'}]
I want to get the value of the dictionary that has the key 'Name'
Initially I wrote a function as below:
def get_json_val(l, key):
for item in l:
for k, v in item.iteritems():
if k == key:
return v
But I want to do it using lambdas and filter in a single line. So, I tried as below to filter out the dictionary:
name_lst = filter(lambda item: (k == 'Name' for k in item.iteritems()), l)
It is showing the whole list again. Is there a way to get only the value (not dictionary) using lambdas and filter? Thanks in advance
Edit 1:
In the first function l is list and 'Name' is passed as key.
Seeing your initial code, this should do the same thing
>>> l = [{'abc': 'def'}, {'ghi': 'jul'}, {'Name': 'my-name'}]
>>> [i["Name"] for i in l if "Name" in i][0]
'my-name'
Your code returns only the first occurence.Using my approach, and probably also if you would use filter, you'll get (first) a list of all occurrences than only get the first (if any). This would make it less "efficient" IMHO, so I would probably change your code to be something more like this:
def get_json_val(l, key):
for item in l:
if not key in item: continue
return item[key]
Why are you iterating over your dict? That defeats the purpose. Just do
[d for d in l if key in d]
Or if you feel some compulsion to use lambda and filter:
filter(lambda d: key in d, l)
Note, these return lists instead of the corresponding value. If you want the value, you'll have to index into the list. Simply out, using a loop is probably the most reasonable approach, just don't iterate over your dict:
def f(key, l):
for d in l:
if key in d:
return d[key]
l = [{'abc': 'def'}, {'ghi': 'jul'}, {'Name': 'my-name', "no-name": "bbb"}]
I was able to do it in this way,
print filter(lambda item: item, map(lambda item: item.get("Name"), l))
if I break this down.
First check if item is in the dict or not?
fna = lambda item: item.get("Name")
then map list of dict to above function.
map(fna, l)
for above list it will print
[None, None, 'my-name']
Now we need to eliminate all the None. I am using filter with following function
fnb = lambda item: item
Now if you put all together you have your code in single line.
Here is an easy way to do it without lambdas or filter or any of that stuff. Just combine the list of dictionaries to one dictionary, then use that.
l = [{'abc': 'def'}, {'ghi': 'jul'}, {'Name': 'my-name'}]
k = {m.keys()[0] : m[m.keys()[0]] for m in l}['Name']
Note that this does not change the structure of l.
Don't underestimate the power of dict.get(). What follows is a simplification of a previous answer. In this case, there is no need for map. For the list:
l = [{'abc': 'def'}, {'ghi': 'jul'}, {'Name': 'my-name'}]
The following produces a list of dictionaries that have the key 'Name':
filtered = list(filter(lambda x: x.get('Name'), l))
From here, you can retrieve the name values for all items in the list:
values = [x['Name'] for x in filtered]
If there is only one target, as in your example, wrap this all up in one line like this:
name = list(filter(lambda x: x.get('Name'), l))[0]['Name']

Categories