Related
I have a json below, and I want to parse out value from this dict.
I can do something like this to get one specific value
print(abc['everything']['A']['1']['tree']['value'])
But, what is best way to parse out all "value?"
I want to output good, bad, good.
abc = {'everything': {'A': {'1': {'tree': {'value': 'good'}}},
'B': {'5': {'tree1': {'value': 'bad'}}},
'C': {'30': {'tree2': {'value': 'good'}}}}}
If you are willing to use pandas, you could just use pd.json_normalize, which is actually quite fast:
import pandas as pd
abc = {'everything': {'A': {'1': {'tree': {'value': 'good'}}},
'B': {'5': {'tree1': {'value': 'bad'}}},
'C': {'30': {'tree2': {'value': 'good'}}}}}
df = pd.json_normalize(abc)
print(df.values[0])
['good' 'bad' 'good']
Without any extra libraries, you will have to iterate through your nested dictionary:
values = [abc['everything'][e][k][k1]['value'] for e in abc['everything'] for k in abc['everything'][e] for k1 in abc['everything'][e][k]]
print(values)
['good', 'bad', 'good']
Provided your keys and dictionaries have a value somewhere, you can try this:
Create a function (or reuse the code) that gets the first element of the dictionary until the value key exists, then return that. Note that there are other ways of doing this.
Iterate through, getting the result under each value key and return.
# Define function
def get(d):
while not "value" in d:
d = list(d.values())[0]
return d["value"]
# Get the results from your example
results = [get(v) for v in list(abc["everything"].values())]
['good', 'bad', 'good']
A Recursive way:
def fun(my_dict, values=[]):
if not isinstance(my_dict, dict):
return values
for i, j in my_dict.items():
if i == 'value':
values.append(j)
else:
values = fun(j, values)
return values
abc = {'everything': {'A': {'1': {'tree': {'value': 'good'}}},
'B': {'5': {'tree1': {'value': 'bad'}}},
'C': {'30': {'tree2': {'value': 'good'}}}}}
data = fun(abc)
print(data)
Output:
['good', 'bad', 'good']
Firstly, the syntax you are using is incorrect.
If you are using pandas, you can code like
import pandas as pd
df4 = pd.DataFrame({"TreeType": ["Tree1", "Tree2", "Tree3"],
"Values": ["Good", "Bad","Good"]})
df4.index = ["A","B","C"]
next just run the code df4, you would get the correct output.
output:
TreeType Values
A Tree1 Good
B Tree2 Bad
C Tree3 Good
Does any one knows how to get the index of the values from dictionary 2 on dictionary 1.. like this:
Dictionary_1= {A: [Tom, Jane, Joe]; B: [Joana, Clare, Tom]; C: [Clare, Jane, Joe]}
Dictionary_2 = {A: Tom; B: Clare; C: Jane}
RESULT = {A: 1; B: 2; C: 2}
EDIT:
Sorry guys.. first of all I got confused and forgot that I needed it starting with "0" instead of "1".
I was having a problem, but it was because my list inside of dictionary 1 was in unicode format instead of list.
Also.. in the example I used here, I noticed later that the keys existed in both dictionaries, but in the code Im writting it wasnt the same thing. I didnt post the original here because it was bigger, so I tried to resume the most I could. Sorry for that too.
So I got it working with this code:
RESULT = {}
for x, y in Dictionary_1.items():
for a, b in Dictionary_2 .items():
if x == a:
z = Dictionary_1[x]
r = eval(z)
if '{0}'.format(b) in r:
RESULT [a] = r.index('{0}'.format(b))
I know that its looks messy but im still learning.
I really appreciate your help guys!
You can try using dict comprehension.
dict1={'A':['Tom','Jane','Joe'],'B':['Joana','Clare','Tom'],'C':['Clare','Jane','Joe']}
dict2={'A':'Tom','B':'Clare','C':'Jane'}
result={k:dict1[k].index(v)+1 for k,v in dict2.values()}
# {'A': 1, 'B': 2, 'C': 2}
#Or
# {k:dict1.get(k).index(v)+1 for k,v in dict2.items()}
Assuming you want 0-based indices, you can use list.index() with a dict comprehension:
d1 = {'A': ['Tom', 'Jane', 'Joe'], 'B': ['Joana', 'Clare', 'Tom'], 'C': ['Clare', 'Jane', 'Joe']}
d2 = {'A': 'Tom', 'B': 'Clare', 'C': 'Jane'}
result = {k: d1[k].index(v) for k, v in d2.items()}
print(result)
# {'A': 0, 'B': 1, 'C': 1}
If you want to have indices starting at 1, then you can do d1[k].index(v) + 1.
An easy to understand solution for you
d1 = {'A': ['Tom', 'Jane', 'Joe'], 'B': ['Joana', 'Clare', 'Tom'], 'C': ['Clare', 'Jane', 'Joe']}
d2 = {'A': 'Tom', 'B': 'Clare', 'C': 'Jane'}
output = {}
for k,v in d2.items():
output[k] = d1[k].index(v)+1
print(output)
This is certainly not the best approach but this is what I did quickly:
dict1 = {0: ['Tom', 'Jane', 'Joe'], 1: ['Joana', 'Clare', 'Tom'], 2: ['Clare', 'Jane', 'Joe']}
dict2 ={0: 'Tom', 1: 'Clare', 2: 'Jane'}
result = {}
val_list = list(dict1.values())
for i in range(0,len(dict1)):
result.update({i : val_list[i].index(dict2[i])})
print(result)
I want to create two dictionaries in python by dictionary comprehension at the same time. The two dictionaries share the same key set, but have different values for each key. Therefore, I use a function to return a tuple of two values, and hoping a dictionary comprehension can create these two dictionaries at the same time.
Say, I have a function
def my_func(foo):
blablabla...
return a, b
And I will create two dictionaries by
dict_of_a, dict_of_b = ({key:my_func(key)[0]}, {key:my_func(key)[1]} for key in list_of_keys)
Is there any better code to improve it? In my opinion, my_func(key) will be called twice in each iteration, slowing down the code. What is the correct way to do it?
With ordered slicing:
def myfunc(k):
return k + '0', k + '1'
list_of_keys = ['a', 'b', 'c']
groups = [(k,v) for k in list_of_keys for v in myfunc(k)]
dict_of_a, dict_of_b = dict(groups[::2]), dict(groups[1::2])
print(dict_of_a) # {'a': 'a0', 'b': 'b0', 'c': 'c0'}
print(dict_of_b) # {'a': 'a1', 'b': 'b1', 'c': 'c1'}
for key in list_of_keys:
dict_of_a[key],dict_of_b[key] = my_func(key)
The regular loop is probably the best way to go. If you want to play with functools, you can write:
>>> def func(foo): return foo[0], foo[1:]
...
>>> L = ['a', 'ab', 'abc']
>>> functools.reduce(lambda acc, x: tuple({**d, x: v} for d, v in zip(acc, func(x))), L, ({}, {}))
({'a': 'a', 'ab': 'a', 'abc': 'a'}, {'a': '', 'ab': 'b', 'abc': 'bc'})
The function reduce is a fold: it takes the current accumulator (here the dicts being built) and the next value from L:
d, v in zip(acc, func(x)) extracts the dicts one at a time and the matching element of the return value of func;
{**d, x: v} update the dict with the current value.
I don't recommend this kind of code since it's hard to maintain.
my_func(key) will be called twice in each iteration, slowing down the code
Dont worry about it. Unless you need to do thousands/millions of iterations and the script takes an unreasonably long time to complete, you shouldn't concern with negligible optimization gains.
That said, I'd use something like this:
if __name__ == '__main__':
def my_func(k):
return f'a{k}', f'b{k}'
keys = ['x', 'y', 'z']
results = (my_func(k) for k in keys)
grouped_values = zip(*results)
da, db = [dict(zip(keys, v)) for v in grouped_values]
print(da)
print(db)
# Output:
# {'x': 'ax', 'y': 'ay', 'z': 'az'}
# {'x': 'bx', 'y': 'by', 'z': 'bz'}
You cannot create two dicts in one dict comprehension.
If your primary goal is to just call my_func once to create both dicts, use a function for that:
def mkdicts(keys):
dict_of_a = {}
dict_of_b = {}
for key in keys:
dict_of_a[key], dict_of_b[key] = my_func(key)
return dict_of_a, dict_of_b
I have a list which is structured as follows:
arr = [ ['a'],
['a','b'],
['a','x','y'],
['a','c'],
['a','c','a'],
['a','c','b'],
['a','c','b','a'],
['a','c','b','b'],
['a','d'],
['b'],
['b','c'],
['b','c','a'],
['b','c','b'],
['c','d'],
['c','d','e'],
['c','d','f'],
['c','d','f','a'],
['c','d','f','b'],
['c','d','f','b','a'],
]
As you would observe that the list has some unique elements and then following elements are building upon the unique element till a new unique element appears. These are supposed to categories and subcategories. So [a] , [b] , ['c','d'] are the broad level main categories and then there are further sub categories within sub categories based on the same priciple as above. Ideally I want the categories and sub categories as a dictionary. the end result should look something like:
{'a': ['a-b',
'a-x-y',
{'a-c':
['a-c-a',
{'a-c-b':
['a-c-b-a',
'a-c-b-b']
}]
}
],
'b' : ................
'c-d': ...............}
I may also be able to work with just the first level of sub-classification and discarding the rest altogether. In that case, the output would be:
{'a': ['a-b', 'a-x-y', 'a-c', 'a-d'], 'b': ['b-c'], 'c-d': ['c-d-e', 'c-d-f']}
I have written a code for the second scenario but I am not sure if this is a robust way to solve this:
def arrange(arr):
cat = {"-".join(arr[0]): ["-".join(arr[1])]}
main = 0
for i in range(2,len(arr)):
l = len(arr[main])
if arr[main] == arr[i][0:l]:
cat["-".join(arr[main])].append("-".join(arr[i]))
else:
cat["-".join(arr[i])] = []
main = i
for k,v in cat.items():
found = True
i = 0
while i < len(v)-1:
f_idx = i + 1
while v[i] in v[f_idx]:
v.pop(f_idx)
i += 1
return cat
Output-:
{'a': ['a-b', 'a-x-y', 'a-c', 'a-d'], 'b': ['b-c'], 'c-d': ['c-d-e', 'c-d-f']}
Please help me make this code better and or help me with a dictionary that has the complete structure where I have all the sub-classifications. Thanks
Finally , I believe I've what you describe as first level of sub-classification and discarding the rest altogether.
The trick was to create action based upon when an item in the list (keys) was not a sublist of subsequent items (values).
The same logic was used for removing duplicates.
from collections import defaultdict
#Function that compares two lists even with duplicate items
def contains_sublist(lst, sublst):
n = len(sublst)
return any((sublst == lst[i:i+n]) for i in xrange(len(lst)-n+1))
#Define default dict of list
aDict = defaultdict(list)
it = iter(arr)
#Format key
key = '-'.join(next(it))
s = list(key)
# Loop that collects keys if key is not sublist else values
for l in it:
if contains_sublist(l, s):
aDict[key].append(l)
else:
key = '-'.join(l)
s = l
#Loop to remove duplicate items based upon recurrance of sublist
it = iter(aDict.keys())
for k in it:
dellist = []
for s in aDict[k]:
for l in aDict[k]:
if l != s:
if contains_sublist(l, s):
if not l in dellist:
dellist.append(l)
for l in dellist:
try:
aDict[k].remove(l)
except ValueError:
pass
#Create final dict by concatenating list of list with '-'
finaldict = {k:[ '-'.join(i) for i in v ] for k,v in aDict.iteritems()}
Result:
Python 2.7.9 (default, Dec 10 2014, 12:24:55) [MSC v.1500 32 bit (Intel)] on win32
Type "copyright", "credits" or "license()" for more information.
>>> ================================ RESTART ================================
>>>
>>> finaldict
{'a': ['a-b', 'a-x-y', 'a-c', 'a-d'], 'b': ['b-c'], 'c-d': ['c-d-e', 'c-d-f']}
>>>
You're describing a Trie.
Here's a very basic implementation:
def make_trie(words):
root = dict()
for word in words:
current_dict = root
for letter in word:
current_dict = current_dict.setdefault(letter, {})
current_dict[1] = 1
return root
trie = make_trie(arr)
print(trie)
# {'a': {1: 1, 'c': {'a': {1: 1}, 1: 1, 'b': {'a': {1: 1}, 1: 1, 'b': {1: 1}}}, 'b': {1: 1}, 'd': {1: 1}, 'x': {'y': {1: 1}}}, 'c': {'d': {1: 1, 'e': {1: 1}, 'f': {'a': {1: 1}, 1: 1, 'b': {'a': {1: 1}, 1: 1}}}}, 'b': {1: 1, 'c': {'a': {1: 1}, 1: 1, 'b': {1: 1}}}}
print(trie.get('a',{}).get('x',{}))
# {'y': {1: 1}}
This trie is just nested dicts, so it's easy to iterate over all the children of ['a', 'x'] or select all the dicts that have a max-depth of 2 for example.
1 is used for leaf words: for example if you have ['a', 'x', 'y'] as sub-array, but not ['a', 'x'].
There are more complete Trie libraries for Python, such as pygtrie.
Is there a library that would help me achieve the task to rearrange the levels of a nested dictionary
Eg: From this:
{1:{"A":"i","B":"ii","C":"i"},2:{"B":"i","C":"ii"},3:{"A":"iii"}}
To this:
{"A":{1:"i",3:"iii"},"B":{1:"ii",2:"i"},"C":{1:"i",2:"ii"}}
ie first two levels on a 3 levelled dictionary swapped. So instead of 1 mapping to A and 3 mapping to A, we have A mapping to 1 and 3.
The solution should be practical for an arbitrary depth and move from one level to any other within.
>>> d = {1:{"A":"i","B":"ii","C":"i"},2:{"B":"i","C":"ii"},3:{"A":"iii"}}
>>> keys = ['A','B','C']
>>> e = {key:{k:d[k][key] for k in d if key in d[k]} for key in keys}
>>> e
{'C': {1: 'i', 2: 'ii'}, 'B': {1: 'ii', 2: 'i'}, 'A': {1: 'i', 3: 'iii'}}
thank god for dict comprehension
One way to think about this would be to consider your data as a (named) array and to take the transpose. An easy way to achieve this would be to use the data analysis package Pandas:
import pandas as pd
df = pd.DataFrame({1: {"A":"i","B":"ii","C":"i"},
2: {"B":"i","C":"ii"},
3: {"A":"iii"}})
df.transpose().to_dict()
{'A': {1: 'i', 2: nan, 3: 'iii'},
'B': {1: 'ii', 2: 'i', 3: nan},
'C': {1: 'i', 2: 'ii', 3: nan}}
I don't really care about performance for my application of this so I haven't bothered checking how efficient this is. Its based on bubblesort so my guess is ~O(N^2).
Maybe this is convoluted, but essentially below works by:
- providing dict_swap_index a nested dictionary and a list. the list should be of the format [i,j,k]. The length should be the depth of the dictionary. Each element corresponds to which position you'd like to move each element to. e.g. [2,0,1] would indicate move element 0 to position 2, element 1 to position 0 and element 2 to position 1.
- this function performs a bubble sort on the order list and dict_, calling deep_swap to swap the levels of the dictionary which are being swapped in the order list
- deep_swap recursively calls itself to find the level provided and returns a dictionary which has been re-ordered
- swap_two_level_dict is called to swap any two levels in a dictionary.
Essentially the idea is to perform a bubble sort on the dictionary, but instead of swapping elements in a list swap levels in a dictionary.
from collections import defaultdict
def dict_swap_index(dict_, order):
for pas_no in range(len(order)-1,0,-1):
for i in range(pas_no):
if order[i] > order[i+1]:
temp = order[i]
order[i] = order[i+1]
order[i+1] = temp
dict_ = deep_swap(dict_, i)
return dict_, order
def deep_swap(dict_, level):
dict_ = deepcopy(dict_)
if level==0:
dict_ = swap_two_level_dict(dict_)
else:
for key in dict_:
dict_[key] = deep_swap(dict_[key], level-1)
return dict_
def swap_two_level_dict(a):
b = defaultdict(dict)
for key1, value1 in a.items():
for key2, value2 in value1.items():
b[key2].update({key1: value2})
return b
e.g.
test_dict = {'a': {'c': {'e':0, 'f':1}, 'd': {'e':2,'f':3}}, 'b': {'c': {'g':4,'h':5}, 'd': {'j':6,'k':7}}}
result = dict_swap_index(test_dict, [2,0,1])
result
(defaultdict(dict,
{'c': defaultdict(dict,
{'e': {'a': 0},
'f': {'a': 1},
'g': {'b': 4},
'h': {'b': 5}}),
'd': defaultdict(dict,
{'e': {'a': 2},
'f': {'a': 3},
'j': {'b': 6},
'k': {'b': 7}})}),
[0, 1, 2])