I am using this SSM method to fetch a bunch of sensitive credentials from my AWS parameter store. The path syntax allows me to pull a bunch of related credentials all at once, which is cool, but as you can see the return format is more of a list of dicts. I have stripped away things so that I can grab all my parameters in a format like this:
{
'/a/b/c/d/e': 'xxxxxxxx',
'/a/b/c/d/f': 'xxxxxx',
'/a/b/c/g': 'xxxxxx'
}
But I am having trouble passing that format, into some sort of (im assuming recursive) function that re-formats that dict with path keys into an actual nested dict like this:
{
'a': {
'b': {
'c': {
'd': {
'e': 'xxxxxxx',
'f': 'xxxxxxx'
},
'g': 'xxxxxxx'
}
}
}
}
Anyone done something like this before?
Try this:
d = {
'/a/b/c/d/e': 'xxxxxxxx',
'/a/b/c/d/f': 'xxxxxx',
'/a/b/c/g': 'xxxxxx'
}
output = {}
for key, value in d.items():
directories = key.split('/')[1:]
helper = output
for dire in directories[:-1]:
if dire not in helper:
helper[dire] = {}
helper = helper[dire]
helper[directories[-1]] = value
print(output)
The output was:
{
'a': {
'b': {
'c': {
'd': {
'e': 'xxxxxxxx',
'f': 'xxxxxx'
},
'g': 'xxxxxx'
}
}
}
}
You may inherit dictionary like follwoing
class RecursiveDict(dict):
def __missing__(self, key):
self.__setleaf__(key, RecursiveDict())
return self[key]
def __setleaf__(self,key,value):
super(RecursiveDict, self).__setitem__(key,value)
def __setitem__(self, key,value):
keys = key.strip('/').split('/')
tmp = self
for k in keys[:-1]:
tmp = tmp[k]
tmp.__setleaf__(keys[-1],value)
You may use it as follows
d = RecursiveDict()
d['/a/b/c/d/e'] = 'xxxxxxxx'
d['/a/b/c/d/f'] = 'xxxxxx'
print(d)
This approach doesn't work perfectly with what you're looking for, but if your separator is a period instead of a forward slash, it will work perfectly.
Install unflatten from Pypi (https://pypi.org/project/unflatten/)
pip install unflatten
Code:
from unflatten import unflatten
t = {
'a.b.c.d.e': 'xxxxxxxx',
'a.b.c.d.f': 'xxxxxx',
'a.b.c.g': 'xxxxxx'
}
print(unflatten(t))
Input:
{
'a.b.c.d.e': 'xxxxxxxx',
'a.b.c.d.f': 'xxxxxx',
'a.b.c.g': 'xxxxxx'
}
Output:
{'a': {'b': {'c': {'g': 'xxxxxx', 'd': {'f': 'xxxxxx', 'e': 'xxxxxxxx'}}}}}
I wrote library (ssmenv) that I am using for reading values from parameter store:
from ssmenv import SSMEnv
params = SSMEnv("/service/my-service", "/resource/mysql")
This way you will fetch all keys under "/service/my-service" and "/resource/mysql" namespace as normalised keys e.g. "SERVICE_MY_SERVICE_DEBUG"
Related
I'm working with dictionary structure of Python 3 and I want to some change in a dictionary with desired and changeable key.
For instance, let's create a dictionary as follow:
myDict = {
'foo': {
'a':12,
'b':14
},
'bar': {
'c':12,
'b':14
},
'moo': {
'a':12,
'd':14
},
}
In this point, a key that is unknown will have and will used to find desired data path.
So, if received key is "myDict.foo.a", I must change value of "a" variable of foo, or if the key is "myDict.moo.a", I will change value of "a" variable of moo. As in this example, the key to use is unknown and I have a value that will put to the identified key (data path).
Under these conditions, how to change dictionary value with unknown key (data path).
To better explain, I described a dysfunctional code snippet about the solve of this question:
dictionary = init_dic() # initialization step for dictionary
desired_value = 1 # a variable to use for change operation in dictionary
received_key = get_key() # receive unknown key group (exp: myDict.foo.a)
dictionary[received_key] = desire_value # The question of this topic
Thank you for reading, have a good day!
For the question I asked, I found a solution like below using "exec".
myDict = {}
myDict["myDict"] = { 'foo': { 'a':12, 'b':14 }, 'bar': { 'c':12, 'b':14 }, 'moo': { 'a':12, 'd':14 }, }
print(myDict)
{'myDict': {'foo': {'a': 12, 'b': 14}, 'bar': {'c': 12, 'b': 14},
'moo': {'a': 12, 'd': 14}}}
data_path = "myDict.foo.a"
desired_value = 1
exec_string = "myDict"
for path in data_path.split("."):
exec_string += "[\"{}\"]".format(path)
exec_string += " = {}".format(desired_value)
exec(exec_string)
print(myDict)
{'myDict': {'foo': {'a': 1, 'b': 14}, 'bar': {'c': 12, 'b': 14},
'moo': {'a': 12, 'd': 14}}}
This perspective is required some control steps but, for me, enough for now. For instance, incoming sub data path can be unwanted command such as open("file.txt","w"). Therefore a control mechanism is required.
If you have different suggestion as solution, could you write here?
I have a dictionary like this
d = {
'Benefits': {
1: {
'BEN1': {
'D': [{'description': 'D1'}],
'C': [{'description': 'C1'}]
}
},
2: {
'BEN2': {
'D': [{'description': 'D2'}],
'C': [{'description': 'C2'}]
}
}
}
}
I am trying to sort dictionary based on KEY OF LAST VALUES(LIST).
FOR EXAMPLE
I am looking for get dictionary value like 'C' IN first and 'D' in second
I'm trying to get correct order. Here is code:
d1 = collections.OrderedDict(sorted(d.items()))
Unfortunately didn't get correct result
This is my expected output
{'Benefits':
{1:
{'BEN1':
{'C':[{'description': 'C1'}], 'D': [{'description': 'D1'}]
}
},
2:
{'BEN2':
{'C': [{'description': 'C2'}], 'D': [{'description': 'D2'}]
}
}
}
}
I am using python 3.5 . I am trying to get order like this
{'C':[{'description': 'C1'}], 'D': [{'description': 'D1'}]}
The following code will sort any dictionary by its key and recursively sort any dictionary value that is also a dictionary by its key and makes no assumption as to the content of the dictionary being sorted. It uses an OrderedDict but if you can be sure it will always run on Python 3.6 or greater, a simple change can be made to use a dict.
from collections import OrderedDict
d = {
'Benefits': {
1: {
'BEN1': {
'D': [{'description': 'D1'}],
'C': [{'description': 'C1'}]
}
},
2: {
'BEN2': {
'D': [{'description': 'D2'}],
'C': [{'description': 'C2'}]
}
}
}
}
def sort_dict(d):
items = [[k, v] for k, v in sorted(d.items(), key=lambda x: x[0])]
for item in items:
if isinstance(item[1], dict):
item[1] = sort_dict(item[1])
return OrderedDict(items)
#return dict(items)
print(sort_dict(d))
See demo
d1 = collections.OrderedDict(sorted(d.items()))
This is not working because it is sorting only on the Benefits item. Here you want to sort inner items, so we have to reach the inner items and sort them.
d1 = {'Benefits': {}}
for a_benefit in d['Benefits']:
d1['Benefits'][a_benefit] = {}
for a_ben in d['Benefits'][a_benefit]:
d1['Benefits'][a_benefit][a_ben] = dict(collections.OrderedDict(sorted(d['Benefits'][a_benefit][a_ben].items())))
I have a JSON data with structure like this:
{ "a":"1",
"b":[{ "a":"4",
"b":[{}],
"c":"6"}]
"c":"3"
}
Here the key a is always unique even if nested.
I want to separate my JSON data so that it should look like this:
{"a":"1"
"b":[]
"c":"3"
},
{"a":"4",
"b":[],
"c":"6"
}
JSON data can be nested up to many times.
How to do that?
I'd use an input and output stack:
x = {
"a":1,
"b":[
{
"a":2,
"b":[ { "a":3, }, { "a":4, } ]
}
]
}
input_stack = [x]
output_stack = []
while input_stack:
# for the first element in the input stack
front = input_stack.pop(0)
b = front.get('b')
# put all nested elements onto the input stack:
if b:
input_stack.extend(b)
# then put the element onto the output stack:
output_stack.append(front)
output_stack ==
[{'a': 1, 'b': [{'a': 2, 'b': [{'a': 3}, {'a': 4}]}]},
{'a': 2, 'b': [{'a': 3}, {'a': 4}]},
{'a': 3},
{'a': 4}]
output_stack can be a dict of cause. Then replace
output_stack.append(front)
with
output_dict[front['a']] = front
Not sure about a Python implementation, but in JavaScript this could be done using recursion:
function flatten(objIn) {
var out = [];
function unwrap(obj) {
var arrayItem = {};
for(var idx in obj) {
if(!obj.hasOwnProperty(idx)) {continue;}
if(typeof obj[idx] === 'object') {
if(isNaN(parseInt(idx)) === true) {
arrayItem[idx] = [];
}
unwrap(obj[idx]);
continue;
}
arrayItem[idx] = obj[idx];
}
if(JSON.stringify(arrayItem) !== '{}') {
out.unshift(arrayItem);
}
}
unwrap(objIn);
return out;
}
This will only work as expected if the object key names are not numbers.
See JSFiddle.
I have a dictionary that looks like this:
d = {'dev':
{<dev1>:
{'mod':
{<mod1>:
{'port': [1, 2, 3]
}
}
}
<dev2>:
{'mod':
{<mod3>:
{'port': [] }
}
}
}
}
I want to be able to write a function, such that if i provide a search object such as 'mod1', it provides me the parent key as 'dev1'.
I have searched all over and tried a bunch of things, but couldnt seem to get this to work. Any help will be appreciated!
I have tried the stuff mentioned at the link below:
Python--Finding Parent Keys for a specific value in a nested dictionary
Find a key in a python dictionary and return its parents
This should work:
def find_parent_keys(d, target_key, parent_key=None):
for k, v in d.items():
if k == target_key:
yield parent_key
if isinstance(v, dict):
for res in find_parent_keys(v, target_key, k):
yield res
Usage:
d = {
'dev': {
'dev1': {
'mod': {
'mod1': {'port': [1, 2, 3]},
},
},
'dev2': {
'mod': {
'mod3': {'port': []},
},
},
},
}
print list(find_parent_keys(d, 'mod'))
print list(find_parent_keys(d, 'dev'))
Output:
['dev2', 'dev1']
[None]
I have two dictionary objects which are very complex and created by converting large xml files into python dictionaries.
I don't know the depth of the dictionaries and just want to compare and want the following output...
e.g. My dictionaries are like this
d1 = {"great grand father":
{"name":"John",
"grand father":
{"name":"Tom",
"father":
{"name":"Andy",
"Me":
{"name":"Mike",
"son":
{"name":"Tom"}
}
}
}
}
}
d2 is also a similar but could be possible any one of the field is missing or changed as below
d2 = {"great grand father":
{"name":"John",
"grand father":
{"name":"Tom",
"father":
{"name":"Andy",
"Me":
{"name":"Tonny",
"son":
{"name":"Tom"}
}
}
}
}
}
The dictionary comparison should give me results like this -
Expected Key/Val : Me->name/"Mike"
Actual Key/Val : Me->name/"Tonny"
If the key "name" does not exists in "Me" in d2, it should give me following output
Expected Key/Val : Me->name/"Mike"
Actual Key/Val : Me->name/NOT_FOUND
I repeat the dictionary depth could be variable or dynamically generated. The two dictionaries here are given as examples...
All the dictionary comparison questions and their answers which I have seen in SO are related fixed depth Dictionaries.....
You're in luck, I did this as part of a project where I worked.
You need a recursive function something like:
def checkDifferences(dict_a,dict_b,differences=[])
You can first check for keys that don't exist in one or the other.
e.g
Expected Name/Tom Actual None
Then you compare the types of the values i.e check if the value is a dict or a list etc.
If it is then you can recursively call the function using the value as dict_a/b. When calling recursively pass the differences array.
If the type of the value is a list and the list may have dictionaries within it then you need to covert the list to a dict and call the function on the converted dictionary.
I'm sorry I can't help more but I no longer have access to the source code. Hopefully this is enough to get you started.
Here I found a way to compare any two dictionaries -
I have tried with various dictionaries of any depths and worked for me. The code is not so modular but just for the reference -
import pprint
pp = pprint.PrettyPrinter(indent=4)
dict1 = { 'Person' : { 'Male' : {'Boys' : {'Roger' : {'age' : 20},
'Rafa' : {'age' : 25}
}
},
'Female' : { 'Girls' : {'Serena' : {'age' : 23},
'Maria' : {'age' : 15}
}
}
},
'Animal' : { 'Huge' : {'Elephant' : {'color' : 'black' }
}
}
}
'''
dict2 = { 'Person' : { 'Male' : {'Boys' : {'Roger' : {'age' : 20}
}
},
'Female' : { 'Girls' : {'Serena' : {'age' : 23},
'Maria' : {'age' : 1}
}
}
}
}
dict2 = { 'Person' : { 'Male' : {'Boys' : {'Roger' : {'age' : 20},
'Rafa' : {'age' : 2}
}
}
}
}
'''
dict2 = { 'Person' : { 'Male' : {'Boys' : {'Roger' : {'age' : 2}}},
'Female' : 'Serena'}
}
key_list = []
err_list = {}
def comp(exp,act):
for key in exp:
key_list.append(key)
exp_val = exp[key]
try:
act_val = act[key]
is_dict_exp = isinstance(exp_val,__builtins__.dict)
is_dict_act = isinstance(act_val,__builtins__.dict)
if is_dict_exp == is_dict_act == True:
comp(exp_val,act_val)
elif is_dict_exp == is_dict_act == False:
if not exp_val == act_val:
temp = {"Exp" : exp_val,"Act" : act_val}
err_key = "-->".join(key_list)
if err_list.has_key(key):
err_list[err_key].update(temp)
else:
err_list.update({err_key : temp})
else:
temp = {"Exp" : exp_val, "Act" : act_val}
err_key = "-->".join(key_list)
if err_list.has_key(key):
err_list[err_key].update(temp)
else:
err_list.update({err_key : temp})
except KeyError:
temp = {"Exp" : exp_val,"Act" : "NOT_FOUND"}
err_key = "-->".join(key_list)
if err_list.has_key(key):
err_list[err_key].update(temp)
else:
err_list.update({err_key : temp})
key_list.pop()
comp(dict1,dict2)
pp.pprint(err_list)
Here is the output of my code -
{ 'Animal': { 'Act': 'NOT_FOUND',
'Exp': { 'Huge': { 'Elephant': { 'color': 'black'}}}},
'Person-->Female': { 'Act': 'Serena',
'Exp': { 'Girls': { 'Maria': { 'age': 15},
'Serena': { 'age': 23}}}},
'Person-->Male-->Boys-->Rafa': { 'Act': 'NOT_FOUND', 'Exp': { 'age': 25}},
'Person-->Male-->Boys-->Roger-->age': { 'Act': 2, 'Exp': 20}
}
One can also try with other dictionaries given in commented code..
One more thing - The keys are checked in expected dictionary and the matched with an actual. If we pass dictionaries in alternate order the other way matching is also possible...
comp(dict2,dict1)