We Keep Coding

How can I remove nested keys and create a new dict and link both with an ID?

I have a problem. I have a dict my_Dict. This is somewhat nested. However, I would like to 'clean up' the dict my_Dict, by this I mean that I would like to separate all nested ones and also generate a unique ID so that I can later find the corresponding object again.
For example, I have detail: {...}, this nested, should later map an independent dict my_Detail_Dict and in addition, detail should receive a unique ID within my_Dict. Unfortunately, my list that I give out is empty. How can I remove my slaughtered keys and give them an ID?
my_Dict = {
'_key': '1',
'group': 'test',
'data': {},
'type': '',
'code': '007',
'conType': '1',
'flag': None,
'createdAt': '2021',
'currency': 'EUR',
'detail': {
'selector': {
'number': '12312',
'isTrue': True,
'requirements': [{
'type': 'customer',
'requirement': '1'}]
}
}
}
def nested_dict(my_Dict):
my_new_dict_list = []
for key in my_Dict.keys():
#print(f"Looking for {key}")
if isinstance(my_Dict[key], dict):
print(f"{key} is nested")
# Add id to nested stuff
my_Dict[key]["__id"] = 1
my_nested_Dict = my_Dict[key]
# Delete all nested from the key
del my_Dict[key]
# Add id to key, but not the nested stuff
my_Dict[key] = 1
my_new_dict_list.append(my_Dict[key])
my_new_dict_list.append(my_Dict)
return my_new_dict_list
nested_dict(my_Dict)
[OUT] []
# What I want
[my_Dict, my_Details_Dict, my_Data_Dict]
What I have
{'_key': '1',
'group': 'test',
'data': {},
'type': '',
'code': '007',
'conType': '1',
'flag': None,
'createdAt': '2021',
'currency': 'EUR',
'detail': {'selector': {'number': '12312',
'isTrue': True,
'requirements': [{'type': 'customer', 'requirement': '1'}]}}}
What I want
my_Dict = {'_key': '1',
'group': 'test',
'data': 18,
'type': '',
'code': '007',
'conType': '1',
'flag': None,
'createdAt': '2021',
'currency': 'EUR',
'detail': 22}
my_Data_Dict = {'__id': 18}
my_Detail_Dict = {'selector': {'number': '12312',
'isTrue': True,
'requirements': [{'type': 'customer', 'requirement': '1'}]}, '__id': 22}

The following code snippet will solve what you are trying to do:
my_Dict = {
'_key': '1',
'group': 'test',
'data': {},
'type': '',
'code': '007',
'conType': '1',
'flag': None,
'createdAt': '2021',
'currency': 'EUR',
'detail': {
'selector': {
'number': '12312',
'isTrue': True,
'requirements': [{
'type': 'customer',
'requirement': '1'}]
}
}
}
def nested_dict(my_Dict):
# Initializing a dictionary that will store all the nested dictionaries
my_new_dict = {}
idx = 0
for key in my_Dict.keys():
# Checking which keys are nested i.e are dictionaries
if isinstance(my_Dict[key], dict):
# Generating ID
idx += 1
# Adding generated ID as another key
my_Dict[key]["__id"] = idx
# Adding nested key with the ID to the new dictionary
my_new_dict[key] = my_Dict[key]
# Replacing nested key value with the generated ID
my_Dict[key] = idx
# Returning new dictionary containing all nested dictionaries with ID
return my_new_dict
result = nested_dict(my_Dict)
print(my_Dict)
# Iterating through dictionary to get all nested dictionaries
for item in result.items():
print(item)

If I understand you correctly, you wish to automatically make each nested dictionary it's own variable, and remove it from the main dictionary.
Finding the nested dictionaries and removing them from the main dictionary is not so difficult. However, automatically assigning them to a variable is not recommended for various reasons. Instead, what I would do is store all these dictionaries in a list, and then assign them manually to a variable.
# Prepare a list to store data in
inidividual_dicts = []
id_index = 1
for key in my_Dict.keys():
# For each key, we get the current value
value = my_Dict[key]
# Determine if the current value is a dictionary. If so, then it's a nested dict
if isinstance(value, dict):
print(key + " is a nested dict")
# Get the nested dictionary, and replace it with the ID
dict_value = my_Dict[key]
my_Dict[key] = id_index
# Add the id to previously nested dictionary
dict_value['__id'] = id_index
id_index = id_index + 1 # increase for next nested dic
inidividual_dicts.append(dict_value) # store it as a new dictionary
# Manually write out variables names, and assign the nested dictionaries to it.
[my_Details_Dict, my_Data_Dict] = inidividual_dicts

Python Dict inside List inside Dict - comprehension possible?

I have an object in Python 3 of this format:
a = {
'events': [
{
'timestamp': 123,
'message': 'test'
},
{
'timestamp': 456,
'message': 'foo'
},
{
'timestamp': 789,
'message': 'testbar'
},
],
'first': 'abc',
'last': 'def'
}
I want to create a new object of the same format, but filtered by whether the message key's corresponding value contains a certain string, for example filtering by "test":
a = {
'events': [
{
'timestamp': 123,
'message': 'test'
},
{
'timestamp': 789,
'message': 'testbar'
},
],
'first': 'abc',
'last': 'def'
}
Can I use a nested comprehension for this? I know you can do nested list comprehensions like:
[[y*2 for y in x] for x in l]
But is there a neat way for a dict > list > dict situation?

One option would be to create a new copy of the input dict without events, and then set the filtered events as you require, like this:
copy = {k: v for k, v in a.items() if k != 'events'}
copy['events'] = [e for e in a['events'] if 'test' in e['message']]
Or if you don't mind overwriting the original input, simply do this:
a['events'] = [e for e in a['events'] if 'test' in e['message']]

I would go with a list comprehension with an if-statement like the following:
[event for event in a["events"] if event["message"] == "test" ]
Loop through the values of the "events"-key and add them to the list if the value of their "message" key equals "test".
The result is a list of dictionaries that you can assign back to a["events"] or a copy of a if you would like to preserve a["events"].

So - you can use multiple layers of comprehension, but that doesn't mean you should. I think for such an example, you'd produce cleaner code, by running it through a couple of for loops. Having that said, I think the following is technically achieves the outcome you're asking for.
>>> pprint.pprint(a)
{'events': [{'message': 'test', 'timestamp': 123},
{'message': 'foo', 'timestamp': 456},
{'message': 'testbar', 'timestamp': 789}],
'first': 'abc',
'last': 'def'}
>>> aa = copy.deepcopy(a)
>>> aa['beta'] = aa['events']
>>> pprint.pprint({k:[item for item in v if 'test' in item['message']] if isinstance(v, list) else v for k, v in aa.items()})
{'beta': [{'message': 'test', 'timestamp': 123},
{'message': 'testbar', 'timestamp': 789}],
'events': [{'message': 'test', 'timestamp': 123},
{'message': 'testbar', 'timestamp': 789}],
'first': 'abc',
'last': 'def'}
>>> pprint.pprint({k:[item for item in v if 'test' in item['message']] if isinstance(v, list) else v for k, v in a.items()})
{'events': [{'message': 'test', 'timestamp': 123},
{'message': 'testbar', 'timestamp': 789}],
'first': 'abc',
'last': 'def'}
As said, this is something you can do; I would however on behalf of everyone who's had to read other people code in their careers, respectfully request that you don't use this in production code. A couple of for loops might be more LOC, but would in most cases be much more readable and maintainable.

Create a new dictionary from a nested JSON output after parsing

In python3 I need to get a JSON response from an API call,
and parse it so I will get a dictionary That only contains the data I need.
The final dictionary I ecxpt to get is as follows:
{'Severity Rules': ('cc55c459-eb1a-11e8-9db4-0669bdfa776e', ['cc637182-eb1a-11e8-9db4-0669bdfa776e']), 'auto_collector': ('57e9a4ec-21f7-4e0e-88da-f0f1fda4c9d1', ['0ab2470a-451e-11eb-8856-06364196e782'])}
the JSON response returns the following output:
{
'RuleGroups': [{
'Id': 'cc55c459-eb1a-11e8-9db4-0669bdfa776e',
'Name': 'Severity Rules',
'Order': 1,
'Enabled': True,
'Rules': [{
'Id': 'cc637182-eb1a-11e8-9db4-0669bdfa776e',
'Name': 'Severity Rule',
'Description': 'Look for default severity text',
'Enabled': False,
'RuleMatchers': None,
'Rule': '\\b(?P<severity>DEBUG|TRACE|INFO|WARN|ERROR|FATAL|EXCEPTION|[I|i]nfo|[W|w]arn|[E|e]rror|[E|e]xception)\\b',
'SourceField': 'text',
'DestinationField': 'text',
'ReplaceNewVal': '',
'Type': 'extract',
'Order': 21520,
'KeepBlockedLogs': False
}],
'Type': 'user'
}, {
'Id': '4f6fa7c6-d60f-49cd-8c3d-02dcdff6e54c',
'Name': 'auto_collector',
'Order': 4,
'Enabled': True,
'Rules': [{
'Id': '2d6bdc1d-4064-11eb-8856-06364196e782',
'Name': 'auto_collector',
'Description': 'DO NOT CHANGE!! Created via API coralogix-blocker tool',
'Enabled': False,
'RuleMatchers': None,
'Rule': 'AUTODISABLED',
'SourceField': 'subsystemName',
'DestinationField': 'subsystemName',
'ReplaceNewVal': '',
'Type': 'block',
'Order': 1,
'KeepBlockedLogs': False
}],
'Type': 'user'
}]
}
I was able to create a dictionary that contains the name and the RuleGroupsID, like that:
response = requests.get(url,headers=headers)
output = response.json()
outputlist=(output["RuleGroups"])
groupRuleName = [li['Name'] for li in outputlist]
groupRuleID = [li['Id'] for li in outputlist]
# Create a dictionary of NAME + ID
ruleDic = {}
for key in groupRuleName:
for value in groupRuleID:
ruleDic[key] = value
groupRuleID.remove(value)
break
Which gave me a simple dictionary:
{'Severity Rules': 'cc55c459-eb1a-11e8-9db4-0669bdfa776e', 'Rewrites': 'ddbaa27e-1747-11e9-9db4-0669bdfa776e', 'Extract': '0cb937b6-2354-d23a-5806-4559b1f1e540', 'auto_collector': '4f6fa7c6-d60f-49cd-8c3d-02dcdff6e54c'}
but when I tried to parse it as nested JSON things just didn't work.

In the end, I managed to create a function that returns this dictionary,
I'm doing it by breaking the JSON into 3 lists by the needed elements (which are Name, Id, and Rules from the first nest), and then create another list from the nested JSON ( which listed everything under Rule) which only create a list from the keyword "Id".
Finally creating a dictionary using a zip command on the lists and dictionaries created earlier.
def get_filtered_rules() -> List[dict]:
groupRuleName = [li['Name'] for li in outputlist]
groupRuleID = [li['Id'] for li in outputlist]
ruleIDList = [li['Rules'] for li in outputlist]
ruleIDListClean = []
ruleClean = []
for sublist in ruleIDList:
try:
lstRule = [item['Rule'] for item in sublist]
ruleClean.append(lstRule)
ruleContent=list(zip(groupRuleName, ruleClean))
ruleContentDictionary = dict(ruleContent)
lstID = [item['Id'] for item in sublist]
ruleIDListClean.append(lstID)
# Create a dictionary of NAME + ID + RuleID
ruleDic = dict(zip(groupRuleName, zip(groupRuleID, ruleIDListClean)))
except Exception as e: print(e)
return ruleDic

Dynamically create dictionary keys where I can append additional values?

I have a dictionary that I'll be adding data to that looks like this:
mydict = {
'key1': ['First string message', 'Second string message'],
'key2': ['String message']
}
I am iterating through a json object that has two keys - key and message. I want to add every key into mydict and append all messages that have the same key.
Example:
[
{
'key': 'key1',
'message': 'First string message'
},
{
'key': 'key2',
'message': 'String message'
},
{
'key': 'key1',
'message': 'Second string message'
},
{
'key': 'key3',
'message': 'Brand new string message'
}
]
I am having a problem on building my dictionary dynamically though. How can I dynamically create my keys so that I can append additional strings?
One of the ways I've tried is this:
import json
response = [
{
'key': 'key1',
'message': 'First string message'
},
{
'key': 'key2',
'message': 'String message'
},
{
'key': 'key1',
'message': 'Second string message'
},
{
'key': 'key3',
'message': 'Brand new string message'
}
]
mydict = {}
for r in response:
mydict[r['key']] = mydict.get(r['key'], [r['message']])
print(mydict)
{'key1': ['First string message'], 'key2': ['String message'], 'key3': ['Brand new string message']}
This, obviously, fails my expected result because key1 doesn't contain both strings.
I tried this:
for r in response:
mydict[r['key']] = mydict.get(r['key'], [r['message']]) + r['message']
But this fails with TypeError: can only concatenate list (not "str") to list
How can I dynamically create my keys so that I can append additional strings?

You should use either defaultdict or setdefault. I prefer defaultdict:
from collections import defaultdict
response = [
{
'key': 'key1',
'message': 'First string message'
},
{
'key': 'key2',
'message': 'String message'
},
{
'key': 'key1',
'message': 'Second string message'
},
{
'key': 'key3',
'message': 'Brand new string message'
}
]
mydict = defaultdict(list)
for r in response:
mydict[r['key']].append(r['message'])
With setdefault you can make mydict a regular dictionary:
mydict = {}
for r in response:
mydict.setdefault(r['key'], []).append(r['message'])

python dictionary error AttributeError: 'list' object has no attribute 'keys'

I have an error with this line. I am working with a dictionary from a file with an import. This is the dictionary:
users = [{'id':1010,'name':"Administrator",'type':1},{'id':1011,'name':"Administrator2",'type':1}]
And the method with which the work is as follows:
def addData(dict, entry):
new = {}
x = 0
for i in dict.keys():
new[i] = entry(x)
x += 1
dict.append(new)
Where "dict" would be "users", but the error is that the dictionary does not recognize me as such. Can anyone tell me, I have wrong in the dictionary?

That's not a dicionary, it's a list of dictionaries!
EDIT: And to make this a little more answer-ish:
users = [{'id':1010,'name':"Administrator",'type':1},{'id':1011,'name':"Administrator2",'type':1}]
newusers = dict()
for ud in users:
newusers[ud.pop('id')] = ud
print newusers
#{1010: {'type': 1, 'name': 'Administrator'}, 1011: {'type': 1, 'name': 'Administrator2'}}
newusers[1012] = {'name': 'John', 'type': 2}
print newusers
#{1010: {'type': 1, 'name': 'Administrator'}, 1011: {'type': 1, 'name': 'Administrator2'}, 1012: {'type': 2, 'name': 'John'}}
Which is essentially the same as dawgs answer, but with a simplified approach on generating the new dictionary

Perhaps you are looking to do something along these lines:
users = [{'id':1010,'name':"Administrator",'type':1},{'id':1011,'name':"Administrator2",'type':1}]
new_dict={}
for di in users:
new_dict[di['id']]={}
for k in di.keys():
if k =='id': continue
new_dict[di['id']][k]=di[k]
print new_dict
# {1010: {'type': 1, 'name': 'Administrator'}, 1011: {'type': 1, 'name': 'Administrator2'}}
Then you can do:
>>> new_dict[1010]
{'type': 1, 'name': 'Administrator'}
Essentially, this is turning a list of anonymous dicts into a dict of dicts that are keys from the key 'id'