I have a complicated method that needs to work correctly, that I can't seem to figure out why it doesn't.
I have a dictionary as it follows:
{'view': ['premium_subscribers', 'premium_content'], 'delete': ['admins', 'normal_content', 'premium_content']}
I need to know how to iterate one by one through each specific key values, which are arrays. For example:
key = delete
for loop (that iterates 1 by 1 through key "delete's" values
takes "admins" value and does some processing
in the next iteration takes normal_content and does same processing
and so on ......
its basically checking for a match to be found.
In case if you're interested in my method, I have it below. It has 3 dictionaries and is comparing their key's values to accomplish access permission.
If the for loop iterates correctly through each value of that key, it will start working as expected.
def accessPermission(operation, user, object):
domains = dict()
types = dict()
access = dict()
access = json.load(open("access.txt"))
domains = json.load(open("domains.txt"))
types = json.load(open("types.txt"))
print(types)
print(access)
print(domains)
if operation in access.keys():
firstkeyvalue = access.get(operation)[0]
if firstkeyvalue in domains.keys():
if user in domains.get(firstkeyvalue):
for access.get(operation)[0] in access:
if object in types.values():
print("Success")
else:
print("Error: access denied")
else:
print("Error: access denied")
else:
print("Error: access denied")
else:
print("Error: access denied")
Seems you only need to iterate through the elements like this:
dict = {'view': ['premium_subscribers', 'premium_content'], 'delete': ['admins', 'normal_content', 'premium_content']}
key = 'delete' #set somewhere
for v in dict[key]:
print(v)
#do what you need with that
Output:
admins
normal_content
premium_content
Each Value is a list, so you need an extra loop for iterating over the items of each list:
data_dict = {'view': ['premium_subscribers', 'premium_content'], 'delete': ['admins', 'normal_content', 'premium_content']}
for key in data_dict:
values = data_dict[key]
# iterate over list of values
for v in values:
# Now you have access to each one
print(v)
Output:
premium_subscribers
premium_content
admins
normal_content
premium_content
Related
I have a value, which I need to add it to a multidimensional dict
The problem is that the keys may or may not exist
If they exist, I just want to add it, if not .. I need to create it
What is the best way of doing this, since right now what I have looks pretty bad
if response.get('pages', {}).get(url, {}).get('variations', {}).get('custom_screenshot'):
response['pages'][url]['variations']['custom_screenshot'][command.get('params')[0]] = output
elif response.get('pages', {}).get(url, {}).get('variations'):
response['pages'][url]['variations']['custom_screenshot'] = {command.get('params')[0]: output}
elif response.get('pages', {}).get(url, {}):
response['pages'][url]['variations'] = {'custom_screenshot': {command.get('params')[0]: output}}
elif response.get('pages', {}):
response['pages']['url'] = {'variations': {'custom_screenshot': {command.get('params')[0]: output}}}
else:
response['pages'] = {url: {'variations': {'custom_screenshot': {command.get('params')[0]: output}}}}
return response
Use referential nature of Python dictionary.
Declare intermediate keys that should be in the final response (in proper order)
Loop though the keys calling dict.setdefaut method to set the inner dictionary if it's not there
Set unconditional value output for the custom key command.get('params')[0]
resp_keys = [url, 'variations', 'custom_screenshot']
pages_dict = resp.setdefault('pages', {})
for k in resp_keys:
pages_dict = pages_dict.setdefault(k, {}) # return dict under key k
pages_dict[command.get('params')[0]] = output
def loop_through_list_keys_and_update_or_delete(body_dict, list_key, key_to_manipulate, update_or_delete):
request_body = body_dict
try:
for keys in request_body[list_key]:
for key, value in keys.items():
if key == key_to_manipulate:
if update_or_delete == 'delete':
del request_body[key]
Hi all! Relatively new to Python, so please be gentle and I definitely appreciate your help! In the attached picture are my run time values. Basically what I am trying to do is delete a specific (but one that changes) key/value pair. The request_body is my json that I've read in.
I am looping on the list 'fxTransactions' and the only keys it is getting are financialTransaction and transactionRefId. In this case, I need to get to the request_body[fxTransactions][0][financialTransaction][rule] field that is within financialTransaction and delete it and its value. But I can't send this as a variable to the delete as you can't combine the list name with the keys in a variable. I can't seem to figure out how to get to this field and delete it. Note that the field/value pair to delete is dynamic so I will not always know which one I want to delete (could be debtor, creditor, etc). I'd also like to use this code for other lists, if possible, so trying not to hard-code. But I can if there is no other way.
Thank you in advance!
Here is code that works to do what I'm trying to do, but I'm trying to come up with a way to make it cleaner and reusable:
index_key_int = int(index_key)
split_key = key.split(".")
keys_numb = len(split_key)
if keys_numb == 5:
del request_body[list_key][index_key_int][split_key[0]][split_key[1]][split_key[2]][split_key[3]][
split_key[4]]
if keys_numb == 4:
del request_body[list_key][index_key_int][split_key[0]][split_key[1]][split_key[2]][split_key[3]]
if keys_numb == 3:
del request_body[list_key][index_key_int][split_key[0]][split_key[1]][split_key[2]]
elif keys_numb == 2:
del request_body[list_key][index_key_int][split_key[0]][split_key[1]]
elif keys_numb == 1:
del request_body[list_key][index_key_int][split_key[0]]
You can use a loop instead of hard-coding all the nested indexing.
obj = request_body[list_key][index_key_int]
for key in split_key[:-1]:
obj = obj[key]
del obj[split_key[-1]]
The for loop dives down into the nested dictionaries. Then it deletes the element with the final key.
I am asking an ElasticSearch database to provide me with a list of indices and their creation dates using Python 2.7 and the Requests package. The idea is to quickly calculate which indices have exceeded the retention policy and need to be put to sleep.
The request works perfectly and the results are exactly what I want. However, when I run the code below, when I try to convert the json result to a dict, the type of theDict is correct but it reports a size of 1, when there should be at least a couple dozen entries. What am I doing wrong? I have a feeling it's something really dumb but I just can't snag it! :)
import json
import requests
esEndPoint = "https://localhost:9200"
retrieveString = "/_cat/indices?h=index,creation.date.string&format=json&s=creation.date"
# Gets the current indices and their creation dates
def retrieveIndicesAndDates():
try:
theResult = requests.get(esEndPoint+retrieveString)
print (theResult.content)
except Exception as e:
print("Unable to retrieve list of indices with creation dates.")
print("Error: "+e)
exit(3)
return theResult.content
def main():
theDict = dict(json.loads(retrieveIndicesAndDates()))
print(type(theDict)) # Reports correct type
print(len(theDict)) # Always outputs "1" ??
for index, creationdate in theDict.items():
print("Index: ",index,", Creation date: ",theDict[index])
return
The json the call returns:
[{"index":".kibana","creation.date.string":"2017-09-14T15:01:38.611Z"},{"index":"logstash-2018.07.23","creation.date.string":"2018-07-23T00:00:01.024Z"},{"index":"cwl-2018.07.23","creation.date.string":"2018-07-23T00:00:03.877Z"},{"index":"k8s-testing-internet-2018.07.23","creation.date.string":"2018-07-23T14:19:10.024Z"},{"index":"logstash-2018.07.24","creation.date.string":"2018-07-24T00:00:01.023Z"},{"index":"k8s-testing-internet-2018.07.24","creation.date.string":"2018-07-24T00:00:01.275Z"},{"index":"cwl-2018.07.24","creation.date.string":"2018-07-24T00:00:02.157Z"},{"index":"k8s-testing-internet-2018.07.25","creation.date.string":"2018-07-25T00:00:01.022Z"},{"index":"logstash-2018.07.25","creation.date.string":"2018-07-25T00:00:01.186Z"},{"index":"cwl-2018.07.25","creation.date.string":"2018-07-25T00:00:04.012Z"},{"index":"logstash-2018.07.26","creation.date.string":"2018-07-26T00:00:01.026Z"},{"index":"k8s-testing-internet-2018.07.26","creation.date.string":"2018-07-26T00:00:01.185Z"},{"index":"cwl-2018.07.26","creation.date.string":"2018-07-26T00:00:02.587Z"},{"index":"k8s-testing-internet-2018.07.27","creation.date.string":"2018-07-27T00:00:01.027Z"},{"index":"logstash-2018.07.27","creation.date.string":"2018-07-27T00:00:01.144Z"},{"index":"cwl-2018.07.27","creation.date.string":"2018-07-27T00:00:04.485Z"},{"index":"ctl-2018.07.27","creation.date.string":"2018-07-27T09:02:09.854Z"},{"index":"cfl-2018.07.27","creation.date.string":"2018-07-27T11:12:44.681Z"},{"index":"elb-2018.07.27","creation.date.string":"2018-07-27T11:13:51.340Z"},{"index":"cfl-2018.07.24","creation.date.string":"2018-07-27T11:45:23.697Z"},{"index":"cfl-2018.07.23","creation.date.string":"2018-07-27T11:45:24.646Z"},{"index":"cfl-2018.07.25","creation.date.string":"2018-07-27T11:45:25.700Z"},{"index":"cfl-2018.07.26","creation.date.string":"2018-07-27T11:45:26.341Z"},{"index":"elb-2018.07.24","creation.date.string":"2018-07-27T11:45:27.440Z"},{"index":"elb-2018.07.25","creation.date.string":"2018-07-27T11:45:29.572Z"},{"index":"elb-2018.07.26","creation.date.string":"2018-07-27T11:45:36.170Z"},{"index":"logstash-2018.07.28","creation.date.string":"2018-07-28T00:00:01.023Z"},{"index":"k8s-testing-internet-2018.07.28","creation.date.string":"2018-07-28T00:00:01.316Z"},{"index":"cwl-2018.07.28","creation.date.string":"2018-07-28T00:00:03.945Z"},{"index":"elb-2018.07.28","creation.date.string":"2018-07-28T00:00:53.992Z"},{"index":"ctl-2018.07.28","creation.date.string":"2018-07-28T00:07:19.543Z"},{"index":"k8s-testing-internet-2018.07.29","creation.date.string":"2018-07-29T00:00:01.026Z"},{"index":"logstash-2018.07.29","creation.date.string":"2018-07-29T00:00:01.378Z"},{"index":"cwl-2018.07.29","creation.date.string":"2018-07-29T00:00:04.100Z"},{"index":"elb-2018.07.29","creation.date.string":"2018-07-29T00:00:59.241Z"},{"index":"ctl-2018.07.29","creation.date.string":"2018-07-29T00:06:44.199Z"},{"index":"logstash-2018.07.30","creation.date.string":"2018-07-30T00:00:01.024Z"},{"index":"k8s-testing-internet-2018.07.30","creation.date.string":"2018-07-30T00:00:01.179Z"},{"index":"cwl-2018.07.30","creation.date.string":"2018-07-30T00:00:04.417Z"},{"index":"elb-2018.07.30","creation.date.string":"2018-07-30T00:01:01.442Z"},{"index":"ctl-2018.07.30","creation.date.string":"2018-07-30T00:08:28.936Z"},{"index":"cfl-2018.07.30","creation.date.string":"2018-07-30T06:52:16.739Z"}]
Your error is trying to convert a list of dicts to a dict:
theDict = dict(json.loads(retrieveIndicesAndDates()))
# ^^^^^ ^
That would only work for a dict of lists. It would be redundant, though.
Just use the reply directly. Each entry is a dict with the appropriate keys:
data = json.loads(retrieveIndicesAndDates())
for entry in data:
print("Index: ", entry["index"], ", Creation date: ", entry["creation.date.string"])
So what happens when you do convert that list to a dict? Why is there just one entry?
The dict understands three initialisation methods: keywords, mappings and iterables. A list fits the last one.
Initialisation from an iterable goes through it and expects key-value iterables as elements. If one were to do it manually, it would look like this:
def sequence2dict(sequence):
map = {}
for element in sequence:
key, value = element
map[key] = value
return map
Notice how each element is unpacked via iteration? In the reply each element is a dict with two entries. Iteration on that yields the two keys but ignores the values.
key, value = {"index":".kibana","creation.date.string":"2017-09-14T15:01:38.611Z"}
print(key, '=>', value) # prints "index => creation.date.string"
To the dict constructor, every element in the reply has the same key-value pair: "index" and "creation.date.string". Since keys in a dict are unique, all elements collapse to the same entry: {"index": "creation.date.string"}.
I am trying to create a python script and I am stuck with the dictionaries. I have read through some of the other forums but can't seem to get anywhere. I am a very new python programmer so please be gentle.
What I want to do:
1) set up a dictionary like this: {'Name':'userid','jobid:jobid','walltime:walltime,'nodes:nds'}
2) iterate through a list of entries created from and external function call and extract information to populate the dictionary
3) Problem: I cannot figure out how to append entries to the appropriate keys
For example, I want this:
{‘Name’:’jose’,’jobid’:’001,002,003,005’,’walltime:32:00,240:00,04:00,07:00’,’nodes’:32,32,500’}
Notice for one userid, I have multiple jobids, walltimes and nodes.
(len(jobids)==len(walltimes)==len(nodes) for any one userid but can vary across userids)
I am able to get the script to find the first value for each username, but it never appends. How can I get this to append?
Here is what I have tried
from collections import defaultdict
pdict = defaultdict(list)
start the loop:
# get new values – add these to the dictionary keyed
# on username (create a new entry or append to existing entry)
…
(jobid,userid,jobname, sessid, nds, tsk, walltime,rest)= m.groups()
...
if userid in pdict:
print "DEBUG: %s is currently in the dictionary -- appending entries" %(userid)
pdict[userid][‘jobid’] = pdict[userid][jobid].append(jobid) I
# repeat for nodes, walltime, etc
if not userid in pdict:
print "DEBUG: %s is not in the dictionary creating entry" %(userid)
pdict[userid] = {} # define a dictionary within a dictionary with key off userid
pdict[userid]['jobid'] = jobid
pdict[userid]['jobname'] = jobname
pdict[userid]['nodes'] = nds
pdict[userid]['walltime'] = walltime
I know this is wrong but can’t figure out how to get the append to work. I have tried many of the suggestions offered on this site. I need to append (to the dictionary) the most recent values from the loop keyed to userid
Here is an example of the ouput – it does not append multiple entries for each userid but rather takes only the first value for each userid
userid jmreill contains data: {'nodes': '1', 'jobname':
'A10012a_ReMig_Q', 'walltime': '230:0', 'jobid': '1365582'}
userid igorysh contains data: {'nodes': '20', 'jobname':
'emvii_Beam_fwi6', 'walltime': '06:50', 'jobid': '1398100'}
Any suggestions? This should be easy but I can’t figure it out!
from collections import defaultdict
pdict = defaultdict(dict)
start the loop:
# get new values – add these to the dictionary keyed
# on username (create a new entry or append to existing entry)
…
(jobid,userid,jobname, sessid, nds, tsk, walltime,rest)= m.groups()
...
if userid in pdict:
print "DEBUG: %s is currently in the dictionary -- appending entries" %(userid)
pdict[userid][jobid].append(jobid)
# repeat for nodes, walltime, etc
if userid not in pdict:
print "DEBUG: %s is not in the dictionary creating entry" %(userid)
pdict[userid]['jobid'] = [jobid]
pdict[userid]['jobname'] = jobname
pdict[userid]['nodes'] = nds
pdict[userid]['walltime'] = walltime
The value corresponding to key 'jobid' should be a list of strings rather than a string. If you create your dictionary this way, you can append new jobid's to the list simply by:
pdict[userid]['jobid'].append(jobid)
I cant remember the explanation why to use the lambda expression in the following code, but you have to define a defaultdict of a defaultdict:
pdict = defaultdict(lambda: defaultdict(list))
pdict[userid][‘jobid’].append('1234')
will work.
The append() method does not return the list...it is modified in place. Also, you need to initialize your elements as lists (using square brackets):
if userid in pdict:
print "DEBUG: %s is currently in the dictionary -- appending entries" %(userid)
pdict[userid][jobid].append(jobid) ## just call append here
# repeat for nodes, walltime, etc
if not userid in pdict:
print "DEBUG: %s is not in the dictionary creating entry" %(userid)
pdict[userid] = {} # define a dictionary within a dictionary with key off userid
pdict[userid]['jobid'] = [jobid,] ## initialize with lists
pdict[userid]['jobname'] = [jobname,]
pdict[userid]['nodes'] = [nds,]
pdict[userid]['walltime'] = [walltime,]
append doesn't return a value, it modifies the list in place, and you forgot to quote 'jobid' on the right of equals. So you should replace pdict[userid][‘jobid’] = pdict[userid][jobid].append(jobid) with pdict[userid]['jobid'].append(jobid). Also take into account comment from #Jasper.
You are looking for a dict of dicts? AutoVivification is the perfect solution. Implement the perl’s autovivification feature in Python.
class AutoVivification(dict):
"""Implementation of perl's autovivification feature."""
def __getitem__(self, item):
try:
return dict.__getitem__(self, item)
except KeyError:
value = self[item] = type(self)()
return value
This makes everything easier. Note that the value of pdict[userid]['jobid'] should be a list [jobid] instead of a variable jobid as you have multiple jobid.
pdict = AutoVivification()
if userid in pdict:
pdict[userid]['jobid'].append(jobid)
else:
pdict[userid]['jobid'] = [jobid] # a list
Refer to What is the best way to implement nested dictionaries in Python?.
Here is my code:
for response in responses["result"]:
ids = {}
key = response['_id'].encode('ascii')
print key
for value in response['docs']:
ids[key].append(value)
Traceback:
File "people.py", line 47, in <module>
ids[key].append(value)
KeyError: 'deanna'
I am trying to add multiple values to a key. Throws an error like above
Check out setdefault:
ids.setdefault(key, []).append(value)
It looks to see if key is in ids, and if not, sets that to be an empty list. Then it returns that list for you to inline call append on.
Docs:
http://docs.python.org/2/library/stdtypes.html#dict.setdefault
If I'm reading this correctly your intention is to map the _id of a response to its docs. In that case you can bring down everything you have above to a dict comprehension:
ids = {response['_id'].encode('ascii'): response['docs']
for response in responses['result']}
This also assumes you meant to have id = {} outside of the outermost loop, but I can't see any other reasonable interpretation.
If the above is not correct,
You can use collections.defaultdict
import collections # at top level
#then in your loop:
ids = collections.defaultdict(list) #instead of ids = {}
A dictionary whose default value will be created by calling the init argument, in this case calling list() will produce an empty list which can then be appended to.
To traverse the dictionary you can iterate over it's items()
for key, val in ids.items():
print(key, val)
The reason you're getting a KeyError is this: In the first iteration of your for loop, you look up the key in an empty dictionary. There is no such key, hence the KeyError.
The code you gave will work, if you first insert an empty list into the dictionary under to appropriate key. Then append the values to the list. Like so:
for response in responses["result"]:
ids = {}
key = response['_id'].encode('ascii')
print key
if key not in ids: ## <-- if we haven't seen key yet
ids[key] = [] ## <-- insert an empty list into the dictionary
for value in response['docs']:
ids[key].append(value)
The previous answers are correct. Both defaultdict and dictionary.setdefault are automatic ways of inserting the empty list.