I have two nested dictionaries. Each dictionary has a key/value pair that are the same. I have some code that says hey if these are the same, update an existing dictionary with another key/value combo that exists in one of the dictionaries to the other dictionary. I get the error RuntimeError: dictionary changed size during iteration. I've seen that you can use deepcopy to solve for this, but does anybody else have any other ideas?
performances = [
{'campaign_id': 'bob'},
{'campaign_id': 'alice'},
]
campaign_will_spend = [
{'id': 'bob'},
{'id': 'alice'},
]
for item in campaign_will_spend:
ad_dictt = dict()
for willspendkey, willspendvalue in item.items():
if willspendkey == "id":
for i in performances:
for key, value in i.items():
if key == 'campaign_id' and value == willspendvalue:
i['lifetime_budget'] = item
You're causing yourself a lot of trouble by treating dictionaries like lists and iterating over them in their entirety to find a particular item. Most of the code goes away when you just stop doing that, and the rest of it goes away if you build a dictionary to be able to easily look up entries in campaign_will_spend:
# Easy lookup for campaign_will_spend dictionaries by id.
cws_by_id = {d['id']: d for d in campaign_will_spend}
for p in performances:
p["lifetime_budget"] = cws_by_id[p["campaign_id"]]
Related
I am still relatively new to python and trying to teach myself different things. However, this issue has stumped me. I have a list of dictionaries. I want to extract the value of SourceKey from the dictionary and use that value as a new key within the list that also contains all the rest of the dictionary entries inside the new key (I hope that didn't sound too confusing). For example:
data = [
{'AssetId':'1234',
'CreatedById':'02i3s',
'Billable__c': True,
'SourceKey': '00a1234'},
{'AssetId':'4567',
'CreatedById':'03j8t',
'Billable__c':True,
'SourceKey': '00b4321'}
]
So now I want to take the value from SourceKey to create a dictionary of dictionaries so it looks like this:
new_data = [
{'00a1234': {'AssetId':'1234',
'CreatedById':'02i3s',
'Billable__c': True},
{'00b4321': {'AssetId':'4567',
'CreatedById':'03j8t',
'Billable__c':True}
]
I basically have this starting point, but I'm just stuck on how to put the key-value pairs of the nested dictionary inside the new value from SourceKey because I know I need to replace the ___ with the key-values from the rest of data:
[new_data] = {}
for row in data:
if row['SourceKey']:
new_data.update(row['SourceKey'], ___)
Any help would be fantastic!
You're defining [new_data] as dict but you actually want new_data as dictionary.
Use comprehension to get the content of data without SourceKey:
from pprint import pprint
data = [
{'AssetId':'1234',
'CreatedById':'02i3s',
'Billable__c': True,
'SourceKey': '00a1234'},
{'AssetId':'4567',
'CreatedById':'03j8t',
'Billable__c':True,
'SourceKey': '00b4321'}
]
new_data = dict()
for row in data:
if row['SourceKey']:
new_data[row['SourceKey']] = {k:v for k,v in row.items() if k != 'SourceKey'}
pprint(new_data)
Output:
{'00a1234': {'AssetId': '1234', 'Billable__c': True, 'CreatedById': '02i3s'},
'00b4321': {'AssetId': '4567', 'Billable__c': True, 'CreatedById': '03j8t'}}
I wonder if a existing dictionary instance can add and/or delete multiple items without using iterations.
I mean something like this.
supposition:(it actually doesn't work)
D = {"key1":"value1", "key2":"value2", "key3":"value3"}
tags = ["key1","key2"]
D.pop(tags)
print(D)
{"key3":"value3"}
Thank you in advance.
If so, you could iterate a list instead of iterate the full dict:
D = {"key1":"value1", "key2":"value2", "key3":"value3"}
for i in ["key1", "key2"]:
D.pop(i)
print(D)
If you don't actually need to avoid iteration, but rather just want to do the transformation of the dictionary in an expression, rather than a statement, you could use a dictionary comprehension to create a new dictionary containing only the keys (and the associated values) that don't match your list of things to remove:
D = {key: value for key, value in D.items() if key not in tags}
Unfortunately, this doesn't modify D in place, so if you need to change the value referenced through some other variable this won't help you (and you'd need to do an explicit loop). Note that if you don't care about the values being removed, you probably should use del D[key] instead of D.pop(key).
If all you're wanting to do is show the dictionary where key from list is not present, why not just create a new dic:
D = {"key1":"value1", "key2":"value2", "key3":"value3"}
tags=["key1", "key2"]
dict = {key:value for key, value in D.items() if key not in tags}
print(dict)
I'm looking at converting some Chef run_lists to tags, and would like to automate the process.
So far what I've done is created a variable that runs:
# write to file instead of directly to variable for archival purposes
os.system("knife search '*:*' -a expanded_run_list -F json > /tmp/hostname_runlist.json")
data = json.load(open('/tmp/hostname_runlist.json'))
From there, I have a dict within a dict with list values similar to this:
{u'abc.com': {u'expanded_run_list': None}}
{u'foo.com': {u'expanded_run_list': u'base::default'}}
{u'123.com': {u'expanded_run_list': [u'utils::default', u'base::default']}}
...
I would like to convert that to a more simpler dictionary by removing the 'expanded_run_list' portion, as it it's not required at this point, so in the end it looks like this:
abc.com:None
foo.com:'base::default'
123.com:['utils::default', 'base::default']
I would like to keep the values as a list, or a single value depending on what is returned. When I run a 'for statement' to iterate, I can pull the hostnames from i.keys, but would need to remove the expanded_run_list key from i.values, as well as pair the key values up appropriately.
From there, I should have an easier time to iterate through the new dictionary when running an os.system Chef command to create the new tags. It's been a few years since I've written in python, so am a bit rusty. Any descriptive help would be much appreciated.
Considering that you are having your list of dict objects as:
my_list = [
{u'abc.com': {u'expanded_run_list': None}},
{u'foo.com': {u'expanded_run_list': u'base::default'}},
{u'123.com': {u'expanded_run_list': [u'utils::default', u'base::default']}}
]
Then, in order to achieve your desired result, you may use a combination of list comprehension and dict comprehension as:
For getting the list of nested dictionary
[{k: v.get('expanded_run_list') for k, v in l.items()} for l in my_list]
which will return you the list of dict objects in your desired form as:
[
{u'abc.com': None},
{u'foo.com': u'base::default'},
{u'123.com': [u'utils::default', u'base::default']}
]
Above solution assumes that you only want the value of key 'expanded_run_list' to be picked up from each of your nested dictionary. In case it doesn't exists, dict.get will return None which will be set as value in your resultant dict.
For pulling up your nested dictionary to form single dictionary
{k: v.get('expanded_run_list') for l in my_list for k, v in l.items()}
which will return:
{
'foo.com': 'base::default',
'123.com': ['utils::default', 'base::default'],
'abc.com': None
}
I'm collecting weather data and trying to create a list that has the latest (temperature) value by minute.
I want to add them to a list, and if the list does not contains the "minute index" it should at it as a new element in the list. So the list always keeps the latest temperature value per minute:
def AddValue(arr, value):
timestamp = datetime.datetime.utcnow().strftime("%Y-%m-%d %H:%M")
for v in arr['values']:
try:
e = v[timestamp] # will trigger the try/catch if not there
v[timestamp] = value
except KeyError:
v.append({ timestamp: value })
history = [
{ 'values': [ {'2017-12-22 10:20': 1}, {'2017-12-22 10:21': 2}, {'2017-12-22 10:22': 3} ] },
]
AddValue(history, 99)
However, I'm getting
AttributeError: 'dict' object has no attribute 'append'**
You associate a key k with a value v in a dictionary d with:
d[k] = v
this works regardless whether there is already a key k present in the dictioanry. In case that happens, the value is "overwritten". We can thus rewrite the for loop to:
for v in arr['values']:
v[timestamp] = value
In case you want to update a dictionary with several keys, you can use .update and pass a dictionary object, or named parameters as keys (and the corresponding values as value). So we can write it as:
for v in arr['values']:
v.update({timestamp: value})
which is semantically the same, but will require more computational effort.
Nevertheless since you need to iterate over a dictionary, you perhaps should reconsider the way you structured the data.
I have a list of dictionaries=
a = [{"ID":1, "VALUE":2},{"ID":2, "VALUE":2},{"ID":3, "VALUE":4},...]
"ID" is a unique identifier for each dictionary. Considering the list is huge, what is the fastest way of checking if a dictionary with a certain "ID" is in the list, and if not append to it? And then update its "VALUE" ("VALUE" will be updated if the dict is already in list, otherwise a certain value will be written)
You'd not use a list. Use a dictionary instead, mapping ids to nested dictionaries:
a = {
1: {'VALUE': 2, 'foo': 'bar'},
42: {'VALUE': 45, 'spam': 'eggs'},
}
Note that you don't need to include the ID key in the nested dictionary; doing so would be redundant.
Now you can simply look up if a key exists:
if someid in a:
a[someid]['VALUE'] = newvalue
I did make the assumption that your ID keys are not necessarily sequential numbers. I also made the assumption you need to store other information besides VALUE; otherwise just a flat dictionary mapping ID to VALUE values would suffice.
A dictionary lets you look up values by key in O(1) time (constant time independent of the size of the dictionary). Lists let you look up elements in constant time too, but only if you know the index.
If you don't and have to scan through the list, you have a O(N) operation, where N is the number of elements. You need to look at each and every dictionary in your list to see if it matches ID, and if ID is not present, that means you have to search from start to finish. A dictionary will still tell you in O(1) time that the key is not there.
If you can, convert to a dictionary as the other answers suggest, but in case you you have reason* to not change the data structure storing your items, here's what you can do:
items = [{"ID":1, "VALUE":2}, {"ID":2, "VALUE":2}, {"ID":3, "VALUE":4}]
def set_value_by_id(id, value):
# Try to find the item, if it exists
for item in items:
if item["ID"] == id:
break
# Make and append the item if it doesn't exist
else: # Here, `else` means "if the loop terminated not via break"
item = {"ID": id}
items.append(id)
# In either case, set the value
item["VALUE"] = value
* Some valid reasons I can think of include preserving the order of items and allowing duplicate items with the same id. For ways to make dictionaries work with those requirements, you might want to take a look at OrderedDict and this answer about duplicate keys.
Convert your list into a dict and then checking for values is much more efficient.
d = dict((item['ID'], item['VALUE']) for item in a)
for new_key, new_value in new_items:
if new_key not in d:
d[new_key] = new_value
Also need to update on key found:
d = dict((item['ID'], item['VALUE']) for item in a)
for new_key, new_value in new_items:
d.setdefault(new_key, 0)
d[new_key] = new_value
Answering the question you asked, without changing the datastructure around, there's no real faster way of looking without a loop and checking every element and doing a dictionary lookup for each one - but you can push the loop down to the Python runtime instead of using Python's for loop.
I haven't tried if it ends up faster though.
a = [{"ID":1, "VALUE":2},{"ID":2, "VALUE":2},{"ID":3, "VALUE":4}]
id = 2
tmp = filter(lambda d: d['ID']==id, a)
# the filter will either return an empty list, or a list of one item.
if not tmp:
tmp = {"ID":id, "VALUE":"default"}
a.append(tmp)
else:
tmp = tmp[0]
# tmp is bound to the found/new dictionary