I want to create a dictionary of dictionaries with a list of numbers in the last one. I'm adding the numbers 1 - 254 as strings to a dictionary so the number is the key and the value will be another dictionary with the keys as 1 - 254 as strings again and finally a list as the value with the numbers 1 - 254 as strings. I cant figure out how to do this. There will be a total of around 16 million elements when its all been executed. I can't figure this out.
def fill_ip_list(dict_list):
""" Create an IP list of lists approximately 15 million numbers from 1 - 255, three lists deep. """
ips = []
count = 0
print("Populating the IP store. Please wait...")
for octet_one in dict_list:
ips.append({str(octet_one["prefix"]): {}})
for octet_two in octet_one:
for x in range(1, 254 + 1):
octet_one.update({x: {}})
print(ips)
Output:
[{'1': {}}, {'2': {}}, {'3': {}}, {'4': {}}, {'5': {}}, {'7': {}}...
Expected output:
[{'1': {'1': '1'}, {'2': '2'}, {'3': '3'}...}]
[{'2': {'1': '1'}, {'2': '2'}, {'3': '3'}...}]
[{'3': {'1': '1'}, {'2': '2'}, {'3': '3'}...}]
[{'4': {'1': '1'}, {'2': '2'}, {'3': '3'}...}]
.
.
.
Related
This question already has answers here:
How to copy a dictionary and only edit the copy
(23 answers)
Closed 1 year ago.
I meet a problem when trying to continuously add modified dictionaries into the same list. I simplified what I want to do below:
a = {"1":"1"}
b = [a]
for i in range (2,5):
a["1"] = i
b.append(a)
print(b)
This will give me
[{'1': 4}, {'1': 4}, {'1': 4}, {'1': 4}]
But I want b as
[{'1': 1}, {'1': 2}, {'1': 3}, {'1': 4}]
How should I do this?
Do not reuse the "a" dict inside the loop
and create a new small dict each time it is needed
You can do like this:
a = {"1":"1"}
b = [a]
for i in range (2,5):
b.append({"1":i})
print(b)
the result is :
[{'1': '1'}, {'1': 2}, {'1': 3}, {'1': 4}]
Dict 1 :
{'result2': [{'2': '22'}, {'22': '222'}], 'result1': [{'1': '11'}, {'11': '111'}]}
Dict 2:
{'result2': [{'two': 'twentytwo'}, {'22': '222'}], 'result1': [{'one': 'eleven'}, {'11': '111'}]}
I want to merge them based on keys result1 and result 2 and preserve the same format.
Expected Output
{"result1":[{"1":"11"},{"one":"eleven"},{"11":"111"}]},{"result2":[{"2":"22"},{"two":"twentytwo"},{"22":"222"}]}
And the keys of both dict may not necessarily have same index.
It's always good to add desired format for output. It likes you want to list of dict which contains data from both with respect to result1 & result 2.
Given -
a = [{"result1":[{"1":"11"},{"11":"111"}]},{"result2":[{"2":"22"},{"22":"222"}]}]
b = [{"result1":[{"one":"eleven"},{"11":"111"}]},{"result2":[{"two":"twentytwo"},{"22":"222"}]}]
Here is code-
desired_op = []
for index, item in enumerate(a):
inner_item = {}
for key, value in item.items():
#print (key, value + b[index][key])
inner_item[key] = value + b[index][key]
desired_op.append(inner_item)
print (desired_op)
[{'result1': [{'1': '11'}, {'11': '111'}, {'one': 'eleven'}, {'11': '111'}]}, {'result2': [{'2': '22'}, {'22': '222'}, {'two': 'twentytwo'}, {'22': '222'}]}]
If you need unique items, it will be bit complex-
desired_op = []
for index, item in enumerate(a):
inner_item = {}
for key, value in item.items():
#print (key, value + b[index][key])
inner_item[key] = list({list(litem.keys())[0]:litem for litem in value + b[index][key]}.values())
desired_op.append(inner_item)
print (desired_op)
[{'result1': [{'1': '11'}, {'11': '111'}, {'one': 'eleven'}]}, {'result2': [{'2': '22'}, {'22': '222'}, {'two': 'twentytwo'}]}]
follow up on comment
simple append can work, but for this you must be sure that first index from second list should contain result1 & second index should contain result2 as dict keys.
ex. (but this won't remove duplicates)
a[0]['result1'].extend(b[0]['result1'])
a[1]['result2'].extend(b[1]['result2'])
print(a)
[{'result1': [{'1': '11'}, {'11': '111'}, {'one': 'eleven'}, {'11': '111'}]}, {'result2': [{'2': '22'}, {'22': '222'}, {'two': 'twentytwo'}, {'22': '222'}]}]
if you aren't sure about index position in second list, this will work-
keys_mapping = {list(item.keys())[0]:index for index, item in enumerate(a)}
print (keys_mapping)
{'result1': 0, 'result2': 1}
for item in b:
key = list(item.keys())[0]
a[keys_mapping[key]][key].extend(item[key])
print (a)
[{'result1': [{'1': '11'}, {'11': '111'}, [{'one': 'eleven'}, {'11': '111'}]]}, {'result2': [{'2': '22'}, {'22': '222'}, [{'two': 'twentytwo'}, {'22': '222'}]]}]
You could separate this in two phases:
merge the dictionaries by appending the lists
remove duplicates from the resulting merged dictionary
...
dict1={'result2': [{'2': '22'}, {'22': '222'}],
'result1': [{'1': '11'}, {'11': '111'}]}
dict2={'result2': [{'two': 'twentytwo'}, {'22': '222'}],
'result1': [{'one': 'eleven'}, {'11': '111'}]}
# merge phase
dict3 = {k:dict1.get(k,[])+dict2.get(k,[]) for k in {*dict1,*dict2}}
# uniqueness phase
dict3 = {k:[d for i,d in enumerate(v) if v.index(d)==i]
for k,v in dict3.items()}
print(dict3)
{'result2': [{'2': '22'}, {'22': '222'}, {'two': 'twentytwo'}],
'result1': [{'1': '11'}, {'11': '111'}, {'one': 'eleven'}]}
Note that you could combine the two phases in one large dictionary comprehension:
dict3 = {k: [d for i,d in enumerate(v) if v.index(d)==i]
for k in {*dict1,*dict2}
for v in [dict1.get(k,[])+dict2.get(k,[])] }
If dict1 and dict2 are guaranteed to have the same keys, then the whole process can be performed more concisely:
dict3 = {k:v+[w for w in dict2[k] if w not in v] for k,v in dict1.items()}
I have the following problem. I have a nested dictionary like the following
A = {'A':{'STATES':['0','1','2']}, 'B':{'STATES':['10','20']}}
What I want to build eventually is a nested dictionary with all the possible combinations of STATES. So for a known number of keys in A is trivial as this
_dict = {}
for s in A['A']['STATES']:
if s not in _dict:
_dict[s] = {}
for s1 in A['B']['STATES']:
_dict[s][s1] = 0
This gives
{'0': {'10': 0, '20': 0}, '1': {'10': 0, '20': 0}, '2': {'10': 0, '20': 0}}
which is what I want. However I do not know the number of keys in A beforehand. What it would be an efficient solution to to the same with an arbitrary number of elements in A?
EDIT
For instance with three elements I would have
{'0': {'10': {'100':0}, '20': {'100':0}, '1': {'10': {'100':0}, '20': {'100':0}, '2': {'10': {'100':0}, '20': {'100':0}}
This problem is a little complex, but it can be splitted up in three parts:
Parse all the values, mapping a list to every valid key.
Get the list of all the combinations in the order of dictionary insertion.
Translate the list of tuples into a nested dictionary, looping over the values inside the tuple itself - because we don't know its length.
import itertools
A = {'A':{'STATES':['0','1','2']}, 'B':{'STATES':['10','20']}, 'C':{'STATES':['100']}}
# 1. get your dictionary A, but reduced, so that
# for every key you have a list of states if the key "STATES" exists
Ared = {k: A[k]["STATES"] for k in A if A[k].get("STATES")}
print(Ared) # {'A': ['0', '1', '2'], 'B': ['10', '20'], 'C': ['100']}
# 2. get all the combinations
combs = list(itertools.product(*Ared.values()))
print(combs) # [('0', '10', '100'), ('0', '20', '100'), ('1', '10', '100'), ('1', '20', '100'), ('2', '10', '100'), ('2', '20', '100')]
# 3. translate them into a nested dictionary
d = dict()
for comb in combs:
old_dict = d
for i, key in enumerate(comb):
if i == len(comb) - 1:
old_dict[key] = 0
elif not old_dict.get(key):
old_dict[key] = {}
old_dict = old_dict[key]
print(d) # {'0': {'10': {'100': 0}, '20': {'100': 0}}, '1': {'10': {'100': 0}, '20': {'100': 0}}, '2': {'10': {'100': 0}, '20': {'100': 0}}}
You can use recursion:
A = {'A':{'STATES':['0','1','2']}, 'B':{'STATES':['10','20']}}
def combos(d):
return 0 if not d else {i:combos(d[1:]) for i in d[0]}
print(combos([j['STATES'] for j in A.values()]))
Output:
{'0': {'10': 0, '20': 0}, '1': {'10': 0, '20': 0}, '2': {'10': 0, '20': 0}}
With more than two keys:
A = {'A':{'STATES':['0','1','2']}, 'B':{'STATES':['10','20']}, 'C':{'STATES':['100']}}
print(combos([j['STATES'] for j in A.values()]))
Output:
{'0': {'10': {'100': 0}, '20': {'100': 0}}, '1': {'10': {'100': 0}, '20': {'100': 0}}, '2': {'10': {'100': 0}, '20': {'100': 0}}}
I have string a='51545'
I am finding index of the each char in the string like this
modchar=[{i:a.index(i)} for i in a ]
#modchar=[{'5': 0}, {'1': 1}, {'5': 0}, {'4': 3}, {'5': 0}]
but i need to get it as
#modchar=[{'5': 0}, {'1': 1}, {'5': 2}, {'4': 3}, {'5': 4}]
How can we achieve this?
In this case, you probably want to enumerate the string creating the dictionaries as you go:
[{c: i} for i, c in enumerate(a)]
Note that as a side bonus, this happens in O(n) time opposed to your original solution which is O(n^2)
Try this:
a='51545'
obj = []
for i in range(len(a)):
obj.append({a[i]: i})
This runs as:
>>> a='51545'
>>> obj = []
>>> for i in range(len(a)):
... obj.append({a[i]: i})
...
>>> obj
[{'5': 0}, {'1': 1}, {'5': 2}, {'4': 3}, {'5': 4}]
>>>
You can do list comprehension using enumerate:
[{value: index} for value, index in enumerate(a)]
Which runs as:
>>> [{value: index} for index, value in enumerate(a)]
[{'5': 0}, {'1': 1}, {'5': 2}, {'4': 3}, {'5': 4}]
>>>
Or, you can use a basic list comprehension:
[{a[index]: index} for index in range(len(a))]
Which runs as:
>>> [{a[index]: index} for index in range(len(a))]
[{'5': 0}, {'1': 1}, {'5': 2}, {'4': 3}, {'5': 4}]
>>>
enumerate is basically a combination of using a for loop to get the index, and then accessing the list:
>>> arr = [5, 8, 2, 4]
>>> for index, value in enumerate(arr):
... print index, value
...
0 5
1 8
2 2
3 4
>>>
Im trying to parse a CSV using python and would like to be able to index items in a row so they can be accessed using row[0], row[1] and so on.
So far this is my code:
def get_bitstats():
url = 'http://bitcoincharts.com/t/trades.csv?symbol=mtgoxUSD'
data = urllib.urlopen(url).read()
dictReader = csv.DictReader(data)
obj = BitData()
for row in dictReader:
obj.datetime = datetime.datetime.fromtimestamp(int(row['0'])/1000000)
q = db.Query(BitData).filter('datetime', obj.datetime)
if q != None:
raise ValueError(obj.datetime + 'is already in database')
else:
obj.price = row['1']
obj.amount = row['2']
obj.put()
This returns KeyError: '0' and I have no idea how to set it up. I did input this into an interactive shell and when running
for row in dictReader:
print row
I get this as the output:
{'1': '3'}
{'1': '6'}
{'1': '2'}
{'1': '6'}
{'1': '9'}
{'1': '8'}
{'1': '6'}
{'1': '4'}
{'1': '4'}
{'1': '', None: ['']}
{'1': '4'}
{'1': '2'}
{'1': '.'}
{'1': '0'}
{'1': '5'}
{'1': '7'}
{'1': '1'}
{'1': '6'}
{'1': '0'}
{'1': '0'}
{'1': '0'}
{'1': '0'}
{'1': '0'}
{'1': '0'}
{'1': '0'}
{'1': '', None: ['']}
{'1': '0'}
{'1': '.'}
{'1': '0'}
{'1': '1'}
{'1': '0'}
{'1': '0'}
{'1': '5'}
{'1': '4'}
{'1': '2'}
{'1': '5'}
{'1': '0'}
{'1': '0'}
{'1': '0'}
{'1': '0'}
{'1': '1'}
{'1': '3'}
{'1': '6'}
{'1': '2'}
{'1': '6'}
{'1': '9'}
{'1': '8'}
{'1': '6'}
{'1': '4'}
{'1': '4'}
and on and on for thousands and thousands of lines. ( as Im sure the CSV is thousands of digits)
Why is my CSV printing this way and is there anyway to separate a row into a list of 3 ints such as [130534543, 47.00009, 23001.9000]
EDIT:
as the Answer states I was using the wrong csv function in my code above but even though fixing it gave me a list the list itself was in the same format as the dict such that:
['1']
['2']
['1']
['3']
['8']
['3']
['5']
.
.
.
It turns out I also had to remove the .read() from data = urllib.urlopen(url).read().
csv.reader will return each row as a list
reader = csv.reader(data)
for line_list in reader:
pass
# line_list is a list of the data contained in a row so you can access line_list[0]