Python: how to programmatically create a dictionary of arrays [closed] - python
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I have an array a that I want to store every 5 values of in a dictionary, while programmatically creating keys.
For example:
if we have
a=[1,2,3,4,5,6,7,8,9,10]
I want the dictionary to look like this:
d={"first_title":[1,2,3,4,5], "second_title":[6,7,8,9,10]}
Edit: I have thousands of terms so I want to do this in a loop
Dictionary values can be any object, including arrays. So, you just start with an empty dict, and add array values to it.
my_dict = {}
year = 1984
for index in range(0, len(a), 5):
my_dict[year] = a[index: index+5]
year += 1
How about this?
Input:
a = [1,2,3,4,5,6,7,8,9,10]
keys = [1984, 1985, 1986, 1987]
n = 5
Code:
my_dict = {}
for i in range(len(a)//n):
key = str(keys[i]) if i < len(keys) else "after " + str(keys[-1])
my_dict[key] = a[i*n: (i+1)*n]
print(my_dict)
Output:
{'1984': [1, 2, 3, 4, 5], '1985': [6, 7, 8, 9, 10]}
Depending on your use case you could also do something like this:
# Input
a = range(22)
keys = [1984, 1985, 1986] # maybe replace it with range(1984, 2000)
n = 5
# Code
b = a
my_dict = {}
for i in range(min(len(keys), len(a)//n)):
key = keys[min(i, len(keys)-1)]
my_dict[key] = b[:n]
b = b[n:]
my_dict['later'] = b
print(my_dict)
# Output
{
1984: [0, 1, 2, 3, 4],
1985: [5, 6, 7, 8, 9],
1986: [10, 11, 12, 13, 14],
'later': [15, 16, 17, 18, 19, 20, 21]
}
A somewhat general and pythonic solution is the following:
from itertools import izip_longest
def grouper(n, iterable, fillvalue=None):
"""grouper(3, 'ABCDEFG', 'x') --> ABC DEF Gxx"""
args = [iter(iterable)] * n
return izip_longest(fillvalue=fillvalue, *args)
keys = ["first-title", "second-title"]
a = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10]
result = {key: list(group) for key, group in zip(keys, grouper(5, a))}
This uses the grouper recipe (see this answer for a better explanation). A less pythonic solution, is to iterate over the pair of keys and groups using a for loop:
result = {}
for key, group in zip(keys, grouper(5, a)):
result[key] = group
In both cases the output is:
{'first-title': [1, 2, 3, 4, 5], 'second-title': [6, 7, 8, 9, 10]}
Related
Watching a counter, tally total and counting missed counts
I am attempting to create a piece of code that will watch a counter with an output something like: a1 = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30] I want the code to be able to tally the total and tell me how many counts are missed for example if this happened: a1 = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24, 25, 26, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,1,2] I would get a total of 92 still, but get feedback that 8 are missing. I have gotten very close with the following code: Blk_Tot = 0 CBN = 0 LBN = 0 x = 0 y = 0 z = 0 MissedBlocks = 0 for i in range(len(a1)): CBN = a1[i] if CBN - LBN <= 0: if LBN == 30: y = 30 - abs(CBN - LBN) elif LBN < 30: z = 30 - LBN y = 30 - abs(CBN - LBN) + z print(z) Blk_Tot = Blk_Tot + y else: x = CBN - LBN Blk_Tot = Blk_Tot + x if x > 1: MissedBlocks = MissedBlocks - 1 + x LBN = CBN print(Blk_Tot) print(MissedBlocks) If I delete anywhere between 1 and 30 it works perfectly, however if I delete across 30, say 29,30,1,2 it breaks.I don't expect it to be able to miss 30 in a row and still be able to come up with a proper total however. Anyone have any ideas on how this might be achieved? I feel like I am missing an obvious answer :D Sorry I think I was unclear, a1 is a counter coming from an external device that counts from 1 to 30 and then wraps around to 1 again. Each count is actually part of a message to show that the message was received; so say 1 2 4, I know that the 3rd message is missing. What I am trying to do is found out the total that should have been recieved and how many are missing from the count. Update after an idea from the posts below, another method of doing this maybe: Input: 123456 List[1,2,3,4,5,6] 1.Check first input to see which part of the list it is in and start from there (in case we don't start from zero) 2.every time an input is received check if that matches the next value in the array 3.if not then how many steps does it take to find that value
You don't need to keep track if you past the 30 line.
Just compare with the ideal sequence and count the missing numbers.
No knowledge if parts missing at the end.
No knowledge if more than 30 parts are missing in a block.
from itertools import cycle
def idealSeqGen():
for i in cycle(range(1,31)):
yield(i)
def receivedSeqGen():
a1 = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,
5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,
1,2]
for i in a1:
yield(i)
receivedSeq = receivedSeqGen()
idealSeq = idealSeqGen()
missing = 0
ideal = next(idealSeq)
try:
while True:
received = next(receivedSeq)
while received != ideal:
missing += 1
ideal = next(idealSeq)
ideal = next(idealSeq)
except StopIteration:
pass
print (f'There are {missing} items missing')
Edit
The loop part can be a little bit simpler
missing = 0
try:
while True:
ideal = next(idealSeq)
received = next(receivedSeq)
while received != ideal:
missing += 1
ideal = next(idealSeq)
except StopIteration:
pass
print (f'There are {missing} items missing')
In general, if you want to count the number of differences between two lists, you can easily use a dictionary. The other answer would also work, but it is highly inefficient for even slightly larger lists.
def counter(lst):
# create a dictionary with count of each element
d = {}
for i in lst:
if d.get(i, None):
d[i] += 1
else:
d[i] = 1
return d
def compare(d1, d2):
# d1 and d2 are dictionaries
ans = 0
for i in d1.values():
if d2.get(i, None):
# comapares the common values in both lists
ans += abs(d1[i]-d2[i])
d2[i] = 0
else:
#for elements only in the first list
ans += d1[i]
for i in d2.values():
# for elements only in the second list
if d2[i]>0:
ans += d2[i]
return ans
l1 = [...]
l2 = [...]
print(compare(counter(l1), counter(l2)))
New code to check for missing elements from a repeating sequence pattern
Now that I have understood your question more clearly, here's the code. The assumption in this code is the list will always be in ascending order from 1 thru 30 and then repeats again from 1. There can be missing elements between 1 and 30 but the order will always be in ascending order between 1 and 30.
If the source data is as shown in list a1, then the code will result in 8 missing elements.
a1 = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,
5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,
1,2]
a2 = a1.copy()
c = 1
missing = 0
while a2:
if a2[0] == c:
c+=1
a2.pop(0)
elif a2[0] > c:
missing +=1
c+=1
elif a2[0] < c:
missing += 31-c
c = 1
if c == 31: c=1
print (f'There are {missing} items missing in the list')
The output of this will be:
There are 8 items missing in the list
Let me know if this addresses your question
earlier code to compare two lists
You cannot use set as the items are repeated. So you need to sort them and find out how many times each element is in both lists. The difference will give you the missing counts. You may have an element in a1 but not in a2 or vice versa. So finding out the absolute count of missing items will give you the results.
I will update the response with better variables in my next update.
Here's how I did it:
code with comments:
a1 = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30]
a2 = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,
5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,1,2]
#step 1: Find out which list is longer. We will use that as the master list
if len(a1) > len(a2):
master_list = a1.copy()
second_list = a2.copy()
else:
master_list = a2.copy()
second_list = a1.copy()
#step 2: We must sort both master and second list
# so we can compare against each other
master_list.sort()
second_list.sort()
#set the counter to zero
missing = 0
#iterate through the master list and find all values in master against second list
#for each iteration, remove the value in master[0] from both master and second list
#when you have iterated through the full list, you will get an empty master_list
#this will help you to use while statement to iterate until master_list is empty
while master_list:
#pick the first element of master list to search for
x = master_list[0]
#count the number of times master_list[0] is found in both master and second list
a_count = master_list.count(x)
b_count = second_list.count(x)
#absolute difference of both gives you how many are missing from each other
#master may have 4 occurrences and second may have 2 occurrences. abs diff is 2
#master may have 2 occurrences and second may have 5 occurrences. abs diff is 3
missing += abs(a_count - b_count)
#now remove all occurrences of master_list[0] from both master and second list
master_list = [i for i in master_list if i != x]
second_list = [i for i in second_list if i != x]
#iterate until master_list is empty
#you may end up with a few items in second_list that are not found in master list
#add them to the missing items list
#thats your absolute total of all missing items between lists a1 and a2
#if you want to know the difference between the bigger list and shorter one,
#then don't add the missing items from second list
missing += len(second_list)
#now print the count of missig elements between the two lists
print ('Total number of missing elements are:', missing)
The output from this is:
Total number of missing elements are: 7
If you want to find out which elements are missing, then you need to add a few more lines of code.
In the above example, elements 27,28,29,30, 4, 5 are missing from a2 and 31 from a1. So total number of missing elements is 7.
code without comments:
a1 = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30]
a2 = [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,
5, 6, 7, 8, 9, 10, 11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,1,2]
if len(a1) > len(a2):
master_list = a1.copy()
second_list = a2.copy()
else:
master_list = a2.copy()
second_list = a1.copy()
master_list.sort()
second_list.sort()
missing = 0
while master_list:
x = master_list[0]
a_count = master_list.count(x)
b_count = second_list.count(x)
missing += abs(a_count - b_count)
master_list = [i for i in master_list if i != x]
second_list = [i for i in second_list if i != x]
missing += len(second_list)
print ('Total number of missing elements are:', missing)
How to account for ties in returning top n values?
I have a dictionary of top values that I have collected from Twitter, here's a small example:
{'aberdeen': 5,
'amsterdam': 6,
'amsterdam?fussa': 6,
'anchorage': 12,
'andalucia?granada': 5,
'ann arbor': 6,
'aral': 6,
'arlington': 6,
'asia?london': 6,
I have two functions of returning the top 'n' results:
def top_items(item_counts, n=3):
counts = Counter(item_counts)
most_common = counts.most_common(n)
for item in zip(*most_common):
return list(item)
and
def top_items2(item_counts, n=3):
top_count = sorted(item_counts.items(), key=lambda x: x[1], reverse=True)[:n]
return [x[0] for x in top_count]
Both retrieve the top n results. However, I'm having issues trying to account for ties. In this example, the top cities are ['los angeles', 'chicago', 'denver'], however denver and nyc both have a count of 8, but it returns denver because its first alphabetically.
How could I edit my code to return the keys of my dictionary of the top n values including ties?
Thanks!
def top_items(item_counts, n=3):
counts = Counter(item_counts)
top_n_values = set([x[1] for x in c.most_common(n)])
output = {k:v for k,v in d.items() if v in top_n_values}
return sorted(output, key=output.get, reverse=True)
d = {'aberdeen': 5,
'amsterdam': 6,
'amsterdam?fussa': 6,
'anchorage': 12,
'andalucia?granada': 5,
'ann arbor': 6,
'aral': 6,
'arlington': 6,
'asia?london': 6,}
top_items(d)
Output
['anchorage',
'amsterdam',
'amsterdam?fussa',
'ann arbor',
'aral',
'arlington',
'asia?london']
You can first find the top values by sorting and compare them to all the values:
di = {'aberdeen': 5,
'amsterdam': 6,
'amsterdam?fussa': 6,
'anchorage': 12,
'andalucia?granada': 5,
'ann arbor': 6,
'aral': 6,
'arlington': 6,
'asia?london': 6
}
def getTopResults(di: dict, n):
final = {}
topValues = sorted(di.values())[:n+1]
for k, v in di.items():
if v in topValues:
final[k] = v
return final
print(getTopResults(di, 2)) # {'aberdeen': 5, 'amsterdam': 6, 'amsterdam?fussa': 6, 'andalucia?granada': 5, 'ann arbor': 6, 'aral': 6, 'arlington': 6, 'asia?london': 6}
This function does same in a recursive way, assuming input list is already sorted in asc or desc order: get_next(3, []) get_next(3, [1,2,3,4,5]) get_next(3, [1,1,1,1,1,2,2,3,4,5,6]) get_next(3, [1,2,2,2,2,3,4,5]) def get_next(n,xs, last=None): if len(xs)>0: head, *tail=xs if head == last: n+=1 # need to go 1 further if tie if n>0: return [head] + get_next(n-1, tail, head) return []
Construct a superset from pandas groupby operation result
name_region bahia [10, 11, 12, 1, 2, 3, 4] distrito_federal [9, 10, 11, 12, 1, 2, 3, 4] goias [9, 10, 11, 12, 1, 2, 3, 4] maranhao [10, 11, 12, 1, 2, 3, 4] mato_grosso [9, 10, 11, 12, 1, 2, 3, 4] mato_grosso_do_sul [8, 9, 10, 11, 12, 1, 2, 3] I have a pandas series above, obtained from a groupby operation. The 2nd column represents months of the year. How do I construct a superset of months i.e. [8, 9, 10, 11, 12, 1, 2, 3, 4] since that represents all possible months present in the dataset --NOTE: I do want to preserve order
You can use the itertools recipe unique_everseen (which preserves order) like so:
>>> [i for i in unique_everseen([z for z in y['months'] for x,y in df.iterrows()])]
[9, 10, 11, 12, 1, 2, 3, 4]
Definition of unique_everseen:
import itertools as it
def unique_everseen(iterable, key=None):
"List unique elements, preserving order. Remember all elements ever seen."
# unique_everseen('AAAABBBCCDAABBB') --> A B C D
# unique_everseen('ABBCcAD', str.lower) --> A B C D
seen = set()
seen_add = seen.add
if key is None:
for element in it.ifilterfalse(seen.__contains__, iterable):
seen_add(element)
yield element
else:
for element in iterable:
k = key(element)
if k not in seen:
seen_add(k)
yield element
I seem to have misinterpreted the data structure in the question, but as it might be useful for similar cases, I will keep this answer here for future reference.
You can use numpy's unique function.
import pandas as pd
import numpy as np
df = pd.DataFrame({"x": [1,3,5], "y": [3,4,5]})
print np.unique(df) # prints [1 3 4 5]
I don't know if there is a way to do this more cleanly in Pandas so if anyone else knows please answer... Looking at the types this seems like a time for folding over that column. I didn't see a fold operation in pandas, so maybe just a for loop that accumulates.. i.e. all_months = [] for row in df.iterrows(): months = row['months'] all_months += [e for e in months if not e in all_months] on second thought.. would use set instead of complicated for comprehension all_months = set() for row in df.iterrows(): months = set(row['months']) all_months = all_months.union(months) hmm just saw the other guys answer, haven't tested it.. but it looks better! choose that one :). Posting this just in case it helps someone...
Count number of complementary pairs in array [closed]
Closed. This question needs details or clarity. It is not currently accepting answers. Want to improve this question? Add details and clarify the problem by editing this post. Closed 6 years ago. Improve this question I am rookie with python and have question about counting pairs in array that complement to some certain number. Example Given an integer array A find all the complementary pairs in A that sums up to a provided value K, so for example given the array [2, 45, 7, 3, 5, 1,8,9] and an integer K = 10 the result must be 3 because the input array contains [9, 1] [7, 3] [8, 2] Here is the code finding complementary pairs def find_pairs(x, sum): s = set(L) for i in L: diff = 10-i s.remove(i) if diff in s: print i, diff L = [2, 45, 7, 3, 5, 1, 8, 9] sum = 10 find_pairs(L, sum) How can I add count command at the end?
Use a simple counter variable def find_pairs(list_, sum_): s = set(list_) pairs = 0 for i in list_: diff = sum_ - i s.remove(i) if diff in s: pairs += 1 return pairs Or you can use a helper list for storing pairs if you also want to know the pairs def find_pairs(list_, sum_): s = set(list_) pairs = [] for i in list_: diff = sum_ - i s.remove(i) if diff in s: pairs.append([i, diff]) return pairs L = [2, 45, 7, 3, 5, 1, 8, 9] pairs = find_pairs(L, 10) print pairs >>> [[2, 8], [7, 3], [1, 9]] print len(pairs) >>> 3
Using numpy: A = np.asarray(A) np.count_nonzero((A[None, :] + A[:, None]) == K) // 2
Split list into dictionary
I am trying to split my list into a dictionary and I wrote a function for it. It takes in a list and gets the list length. If the length of the list is 253 , it creates a dictionary with 26 keys - this is calculated by rounding up the number to a higher 10 (250 will create 25 keys, 251-259 will create 26 keys). Each key will store a chuck of the original list (as of now I am storing 10 elements per list). I feel def limit_files(file_list, at_a_time=10): l = file_list n = len(l) d, r = divmod(n, at_a_time) num_keys = d + 1 if r else d slice = n // num_keys vals = (l[i:i+slice] for i in range(0, n, slice+1)) dct = dict(zip(range(1,num_keys+1),vals)) return (dct) I just wanted to know if there is way to improve the code
You can use itertools.izip_longest to group items in your list as equally divided chunks.
import itertools
def limit_files(file_list, at_a_time=10):
d, r = divmod(len(file_list), at_a_time)
num_keys = d + 1 if r > 0 else d
chunks = itertools.izip_longest(*([iter(x)] * at_a_time))
return dict((x, y) for x, y in enumerate(chunks))
Note that, it will be padding with None values to fill out the last chunk.
>>> limit_files([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12])
{0: (1, 2, 3, 4, 5, 6, 7, 8, 9, 10),
1: (11, 12, None, None, None, None, None, None, None, None)}
Alternatively, without using padding values, you can just iterate through the range and call itertools.islice until the iterator exhausts as follows:
def limit_files(file_list, at_a_time=10):
d, r = divmod(len(file_list), at_a_time)
num_keys = d + 1 if r > 0 else d
iterator = iter(file_list)
dictionary = {}
for chunk_id in xrange(num_keys):
chunk = list(itertools.islice(iterator, at_a_time))
if not chunk:
break
dictionary[chunk_id] = chunk
return dictionary
>>> limit_files([1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12])
{0: [1, 2, 3, 4, 5, 6, 7, 8, 9, 10], 1: [11, 12]}