Related
I want to add multiple values to a specific key in a python dictionary. How can I do that?
a = {}
a["abc"] = 1
a["abc"] = 2
This will replace the value of a["abc"] from 1 to 2.
What I want instead is for a["abc"] to have multiple values (both 1 and 2).
Make the value a list, e.g.
a["abc"] = [1, 2, "bob"]
UPDATE:
There are a couple of ways to add values to key, and to create a list if one isn't already there. I'll show one such method in little steps.
key = "somekey"
a.setdefault(key, [])
a[key].append(1)
Results:
>>> a
{'somekey': [1]}
Next, try:
key = "somekey"
a.setdefault(key, [])
a[key].append(2)
Results:
>>> a
{'somekey': [1, 2]}
The magic of setdefault is that it initializes the value for that key if that key is not defined. Now, noting that setdefault returns the value, you can combine these into a single line:
a.setdefault("somekey", []).append("bob")
Results:
>>> a
{'somekey': [1, 2, 'bob']}
You should look at the dict methods, in particular the get() method, and do some experiments to get comfortable with this.
How about
a["abc"] = [1, 2]
This will result in:
>>> a
{'abc': [1, 2]}
Is that what you were looking for?
Append list elements
If the dict values need to be extended by another list, extend() method of lists may be useful.
a = {}
a.setdefault('abc', []).append(1) # {'abc': [1]}
a.setdefault('abc', []).extend([2, 3]) # a is now {'abc': [1, 2, 3]}
This can be especially useful in a loop where values need to be appended or extended depending on datatype.
a = {}
some_key = 'abc'
for v in [1, 2, 3, [2, 4]]:
if isinstance(v, list):
a.setdefault(some_key, []).extend(v)
else:
a.setdefault(some_key, []).append(v)
a
# {'abc': [1, 2, 3, 2, 4]}
Append list elements without duplicates
If there's a dictionary such as a = {'abc': [1, 2, 3]} and it needs to be extended by [2, 4] without duplicates, checking for duplicates (via in operator) should do the trick. The magic of get() method is that a default value can be set (in this case empty set ([])) in case a key doesn't exist in a, so that the membership test doesn't error out.
a = {some_key: [1, 2, 3]}
for v in [2, 4]:
if v not in a.get(some_key, []):
a.setdefault(some_key, []).append(v)
a
# {'abc': [1, 2, 3, 4]}
This question already has answers here:
How do I remove duplicates from a list, while preserving order?
(31 answers)
Closed 7 years ago.
I have a dict containing lists and need a fast way to dedupe the lists.
I know how to dedupe a list in isolation using the set() function, but in this case I want a fast way of iterating through the dict, deduping each list on the way.
hello = {'test1':[2,3,4,2,2,5,6], 'test2':[5,5,8,4,3,3,8,9]}
I'd like it to appear like;
hello = {'test1':[2,3,4,5,6], 'test2':[5,8,4,3,9]}
Though I don't necessarily need to have the original order of the lists preserved.
I've tried using a set like this, but it's not quite correct (it's not iterating properly and I'm losing the first key)
for key, value in hello.items(): goodbye = {key: set(value)}
>>> goodbye
{'test2': set([8, 9, 3, 4, 5])}
EDIT: Following PM 2Ring's comment below, I'm now populating the dict differently to avoid duplicates in the first place. Previously I was using lists, but using sets prevents dupes to be appended by default;
>>> my_numbers = {}
>>> my_numbers['first'] = [1,2,2,2,6,5]
>>> from collections import defaultdict
>>> final_list = defaultdict(set)
>>> for n in my_numbers['first']: final_list['test_first'].add(n)
...
>>> final_list['test_first']
set([1, 2, 5, 6])
As you can see, the final output is a deduped set, as required.
It's not iterating wrong, you're just assigning goodbye as a new dict each time. You need to assign as an empty dict then assign the values to keys in each iteration.
goodbye = {}
for key, value in hello.items(): goodbye[key] = set(value)
>>> goodbye
{'test1': set([2, 3, 4, 5, 6]), 'test2': set([8, 9, 3, 4, 5])}
Also since sets don't preserve order, if you do want to preserve order it's best to make a simple iterating function that will return a new list that skips over already added values.
def uniqueList(li):
newList = []
for x in li:
if x not in newList:
newList.append(x)
return newList
goodbye = {}
for key, value in hello.items(): goodbye[key] = uniqueList(value)
>>> goodbye
{'test1': [2, 3, 4, 5, 6], 'test2': [5, 8, 4, 3, 9]}
You can use a list comprehension with a deduplicate function that preserves the order:
def deduplicate(seq):
seen = set()
seen_add = seen.add
return [ x for x in seq if not (x in seen or seen_add(x))]
{key: deduplicate(value) for key, value in hello.items()}
>>>hello = {'test1':[2,3,4,2,2,5,6], 'test2':[5,5,8,4,3,3,8,9]}
>>>for key,value in hello.iteritems():
hello[key] = list(set(value))
>>>hello
{'test1': [2, 3, 4, 5, 6], 'test2': [8, 9, 3, 4, 5]}
This is a more verbose way of doing it, which preserves order and works in all Python versions:
for key in hello:
s = set()
l = []
for subval in hello[key]:
if subval not in s:
l.append(subval)
s.add(subval)
hello[key] = l
my_list = [1,2,2,2,3,4,5,6,7,7,7,7,7,8,9,10]
seen = set()
print list(filter(lambda x:x not in seen and not seen.add(x),my_list))
I need help writing a function that will take a single list and return a different list where every element in the list is in its own original list.
I know that I'll have to iterate through the original list that I pass through and then append the value depending on whether or not the value is already in my list or create a sublist and add that sublist to the final list.
an example would be:
input:[1, 2, 2, 2, 3, 1, 1, 3]
Output:[[1,1,1], [2,2,2], [3,3]]
I'd do this in two steps:
>>> import collections
>>> inputs = [1, 2, 2, 2, 3, 1, 1, 3]
>>> counts = collections.Counter(inputs)
>>> counts
Counter({1: 3, 2: 3, 3: 2})
>>> outputs = [[key] * count for key, count in counts.items()]
>>> outputs
[[1, 1, 1], [2, 2, 2], [3, 3]]
(The fact that these happen to be in sorted numerical order, and also in the order of first appearance, is just a coincidence here. Counters, like normal dictionaries, store their keys in arbitrary order, and you should assume that [[3, 3], [1, 1, 1], [2, 2, 2]] would be just as possible a result. If that's not acceptable, you need a bit more work.)
So, how does it work?
The first step creates a Counter, which is just a special subclass of dict made for counting occurrences of each key. One of the many nifty things about it is that you can just pass it any iterable (like a list) and it will count up how many times each element appears. It's a trivial one-liner, it's obvious and readable once you know how Counter works, and it's even about as efficient as anything could possibly be.*
But that isn't the output format you wanted. How do we get that? Well, we have to get back from 1: 3 (meaning "3 copies of 1") to [1, 1, 1]). You can write that as [key] * count.** And the rest is just a bog-standard list comprehension.
If you look at the docs for the collections module, they start with a link to the source. Many modules in the stdlib are like this, because they're meant to serve as source code for learning from as well as usable code. So, you should be able to figure out how the Counter constructor works. (It's basically just calling that _count_elements function.) Since that's the only part of Counter you're actually using beyond a basic dict, you could just write that part yourself. (But really, once you've understood how it works, there's no good reason not to use it, right?)
* For each element, it's just doing a hash table lookup (and insert if needed) and a += 1. And in CPython, it all happens in reasonably-optimized C.
** Note that we don't have to worry about whether to use [key] * count vs. [key for _ in range(count)] here, because the values have to be immutable, or at least of an "equality is as good as identity" type, or they wouldn't be usable as keys.
The most time efficient would be to use a dictionary:
collector = {}
for elem in inputlist:
collector.setdefault(elem, []).append(elem)
output = collector.values()
The other, more costly option is to sort, then group using itertools.groupby():
from itertools import groupby
output = [list(g) for k, g in groupby(sorted(inputlist))]
Demo:
>>> inputlist = [1, 2, 2, 2, 3, 1, 1, 3]
>>> collector = {}
>>> for elem in inputlist:
... collector.setdefault(elem, []).append(elem)
...
>>> collector.values()
[[1, 1, 1], [2, 2, 2], [3, 3]]
>>> from itertools import groupby
>>> [list(g) for k, g in groupby(sorted(inputlist))]
[[1, 1, 1], [2, 2, 2], [3, 3]]
What about this, as you said you wanted a function:
def makeList(user_list):
user_list.sort()
x = user_list[0]
output = [[]]
for i in user_list:
if i == x:
output[-1].append(i)
else:
output.append([i])
x = i
return output
>>> print makeList([1, 2, 2, 2, 3, 1, 1, 3])
[[1, 1, 1], [2, 2, 2], [3, 3]]
I have a dict like this:
dic = {'01':[1,2], '02':[1], '03':[2,3]}
what I want to achieve is a new dict, its keys are combinations of the keys (group in 2 only), and without duplicate values.
in this simple example, the output will be:
newDic = {'0102':[1,2], '0103':[1,2,3],'0203':[1,2,3]}
thanks a bunch!!
You can use a itertools.combinations to get the different combo's of keys. and then use set to get unique values of the list. Put it all into a dictionary-comprehension like this:
>>> dic = {'01':[1,2], '02':[1], '03':[2,3]}
>>> import itertools as IT
>>> {a+b: list(set(dic[a]+dic[b])) for a,b in IT.combinations(dic, 2)}
{'0203': [1, 2, 3], '0301': [1, 2, 3], '0201': [1, 2]}
You can also use join and sorted to have the keys the way you want them:
>>> {''.join(sorted([a,b])): list(set(dic[a]+dic[b])) for a,b in IT.combinations(dic, 2)}
{'0203': [1, 2, 3], '0103': [1, 2, 3], '0102': [1, 2]}
newDic = { a+b : list(set(dic[a] + dic[b])) for a in dic for b in dic if b>a }
I was wondering how to sort like values in a list, and then break like values into a sub-list.
For example: I would want a function that probably does something like
def sort_by_like_values(list):
#python magic
>>>list=[2,2,3,4,4,10]
>>>[[2,2],[3],[4,4],[10]]
OR
>>>[2,2],[3],[4,4],[10]
I read up on the sorted api and it works well for sorting things within their own list, but doesn't break lists up into sub-lists. What module would help me out here?
Use groupby from the itertools module.
from itertools import groupby
L = [2, 2, 3, 4, 4, 10]
L.sort()
for key, iterator in groupby(L):
print key, list(iterator)
Result:
2 [2, 2]
3 [3]
4 [4, 4]
10 [10]
A couple of things to be aware of: groupby needs the data it works on to be sorted by the same key you wish to group by, or it won't work. Also, the iterator needs to be consumed before continuing to the next group, so make sure you store list(iterator) to another list or something. One-liner giving you the result you want:
>>> [list(it) for key, it in groupby(sorted(L))]
[[2, 2], [3], [4, 4], [10]]
Check the itertools module, it has the useful groupby function:
import itertools as i
for k,g in i.groupby(sorted([2,2,3,4,4,10])):
print list(g)
....
[2, 2]
[3]
[4, 4]
[10]
You should be able to modify this to get the values in a list.
As everyone else has suggested itertools.groupby (which would be my first choice) - it's also possible with collections.Counter to obtain key and frequency, sort by the key, then expand back out freq times.
from itertools import repeat
from collections import Counter
grouped = [list(repeat(key, freq)) for key, freq in sorted(Counter(L).iteritems())]
itertools.groupby() with a list comprehension works fine.
In [20]: a = [1, 1, 2, 3, 3, 4, 5, 5, 5, 6]
In [21]: [ list(subgroup) for key, subgroup in itertools.groupby(sorted(a)) ]
Out[21]: [[1, 1], [2], [3, 3], [4], [5, 5, 5], [6]]
Note that groupby() returns a list of iterators, and you have to consume these iterators in order. As per the docs:
The returned group is itself an iterator that shares the underlying iterable with groupby(). Because the source is shared, when the groupby() object is advanced, the previous group is no longer visible. So, if that data is needed later, it should be stored as a list:
If you do not wish to use itertools and can wrap your head around list comprehensions, this should also do the trick :
def group(a):
a = sorted(a)
d = [0] + [x+1 for x in range(len(a)-1) if a[x]!=a[x+1]] + [len(a)]
return [a[(d[x]):(d[x+1])] for x in range(len(d)-1)]
where ais your list