I have a set() with terms like 'A' 'B' 'C'. I want a 2-d associative array so that i can perform an operation like d['A']['B'] += 1 . What is the pythonic way of doing this, I was thinking a dicts of dicts. Is there a better way.
There are two obvious solutions: One, use defaultdict to nest a dict automatically inside another dict
>>> d = collections.defaultdict(dict)
>>> d['a']['b'] = 'abc'
>>> d
defaultdict(<type 'dict'>, {'a': {'b': 'abc'}})
>>>
The other is to just use tuples for your dict keys:
>>> d = {}
>>> d['a','b'] = 'abc'
>>> d
{('a', 'b'): 'abc'}
>>>
To get the += behavior, substitute a defaultdict(int) for the dicts above:
>>> d = collections.defaultdict(lambda:collections.defaultdict(int))
>>> d['a']['b'] += 1
>>> d
defaultdict(<function <lambda> at 0x18d31b8>, {'a': defaultdict(<type 'int'>, {'b': 1})})
>>>
>>> d = collections.defaultdict(int)
>>> d['a','b'] += 1
>>> d
defaultdict(<type 'int'>, {('a', 'b'): 1})
>>>
A dict of dict is one way.
An alternative is to simply use a tuple:
d[('A','B')] += 1
As pointed out by TokenMacGuy, the parentheses are optional:
d['A','B'] += 1
Depending on your code, this might simplify things.
Is there any reason not to use a dict of dicts? It does what you want (though note that there's no such thing as ++ in Python), after all.
There's nothing stylistically poor or non-Pythonic about using a dict of dicts.
Related
How can I turn a list of dicts like [{'a':1}, {'b':2}, {'c':1}, {'d':2}], into a single dict like {'a':1, 'b':2, 'c':1, 'd':2}?
Answers here will overwrite keys that match between two of the input dicts, because a dict cannot have duplicate keys. If you want to collect multiple values from matching keys, see How to merge dicts, collecting values from matching keys?.
This works for dictionaries of any length:
>>> result = {}
>>> for d in L:
... result.update(d)
...
>>> result
{'a':1,'c':1,'b':2,'d':2}
As a comprehension:
# Python >= 2.7
{k: v for d in L for k, v in d.items()}
# Python < 2.7
dict(pair for d in L for pair in d.items())
In case of Python 3.3+, there is a ChainMap collection:
>>> from collections import ChainMap
>>> a = [{'a':1},{'b':2},{'c':1},{'d':2}]
>>> dict(ChainMap(*a))
{'b': 2, 'c': 1, 'a': 1, 'd': 2}
Also see:
What is the purpose of collections.ChainMap?
Little improvement for #dietbuddha answer with dictionary unpacking from PEP 448, for me, it`s more readable this way, also, it is faster as well:
from functools import reduce
result_dict = reduce(lambda a, b: {**a, **b}, list_of_dicts)
But keep in mind, this works only with Python 3.5+ versions.
This is similar to #delnan but offers the option to modify the k/v (key/value) items and I believe is more readable:
new_dict = {k:v for list_item in list_of_dicts for (k,v) in list_item.items()}
for instance, replace k/v elems as follows:
new_dict = {str(k).replace(" ","_"):v for list_item in list_of_dicts for (k,v) in list_item.items()}
unpacks the k,v tuple from the dictionary .items() generator after pulling the dict object out of the list
For flat dictionaries you can do this:
from functools import reduce
reduce(lambda a, b: dict(a, **b), list_of_dicts)
You can use join function from funcy library:
from funcy import join
join(list_of_dicts)
>>> L=[{'a': 1}, {'b': 2}, {'c': 1}, {'d': 2}]
>>> dict(i.items()[0] for i in L)
{'a': 1, 'c': 1, 'b': 2, 'd': 2}
Note: the order of 'b' and 'c' doesn't match your output because dicts are unordered
if the dicts can have more than one key/value
>>> dict(j for i in L for j in i.items())
If you don't need the singleton dicts anymore:
>>> L = [{'a':1}, {'b':2}, {'c':1}, {'d':2}]
>>> dict(map(dict.popitem, L))
{'a': 1, 'b': 2, 'c': 1, 'd': 2}
dict1.update( dict2 )
This is asymmetrical because you need to choose what to do with duplicate keys; in this case, dict2 will overwrite dict1. Exchange them for the other way.
EDIT: Ah, sorry, didn't see that.
It is possible to do this in a single expression:
>>> from itertools import chain
>>> dict( chain( *map( dict.items, theDicts ) ) )
{'a': 1, 'c': 1, 'b': 2, 'd': 2}
No credit to me for this last!
However, I'd argue that it might be more Pythonic (explicit > implicit, flat > nested ) to do this with a simple for loop. YMMV.
this way worked for me:
object = [{'a':1}, {'b':2}, {'c':1}, {'d':2}]
object = {k: v for dct in object for k, v in dct.items()}
printing object:
object = {'a':1,'b':2,'c':1,'d':2}
thanks Axes
>>> dictlist = [{'a':1},{'b':2},{'c':1},{'d':2, 'e':3}]
>>> dict(kv for d in dictlist for kv in d.iteritems())
{'a': 1, 'c': 1, 'b': 2, 'e': 3, 'd': 2}
>>>
Note I added a second key/value pair to the last dictionary to show it works with multiple entries.
Also keys from dicts later in the list will overwrite the same key from an earlier dict.
I have a dictionary where key is string and value is list.
Now while adding a value associated with given key, I always have to check if there is any list yet, otherwise I have to initialize as empty list somewhat like following snippet:
if not k in myDict:
myDict[k] = []
myDict[k].append(v)
I am wondering if there is any way to combine these steps into single one in python 3.7.
There are at least three ways:
Use dict.setdefault
>>> data = {}
>>> data.setdefault('foo', []).append(42)
>>> data
{'foo': [42]}
Use defaultdict, which unlike .setdefault, takes a callable:
>>> from collections import defaultdict
>>> data = defaultdict(list)
>>> data
defaultdict(<class 'list'>, {})
>>> data['foo'].append(42)
>>> data
defaultdict(<class 'list'>, {'foo': [42]})
Finally, subclass dict and implement __missing__:
>>> class MyDict(dict):
... def __missing__(self, key):
... self[key] = value = []
... return value
...
>>> data = MyDict()
>>> data['foo'].append(42)
>>> data
{'foo': [42]}
Note, you can think of the last one as the most flexible, you have access to the actual key that's missing when you deal with it. defaultdict is a class factory, and it generates a subclass of dict as well. But, the callable is not passed any arguments, nevertheless, it is sufficient for most needs.
Further, note that the defaultdict and __missing__ approaches will keep the default behavior, this may be undesirable after you create your data structure, you probably want a KeyError usually, or at least, you don't want mydict[key] to add a key anymore.
In both cases, you can just create a regular dict from the dict subclasses, e.g. dict(data). This should generally be very fast, even for large dict objects, especially if it is a one-time cost. For defaultdict, you can also set the default_factory to None and the old behavior returns:
>>> data = defaultdict(list)
>>> data
defaultdict(<class 'list'>, {})
>>> data['foo']
[]
>>> data['bar']
[]
>>> data
defaultdict(<class 'list'>, {'foo': [], 'bar': []})
>>> data.default_factory = None
>>> data['baz']
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
KeyError: 'baz'
>>>
You can use dict.setdefault:
>>> d = {}
>>> d.setdefault('k', []).append(1)
>>> d
{'k': [1]}
>>> d.setdefault('k', []).append(2)
>>> d
{'k': [1, 2]}
Help on method_descriptor in dict:
dict.setdefault = setdefault(...)
D.setdefault(k[,d]) -> D.get(k,d), also set D[k]=d if k not in D
is there any way to dynamically create missing keys if i want to want to set a variable in a subdictionary.
essentially I want to create any missing keys and set my value.
self.portdict[switchname][str(neighbor['name'])]['local']['ports'] = []
currently i'm doing it but its messy:
if not switchname in self.portdict:
self.portdict[switchname] = {}
if not str(neighbor['name']) in self.portdict[switchname]:
self.portdict[switchname][str(neighbor['name'])] = {}
if not 'local' in self.portdict[switchname][str(neighbor['name'])]:
self.portdict[switchname][str(neighbor['name'])]['local'] = {}
if not 'ports' in self.portdict[switchname][str(neighbor['name'])]['local']:
self.portdict[switchname][str(neighbor['name'])]['local']['ports'] = []
Is there any way to do this in one or two lines instead?
This is easier to do without recursion:
def set_by_path(dct, path, value):
ipath = iter(path)
p_last = next(ipath)
try:
while True:
p_next = next(ipath)
dct = dct.setdefault(p_last, {})
p_last = p_next
except StopIteration:
dct[p_last] = value
And a test case:
d = {}
set_by_path(d, ['foo', 'bar', 'baz'], 'qux')
print d # {'foo': {'bar': {'baz': 'qux'}}}
If you want to have it so you don't need a function, you can use the following defaultdict factory which allows you to nest things arbitrarily deeply:
from collections import defaultdict
defaultdict_factory = lambda : defaultdict(defaultdict_factory)
d = defaultdict_factory()
d['foo']['bar']['baz'] = 'qux'
print d
Use collections.defaultdict
self.portdict = defaultdict(lambda: defaultdict(lambda: defaultdict(lambda: defaultdict(lambda: []))))
I've run into a similar problem in the past. I found that defaultdict was the right answer for me—but writing the super long definitions (like the one in #o11c's answer or #Apero's answer) was no good. Here's what I came up with instead:
from collections import defaultdict
from functools import partial
def NestedDefaultDict(levels, baseFn):
def NDD(lvl):
return partial(defaultdict, NDD(lvl-1)) if lvl > 0 else baseFn
return defaultdict(NDD(levels-1))
This creates a dictionary with levels of nested dictionaries. So if you have levels=3, then you need 3 keys to access the bottom-level value. The second argument is a function which is used to create the bottom-level values. Something like list or lambda: 0 or even dict would work well.
Here's an example of using the "automatic" keys with 4 levels, and list as the default function:
>>> x = NestedDefaultDict(4, list)
>>> x[1][2][3][4].append('hello')
>>> x
defaultdict(<functools.partial object at 0x10b5c22b8>, {1: defaultdict(<functools.partial object at 0x10b5c2260>, {2: defaultdict(<functools.partial object at 0x10b5c2208>, {3: defaultdict(<type 'list'>, {4: ['hello']})})})})
I think that's basically what you'd want for the case in your question. Your 4 "levels" are switch-name, neighbor-name, local, & ports—and it looks like you want a list at the bottom-level to store your ports.
Another example using 2 levels and lambda: 0 as the default:
>>> y = NestedDefaultDict(2, lambda: 0)
>>> y['foo']['bar'] += 7
>>> y['foo']['baz'] += 10
>>> y['foo']['bar'] += 1
>>> y
defaultdict(<functools.partial object at 0x1021f1310>, {'foo': defaultdict(<function <lambda> at 0x1021f3938>, {'baz': 10, 'bar': 8})})
Have a close look to collections.defaultdict:
from collections import defaultdict
foo = defaultdict(dict)
foo['bar'] = defaultdict(dict)
foo['bar']['baz'] = defaultdict(dict)
foo['bar']['baz']['aaa'] = 1
foo['bor'] = 0
foo['bir'] = defaultdict(list)
foo['bir']['biz'].append(1)
foo['bir']['biz'].append(2)
print foo
defaultdict(<type 'dict'>, {'bir': defaultdict(<type 'list'>, {'biz': [1, 2]}), 'bor': 0, 'bar': defaultdict(<type 'dict'>, {'baz': defaultdict(<type 'dict'>, {'aaa': 1})})})
I want to add to a value in a dictionary storing counters:
d[key] += 1
but sometimes the key will not exist yet. Checking to see if the key exists seems too ugly. Is there a nice and pythonic one liner for this - add if the key exists, or create the value 1 if the key is not in the dict.keys ?
thanks
you can use
d={}
key='sundar'
d[key]=d.get(key,0)+1
print d
#output {'sundar': 1}
d[key]=d.get(key,0)+1
print d
#output {'sundar': 2}
You can use collections.Counter - this guarantees that all values are 1 or more, supports various ways of initialisation, and supports certain other useful abilities that a dict/defaultdict don't:
from collections import Counter
values = ['a', 'b', 'a', 'c']
# Take an iterable and automatically produce key/value count
counts = Counter(values)
# Counter({'a': 2, 'c': 1, 'b': 1})
print counts['a'] # 2
print counts['d'] # 0
# Note that `counts` doesn't have `0` as an entry value like a `defaultdict` would
# a `dict` would cause a `KeyError` exception
# Counter({'a': 2, 'c': 1, 'b': 1})
# Manually update if finer control is required
counts = Counter()
for value in values:
counts.update(value) # or use counts[value] += 1
# Counter({'a': 2, 'c': 1, 'b': 1})
>>> import collections
>>> d = collections.defaultdict(int)
>>> key = 'foo'
>>> d[key] += 1
>>> d
defaultdict(<type 'int'>, {'foo': 1})
>>> d[key]
1
>>> d[key] += 1
>>> d[key]
2
collections.Counter is the best for the specific use case you gave, but for a more general solution that doesn't require importing anything, use dict.setdefault():
d[key] = d.setdefault(key, 0) + 1
After reading What is the best way to implement nested dictionaries? why is it wrong to do:
c = collections.defaultdict(collections.defaultdict(int))
in python? I would think this would work to produce
{key:{key:1}}
or am I thinking about it wrong?
The constructor of defaultdict expects a callable. defaultdict(int) is a default dictionary object, not a callable. Using a lambda it can work, however:
c = collections.defaultdict(lambda: collections.defaultdict(int))
This works since what I pass to the outer defaultdict is a callable that creates a new defaultdict when called.
Here's an example:
>>> import collections
>>> c = collections.defaultdict(lambda: collections.defaultdict(int))
>>> c[5][6] += 1
>>> c[5][6]
1
>>> c[0][0]
0
>>>
Eli Bendersky provides a great direct answer for this question. It might also be better to restructure your data to
>>> import collections
>>> c = collections.defaultdict(int)
>>> c[1, 2] = 'foo'
>>> c[5, 6] = 'bar'
>>> c
defaultdict(<type 'int'>, {(1, 2): 'foo', (5, 6): 'bar'})
depending on what you actually need.