Related
This question already has answers here:
Elegant way to unpack limited dict values into local variables in Python
(5 answers)
Closed 9 months ago.
Is there a Pythonic way to assign the values of a dictionary to its keys, in order to convert the dictionary entries into variables?
I tried this out:
>>> d = {'a':1, 'b':2}
>>> for key,val in d.items():
exec('exec(key)=val')
exec(key)=val
^
SyntaxError: invalid syntax
I am certain that the key-value pairs are correct because they were previously defined as variables by me before. I then stored these variables in a dictionary (as key-value pairs) and would like to reuse them in a different function. I could just define them all over again in the new function, but because I may have a dictionary with about 20 entries, I thought there may be a more efficient way of doing this.
You can do it in a single line with:
>>> d = {'a': 1, 'b': 2}
>>> locals().update(d)
>>> a
1
However, you should be careful with how Python may optimize locals/globals access when using this trick.
Note
I think editing locals() like that is generally a bad idea. If you think globals() is a better alternative, think it twice! :-D
Instead, I would rather always use a namespace.
With Python 3 you can:
>>> from types import SimpleNamespace
>>> d = {'a': 1, 'b': 2}
>>> n = SimpleNamespace(**d)
>>> n.a
1
If you are stuck with Python 2 or if you need to use some features missing in types.SimpleNamespace, you can also:
>>> from argparse import Namespace
>>> d = {'a': 1, 'b': 2}
>>> n = Namespace(**d)
>>> n.a
1
If you are not expecting to modify your data, you may as well consider using collections.namedtuple, also available in Python 3.
This was what I was looking for:
>>> d = {'a':1, 'b':2}
>>> for key,val in d.items():
exec(key + '=val')
You already have a perfectly good dictionary. Just use that. If you know what the keys are going to be, and you're absolutely sure this is a reasonable idea, you can do something like
a, b = d['a'], d['b']
but most of the time, you should just use the dictionary. (If using the dictionary is awkward, you are probably not organizing your data well; ask for help reorganizing it.)
you can use operator.itemgetter
>>> from operator import itemgetter
>>> d = {'a':1, 'b':2}
>>> a, b = itemgetter('a', 'b')(d)
>>> a
1
>>> b
2
Consider the "Bunch" solution in Python: load variables in a dict into namespace. Your variables end up as part of a new object, not locals, but you can treat them as variables instead of dict entries.
class Bunch(object):
def __init__(self, adict):
self.__dict__.update(adict)
d = {'a':1, 'b':2}
vars = Bunch(d)
print vars.a, vars.b
Python has great support for list unpacking, but not dict or object unpacking. The most unsurprising and Pythonic approach seems to be accessing each item by hand to build an intermediate tuple as described in this answer:
a, b = d['a'], d['b']
However, if you have a lot of properties, or variable names are long, it can get nasty to do:
great, wow, awesome = dictionary['great'], dictionary['wow'], dictionary['awesome']
For context, the JavaScript equivalent of the above (destructuring) is:
const {great, wow, awesome} = dictionary;
Here's an option that is a bit more dynamic:
>>> dictionary = dict(great=0, wow=1, awesome=2)
>>> great, wow, awesome = (dictionary[k] for k in ("great", "wow", "awesome"))
>>> great
0
>>> awesome
2
This is still verbose; you could write a function to abstract things a bit, but unfortunately you still have to type everything twice:
>>> def unpack(dct, *keys):
... return (dct[k] for k in keys)
...
>>> dictionary = dict(great=0, wow=1, awesome=2)
>>> great, wow, awesome = unpack(dictionary, "great", "wow", "awesome")
You can generalize this to work on objects too:
>>> def unpack(x, *keys):
... if isinstance(x, dict):
... return (x[k] for k in keys)
... return (getattr(x, k) for k in keys)
...
>>> from collections import namedtuple
>>> Foo = namedtuple("Foo", "a b c d e")
>>> foo = Foo(a=0, b=1, c=2, d=3, e=4)
>>> c, b, d, a = unpack(foo, "c", "b", "d", "a")
>>> d
3
After all is said and done, unpacking by hand on multiple lines is probably best for real production code that you need to be safe and comprehensible:
>>> great = dictionary["great"]
>>> wow = dictionary["wow"]
>>> awesome = dictionary["awesome"]
Use pandas:
import pandas as pd
var=pd.Series({'a':1, 'b':2})
#update both keys and variables
var.a=3
print(var.a,var['a'])
I have a dictionary where key is string and value is list.
Now while adding a value associated with given key, I always have to check if there is any list yet, otherwise I have to initialize as empty list somewhat like following snippet:
if not k in myDict:
myDict[k] = []
myDict[k].append(v)
I am wondering if there is any way to combine these steps into single one in python 3.7.
There are at least three ways:
Use dict.setdefault
>>> data = {}
>>> data.setdefault('foo', []).append(42)
>>> data
{'foo': [42]}
Use defaultdict, which unlike .setdefault, takes a callable:
>>> from collections import defaultdict
>>> data = defaultdict(list)
>>> data
defaultdict(<class 'list'>, {})
>>> data['foo'].append(42)
>>> data
defaultdict(<class 'list'>, {'foo': [42]})
Finally, subclass dict and implement __missing__:
>>> class MyDict(dict):
... def __missing__(self, key):
... self[key] = value = []
... return value
...
>>> data = MyDict()
>>> data['foo'].append(42)
>>> data
{'foo': [42]}
Note, you can think of the last one as the most flexible, you have access to the actual key that's missing when you deal with it. defaultdict is a class factory, and it generates a subclass of dict as well. But, the callable is not passed any arguments, nevertheless, it is sufficient for most needs.
Further, note that the defaultdict and __missing__ approaches will keep the default behavior, this may be undesirable after you create your data structure, you probably want a KeyError usually, or at least, you don't want mydict[key] to add a key anymore.
In both cases, you can just create a regular dict from the dict subclasses, e.g. dict(data). This should generally be very fast, even for large dict objects, especially if it is a one-time cost. For defaultdict, you can also set the default_factory to None and the old behavior returns:
>>> data = defaultdict(list)
>>> data
defaultdict(<class 'list'>, {})
>>> data['foo']
[]
>>> data['bar']
[]
>>> data
defaultdict(<class 'list'>, {'foo': [], 'bar': []})
>>> data.default_factory = None
>>> data['baz']
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
KeyError: 'baz'
>>>
You can use dict.setdefault:
>>> d = {}
>>> d.setdefault('k', []).append(1)
>>> d
{'k': [1]}
>>> d.setdefault('k', []).append(2)
>>> d
{'k': [1, 2]}
Help on method_descriptor in dict:
dict.setdefault = setdefault(...)
D.setdefault(k[,d]) -> D.get(k,d), also set D[k]=d if k not in D
I'd like to be able to setup a mock that allows me to return something when I apply the builtin dict method.
I've tried using __iter__ to no avail. I can't seem to get anything but an empty dictionary:
import mock
mocked_object = mock.MagicMock()
mocked_object.__iter__.return_value = [1, 2, 3]
dict(mocked_object)
# {}
From dict documentation
If a positional argument is given and it is a mapping object, a dictionary is created with the same key-value pairs as the mapping object. Otherwise, the positional argument must be an iterable object. Each item in the iterable must itself be an iterable with exactly two objects. The first object of each item becomes a key in the new dictionary, and the second object the corresponding value.
MagicMock objects expose a keys method just because are mock objects and so dict() will consider they mapping objects. Unfortunately that way is little bit complicated to use if we want that on dict call mock become a dictionary with predefined key-values. The follow examples show how to implement a dict conversion to a predefined dictionary by use mapping object protocol:
>>> m = MagicMock()
>>> d = {"a":"A", "b":"B", "c":"C"}
>>> m.keys.return_value.__iter__.return_value = ["a", "b", "c"]
>>> m.__getitem__.side_effect = ["A","B","C"]
>>> dict(m)
{'a': 'A', 'c': 'C', 'b': 'B'}
>>> #Little bit generic
>>> m.keys.return_value.__iter__.return_value = d.keys()
>>> m.__getitem__.side_effect = lambda k:d[k]
>>> dict(m)
{'a': 'A', 'c': 'C', 'b': 'B'}
Both are little bit hard to read and in our test we would like something of simpler to read.
To lead dict to use iterator instead mapping we can just remove keys method from our mock and set __iter__.return_value:
>>> del m.keys
>>> m.__iter__.return_value = [("a","A"),("b","B"),("c","C")]
>>> dict(m)
{'a': 'A', 'c': 'C', 'b': 'B'}
>>> #Little bit generic
>>> m.__iter__.return_value = d.items()
>>> dict(m)
{'a': 'A', 'c': 'C', 'b': 'B'}
IMHO that is a simple and neat way to set up your mock and get a predefined dictionary from dict call.
Actually I think you need to do something like this:
mocked_object.keys.return_value.__iter__.return_value = [1, 2, 3]
With this, the dict method will give you an object with those keys, and the result of getattr(mocked_object, '1') (so, another mocked method) as values. I think you might be able to do what you want by mocking the keys() method as well if you want more control on the result.
I am struggling with the following problem:
I want to convert an OrderedDict like this:
OrderedDict([('method', 'constant'), ('data', '1.225')])
into a regular dict like this:
{'method': 'constant', 'data':1.225}
because I have to store it as string in a database. After the conversion the order is not important anymore, so I can spare the ordered feature anyway.
Thanks for any hint or solutions,
Ben
>>> from collections import OrderedDict
>>> OrderedDict([('method', 'constant'), ('data', '1.225')])
OrderedDict([('method', 'constant'), ('data', '1.225')])
>>> dict(OrderedDict([('method', 'constant'), ('data', '1.225')]))
{'data': '1.225', 'method': 'constant'}
>>>
However, to store it in a database it'd be much better to convert it to a format such as JSON or Pickle. With Pickle you even preserve the order!
Even though this is a year old question, I would like to say that using dict will not help if you have an ordered dict within the ordered dict. The simplest way that could convert those recursive ordered dict will be
import json
from collections import OrderedDict
input_dict = OrderedDict([('method', 'constant'), ('recursive', OrderedDict([('m', 'c')]))])
output_dict = json.loads(json.dumps(input_dict))
print output_dict
It is easy to convert your OrderedDict to a regular Dict like this:
dict(OrderedDict([('method', 'constant'), ('data', '1.225')]))
If you have to store it as a string in your database, using JSON is the way to go. That is also quite simple, and you don't even have to worry about converting to a regular dict:
import json
d = OrderedDict([('method', 'constant'), ('data', '1.225')])
dString = json.dumps(d)
Or dump the data directly to a file:
with open('outFile.txt','w') as o:
json.dump(d, o)
If you are looking for a recursive version without using the json module:
def ordereddict_to_dict(value):
for k, v in value.items():
if isinstance(v, dict):
value[k] = ordereddict_to_dict(v)
return dict(value)
Here is what seems simplest and works in python 3.7
from collections import OrderedDict
d = OrderedDict([('method', 'constant'), ('data', '1.225')])
d2 = dict(d) # Now a normal dict
Now to check this:
>>> type(d2)
<class 'dict'>
>>> isinstance(d2, OrderedDict)
False
>>> isinstance(d2, dict)
True
NOTE: This also works, and gives same result -
>>> {**d}
{'method': 'constant', 'data': '1.225'}
>>> {**d} == d2
True
As well as this -
>>> dict(d)
{'method': 'constant', 'data': '1.225'}
>>> dict(d) == {**d}
True
Cheers
You can use "dict_constructor" parameters.
xmltodict.parse(text, attr_prefix='',dict_constructor=dict)
If your data structure might contain internal (nested) OrderedDict instances, you should leverage Python's builtin copy mechanism.
You can override copying behavior for OrderedDict via Python's copyreg module (also used by pickle). Then you can use Python's builtin copy.deepcopy() function to perform the conversion.
import copy
import copyreg
from collections import OrderedDict
def convert_nested_ordered_dict(x):
"""
Perform a deep copy of the given object, but convert
all internal OrderedDicts to plain dicts along the way.
Args:
x: Any pickleable object
Returns:
A copy of the input, in which all OrderedDicts contained
anywhere in the input (as iterable items or attributes, etc.)
have been converted to plain dicts.
"""
# Temporarily install a custom pickling function
# (used by deepcopy) to convert OrderedDict to dict.
orig_pickler = copyreg.dispatch_table.get(OrderedDict, None)
copyreg.pickle(
OrderedDict,
lambda d: (dict, ([*d.items()],))
)
try:
return copy.deepcopy(x)
finally:
# Restore the original OrderedDict pickling function (if any)
del copyreg.dispatch_table[OrderedDict]
if orig_pickler:
copyreg.dispatch_table[OrderedDict] = orig_pickler
Merely by using Python's builtin copying infrastructure, this solution is superior to all other answers presented here, in the following ways:
Works for arbitrary data hierarchies, including nested OrderedDicts.
Works for more than just JSON data.
Does not require you to implement special logic for each possible element type (e.g. list, tuple, etc.)
deepcopy() will properly handle duplicate objects within the collection:
x = [1,2,3]
d = {'a': x, 'b': x}
assert d['a'] is d['b']
d2 = copy.deepcopy(d)
assert d2['a'] is d2['b']
Since our solution is based on deepcopy() we'll have the same advantage.
This solution also converts attributes that happen to be OrderedDict, not only collection elements:
class C:
def __init__(self, a):
self.a = a
def __repr__(self):
return f"C(a={self.a})"
c = C(OrderedDict([(1, 'one'), (2, 'two')]))
print("original: ", c)
print("converted:", convert_nested_ordered_dict(c))
original: C(a=OrderedDict([(1, 'one'), (2, 'two')]))
converted: C(a={1: 'one', 2: 'two'})
Its simple way
>>import json
>>from collection import OrderedDict
>>json.dumps(dict(OrderedDict([('method', 'constant'), ('data', '1.225')])))
A version that handles nested dictionaries and iterables but does not use the json module. Nested dictionaries become dict, nested iterables become list, everything else is returned unchanged (including dictionary keys and strings/bytes/bytearrays).
def recursive_to_dict(obj):
try:
if hasattr(obj, "split"): # is string-like
return obj
elif hasattr(obj, "items"): # is dict-like
return {k: recursive_to_dict(v) for k, v in obj.items()}
else: # is iterable
return [recursive_to_dict(e) for e in obj]
except TypeError: # return everything else
return obj
This question already has answers here:
Elegant way to unpack limited dict values into local variables in Python
(5 answers)
Closed 9 months ago.
Is there a Pythonic way to assign the values of a dictionary to its keys, in order to convert the dictionary entries into variables?
I tried this out:
>>> d = {'a':1, 'b':2}
>>> for key,val in d.items():
exec('exec(key)=val')
exec(key)=val
^
SyntaxError: invalid syntax
I am certain that the key-value pairs are correct because they were previously defined as variables by me before. I then stored these variables in a dictionary (as key-value pairs) and would like to reuse them in a different function. I could just define them all over again in the new function, but because I may have a dictionary with about 20 entries, I thought there may be a more efficient way of doing this.
You can do it in a single line with:
>>> d = {'a': 1, 'b': 2}
>>> locals().update(d)
>>> a
1
However, you should be careful with how Python may optimize locals/globals access when using this trick.
Note
I think editing locals() like that is generally a bad idea. If you think globals() is a better alternative, think it twice! :-D
Instead, I would rather always use a namespace.
With Python 3 you can:
>>> from types import SimpleNamespace
>>> d = {'a': 1, 'b': 2}
>>> n = SimpleNamespace(**d)
>>> n.a
1
If you are stuck with Python 2 or if you need to use some features missing in types.SimpleNamespace, you can also:
>>> from argparse import Namespace
>>> d = {'a': 1, 'b': 2}
>>> n = Namespace(**d)
>>> n.a
1
If you are not expecting to modify your data, you may as well consider using collections.namedtuple, also available in Python 3.
This was what I was looking for:
>>> d = {'a':1, 'b':2}
>>> for key,val in d.items():
exec(key + '=val')
You already have a perfectly good dictionary. Just use that. If you know what the keys are going to be, and you're absolutely sure this is a reasonable idea, you can do something like
a, b = d['a'], d['b']
but most of the time, you should just use the dictionary. (If using the dictionary is awkward, you are probably not organizing your data well; ask for help reorganizing it.)
you can use operator.itemgetter
>>> from operator import itemgetter
>>> d = {'a':1, 'b':2}
>>> a, b = itemgetter('a', 'b')(d)
>>> a
1
>>> b
2
Consider the "Bunch" solution in Python: load variables in a dict into namespace. Your variables end up as part of a new object, not locals, but you can treat them as variables instead of dict entries.
class Bunch(object):
def __init__(self, adict):
self.__dict__.update(adict)
d = {'a':1, 'b':2}
vars = Bunch(d)
print vars.a, vars.b
Python has great support for list unpacking, but not dict or object unpacking. The most unsurprising and Pythonic approach seems to be accessing each item by hand to build an intermediate tuple as described in this answer:
a, b = d['a'], d['b']
However, if you have a lot of properties, or variable names are long, it can get nasty to do:
great, wow, awesome = dictionary['great'], dictionary['wow'], dictionary['awesome']
For context, the JavaScript equivalent of the above (destructuring) is:
const {great, wow, awesome} = dictionary;
Here's an option that is a bit more dynamic:
>>> dictionary = dict(great=0, wow=1, awesome=2)
>>> great, wow, awesome = (dictionary[k] for k in ("great", "wow", "awesome"))
>>> great
0
>>> awesome
2
This is still verbose; you could write a function to abstract things a bit, but unfortunately you still have to type everything twice:
>>> def unpack(dct, *keys):
... return (dct[k] for k in keys)
...
>>> dictionary = dict(great=0, wow=1, awesome=2)
>>> great, wow, awesome = unpack(dictionary, "great", "wow", "awesome")
You can generalize this to work on objects too:
>>> def unpack(x, *keys):
... if isinstance(x, dict):
... return (x[k] for k in keys)
... return (getattr(x, k) for k in keys)
...
>>> from collections import namedtuple
>>> Foo = namedtuple("Foo", "a b c d e")
>>> foo = Foo(a=0, b=1, c=2, d=3, e=4)
>>> c, b, d, a = unpack(foo, "c", "b", "d", "a")
>>> d
3
After all is said and done, unpacking by hand on multiple lines is probably best for real production code that you need to be safe and comprehensible:
>>> great = dictionary["great"]
>>> wow = dictionary["wow"]
>>> awesome = dictionary["awesome"]
Use pandas:
import pandas as pd
var=pd.Series({'a':1, 'b':2})
#update both keys and variables
var.a=3
print(var.a,var['a'])