This question is similar to the one asked here:
However in my case I have a large dictionary filled with many keys and values, and I will not be able to enumerate each key to implement this solution:
from operator import itemgetter
params = {'a': 1, 'b': 2}
a, b = itemgetter('a', 'b')(params)
In my case:
from operator import itemgetter
params = {'a': 1, 'b': 2 ........... 'y': 25, 'z': 26 }
a, b ..... y, z = itemgetter('a', 'b'.... 'y', 'z')(params)
Is there any way I can unpack and assign each key to it's value, and associate values with variables names after its keys on a large scale?
Please Advise.
# Function
a = 1
b = 2
def foo(z):
return a + b + z
foo(3)
# should return 6
Either state the variables as input argument and unpack the dictionary in the call
def foo(z, a, b):
return a + b + z
param = {'a': 1, 'b': 2}
foo(3, **param)
>> 6
then you could also provide default values (for example 0) if any input should be missing.
Another possibility is to provide provide the entire dictionary as input and let the function handle the unpacking.
def bar(z, p):
return p['a'] + p['b'] + z
bar(3, param)
>> 6
Related
I'm a Python Newb and trying to create a dictionary with ordered values.
Since dict.fromkeys only allows me to copy the same value for each key, I've set all values to 0 and tried something like this:
def Ord_Values_in_Dic(D):
c = 0
for value in D.values():
c += 1
value += c
return D
My output only changes the first value of the dictionary to 1 though, instead I'd want the second value to also change to 2, the third value to change to 3 and so on...
I don't get if the loop isn't iterating correctly through the dictionary or there's something else wrong.
Since dict.fromkeys only allows me to copy the same value for each key
then it is not right tool for you task. You might use zip to prepare dict from 2 iterables - one for keys, one for values, consider following simple example
keys = ["x","y","z"]
d = dict(zip(keys,range(3)))
print(d) # {'x': 0, 'y': 1, 'z': 2}
range with single arguments gives subsequent numbers from 0 (inclusive) to given value (exclusive), so in above example: 0,1,2
Got it!
import numpy as np
a = np.linspace(0,100,100)
b = np.sin(a)
c = np.cos(a)
idx = list(range(1,101))
X = dict(zip(b, idx))
Y = dict(zip(c, idx))
This solved it!
Thank you :)
Suppose I have some variables:
a, b, c, d, e = range(5)
I want to save the values of these variables in a file for later inspection. One way I thought to do this was:
lookup = {
'a': a,
'b': b,
'c': c,
'd': d,
'e': e
}
As you might imagine, with a large number of variables, this could get tedious. And, yes, I know many editors have functionality to make this kind of copy-paste action easy. But I'm looking for the standard, "Pythonic" way of dynamically building a key: value lookup where the key is the variable's name and the value is the variable's, well, value!
I thought about this:
>>> {var.__name__: var for var in [a, b, c, d, e]}
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 1, in <dictcomp>
AttributeError: 'int' object has no attribute '__name__'
I'm not surprised this didn't work, because integer variables are constants (I'm not sure of the exact way to describe things):
>>> a = 1
>>> b = 1
>>> a is b
True
>>> b is a
True
>>> a == b
True
How might I accomplish this?
import itertools
data = range(5)
result = {f"a{count}": value for count, value in zip(itertools.count(1), data)}
print(result)
Output:
{'a1': 0, 'a2': 1, 'a3': 2, 'a4': 3, 'a5': 4}
You might want to look into locals() and inspect. The result could be i.e.:
>>> from inspect import ismodule
>>> a = 1
>>> b = 1
>>> dict((k, v) for k, v in locals().items() if not k.startswith("__") and not callable(v) and not ismodule(v))
{'a': 1, 'b': 1}
But to get it right you might need to add some additional conditions, and also you will have to watch out for mutable objects or values, as in this case those would mutate and you would not preserve the earlier value for later inspection. Serialization or copying them could help.
You could tackle it from the other direction and save all the local variables, using locals() For example
import json
def foo(a=None, bb=None):
ccc='lots of c'; de=42
print( json.dumps( locals() ))
foo() generates {"a": null, "bb": null, "ccc": "lots of c", "de": 42}
( json.dumps is one way to serialize a dict, and will work only for simple variables that can be converted to JSON)
Another way to just get some variables would be
print( json.dumps( dict( a=a, b=b, c=c) ))
Here is another way using ascii_lowercase from string module:
import string
alphabets = iter(string.ascii_lowercase)
lookup = {next(alphabets): x for x in range(5)}
print(lookup)
# {'a': 0, 'b': 1, 'c': 2, 'd': 3, 'e': 4}
Assume now I have a well-defined function
def f1(a):
...
and i can not change its definition. Now I want to call it with a argument from a dictionary
D={ 'a':1 , 'b':2}
If I call it like
f1(**D)
there will be a syntax error (because b is not defined in f1). My question is if I can find a smart way to let the function to find the argument it needed only?
You can use inspect.getargspec:
d = {'a': 5, 'b': 6, 'c': 7}
def f(a,b):
return a + b
f(*[d[arg] for arg in inspect.getargspec(f).args])
which gives 11.
Thanks Eugene, getargspec is legacy, instead you should use getfullargspec which has the same usage in this case.
You could get the argument names from the function's code object, then lookup the values for them in the dictionary:
>>> def f(a):
... b = 2
... return a + b
...
>>> D = {'a': 1, 'b': 2, 'c': 3}
>>> nargs = f.__code__.co_argcount
>>> varnames = f.__code__.co_varnames
>>> f(*(D[arg] for arg in varnames[:nargs]))
3
Here is an example of function that got a dictionary and use only the 'b' key :
In [14]: def f1(mydict):
...: return mydict['b']
...:
In [15]: x={'a': 1, 'b':2}
In [16]: f1(x)
Out[16]: 2
I'd like to call a function in python using a dictionary with matching key-value pairs for the parameters.
Here is some code:
d = dict(param='test')
def f(param):
print(param)
f(d)
This prints {'param': 'test'} but I'd like it to just print test.
I'd like it to work similarly for more parameters:
d = dict(p1=1, p2=2)
def f2(p1, p2):
print(p1, p2)
f2(d)
Is this possible?
Figured it out for myself in the end. It is simple, I was just missing the ** operator to unpack the dictionary
So my example becomes:
d = dict(p1=1, p2=2)
def f2(p1,p2):
print p1, p2
f2(**d)
In[1]: def myfunc(a=1, b=2):
In[2]: print(a, b)
In[3]: mydict = {'a': 100, 'b': 200}
In[4]: myfunc(**mydict)
100 200
A few extra details that might be helpful to know (questions I had after reading this and went and tested):
The function can have parameters that are not included in the dictionary
You can not override a function parameter that is already in the dictionary
The dictionary can not have values that aren't in the function.
Examples:
Number 1: The function can have parameters that are not included in the dictionary
In[5]: mydict = {'a': 100}
In[6]: myfunc(**mydict)
100 2
Number 2: You can not override a function parameter that is already in the dictionary
In[7]: mydict = {'a': 100, 'b': 200}
In[8]: myfunc(a=3, **mydict)
TypeError: myfunc() got multiple values for keyword argument 'a'
Number 3: The dictionary can not have values that aren't in the function.
In[9]: mydict = {'a': 100, 'b': 200, 'c': 300}
In[10]: myfunc(**mydict)
TypeError: myfunc() got an unexpected keyword argument 'c'
How to use a dictionary with more keys than function arguments:
A solution to #3, above, is to accept (and ignore) additional kwargs in your function (note, by convention _ is a variable name used for something being discarded, though technically it's just a valid variable name to Python):
In[11]: def myfunc2(a=None, **_):
In[12]: print(a)
In[13]: mydict = {'a': 100, 'b': 200, 'c': 300}
In[14]: myfunc2(**mydict)
100
Another option is to filter the dictionary based on the keyword arguments available in the function:
In[15]: import inspect
In[16]: mydict = {'a': 100, 'b': 200, 'c': 300}
In[17]: filtered_mydict = {k: v for k, v in mydict.items() if k in [p.name for p in inspect.signature(myfunc).parameters.values()]}
In[18]: myfunc(**filtered_mydict)
100 200
Example with both positional and keyword arguments:
Notice further than you can use positional arguments and lists or tuples in effectively the same way as kwargs, here's a more advanced example incorporating both positional and keyword args:
In[19]: def myfunc3(a, *posargs, b=2, **kwargs):
In[20]: print(a, b)
In[21]: print(posargs)
In[22]: print(kwargs)
In[23]: mylist = [10, 20, 30]
In[24]: mydict = {'b': 200, 'c': 300}
In[25]: myfunc3(*mylist, **mydict)
10 200
(20, 30)
{'c': 300}
In python, this is called "unpacking", and you can find a bit about it in the tutorial. The documentation of it sucks, I agree, especially because of how fantasically useful it is.
Here ya go - works just any other iterable:
d = {'param' : 'test'}
def f(dictionary):
for key in dictionary:
print key
f(d)
TL;DR: How can you compare two python dictionaries if some of them have values which are unhashable/mutable (e.g. lists or pandas Dataframes)?
I have to compare dictionary pairs for equality. In that sense, this question is similar to these two, but their solutions only seem to work for immutable objects...
Is there a better way to compare dictionary values
Comparing two dictionaries in Python
My problem, is that I'm dealing with pairs of highly nested dictionaries where the unhashable objects could be found in different places depending on which pair of dictionaries I'm comparing. My thinking is that I'll need to iterate across the deapest values contained in the dictionary and can't just rely on the dict.iteritems() which only unrolls the highest key-value pairs. I'm not sure how iterate across all the possible key-value pairs contained in the dictionary and compare them either using sets/== for the hashable objects and in the cases of pandas dataframes, running df1.equals(df2). (Note for pandas dataframe, just running df1==df2 does a piecewise comparison and NA's are poorly handled. df1.equals(df2) gets around that does the trick.)
So for example:
a = {'x': 1, 'y': {'z': "George", 'w': df1}}
b = {'x': 1, 'y': {'z': "George", 'w': df1}}
c = {'x': 1, 'y': {'z': "George", 'w': df2}}
At a minimum, and this would be pretty awesome already, the solution would yield TRUE/FALSE as to whether their values are the same and would work for pandas dataframes.
def dict_compare(d1, d2):
if ...
return True
elif ...
return False
dict_compare(a,b)
>>> True
dict_compare(a,c)
>>> False
Moderately better: the solution would point out what key/values would be different across the dictionaries.
In the ideal case: the solution could separate the values into 4 groupings:
added,
removed,
modified
same
Well, there's a way to make any type comparable: Simply wrap it in a class that compares like you need it:
class DataFrameWrapper():
def __init__(self, df):
self.df = df
def __eq__(self, other):
return self.df.equals(other.df)
So when you wrap your "uncomparable" values you can now simply use ==:
>>> import pandas as pd
>>> df1 = pd.DataFrame({'a': [1,2,3]})
>>> df2 = pd.DataFrame({'a': [3,2,1]})
>>> a = {'x': 1, 'y': {'z': "George", 'w': DataFrameWrapper(df1)}}
>>> b = {'x': 1, 'y': {'z': "George", 'w': DataFrameWrapper(df1)}}
>>> c = {'x': 1, 'y': {'z': "George", 'w': DataFrameWrapper(df2)}}
>>> a == b
True
>>> a == c
False
Of course wrapping your values has it's disadvantages but if you only need to compare them that would be a very easy approach. All that may be needed is a recursive wrapping before doing the comparison and a recursive unwrapping afterwards:
def recursivewrap(dict_):
for key, value in dict_.items():
wrapper = wrappers.get(type(value), lambda x: x) # for other types don't wrap
dict_[key] = wrapper(value)
return dict_ # return dict_ so this function can be used for recursion
def recursiveunwrap(dict_):
for key, value in dict_.items():
unwrapper = unwrappers.get(type(value), lambda x: x)
dict_[key] = unwrapper(value)
return dict_
wrappers = {pd.DataFrame: DataFrameWrapper,
dict: recursivewrap}
unwrappers = {DataFrameWrapper: lambda x: x.df,
dict: recursiveunwrap}
Sample case:
>>> recursivewrap(a)
{'x': 1,
'y': {'w': <__main__.DataFrameWrapper at 0x2affddcc048>, 'z': 'George'}}
>>> recursiveunwrap(recursivewrap(a))
{'x': 1, 'y': {'w': a
0 1
1 2
2 3, 'z': 'George'}}
If you feel really adventurous you could use wrapper classes that depending on the comparison result modify some variable that holds the information what wasn't equal.
This part of the answer was based on the original question that didn't include nestings:
You can seperate the unhashable values from the hashable values and do a set-comparison for the hashable values and a "order-independant" list-comparison for the unhashables:
def split_hashable_unhashable(vals):
"""Seperate hashable values from unhashable ones and returns a set (hashables)
and list (unhashable ones)"""
set_ = set()
list_ = []
for val in vals:
try:
set_.add(val)
except TypeError: # unhashable
list_.append(val)
return set_, list_
def compare_lists_arbitary_order(l1, l2, cmp=pd.DataFrame.equals):
"""Compare two lists using a custom comparison function, the order of the
elements is ignored."""
# need to have equal lengths otherwise they can't be equal
if len(l1) != len(l2):
return False
remaining_indices = set(range(len(l2)))
for item in l1:
for cmpidx in remaining_indices:
if cmp(item, l2[cmpidx]):
remaining_indices.remove(cmpidx)
break
else:
# Run through the loop without finding a match
return False
return True
def dict_compare(d1, d2):
if set(d1) != set(d2): # compare the dictionary keys
return False
set1, list1 = split_hashable_unhashable(d1.values())
set2, list2 = split_hashable_unhashable(d2.values())
if set1 != set2: # set comparison is easy
return False
return compare_lists_arbitary_order(list1, list2)
It got a bit longer than expected. For your test-cases it definetly works:
>>> import pandas as pd
>>> df1 = pd.DataFrame({'a': [1,2,3]})
>>> df2 = pd.DataFrame({'a': [3,2,1]})
>>> a = {'x': 1, 'y': df1}
>>> b = {'y': 1, 'x': df1}
>>> c = {'y': 1, 'x': df2}
>>> dict_compare(a, b)
True
>>> dict_compare(a, c)
False
>>> dict_compare(b, c)
False
The set-operations can also be used to find differences (see set.difference). It's a bit more complicated with the lists, but not really impossible. One could add the items where no match was found to a seperate list instead of instantly returning False.
Deepdiff library provides extensive ability to diff two python dictionaries
https://github.com/seperman/deepdiff
DeepDiff: Deep Difference of dictionaries, iterables, strings and other objects. It will recursively look for all the changes.
pip install deepdiff