New to python here:
I'm trying to create a new list where every dict from initial list has an element removed, it exists:
arraylist = [{"x":1, "y":2}, {"x":3, "y":2}, {"x":5, "y":2}, {"x":33, "y":2}, {"x":1, "y":8}]
arraylist = map(lambda d: del d["y"] if "y" in d, arraylist)
I know I can do it using for, using del. But I'm looking to learn something new.
Use a list comprehension:
In [26]: [{x:d[x] for x in d if x != 'y'} for d in arraylist]
Out[26]: [{'x': 1}, {'x': 3}, {'x': 5}, {'x': 33}, {'x': 1}]
You can use filter like this
arraylist = [{"x":1, "y":2}, {"x":3, "y":2}, {"x":5, "y":2}, {"x":33, "y":2}, {"x":1, "y":8}]
arraylist = map(lambda d: dict(filter(lambda (k,v): k != "y", d.iteritems())), arraylist)
You can't use del in a lambda function because del is a statement and a lambda's body can only be a single expression. You can't put a statement inside an expression. You could make it work with an ordinary function:
def delete_y(d):
if "y" in d:
del d['y']
return d
Note that using del like this modifies the dictionary d in place. This means that returning the modified dictionary (and using the return value to build a new list) is sort of redundant. The original data structure the dictionary came from will already have the modified version.
Maybe its not the shortest way, but it is definitely a convenient way to remove
items from a list:
arraylist = [{"x":1, "y":2}, {"x":3, "y":2}, {"x":5, "y":2}, {"x":33, "y":2}, {"x":1, "y":8}]
print arraylist
def containsY(d):
if 'y' in d.keys():
del d['y']
return True
return False
filter(containsY, arraylist)
print arraylist
output:
[{'y': 2, 'x': 1}, {'y': 2, 'x': 3}, {'y': 2, 'x': 5}, {'y': 2, 'x': 33}, {'y': 8, 'x': 1}]
[{'x': 1}, {'x': 3}, {'x': 5}, {'x': 33}, {'x': 1}]
Related
As of python 3.7, dict is guaranteed to be insertion ordered. I have a unit test using pytest where I would like to compare two dicts but with order of elements in mind.
My application will never run on python older then 3.7 so backward compatibility is not an issue for me.
Question
Is there any possibility to compare regular dicts, preserving order and nice diff output in pytest?
My unsuccesful tries
a = {'x': 1, 'y': {'u': 3, 'v': 4}}
b = {'y': {'v': 4, 'u': 3}, 'x': 1}
assert a == b
This assertion passes, but I don't want it to. These dicts are equal but with wrong order.
I can convert everything to OrderedDict but it is
ugly code
produces ugly pytest diff output
from collections import OrderedDict
def deep_ordered_dict(value):
if isinstance(value, dict):
value = OrderedDict(value)
for k, v in value.items():
value[k] = deep_ordered_dict(v)
if isinstance(value, list):
for k, v in enumerate(value):
value[k] = deep_ordered_dict(v)
return value
a = {'x': 1, 'y': {'u': 3, 'v': 4}}
b = {'y': {'v': 4, 'u': 3}, 'x': 1}
assert deep_ordered_dict(a) == deep_ordered_dict(b)
Comparing it as JSON is not better
produces ugly pytest diff output
import json
a = {'x': 1, 'y': {'u': 3, 'v': 4}}
b = {'y': {'v': 4, 'u': 3}, 'x': 1}
assert json.dumps(a, indent=2) == json.dumps(b, indent=2)
Even though dictionaries are required to preserve insertion order as of Python 3.7, for backward compatibility equality comparisons are order-independent. This is one of the documented differences between dict and OrderedDict, so if you want order-sensitive comparisons you should use the latter.
I have a function for filtering a list of dict based on the value of certain keys, like a SELECT * WHERE xxx query in SQL
list_of_dict = [
{'key1':val, 'key2':val},
{'key1':val, 'key2':val},
...
]
def filter_list(list_of_dict, key1, key2=None):
if key2:
filtered_list = [i for i in list_of_dict if i['key1']==key1 and i['key2']==key2]
else:
filtered_list = [i for i in list_of_dict if i['key1']==key1]
but when I have more keys as arguments to the function the if ...else... could go really long.
Is there a more pythonic way to do this?
If you have a large or variable number of keys, you can use all to loop over them. Here's an example where the key values are provided as keyword arguments:
def filter_dicts(dicts, **keys):
return [
d for d in dicts
if all(d[k] == v for k, v in keys.items())
]
As #juanpa.arrivillaga points out, dict_items objects behave like sets in many regards, so you can alternatively filter for dictionaries which have keys as a subset:
def filter_dicts(dicts, **keys):
return [d for d in dicts if keys.items() <= d.items()]
Example:
>>> dicts = [{'x': 1, 'y': 2}, {'x': 1, 'y': 3}]
>>> filter_dicts(dicts, x=1)
[{'x': 1, 'y': 2}, {'x': 1, 'y': 3}]
>>> filter_dicts(dicts, x=1, y=2)
[{'x': 1, 'y': 2}]
>>> filter_dicts(dicts, y=3)
[{'x': 1, 'y': 3}]
I have dict like this
d=[{'a':1,'b':2},{'a':2,'b':2},{'a':3},{'a':1}]
So i need like
d=[{'a':1,'b':2},{'a':2,'b':2},{'a':3}]
Remove duplicate
Learn more about Python data types and theirs functions. Here is one tutorial for dictionaries
As what I know using list of dictionaries is a bad practice
Here is my solution it could be not so elegant, but it is working. Also remove duplicates means remove ALL of them, so I remove all list elements where they were repeated.
d = [{'a': 1, 'b': 2}, {'a': 2, 'b': 2}, {'a': 3}, {'a': 1}]
temp = []
for i in d:
for x, y in i.items():
temp_var = [x, y]
if temp_var in temp:
d.pop(d.index(i))
else:
temp.append(temp_var)
print(d)
# result [{'a': 1, 'b': 2}, {'a': 3}]
P.S.: Keep learning and have a nice day :)
I would like to check the intersection of two dictionaries. If I do this, I get exactly what I expected:
dict1 = {'x':1, 'y':2, 'z':3}
dict2 = {'x':1, 'y':2, 'z':4}
set(dict1.items()).intersection(dict2.items())
>> {('x', 1), ('y', 2)}
However, if the items within the dictionary are nonhashable, I get an error.
dict1 = {'x':{1,2}, 'y':{2,3}, 'z':3}
dict2 = {'x':{1,3}, 'y':{2,4}, 'z':4}
TypeError Traceback (most recent call
last)
<ipython-input-56-33fdb931ef54> in <module>
----> 1 set(dict1.items()).intersection(dict2.items())
TypeError: unhashable type: 'set'
Of course, I get the same error for tuples and lists, as they are also not hashable.
Is there a work around or an existing class I can use to check the intersection of nonhashable dictionary values?
You could create a "makeHashable" function to apply to dictionary items for comparison purposed and used it to build a set that you can then check in a list comprehension:
dict1 = {'x':{1,2}, 'y':{2,3}, 'z':3}
dict2 = {'x':{1,3}, 'y':{3,2}, 'z':4}
def makeHashable(x):
if isinstance(x,(list,tuple)): return tuple(map(makeHashable,x))
if isinstance(x,set): return makeHashable(sorted(x))
if isinstance(x,dict): return tuple(map(makeHashable,x.items()))
return x
dict1Set = set(map(makeHashable,dict1.items()))
intersect = [ kv for kv in dict2.items() if makeHashable(kv) in dict1Set]
output:
print(intersect)
# [('y', {2, 3})]
Maybe try:
#!/usr/local/cpython-3.8/bin/python3
def intersection1(dict1, dict2):
intersection = set(dict1.items()).intersection(dict2.items())
return intersection
def intersection2(dict1, dict2):
result = {}
for key1 in dict1:
if key1 in dict2 and dict1[key1] == dict2[key1]:
result[key1] = dict1[key1]
return result
def main():
dict1 = {'x': 1, 'y': 2, 'z': 3}
dict2 = {'x': 1, 'y': 2, 'z': 4}
print(intersection2(dict1, dict2))
print(intersection1(dict1, dict2))
# >> {('x', 1), ('y', 2)}
dict3 = {'x': [1, 2], 'y': [2, 3], 'z': [3, 4]}
dict4 = {'x': [1, 2], 'y': [2, 3], 'z': [4, 5]}
print(intersection2(dict3, dict4))
print(intersection1(dict3, dict4))
main()
You of course cannot put an unhashable type in a set, so I've done the next best thing with intersection2()
You can serialize the dict values before performing set intersection, and deserialize the values in the resulting set. The following example uses pickle for serialization:
import pickle
{k: pickle.loads(v) for k, v in set.intersection(
*({(k, pickle.dumps(v)) for k, v in i} for i in map(dict.items, (dict1, dict2))))}
so that given:
dict1 = {'x': {1, 2}, 'y': {2, 3}, 'z': 3}
dict2 = {'x': {2, 1}, 'y': {2, 4}, 'z': 4}
the expression would return:
{'x': {1, 2}}
I have a list of dictionaries like so:
l = [{"integer":"1"},{"integer":"2"},{"integer":"3"},{"integer":"4"}]
I would like to do something similar to the following so that each of the numbers which are the value pair for the "integer" key are returned as integers:
l = [{"integer":"1"},{"integer":"2"},{"integer":"3"},{"integer":"4"}]
r = map(lambda x: x["integer"]=int(x["integer"]), l)
print r
#[{"integer":1},{"integer":2},{"integer":3},{"integer":4}]
But this causes an error:
SyntaxError: lambda cannot contain assignment
Does anyone know of a clean way to do this in python? Preferably a oneliner using map or something similar?
Use a list comprehension comprehension
You will iterate through the dictionaries in the list and have them returned as x, then insert a new dictionary with your desired key and the integer value of the return within a new list
r = [{'integer': int(x['integer'])} for x in l]
You should just use a loop:
l = [{"integer":"1"},{"integer":"2"},{"integer":"3"},{"integer":"4"}]
for d in l:
d["integer"] = int(d["integer"])
print(l)
#[{'integer': 1}, {'integer': 2}, {'integer': 3}, {'integer': 4}]
However, here is a one-liner that should work for you:
l = [{"integer":"1"},{"integer":"2"},{"integer":"3"},{"integer":"4"}]
[d.update({"integer": int(d["integer"])}) for d in l]
print(l)
#[{'integer': 1}, {'integer': 2}, {'integer': 3}, {'integer': 4}]
Be aware that dict.update() returns None, so if you assigned the output of the list comprehension to a variable it would be a list containing all Nones.
print([d.update({"integer": int(d["integer"])}) for d in l])
#[None, None, None, None]
The following works in one line:
r = [{'integer':int(x['integer'])} for x in l]
print(r)
# [{'integer': 1}, {'integer': 2}, {'integer': 3}, {'integer': 4}]
This utilizes a dict comprehension inside a list comprehension.
[i.update({w:int(k)}) for i in l for w,k in i.items()]
it the second loop is only looping over one key set, so take the two loops with gram of salt :)
Awesome solution by one of my friends in a chat:
>>> listOfDict = [{1:1338}, {1:1338}, {1:1338}]
>>> y = 1337
>>> value = 1
>>> map(lambda x: x.update({value: y}), listOfDict)
[None, None, None]
>>> listOfDict
[{1: 1337}, {1: 1337}, {1: 1337}]
You can try the following
l = [{"integer":"1"},{"integer":"2"},{"integer":"3"},{"integer":"4"}]
a = [dict(d, **{'abcd':5}) for d in l]
print(a)
[{'integer': '1', 'abcd': 4}, {'integer': '2', 'abcd': 4}, {'integer': '3', 'abcd': 4}, {'integer': '4', 'abcd': 4}]