I want to compare two lists of dictionaries in python, I have a list sent from the frontend and a query result stored in the same function, so all I want is to compare both of the lists with the key barcode and if they matches I want to append the name from the second dictionary to the first one
for example:
data_from_frontend = [
{ barcode: '1', name_en: 'milk' },
{ barcode: '2', name_en: 'water' },
{ barcode: '3', name_en: 'cheese' },
{ barcode: '10', name_en: 'pepsi' },
]
result_from_query = [
{ barcode: '1', name: 'PID012343' },
{ barcode: '2', name: 'PID123454' },
{ barcode: '10', name: 'PID123432' },
]
I want to compare both of the lists and by the barcode and if they match I want to merge the the pair of both variables to a new one + adding the one the doesn't match to another list, so the outcome would be two new variables with the [matched + name] and not_found, how can I achieve that ?
Here is what i've tried
equal = []
not_equal = []
no_barcode = []
x = [ { "age": "22" }, { "name": "John Doe" }, { "name": "Jane Doe" }, { "name": "Doctor" }, { "name": "Engineer" } ]
y = [ { "name": "Engineer" }, { "name": "Jane Doe" }, { "name": "Doctor" } ]
x_sort = sorted(x, key=lambda k: ("name" not in k, k.get("name", None)))
y_sort = sorted(y, key=lambda k: ("name" not in k, k.get("name", None)))
print(y_sort)
for x_val in x_sort:
if "name" not in x_val.keys():
no_barcode.append(x_val)
else:
for y_val in y_sort:
if x_val["name"] == y_val["name"]:
equal.append(x_val)
mapped = map(lambda k: k["name"], y_sort)
if x_val["name"] not in mapped:
not_equal.append(x_val)
print('equal')
print(equal)
print('not equal')
print(not_equal)
First of all you should fix your dict keys and enclose them into quotes.
Then you can use generator expression to find items, for example:
print('initial dict:')
pprint.pprint(data_from_frontend)
for item in result_from_query:
item_found = next((i for i in data_from_frontend if i['barcode'] == item['barcode']), False)
if item_found:
item_found['name'] = item['name']
print('dict after search:')
pprint.pprint(data_from_frontend)
will produce:
initial dict:
[{'barcode': '1', 'name_en': 'milk'},
{'barcode': '2', 'name_en': 'water'},
{'barcode': '3', 'name_en': 'cheese'},
{'barcode': '10', 'name_en': 'pepsi'}]
dict after search:
[{'barcode': '1', 'name': 'PID012343', 'name_en': 'milk'},
{'barcode': '2', 'name': 'PID123454', 'name_en': 'water'},
{'barcode': '3', 'name_en': 'cheese'},
{'barcode': '10', 'name': 'PID123432', 'name_en': 'pepsi'}]
Using False in generator will avoid error when searching by barcode value not existing in target dict.
P.S. dont forget to import pprint if you want to use it
P.P.S. and sure you can create new dict instead of modifying existing one, using same logic
Your barcode matching results can be get like this.
barcode = 'barcode'
name_en = 'name_en'
name = 'name'
matching_result = data_from_frontend[:] #get a copy of front end data to use as the output
for i in range(len(data_from_frontend)):
for j in range(len(result_from_query)):
if(data_from_frontend[i][barcode] == result_from_query[j][barcode]):
matching_result[i][name] = result_from_query[j][name]
break
print(matching_result)
Related
There are two dict main and input, I want to validate the "input" such that all the keys in the list of dictionary and nested dictionary (if present/all keys are optional) matches that of the main if not the wrong/different key should be returned as the output.
main = "app":[{
"name": str,
"info": [
{
"role": str,
"scope": {"groups": list}
}
]
},{
"name": str,
"info": [
{"role": str}
]
}]
input_data = "app":[{
'name': 'nms',
'info': [
{
'role': 'user',
'scope': {'groups': ['xyz']
}
}]
},{
'name': 'abc',
'info': [
{'rol': 'user'}
]
}]
when compared input with main the wrong/different key should be given as output, in this case
['rol']
The schema module does exactly this.
You can catch SchemaUnexpectedTypeError to see which data doesn't match your pattern.
Also, make sure you don't use the word input as a variable name, as it's the name of a built-in function.
keys = []
def print_dict(d):
if type(d) == dict:
for val in d.keys():
df = d[val]
try:
if type(df) == list:
for i in range(0,len(df)):
if type(df[i]) == dict:
print_dict(df[i])
except AttributeError:
pass
keys.append(val)
else:
try:
x = d[0]
if type(x) == dict:
print_dict(d[0])
except:
pass
return keys
keys_input = print_dict(input)
keys = []
keys_main = print_dict(main)
print(keys_input)
print(keys_main)
for i in keys_input[:]:
if i in keys_main:
keys_input.remove(i)
print(keys_input)
This has worked for me. you can check above code snippet and if any changes provide more information so any chances if required.
Dictionary and lists compare theire content nested by default.
input_data == main should result in the right output if you format your dicts correctly. Try adding curly brackets "{"/"}" arround your dicts. It should probably look like something like this:
main = {"app": [{
"name": str,
"info": [
{
"role": str,
"scope": {"groups": list}
}
]
},{
"name": str,
"info": [
{"role": str}
]
}]}
input_data = {"app":[{
'name': 'nms',
'info': [
{
'role': 'user',
'scope': {'groups': ['xyz']
}
}]
},{
'name': 'abc',
'info': [
{'rol': 'user'}
]
}]}
input_data2 = {"app": [{
'name': 'nms',
'info': [
{
'role': 'user',
'scope': {'groups': ['xyz']
}
}]
}, {
'name': 'abc',
'info': [
{'rol': 'user'}
]
}]}
Comparision results should look like this:
input_data2 == input_data # True
main == input_data # False
I have some json that I would like to transform from this:
[
{
"name":"field1",
"intValue":"1"
},
{
"name":"field2",
"intValue":"2"
},
...
{
"name":"fieldN",
"intValue":"N"
}
]
into this:
{ "field1" : "1",
"field2" : "2",
...
"fieldN" : "N",
}
For each pair, I need to change the value of the name field to a key, and the values of the intValue field to a value. This doesn't seem like flattening or denormalizing. Are there any tools that might do this out-of-the-box, or will this have to be brute-forced? What's the most pythonic way to accomplish this?
parameters = [ # assuming this is loaded already
{
"name":"field1",
"intValue":"1"
},
{
"name":"field2",
"intValue":"2"
},
{
"name":"fieldN",
"intValue":"N"
}
]
field_int_map = dict()
for p in parameters:
field_int_map[p['name']] = p['intValue']
yields {'field1': '1', 'field2': '2', 'fieldN': 'N'}
or as a dict comprehension
field_int_map = {p['name']:p['intValue'] for p in parameters}
This works to combine the name attribute with the intValue as key:value pairs, but the result is a dictionary instead of the original input type which was a list.
Use dictionary comprehension:
json_dct = {"parameters":
[
{
"name":"field1",
"intValue":"1"
},
{
"name":"field2",
"intValue":"2"
},
{
"name":"fieldN",
"intValue":"N"
}
]}
dct = {d["name"]: d["intValue"] for d in json_dct["parameters"]}
print(dct)
# {'field1': '1', 'field2': '2', 'fieldN': 'N'}
I have a cte query that returns me results of values that are linked (i.e. child -> parent).
Then in Python I am trying to create a nested dictionary that would represent something like this:
{
"name": "Child_example",
"parents": [
{
"name": "child_parent_1",
"parents": [{"name": "child_parent_1_parent", "parents": [{"name": "end", "parents": []}]}]
},
{
"name": "child_parent_2",
"parents": [{"name": "end", "parents": []}]
},
{
"name": "child_parent_3",
"parents": [{"name": "child_parent_3_parent", "parents": [{"name": "end", "parents": []}]}]
}
]
}
My input data looks something like so (it can have more data):
child_col
parent_col
name
depth
Child_example
child_parent_1_col
child_parent_1
0
Child_example
child_parent_2_col
child_parent_2
0
Child_example
child_parent_3_col
child_parent_3
0
child_parent_1_col
child_parent_1_parent
1_parent
1
child_parent_2_col
end
1_parent
1
child_parent_3_col
child_parent_3_parent
3_parent
1
child_parent_3_parent
end
end_3
2
child_parent_1_parent
end
end_1
2
However with my code so far:
r_dict = defaultdict(list)
depth_zero = [x for x in rows if x.depth == 0]
for row in depth_zero:
r_dict['name'] = row.path_key
r_dict['parents'].append({'name': row.path_parent_key, 'parents': []})
depth_not_zero = [x for x in rows if x.depth != 0]
# Set inner levels
for parent in r_dict['parents']:
name = parent['name']
inner_parent = parent['parents'].copy()
for row in depth_not_zero:
if row.path_key == name:
inner_parent.append({'name': row.path_parent_key, 'parents': []})
name = row.path_parent_key
parent['parents'] = inner_parent
I only manage to achieve to append it to initial "parents", instead of setting the ultimate nested "parents". I know it is to do with this line of code:
inner_parent.append({'name': row.path_parent_key, 'parents': []})
But I cannot work out how to essentially get and set it. Would this be a case for recursion instead of the way I am doing it?
Below is an example of the first nested dictionary output that I am currently creating with my code:
{
"name": "Child_example",
"parents": [
{
"name": "child_parent_1",
"parents": [
{"name": "child_parent_1", "parents": []}, {"name": "end", "parents": []}
]
}
]
}
I'm a bit baffled by the way you are assigning the "name" value: "Child_example" comes from child_col, "child_parent_1" from name, "child_parent_3_parent" from parent_col. So I simplified it a bit: I put in the second column of the child row the same value as in the first column of its parents rows. That said, if you really need to take the names from different columns it's just a matter of adding some ifs.
My proposal is to loop over the rows in reverse order, creating the inner dicts and then moving them into the outer ones:
rows = [["c1","p1c1",0],
["c1","p2c1",0],
["c1","p3c1",0],
["p1c1","p1p1c1",1],
["p2c1","end",1],
["p3c1","p1p3c1",1],
["p1p3c1","end",2],
["p1p1c1","end",2]]
r_dict = {}
for row in reversed(rows):
if row[1] == "end":
r_dict[row[0]] = {"name":row[0], "parents":[]}
else:
if not row[0] in r_dict:
r_dict[row[0]] = {"name":row[0], "parents":[]}
r_dict[row[0]]["parents"].append(r_dict[row[1]])
del r_dict[row[1]]
r_dict
{'c1': {'name': 'c1', 'parents': [{'name': 'p3c1', 'parents': [{'name': 'p1p3c1', 'parents': []}]}, {'name': 'p2c1', 'parents': []}, {'name': 'p1c1', 'parents': [{'name': 'p1p1c1', 'parents': []}]}]}}
I´m trying to extract several keys/values out of a List.
My List:
a = [
{
"id": "1",
"system": "2",
},
{
"id": "3",
"system": "4",
}
]
Now i need to parse this into a function (next function) and it returns a[current] or a[0].
But now is a[current] or a[0] a dict.
Next step is just to extract the ID and the value of it. But this below only works if a is a list! So i need to convert the a[current] or a[0] into a list. The code below has to be the same because it´s a function and if i cannot change this for several reasons, so i need to convert the dict a into a list a.
c = list()
for data in a:
value = dict()
value["id"] = data.get("id")
c.append(value)
And here i stuck, i tried several methods like .keys(), .values(), but i can´t put them together to a list anymore. It needs to be scaleable/configurable because a changes from time to time (not a[0]["id"], ...). Currently a[0] looks like this: {'id': '1', 'system': '2'}, but it needs to be like this: [{'id': '1', 'system': '2'},], that i can parse it to my search function.
I need a new list like c:
c = [
{
"id": "1",
},
{
"id": "3",
}
]
Is this your your expected output:
a = [
{
"id": "1",
"system": "2",
},
{
"id": "3",
"system": "4",
}
]
c = list()
for data in a:
value={}
value["id"]=data.get("id")
c.append([value])
# c.extend([value])
print(c)
# [[{'id': '1'}], [{'id': '3'}]]
# print(c) # Extend's output
# [{'id': '1'},{'id': '3'}]
Or you can try one-line solution
print([[{"id":val.get("id")}] for val in a])
# [[{'id': '1'}], [{'id': '3'}]]
Or as your comment if you just want
[{'id': '1', 'system': '2'},]
Then :
print([a[0]])
code updated:
a = [
{
"id": "1",
"system": "2",
},
{
"id": "3",
"system": "4",
}
]
print([[value] for value in a ])
Result:
[[{'id': '1', 'system': '2'}], [{'id': '3', 'system': '4'}]]
Here is a function to filter your dicts list:
def filter_dicts(key, a):
return [{key: element[key]} for element in a]
Use it like this:
c = filter_dicts("id", a)
Note that this will cause an error if there is a dict without the specified key, which may or may not be what you want. To avoid this, replace element[key] with element.get(key, None).
I am trying to compare 2 json files to do a delta check between them.
Exising json:
{
"rules": [
{
"customer_id": "abc123",
"id": "1",
},
{
"customer_id": "xyz456",
"id": "2",
}
]
}
Updated json:
{
"rules": [
{
"customer_id": "abc123",
"id": "1",
},
{
"customer_id": "def456",
"id": "3",
},
{
"customer_id": "xyz789",
"id": "2",
}
]
}
What i want is my code to get the new objects from the new json(in this case id:3 and customer id def456)
however i also want to keep the original existing values (id:2 customer id should remain as xyz456 instead of updated to the new value xyz789)
Here is my current code:
import json
# Opening JSON file
f = open('1.json',)
y = open('2.json',)
# returns JSON object as a dictionary
less_data = json.load(f)
more_data = json.load(y)
# Iterating through the json list
for x in more_data['rules']:
for y in less_data['rules']:
if x['id']== y['id']:
print("x:" + x['id'],"y:" + y['id'])
break
print(x['id'] + " is not found")
//take action to add in new objects into json output
running the program i get the following output:
x:1 y:1
1 is found
x:3 y:1
3 is not found
x:3 y:2
3 is not found
x:2 y:1
2 is not found
x:2 y:2
2 is found
I only want 3 is not found to be printed once after running till the end of the inner for loop instead of printing it out every iteration. Any help would be appreaciated
You can try flatten the JSON & compare its keys as dict\json datatype is unordered kind of datatype:
If you list ordering matters
import collections
from itertools import chain
def flatten(d, parent_key='', sep='__'):
items = []
for k, v in d.items():
new_key = parent_key + sep + k if parent_key else k
if isinstance(v, collections.MutableMapping):
items.extend(flatten(v, new_key, sep=sep).items())
elif isinstance(v, list):
for idx, value in enumerate(v):
items.extend(flatten(value, new_key + sep + str(idx), sep).items())
else:
items.append((new_key, v))
return dict(items)
a1 = {'rules': [{'customer_id': 'abc123', 'id': '1'}, {'customer_id': 'xyz456', 'id': '2'}]}
a2 = {'rules': [{'customer_id': 'abc123', 'id': '1'}, {'customer_id': 'def456', 'id': '3'}, {'customer_id': 'xyz789', 'id': '2'}]}
flatten_a1 = flatten(a1)
flatten_a2 = flatten(a2)
print(flatten_a1)
print(flatten_a2)
Output:
>>> flatten_a1 => {'rules__0__customer_id': 'abc123', 'rules__0__id': '1', 'rules__1__customer_id': 'xyz456', 'rules__1__id': '2'}
>>> flatten_a2 => {'rules__0__customer_id': 'abc123', 'rules__0__id': '1', 'rules__1__customer_id': 'def456', 'rules__1__id': '3', 'rules__2__customer_id': 'xyz789', 'rules__2__id': '2'}
from flatten_a1 & flatten_a2 value, you can easily find the differences as data structure comes in single level (i.e. not in nested format)
You can find:
which keys are missing
keys are present but values are different