We Keep Coding

replace nested document array mongodb with python

i have this document in mongodb
{
"_id": {
"$oid": "62644af0368cb0a46d7c2a95"
},
"insertionData": "23/04/2022 19:50:50",
"ipfsMetadata": {
"Name": "data.json",
"Hash": "Qmb3FWgyJHzJA7WCBX1phgkV93GiEQ9UDWUYffDqUCbe7E",
"Size": "431"
},
"metadata": {
"sessionDate": "20220415 17:42:55",
"dataSender": "user345",
"data": {
"height": "180",
"weight": "80"
},
"addtionalInformation": [
{
"name": "poolsize",
"value": "30m"
},
{
"name": "swimStyle",
"value": "mariposa"
},
{
"name": "modality",
"value": "swim"
},
{
"name": "gender-title",
"value": "schoolA"
}
]
},
"fileId": {
"$numberLong": "4"
}
}
I want to update nested array document, for instance the name with gender-tittle. This have value schoolA and i want to change to adult like the body. I give the parameter number of fileId in the post request and in body i pass this
post request : localhost/sessionUpdate/4
and body:
{
"name": "gender-title",
"value": "adultos"
}
flask
#app.route('/sessionUpdate/<string:a>', methods=['PUT'])
def sessionUpdate(a):
datas=request.json
r=str(datas['name'])
r2=str(datas['value'])
print(r,r2)
r3=collection.update_one({'fileId':a, 'metadata.addtionalInformation':r}, {'$set':{'metadata.addtionalInformation.$.value':r2}})
return str(r3),200
i'm getting the 200 but the document don't update with the new value.

As you are using positional operator $ to work with your array, make sure your select query is targeting array element. You can see in below query that it is targeting metadata.addtionalInformation array with the condition that name: "gender-title"
db.collection.update({
"fileId": 4,
"metadata.addtionalInformation.name": "gender-title"
},
{
"$set": {
"metadata.addtionalInformation.$.value": "junior"
}
})
Here is the Mongo playground for your reference.

How to convert a complete JSON form into XML

If I have the following code and want to convert to XML:
Note: I tried using json2xml, but it doesn't convert the complete set, rather just converts a segment of it.
{
"odoo": {
"data": {
"record": [
{
"model": "ir.ui.view",
"id": "lab_tree_view",
"field": [
{
"name": "name",
"#text": "human.name.tree"
},
{
"name": "model",
"#text": "human.name"
},
{
"name": "priority",
"eval": "16"
},
{
"name": "arch",
"type": "xml",
"tree": {
"string": "Human Name",
"field": [
{"name": "name"},
{"name": "family"},
{"name": "given"},
{"name": "prefix"}
]
}
}
]
},
{
"model": "ir.ui.view",
"id": "human_name_form_view",
"field": [
{
"name": "name",
"#text": "human.name.form"
},
{
"name": "model",
"#text": "human.name"
},
{
"name": "arch",
"type": "xml",
"form": {
"string": "Human Name Form",
"sheet": {
"group": {
"field": [
{"name": "name"},
{"name": "family"},
{"name": "given"},
{"name": "prefix"}
]
}
}
}
}
]
}
],
"#text": "\n\n\n #ACTION_WINDOW_FOR_PATIENT\n ",
"record#1": {
"model": "ir.actions.act_window",
"id": "action_human_name",
"field": [
{
"name": "name",
"#text": "Human Name"
},
{
"name": "res_model",
"#text": "human.name"
},
{
"name": "view_mode",
"#text": "tree,form"
},
{
"name": "help",
"type": "html",
"p": {
"class": "o_view_nocontent_smiling_face",
"#text": "Create the Human Name\n "
}
}
]
},
"menuitem": [
{
"id": "FHIR_root",
"name": "FHIR"
},
{
"id": "FHIR_human_name",
"name": "Human Name",
"parent": "FHIR_root",
"action": "action_human_name"
}
]
}
}
}
Is there any Python library or dedicated code to do this?
I tried building custom functions to break this out and convert them all, but, I am rather stuck in this problem.
The use case here is the code above input and the output should be the code generated by any online converter
EDIT:
from json2xml import json2xml
from json2xml.utils import readfromurl, readfromstring, readfromjson
data = readfromstring(string)
print(json2xml.Json2xml(data).to_xml()
Above code only converts a part of the json like the below code to xml:
{
"record": {
"model": "ir.ui.view",
"id": "address_tree_view",
"field": [
{
"name": "name",
"#text": "address.tree.view"
},
{
"name": "model",
"#text": "address"
},
{
"name": "priority",
"eval": "16"
},
{
"name": "arch",
"type": "xml",
"tree": {
"string": "Address",
"field": [
{
"name": "text_address"
},
{
"name": "address_line1"
},
{
"name": "country_id"
},
{
"name": "state_id"
},
{
"name": "address_district"
},
{
"name": "address_city"
},
{
"name": "address_postal_code"
}
]
}
}
]
}
}
PS: I have used the online converters but, I don't want to do that over here.

Use dicttoxml to convert JSON directly to XML
Installation
pip install dicttoxml
or
easy_install dicttoxml
In [2]: from json import loads
In [3]: from dicttoxml import dicttoxml
In [4]: json_obj = '{"main" : {"aaa" : "10", "bbb" : [1,2,3]}}'
In [5]: xml = dicttoxml(loads(json_obj))
In [6]: print(xml)
<?xml version="1.0" encoding="UTF-8" ?><root><main type="dict"><aaa type="str">10</aaa><bbb type="list"><item type="int">1</item><item type="int">2</item><item type="int">3</item></bbb></main></root>
In [7]: xml = dicttoxml(loads(json_obj), attr_type=False)
In [8]: print(xml)
<?xml version="1.0" encoding="UTF-8" ?><root><main><aaa>10</aaa><bbb><item>1</item><item>2</item><item>3</item></bbb></main></root>
For more information check here

trydicttoxml libary
if you are retrieving data from a JSON file
import json
import dicttoxml
with open("file_name.json", "r") as j:
data = json.load(j);
xml = dicttoxml.dicttoxml(data)
print(xml)

Use Python to iterate through nested JSON data

TASK:
I am using a API call to get JSON data from our TeamCity CI tool.
We need to identify all those builds which are using old version of msbuild.
We can identify from this API call data
{
"name": "msbuild_version",
"value": "15.0"
}
At the moment i am saving the entire API call data to a file; however i will later integrate the API call to the same script.
Now to the question at hand; How can i filter this above property i.e. msbuild_version, to say msbuild_version < 15.0 (i.e. all msbuild less than version 15.0) and display the corresponding 'id' and 'projectName' under 'buildType'; e.g.
"id": "AIntegration_BTool_BToolBuilds_DraftBuild",
"projectName": "A Integration / B Tool / VAR Builds",
here is a part of the JSON data file:-
{
"project": [{
"id": "_Root",
"buildTypes": {
"buildType": []
}
}, {
"id": "AI_BTool_BToolBuilds",
"buildTypes": {
"buildType": [{
"id": "AI_BTool_BToolBuilds_DraftBuild",
"projectName": "A I / B Tool / VAR Builds",
"steps": {
"step": [ {
"id": "RUNNER_213",
"name": "Build",
"type": "MSBuild",
"properties": {
"property": [ {
"name": "msbuild_version",
"value": "16.0"
}, {
"name": "run-platform",
"value": "x64"
}, {
"name": "targets",
"value": "Build"
}, {
"name": "teamcity.step.mode",
"value": "default"
}, {
"name": "toolsVersion",
"value": "15.0"
}]
}
}, {
"id": "RUNNER_228",
"name": "temp",
"type": "VS.Solution",
"properties": {
"property": [{
"name": "build-file-path",
"value": "x"
}, {
"name": "msbuild_version",
"value": "16.0"
}, {
"name": "vs.version",
"value": "vs2019"
}]
}
}]
}
}, {
"id": "AI_BTool_BToolBuilds_ContinuousBuildWithNexusI",
"projectName": "A I / B Tool / VAR Builds",
"steps": {
"step": [ {
"id": "RUNNER_22791",
"name": "Build",
"type": "MSBuild",
"properties": {
"property": [{
"name": "msbuild_version",
"value": "16.0"
}, {
"name": "run-platform",
"value": "x86"
}, {
"name": "teamcity.step.mode",
"value": "default"
}, {
"name": "toolsVersion",
"value": "15.0"
}]
}
}]
}
}]
}
}, {
"id": "AI_BTool_BToolBuilds_VARApiBuilds",
"buildTypes": {
"buildType": [{
"id": "AI_BTool_BToolBuilds_CiVARNewSolutionContinuousBuild",
"projectName": "A I / B Tool / VAR Builds / VAR API builds",
"steps": {
"step": [ {
"id": "RUNNER_22791",
"name": "Build",
"type": "MSBuild",
"properties": {
"property": [{
"name": "msbuilds_version",
"value": "15.0"
}, {
"name": "toolsVersion",
"value": "15.0"
}]
}
}]
}
}, {
"id": "AI_BTool_BToolBuilds_VARApiBuilds_CiVARIngestionWindowsServiceNonReleaseBranchBuild",
"projectName": "A I / B Tool / VAR Builds / VAR API builds",
"steps": {
"step": [{
"id": "RUNNER_22790",
"name": "Nuget Installer",
"type": "jb.nuget.installer",
"properties": {
"property": [{
"name": "nuget.path",
"value": "%teamcity.tool.NuGet.CommandLine.4.9.2%"
}, {
"name": "msbuilds_version",
"value": "16.0"
}, {
"name": "nuget.use.restore",
"value": "restore"
}, {
"name": "sln.path",
"value": "VAR.sln"
}, {
"name": "teamcity.step.mode",
"value": "default"
}]
}
}]
}
}]
}
}]
}
My Solution till now
And my code snippet till now
import json
with open('UnArchivedBuilds.txt') as api_call:
read_content = json.load(api_call)
#for project in read_content['project']:
# print (project.get('buildTypes'))
for project in read_content['project']:
# print (project['id'])
print (project['buildTypes']['buildType'])
I am not able to decide on the hierarchy of the JSON to print the relevant data (i.e id and projectName) wherever msbuild_version is less than 15.0

I had a look at your JSON data which was broken. In order to work with the snippet you provided I fixed the malformed data and removed unneeded parts to decrease clutter:
{
"project": [{
"buildTypes": {
"buildType": [{
"id": "AIntegration_BTool_BToolBuilds_DraftBuild",
"projectName": "A Integration / B Tool / VAR Builds"
},
{
"id": "AIntegration_BTool_BToolBuilds_ContinuousBuildIntegration",
"projectName": "A Integration / B Tool / VAR Builds"
}]
}
}]
}
As per my comment above I suggested recursion or using a schema validator to boil the JSON data down. However, as a quick-and-dirty approach and due to the weird structure of your dataset, I decided to do a multi-iteration over some parts of your data. Iterating over the same data is quite ugly and considered to be bad practice in most cases.
Assuming the data is stored in a file called input.json, the following snippet should give you the desired output:
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
import json
with open('input.json') as f:
data = json.load(f)
projects = (element for element in data.get('project'))
build_types = (element.get('buildTypes') for element in projects)
build_types = (element.get('buildType') for element in build_types)
for item in build_types:
for element in item:
identifier = element.get('id')
project_name = element.get('projectName')
print('{} --> {}'.format(identifier, project_name))
Printing:
AIntegration_BTool_BToolBuilds_DraftBuild --> A Integration / B Tool / VAR Builds
AIntegration_BTool_BToolBuilds_ContinuousBuildIntegration --> A Integration / B Tool / VAR Builds

Python - Parse complex JSON with objectpath

i need parse terraform file, write in JSON format. I have to extract two data, resource and id, this is example file:
{
"version": 1,
"serial": 1,
"modules": [
{
"path": [
"root"
],
"outputs": {
},
"resources": {
"aws_security_group.vpc-xxxxxxx-test-1": {
"type": "aws_security_group",
"primary": {
"id": "sg-xxxxxxxxxxxxxx",
"attributes": {
"description": "test-1",
"name": "test-1"
}
}
},
"aws_security_group.vpc-xxxxxxx-test-2": {
"type": "aws_security_group",
"primary": {
"id": "sg-yyyyyyyyyyyy",
"attributes": {
"description": "test-2",
"name": "test-2"
}
}
}
}
}
]
}
I need export for any resources, the first key and value of id, in this case, aws_security_group.vpc-xxxxxxx-test-1 sg-xxxxxxxxxxxxxx and aws_security_group.vpc-xxxxxxx-test-2 sg-yyyyyyyyyyyy
I have tried to write this in python:
#!/usr/bin/python3.6
import json
import objectpath
with open('file.json') as json_file:
data = json.load(json_file)
json_tree = objectpath.Tree(data['modules'])
result = tuple(json_tree.execute('$..resources[0]'))
result is
('aws_security_group.vpc-xxxxxxx-test-1', 'aws_security_group.vpc-xxxxxxx-test-2')
It's'ok but I can't extract the id, any help is appreciated, also use other methods
Thanks

I don't know objectpath, but I think you need:
tree.execute('$..resources[0]..primary.id')
or even just
tree.execute('$..resources[0]..id')

KeyError while accessing dictionary?

I have a dictionary named json_dict given below.
I need to access the element ==> json_dict['OptionSettings'][3]['Value'].
I need to access the element using the syntax
print(json_dict[parameter]).
When I give a parameter such as
param="['OptionSettings'][3]['Value']" or
param="'OptionSettings'][3]['Value']"
I am getting an error like the one below:
KeyError: "['OptionSettings'][3]['Value']".
I tried to use the below solution but it just printed a string
str1="json_dict"
print(str1+param)
Full Dictionary below:
{
"ApplicationName": "Test",
"EnvironmentName": "ABC-Nodejs",
"CNAMEPrefix": "ABC-Neptune",
"SolutionStackName": "64bit Amazon Linux 2016.03 v2.1.1 running Node.js",
"OptionSettings": [
{
"Namespace": "aws:ec2:vpc",
"OptionName": "AssociatePublicIpAddress",
"Value": "true"
},
{
"Namespace": "aws:elasticbeanstalk:environment",
"OptionName": "EnvironmentType",
"Value": "LoadBalanced"
},
{
"Namespace": "aws:ec2:vpc",
"OptionName": "Subnets",
"Value": "param1"
},
{
"Namespace": "aws:autoscaling:launchconfiguration",
"OptionName": "SecurityGroups",
"Value": "param2"
},
{
"Namespace": "aws:autoscaling:asg",
"OptionName": "MinSize",
"Value": "1"
},
{
"Namespace": "aws:autoscaling:asg",
"OptionName": "MaxSize",
"Value": "4"
},
{
"Namespace": "aws:autoscaling:asg",
"OptionName": "Availability Zones",
"Value": "Any"
},
{
"Namespace": "aws:autoscaling:asg",
"OptionName": "Cooldown",
"Value": "360"
},
{
"Namespace": "aws:autoscaling:launchconfiguration",
"OptionName": "IamInstanceProfile",
"Value": "NepRole"
},
{
"Namespace": "aws:autoscaling:launchconfiguration",
"OptionName": "MonitoringInterval",
"Value": "5 minutes"
},
{
"Namespace": "aws:autoscaling:launchconfiguration",
"OptionName": "RootVolumeType",
"Value": "gp2"
},
{
"Namespace": "aws:autoscaling:launchconfiguration",
"OptionName": "RootVolumeSize",
"Value": "10"
},
{
"Namespace": "aws:elasticbeanstalk:sns:topics",
"OptionName": "Notification Endpoint",
"Value": "sunil.kumar2#pb.com"
},
{
"Namespace": "aws:elasticbeanstalk:hostmanager",
"OptionName": "LogPublicationControl",
"Value": "false"
},
{
"Namespace": "aws:elasticbeanstalk:command",
"OptionName": "DeploymentPolicy",
"Value": "Rolling"
},
{
"Namespace": "aws:elasticbeanstalk:command",
"OptionName": "BatchSizeType",
"Value": "Percentage"
},
{
"Namespace": "aws:elasticbeanstalk:command",
"OptionName": "BatchSize",
"Value": "100"
},
{
"Namespace": "aws:elasticbeanstalk:command",
"OptionName": "HealthCheckSuccessThreshold",
"Value": "Ok"
},
{
"Namespace": "aws:elasticbeanstalk:command",
"OptionName": "IgnoreHealthCheck",
"Value": "false"
},
{
"Namespace": "aws:elasticbeanstalk:command",
"OptionName": "Timeout",
"Value": "600"
},
{
"Namespace": "aws:autoscaling:updatepolicy:rollingupdate",
"OptionName": "RollingUpdateEnabled",
"Value": "false"
},
{
"Namespace": "aws:ec2:vpc",
"OptionName": "ELBSubnets",
"Value": "param3"
},
{
"Namespace": "aws:elb:loadbalancer",
"OptionName": "SecurityGroups",
"Value": "param4"
},
{
"Namespace": "aws:elb:loadbalancer",
"OptionName": "ManagedSecurityGroup",
"Value": "param4"
}
]
}

Unfortunately you can't do that.
When you type param="['OptionSettings'][3]['Value']" and then json_dict[param], you are basically asking for the value represented by the key "['OptionSettings'][3]['Value']" which does not exists.
You´ll have to navigate through the levels until you get to the last one.
But of course, if you need a one-liner, you can always create some logic and extract that to a method.
For example, instead of
print(json_dict[param]).
you could use something like
print(get_json_value(json_dict, param))
and define a function such as
import re
def get_json_value(json_dict, params):
list_of_params = re.findall(r'\[([^]]*)\]', params)
#list_of_params = ['OptionSettings', '3', 'Value']
_ = json_dict
for elem in list_of_params:
_ = _[elem]
return _
I haven't tested it but it should work fine.
(Also, it is just a demo made to guide you through an alternate solution)

This worked for me
str1="json_dict"
params="['OptionSettings'][3]['Value']"
str2=str1+params
print(eval(str5))
Here the use of function eval() is the key to solve this.