Basically I'm generating a json Terraform file from a text file, but I can't get it to format in the correct way:
I want the finished Terraform file to look like this:
{
"resource": {
"aws_route53_record": {
"analytics": {
"name": "analytics",
"records": ["1.2.3.4"],
"ttl": "1800",
"type": "A"
},
"analytics-test": {
"name": "analytics-test",
"records": ["1.2.3.4"],
"ttl": "300",
"type": "A"
}
}
}
}
which is the format Terraform requires to parse json.
So I load the text file in Python, and iterate over each line producing a list of lists that look like so:
records = [["analytics", "1.2.3.4", "1800", "A"],["analytics-test", "1.2.3.4", "300", "A"]]
My code to generate the file at the moment looks like this
I create a dict placeholder containing top level variable like so:
json_object = {'resource': {'aws_route53_record': None}}
Then I look through records and assign the appropriate values:
for each_list in data:
terrarecord = {
each_list[0]:{
"name": each_list[0],
"type": each_list[2],
"ttl": each_list[1],
"records": [each_list[3].replace('\n', '')]
}
}
record_holder.append(terrarecord)
The record_holder object is an empty list that I then use to fill in the json_objects like so:
json_object['resource']['aws_route53_record'] = record_holder
What this gives me in the finished file is:
{
"resource": {
"aws_route53_record": [{
"analytics": {
"ttl": "1800",
"records": ["173.194.245.129"],
"name": "analytics",
"type": "A"
}
}, {
"analytics-test": {
"ttl": "300",
"records": ["130.211.89.168"],
"name": "analytics-test",
"type": "A"
}
}]
}
}
So would there be an easier way to do this without adding the extra [] and {}s that my little loop does?
Why are you creating the intermediate list when you want the dictionary?
terrarecord = {}
for each_list in data:
terrarecord[each_list[0]] = {
"name": each_list[0],
"type": each_list[2],
"ttl": each_list[1],
"records": [each_list[3].replace('\n', '')]
}
}
json_object['resource']['aws_route53_record'] = terrarecord
Related
I have some data which looks like this :
{
"key_value": [
{
"key": "name",
"value": "kapil"
},
{
"key": "age",
"value": "36"
}
]
}
I need to convert it to look like this:
{
"age": "36",
"name": "kapil"
}
Would somebody be able to help with this?
I have already tried using json.dumps()
I'm not sure why you were trying to use json.dumps, but all you need to do is loop through all the pairs and add them to a new dictionary. Like this:
data = {
"key_value": [
{
"key": "name",
"value": "kapil"
},
{
"key": "age",
"value": "36"
}
]
}
res = {}
for pair in data["key_value"]:
res[pair["key"]] = pair["value"]
print(res)
Note that if your data is in JSON, then you need to use json.loads() to convert your JSON to a dictionary, then use json.dumps() to convert that dictionary back to a string that can be written to a file.
I have a nested JSON array, and a separate second array.
Would like perform the equivalent of a SQL UPDATE using a left join.
In other words, keep all items from the main json, and where the same item (key='order') appears in the secondary one, update/append values in the main.
Can obviously achieve this by looping - but really looking for a more elegant & efficient solution.
Most examples of 'merging' json I've seen involve appending new items, or appending - very little regarding 'updating'.
Any pointers appreciated :)
Main JSON object with nested array 'steps'
{
"manifest_header": {
"name": "test",
},
"steps": [
{
"order": "100",
"value": "some value"
},
{
"order": "200",
"value": "some other value"
}
]
}
JSON Array with values to add
{
"steps": [
{
"order": "200",
"etag": "aaaaabbbbbccccddddeeeeefffffgggg"
}
]
}
Desired Result:
{
"manifest_header": {
"name": "test",
},
"steps": [
{
"order": "100",
"value": "some value"
},
{
"order": "200",
"value": "some other value",
"etag": "aaaaabbbbbccccddddeeeeefffffgggg"
}
]
}
I have a csv file and trying to compose JSON from it. There are mulitple records in a file but I am just giving one set of sample records here.This structure is driven on the claimID. There is nesting on the claimLineDetail and claimSpecDiag.I guess I have to create some sort of list to handle this then the problem is how am I going to append it in the required structure. I really need some guidance here to achieve the desired result. Is it possible to break out different sections and append it later, I am not sure just assuming, as there are multiple columns.
Code :
import csv,json
data = []
with open('JsonRequestPricingMedical.csv','r') as f:
reader = csv.DictReader(f)
for row in reader:
print row
csv file :
claimId,subscriberId,claimType,claimSubType,providerId,totalChargeAmt,claimLineNo,pos_code,procedureCode,subdiagnosisCode,svcLineFromDt,svcLineToDt,chargedAmt,clmLineUnits,presentOnAdmit,diagnosisCode
18A000730400,101924200,M,M,002664514003,585,1,11,92014,H43393,2017-06-19,2017-06-19,160,1,U,H43393
18A000730400,101924200,M,M,002664514003,585,2,12,92015,H43395,2017-06-19,2017-06-19,160,2,U,H43394
Desired JSON
[
{
"claimsHeader":" {
"claimId": "18A000730400",
"subscriberId": "101924200",
"claimType":{
"code": "M"
},
"claimSubType": {
"code": "M"
},
"providerId" :"002664514003",
"totalChargeAmt": "585",
"claimLineDetail" :[
{
"claimLineNo": "1",
"placeOfService": {
"code": "11"
},
"procedureCode": {
"code": "92014"
},
"subDiagnosisCd": {
"code": "H43393"
},
"svcLineFromDt": "2017-06-19",
"svcLineToDt": "2017-06-19",
"chargedAmt": "160",
"clmLineUnits": "1",
},
{
"claimLineNo": "2",
"placeOfService": {
"code": "12"
},
"procedureCode": {
"code": "92015"
},
"subDiagnosisCd": {
"code": "H433945
},
"svcLineFromDt": "2017-06-19",
"svcLineToDt": "2017-06-19",
"chargedAmt": "160",
"clmLineUnits": "2",
}
],
{
"claimSpecDiag": [
"presentOnAdmit": "",
"diagnosisCode": "H43393",
},
{
"presentOnAdmit": "",
"diagnosisCode": "H43394",
}
]
}
]
When you read a csv, each line represents variables separated by a special char, in your case, comas: ",".
You can get each variable separated by doing line_variables = row.split(',')
Just pass the first line, and for all the other, do something like:
result = {
"claimsHeader":" {
"claimId": line_variables[0],
"subscriberId": line_variables[1],
"claimType":{
"code": line_variables[2]
}
...
Finaly, just add the result to a list (created just before your for loop) with your_list.append(result).
Are there any python helper libraries I can use to create models that I can use to generate complex json files, such as this. I've read about colander but I'm not sure it does what I need. The tricky bit about the following is that the trigger-rule section may have nested match rules, something as described at https://github.com/adnanh/webhook/wiki/Hook-Rules
[
{
"id": "webhook",
"execute-command": "/home/adnan/redeploy-go-webhook.sh",
"command-working-directory": "/home/adnan/go",
"pass-arguments-to-command":
[
{
"source": "payload",
"name": "head_commit.id"
},
{
"source": "payload",
"name": "pusher.name"
},
{
"source": "payload",
"name": "pusher.email"
}
],
"trigger-rule":
{
"and":
[
{
"match":
{
"type": "payload-hash-sha1",
"secret": "mysecret",
"parameter":
{
"source": "header",
"name": "X-Hub-Signature"
}
}
},
{
"match":
{
"type": "value",
"value": "refs/heads/master",
"parameter":
{
"source": "payload",
"name": "ref"
}
}
}
]
}
}
]
Define a class like this:
class AttributeDictionary(dict):
__getattr__ = dict.__getitem__
__setattr__ = dict.__setitem__
When you load your JSON, pass AttributeDictionary as the object_hook:
import json
data = json.loads(json_str, object_hook=AttributeDictionary)
Then you can access dict entries by specifying the key as an attribute:
print data[0].id
Output
webhook
Note: You will want to replace dashes in keys with underscores. If you don't, this approach won't work on those keys.
I currently have a json file that looks like this....
{
"data": [
{
"tag": "cashandequivalents",
"value": 10027000000.0
},
{
"tag": "shortterminvestments",
"value": 101000000.0
},
{
"tag": "accountsreceivable",
"value": 4635000000.0
},
{
"tag": "netinventory",
"value": 1386000000.0
}...
but what I am trying to get to is this
{
"cashandequivalents": 10027000000.0,
"shortterminvestments":101000000.0 ,
"accountsreceivable":4635000000.0,
"netinventory":1386000000.0
}
I just don't know how to go about this.
Maybe there is an easier way, but this seems the most logical to me because the next step is writer.writerow to csv
So eventually the csv will look like
cashandequivalents | shortterminvestments | accountsreceivable | netinventory
100027000000 101000000000 46350000000 13860000000
########### ############ ########### ...........
(writer.writeheader will be done outside of the loop so I am only writing the values, not the "tags")
Thanks
A naive solution:
import json
json_data = {
"data": [
{
"tag": "cashandequivalents",
"value": 10027000000.0
},
{
"tag": "shortterminvestments",
"value": 101000000.0
},
{
"tag": "accountsreceivable",
"value": 4635000000.0
},
{
"tag": "netinventory",
"value": 1386000000.0
}
]
}
result = dict()
for entry in json_data['data']:
result[entry['tag']] = entry['value']
print json.dumps(result, indent=4)
Output
{
"shortterminvestments": 101000000.0,
"netinventory": 1386000000.0,
"accountsreceivable": 4635000000.0,
"cashandequivalents": 10027000000.0
}
The easiest and cleanest way to do this is with a dictionary comprehension.
d = {
"data": [
{
"tag": "cashandequivalents",
"value": 10027000000.0
},
{
"tag": "shortterminvestments",
"value": 101000000.0
},
{
"tag": "accountsreceivable",
"value": 4635000000.0
},
{
"tag": "netinventory",
"value": 1386000000.0
}
]
}
newDict = {i['tag']: i['value'] for i in d['data']}
# {'netinventory': 1386000000.0, 'shortterminvestments': 101000000.0, 'accountsreceivable': 4635000000.0, 'cashandequivalents': 10027000000.0}
This iterates through the list that is contained within the "data" key of your original dictionary and creates a new one inline with the key being the tag value of each and the value being the value for each during the iterations.