I am currently trying to write something to convert an Ansible inventory file into a JSON array which would allow it to be pulled into awx/tower however I have struggled to build a brand new array from the current inventory file format. I am avoiding use of any of the Ansible python API modules as there is no guarantee that future updates won't break these. One solution I found no longer works as there appears to be a change to the Ansible InventoryParser python module so I'm trying to come up with a Python 2.7 solution.
Example inventory file;
[test]
host1
host2
[test1]
host3
host4
The [] signify groups and the other entries are the hosts which will be the key:value relationship. I have converted this to a list in python and I am then attempting to format this into a key:value setup using the [] as where to split the key from the values.
both = []
f = open(filename, "r")
line = f.readline().strip()
while line:
both.append(line)
line = f.readline().strip()
f.close()
start = '['
end = ']'
json_dict = {'all': [dict(item.split(start)[1].split(end)[0] for item in
both)]}
print json.dumps(json_dict)
Unfortunately this returns the error:
ValueError: dictionary update sequence element #0 has length 4; 2 is required
Although truth be told I'm not sure this will return what I am looking for regardless.
Hoping someone can point me in the right direction or highlight where I've gone wrong so far.
Cheers
EDIT: Adding some code for what output is actually expected;
{
[test]: {
'hosts': ['host1', 'host2'],
},
[test1]: {
'hosts': ['host3', 'host4'],
}
}
A more detailed output example of what I'm trying to achieve;
{
"databases" : {
"hosts" : [ "host1.example.com", "host2.example.com" ],
"vars" : {
"a" : true
}
},
"webservers" : [ "host2.example.com", "host3.example.com" ],
"atlanta" : {
"hosts" : [ "host1.example.com", "host4.example.com",
"host5.example.com" ],
"vars" : {
"b" : false
},
},
"marietta" : [ "host6.example.com" ],
"5points" : [ "host7.example.com" ]
}
So we have a key which holds the group names and within that there are key:value pairs for hosts and vars.
After some more study I am closer to the output I desire with the following code;
both = {}
group = None
with open(filename, "r") as f:
line = f.readline().strip()
while line:
if line.startswith('#') or line.startswith(';') or len(line) == 0:
continue
if line.startswith("["):
# is a group
group = line
both[group] = {}
elif not line.startswith("["):
host = line
both[group][host] = {}
line = f.readline().strip()
f.close()
return both
Which returns the following which isn't quite what I'm after but I feel like I am making progress;
{
"[test2]": {
"host1": {},
"host2": {}
},
"[test3]": {
"host3": {}
},
"[test]": {
"host4": {},
"host5": {}
}
}
It's may help you.
import json
both = {}
start = '['
end = ']'
with open(filename, "r") as f:
line = f.readline().strip()
while line:
if start in line or end in line:
line = line.split(start)[1].split(end)[0]
both[line] = line
line = f.readline().strip()
json_dict = {'all': [both]}
print(json.dumps(json_dict))
Related
I'm trying to create multiple JSON files with different numbers at specific value, this is my code :
import json
json_dict = {
"assetName": "GhostCastle#",
"previewImageNft": {
"mime_Type": "png",
"description": "#",
"fileFromIPFS": "QmNuFreEoJy9CHhXchxaDAwuFXPHu84KYWY9U7S2banxFS/#.png",
"metadataPlaceholder": [
{
"": ""
}
]
}
}
n = 10
for i in range(1, n+1):
json_dict["assetName"] = f"GhostCastle{i}"
json_dict[#What to put here to choose "fileFromIPFS"] = f"QmNuFreEoJy9CHhXchxaDAwuFXPHu84KYWY9U7S2banxFS/{i}.png"
with open(f"{i}.json", 'w') as json_file:
#json.dump() method save dict json to file
json.dump(json_dict, json_file)
so What to put to choose "fileFromIPFS" in the second json_dict
Ive a json file,
{
"IGCSE":[
{
"rolename": "igcsesubject1",
"roleid": 764106550863462431
},
{
"rolename": "igcsesubject2",
"roleid": 764106550863462431
}
],
"AS":[
{
"rolename": "assubject1",
"roleid": 854789476987546
},
{
"rolename": "assubject2",
"roleid": 854789476987546
}
],
"A2":[
{
"rolename": "a2subject1",
"roleid": 854789476987856
},
{
"rolename": "a2subject2",
"roleid": 854789476987856
}
]
}
I want to fetch the keys [igcse, as, a2..] and then fetch the rolename and roleids under the specific keys. How do i do it?
Below is the python code for how i used to do it without the keys.
with open(fileloc) as f:
data = json.load(f)
for s in range(len(data)):
d1 = data[s]
rname = d1["rolename"]
rid = d1["roleid"]
any help would be appreciated :)
First you can have a list of keys, under which you will get them:
l = ['A1','A2']
Then iterate like this:
for x in data:
if x in l:
for y in range(len(data[x])):
print(j[x][y]['rolename'])
print(j[x][y]['roleid'])
hi you can use for and you will get the keys:
with open(fileloc) as f:
data = json.load(f)
for s in data:
d1 = data[s]
rname = d1["rolename"]
rid = d1["roleid"]
The following would work for what you need:
with open(file) as f:
json_dict = json.load(f)
for key in json_dict:
value_list = json_dict[key]
for item in value_list:
rname = item["rolename"]
rid = item["roleid"]
If you need to filter for specific keys in the JSON, you can have a list of keys you want to obtain and filter for those keys as you iterate through the keys (similar to Wasif Hasan's suggestion above).
yaml file contents:
a: a1
b:
b1:
- name: c1-name
type:
ctype:
content: true
I can safe load the above contents using pyyaml module and modify the data (newkey to add, value) as follows
with open(filepath, 'r') as f:
data = yaml.safe_load(f)
data['b]['b1'][0]['type']['ctype'][newkey to add]=value
But I want to do this for some other yaml files where position to add the new key, values are different, so I decided to write this in a function by sending filepath, position and key, value as arguments as follows:
def modify_yaml(filepath,position,key,value):
with open(filepath, 'r') as f:
data = yaml.safe_load(f)
data[position][key]=value
return data
But the above code doesn't work. Please help if there is a way to achieve this.
"b": {
"b1": [
{
"name": "c1-name",
"type": {
"ctype": {
"content": true
}
}
}
]
}
I am trying to create a json object using Dictionary in python. As far as I understand the key part needs to be unique but in my case the array has multiple items with the same key so looks like Dictionary will not work for me here. Trying to understand my options here? Finally I will be saving this json object into a json file on the server.
data = {}
data['key1'] = hostname
for line in pipe.stdout:
parts = line.split() # split line into parts
if len(parts) > 1: # if at least 2 parts/columns
data['package'] = { 'name': parts[0], 'installed': parts[1], 'available': parts[2]}
print(json.dumps(data, indent=4))
Expected Json Output
{
"key1": "xyz-abc-m001",
"package": [
{ "name":"abc", "installed":"1:1", "available":"1:1.2." },
{ "name":"xyz", "installed":"2.02", "available":"2.02" },
{ "name":"zyc", "installed":"1.17.1", "available":"1.17.1" }
]
}
data = {}
data['key1'] = hostname
data['package'] = []
for line in pipe.stdout:
parts = line.split() # split line into parts
if len(parts) > 1: # if at least 2 parts/columns
data['package'].append({ 'name': parts[0], 'installed': parts[1], 'available': parts[2]})
What i have to parse :
I have a tsv file that looks like this :
https://i.stack.imgur.com/yxsXD.png
What is the end goal:
My goal is to read the tsv file and populate the contents of the csv file in a dictionary and nested lists without using csv parser.
In the end the in_memory_table structure would look
like this ( of course with more than two rows ):
{
"header": [
"STATION",
"STATION_ID",
"ELEVATION",
"LAT",
"LONG",
"DATE",
"MNTH_MIN",
"MNTH_MAX"
],
"rows": [
[
"Tukwila",
"12345afbl",
"10",
"47.5463454",
"-122.34234234",
"2016-01-01",
"10",
"41"
],
[
"Tukwila",
"12345afbl",
"10",
"47.5463454",
"-122.34234234",
"2016-02-01",
"5",
"35"
],
]
}
My code looks like this:
in_memory_table = {
'header': [],
'rows': [] }
with open('fahrenheit_monthly_readings.tsv') as f:
in_file = f.readlines()
i = 0
for line in in_file:
temp_list = [line.split('\t')]
if (i == 0):
in_memory_table['header']= line
elif(i != 0):
in_memory_table['rows'].append(line)
i += 1
print("\n",in_memory_table)
Output of the code:
C:\Users\svats\AppData\Local\Programs\Python\Python36-32\python.exe C:/Users/svats/PycharmProjects/BrandNew/module4_lab2/module4_lab2.py
{'header': 'STATION\tSTATION_ID\tELEVATION\tLAT\tLONG\tDATE\tMNTH_MIN\tMNTH_MAX\n', 'rows': ['Tukwila\t12345afbl\t10\t47.5463454\t-122.34234234\t2016-01-01\t10\t41\n', 'Tukwila\t12345afbl\t10\t47.5463454\t-122.34234234\t2016-02-01\t5\t35\n', 'Tukwila\t12345afbl\t10\t47.5463454\t-122.34234234\t2016-03-01\t32\t47\n', 'Tukwila\t12345afbl\t10\t47.5463454\t-122.34234234\t2016-04-01\t35\t49\n', 'Tukwila\t12345afbl\t10\t47.5463454\t-122.34234234\t2016-05-01\t41\t60\n', 'Tukwila\t12345afbl\t10\t47.5463454\t-122.34234234\t2016-06-01\t50\t72\n', 'Tukwila\t12345afbl\t10\t47.5463454\t-122.34234234\t2016-07-01\t57\t70\n', 'Tukwila\t12345afbl\t10\t47.5463454\t-122.34234234\t2016-08-01\t68\t79\n', 'Tukwila\t12345afbl\t10\t47.5463454\t-122.34234234\t2016-09-01\t55\t71\n', 'Tukwila\t12345afbl\t10\t47.5463454\t-122.34234234\t2016-10-01\t47\t77\n', 'Tukwila\t12345afbl\t10\t47.5463454\t-122.34234234\t2016-11-01\t32\t66\n', 'Tukwila\t12345afbl\t10\t47.5463454\t-122.34234234\t2016-12-01\t27\t55\n']}
Help needed:
i am very close towards getting the solution
I have 2 questions :
1. how to get rid of the \t in the o/p?
2. My o/p is little different from the desired o/p. how do i get it ?
If you rewrite your code as:
for line in in_file:
print('repr(line) before :', repr(line) )
temp_list = [line.split()]
#line = line.split()
print('temp_list :',temp_list)
print('repr(line) after :', repr(line) )
print(' %s -----------------' % i)
if ........
and de-comment the line #line = line.split()
you'll understand the reason of the bad result you obtain.
The reason is that line.split() doesn't change the object of name line ,
it creates a new object (the list you want) to which name line must be re-assigned if you want this name to refer to the obtained list.
Note that the method str.split([sep[, maxsplit]]) has a different algorithm according if parameter sep is None or not None, see documentation https://docs.python.org/2/library/stdtypes.html#str.split for this point
.
That said, there's a better way.
with open('fahrenheit_monthly_readings.tsv','r') as f:
in_memory_table = {'header':next(f).split()}
in_memory_table['rows'] = [line.split() for line in f]
or
with open('fahrenheit_monthly_readings.tsv','r') as f:
in_memory_table = {'header':next(f).split()}
in_memory_table['rows'] = list(map(str.split, f))