I am trying to convert my json data into a dictionary with key the id of the json data. for example lets say i have the following json:
{
"id": "1",
"name": "John",
"surname": "Smith"
},
{
"id": "2",
"name": "Steve",
"surname": "Ger"
}
And i want to construct a new dictionary which includes the id as a key and save it into a file so i wrote the following code:
json_dict = []
request = requests.get('http://example.com/...')
with open("data.json", "w") as out:
loaded_data = json.loads(request.text)
for list_item in loaded_data:
json_dict.append({"id": list_item["id"], "data": list_item })
out.write(json.dumps(json_dict))
In the file i get the following output:
[{"data": {"id":"1",
"name":"John",
"Surname":"Smith"
}
},
{"data": {"id":"2",
"name":"Steve",
"Surname":"Ger"
}
},
]
Why the id is not included in my dict before data ?
I'm pretty sure you're looking at a ghost here. You probably tested wrong. It will go away when you try to create a minimal, complete, and verifiable example for us (i. e. with a fixed string as input instead of a request call, with a print instead of an out.write, etc).
This is my test in which I could not reproduce the problem:
entries = [
{
"id": "1",
"name": "John",
"surname": "Smith"
},
{
"id": "2",
"name": "Steve",
"surname": "Ger"
}
]
json_dict = []
for i in entries:
json_dict.append({"id":i["id"], "data": i})
json.dumps(json_dict, indent=2)
This prints as expected:
[
{
"id": "1",
"data": {
"id": "1",
"surname": "Smith",
"name": "John"
}
},
{
"id": "2",
"data": {
"id": "2",
"surname": "Ger",
"name": "Steve"
}
}
]
You could try this instead:
json_dict = []
with open('data.json', 'w) as f:
loaded_data = json.loads(request.text)
for list_item in loaded_data:
json_dict.append({list_item['id'] : list_item})
Related
I have a json file which looks like this
{
"name1": {
"name": "xyz",
"roll": "123",
"id": "A1"
},
"name2":{
"name": "abc",
"roll": "456",
"id": "A2"
},
"name3":{
"name": "def",
"roll": "789",
"id": "B1"
}
}
I want to remove name1, name2, name3 and should be looking like this
{
{
"name": "xyz",
"roll": "123",
"id": "A1"
},
{
"name": "abc",
"roll": "456",
"id": "A2"
},
{
"name": "def",
"roll": "789",
"id": "B1"
}
}
Is there any way to do this in python? If this problem is asked earlier then me please refer to one cause I can't find it. Help will be thankful.
Since you're expecting the outermost (root) level of json document to be an object (curly braces) you should specify at least one key to start with. Refer below example.
Python code Example
import json
in_file = open("/Input/path/to/sample.json", "r")
out_file = open("/Output/path/to/converted_json.json", "w")
input_json = json.load(in_file)
print(input_json)
converted_json = {"names": [input_json[key] for key in input_json]}
print(converted_json)
json.dump(converted_json, out_file)
in_file.close(), out_file.close()
Out put will be a json file with name converetd_json.json with below content.
{
"names": [{
"name": "xyz",
"roll": "123",
"id": "A1"
},
{
"name": "abc",
"roll": "456",
"id": "A2"
},
{
"name": "def",
"roll": "789",
"id": "B1"
}
]
}
If you want to avoid this root key, an alternative is to use an array (Square Brackets) as an outermost level. Refer below example.
Python Code
import json
in_file = open("/Input/path/to/sample.json", "r")
out_file = open("/Output/path/to/converted_json.json", "w")
input_json = json.load(in_file)
print(input_json)
converted_json = [input_json[key] for key in input_json]
print(converted_json)
json.dump(converted_json, out_file)
in_file.close(), out_file.close()
Out put will be a json file with name converetd_json.json with below content.
[
{
"name": "xyz",
"roll": "123",
"id": "A1"
},
{
"name": "abc",
"roll": "456",
"id": "A2"
},
{
"name": "def",
"roll": "789",
"id": "B1"
}
]
Note: The outermost level of a JSON document is either an "object" (curly braces) or an "array" (square brackets). So in this case both are valid json.
You can use simple python script. First, you need to read the json file and then load it as an json object. Then, you can iterate on the whole json object to change it to whatever format you need.
import json
with open("file.json","r+") as file:
s = file.read()
jfile = json.loads(s)
with open("ss.json","w") as file:
json.dump([jfile[i] for i in jfile], file)
I have rather very weird requirement now. I have below json and somehow I have to convert it into flat csv.
[
{
"authorizationQualifier": "SDA",
"authorizationInformation": " ",
"securityQualifier": "ASD",
"securityInformation": " ",
"senderQualifier": "ASDAD",
"senderId": "FADA ",
"receiverQualifier": "ADSAS",
"receiverId": "ADAD ",
"date": "140101",
"time": "0730",
"standardsId": null,
"version": "00501",
"interchangeControlNumber": "123456789",
"acknowledgmentRequested": "0",
"testIndicator": "T",
"functionalGroups": [
{
"functionalIdentifierCode": "ADSAD",
"applicationSenderCode": "ASDAD",
"applicationReceiverCode": "ADSADS",
"date": "20140101",
"time": "07294900",
"groupControlNumber": "123456789",
"responsibleAgencyCode": "X",
"version": "005010X221A1",
"transactions": [
{
"name": "ASDADAD",
"transactionSetIdentifierCode": "adADS",
"transactionSetControlNumber": "123456789",
"implementationConventionReference": null,
"segments": [
{
"BPR03": "ad",
"BPR14": "QWQWDQ",
"BPR02": "1.57",
"BPR13": "23223",
"BPR01": "sad",
"BPR12": "56",
"BPR10": "32424",
"BPR09": "12313",
"BPR08": "DA",
"BPR07": "123456789",
"BPR06": "12313",
"BPR05": "ASDADSAD",
"BPR16": "21313",
"BPR04": "SDADSAS",
"BPR15": "11212",
"id": "aDSASD"
},
{
"TRN02": "2424",
"TRN03": "35435345",
"TRN01": "3435345",
"id": "FSDF"
},
{
"REF02": "fdsffs",
"REF01": "sfsfs",
"id": "fsfdsfd"
},
{
"DTM02": "2432424",
"id": "sfsfd",
"DTM01": "234243"
}
],
"loops": [
{
"id": "24324234234",
"segments": [
{
"N101": "sfsfsdf",
"N102": "sfsf",
"id": "dgfdgf"
},
{
"N301": "sfdssfdsfsf",
"N302": "effdssf",
"id": "fdssf"
},
{
"N401": "sdffssf",
"id": "sfds",
"N402": "sfdsf",
"N403": "23424"
},
{
"PER06": "Wsfsfdsfsf",
"PER05": "sfsf",
"PER04": "23424",
"PER03": "fdfbvcb",
"PER02": "Pedsdsf",
"PER01": "sfsfsf",
"id": "fdsdf"
}
]
},
{
"id": "2342",
"segments": [
{
"N101": "sdfsfds",
"N102": "vcbvcb",
"N103": "dsfsdfs",
"N104": "343443",
"id": "fdgfdg"
},
{
"N401": "dfsgdfg",
"id": "dfgdgdf",
"N402": "dgdgdg",
"N403": "234244"
},
{
"REF02": "23423342",
"REF01": "fsdfs",
"id": "sfdsfds"
}
]
}
]
}
]
}
]
}
]
The column header name corresponding to deeper key-value make take nested form, like functionalGroups[0].transactions[0].segments[0].BPR15.
I am able to do this in java using this github project (here you can find the output format I desire in the explanation) in one line:
flatJson = JSONFlattener.parseJson(new File("files/simple.json"), "UTF-8");
The output was:
date,securityQualifier,testIndicator,functionalGroups[1].functionalIdentifierCode,functionalGroups[1].date,functionalGroups[1].applicationReceiverCode, ...
140101,00,T,HP,20140101,ETIN,...
But I want to do this in python. I tried as suggested in this answer:
with open('data.json') as data_file:
data = json.load(data_file)
df = json_normalize(data, record_prefix=True)
with open('temp2.csv', "w", newline='\n') as csv_file:
csv_file.write(df.to_csv())
However, for column functionalGroups, it dumps json as a cell value.
I also tried as suggested in this answer:
with open('data.json') as f: # this ensures opening and closing file
a = json.loads(f.read())
df = pandas.DataFrame(a)
print(df.transpose())
But this also seem to do the same:
0
acknowledgmentRequested 0
authorizationInformation
authorizationQualifier SDA
date 140101
functionalGroups [{'functionalIdentifierCode': 'ADSAD', 'applic...
interchangeControlNumber 123456789
receiverId ADAD
receiverQualifier ADSAS
securityInformation
securityQualifier ASD
senderId FADA
senderQualifier ASDAD
standardsId None
testIndicator T
time 0730
version 00501
Is it possible to do what I desire in python?
I have the following json:
{
"request": {
"id": "123",
"url": "/aa/bb/cc",
"method": "GET",
"timestamp": "2018-08-09T08:41:38.432Z"
},
"response": {
"status": {
"code": 200,
"message": "OK"
},
"items": [
{
"id": "aaa",
"name": "w1"
},
{
"id": "bbb",
"name": "w2"
},
{
"id": "ccc",
"name": "w3"
}
]
}
}
I need to loop over items and print each name. I've tried the following code which doesn't work.
response = requests.get(url)
data = json.loads(response.content)
for group in data['response']['items']:
print data['response']['items'][group]['name']
When i replace group with 0 for example, I can access the first name:
data['response']['items'][0]['name']
However, I don't know in advanced how many elements are in the array.
As Joel mentioned, in the for loop,
for group in data['response']['items']:
you are assigning group the value from data['response']['items']. Hence group contains the value :
[
{
"id": "aaa",
"name": "w1"
},
{
"id": "bbb",
"name": "w2"
},
{
"id": "ccc",
"name": "w3"
}
]
So all you need to do is
print group['name']
You can use Pandas module and call read_json function.
import pandas as pd
df = pd.read_json(your_json_file.json)
for i in df.response['items']:
print(i['name'])
# w1
# w2
# w3
You could try this:
for i in range (0,len(d['response']['items'])):
print(d['response']['items'][i]['name'])
Output:
w1
w2
w3
I have two dictionaries in python 3, called a and b which when dumped as JSON come out as below:
a = {"person": 26.94, "car": 99.49, "dog": 50.56}
b = {"filename": "1234.jpg", "model": "model1", "prototxt": "prototxt.txt"}
I need to combine these into the JSON format below but I am a bit lost on the approach in python, so welcome any pointers!
{
"payload": {
"config": [{
"model": "model1",
"filename": "1234.jpg",
"prototxt": "prototxt.txt"
}],
"results": [{
"object": "person",
"value": 26.94
},
{
"object": "car",
"value": 99.49
},
{
"object": "dog",
"value": 50.56
}
]
}
}
You can achieve this by the following code:
a = {"person": 26.94, "car": 99.49, "dog": 50.56}
b = {"filename": "1234.jpg", "model": "model1", "prototxt": "prototxt.txt"}
a_list= []
for record in a:
a_list.append({'object':record, 'value':a[record]})
payload = {'config':[b], 'results':a_list}
data = {"payload":payload}
# You can print this in the terminal/notebook with
print json.dumps(data, indent=4)
# Or save with
json_string = json.dumps(data)
with open('/path/to/file/payload.json', 'w') as outfile:
json.dump(data, outfile)
It's a matter of making entries in dictionaries and packaging them up in strings to add into new higher dictionaries.
I have a sample JSON in this format:
JSON FILE:
{
"Name": "ABC",
"Phone":"123",
"Address":[{"City":"City-1"},{"Country":"Country-1"}]
}
{
"Name": "ABC-1",
"Phone":"123-1",
"Address":[{"City":"City-2"},{"Country":"Country-2"}]
}
Is there any approach to parse the JSON and loop through the file and print each key-value pair.
The approach I used was through using
json_open = open(json_file)
json_data = json.load(json_open)
print(json_data[Name]) ##should give ABC
print(json_data[Name]) ##should give ABC-1 - unsure about the syntax and format
But I'm currently able to print only the first object values - i.e. name=ABC and not name=ABC-1
There is error in your json file. I modified your json and written code for traverse each element in it.
Error:
Error: Parse error on line 9:
... "Country-1" }]}{ "Name": "ABC-1",
-------------------^
Expecting 'EOF', '}', ',', ']', got '{'
sample.json
{
"data": [
{
"Name": "ABC",
"Phone": "123",
"Address": [
{
"City": "City-1"
},
{
"Country": "Country-1"
}
]
},
{
"Name": "ABC-1",
"Phone": "123-1",
"Address": [
{
"City": "City-2"
},
{
"Country": "Country-2"
}
]
}
]
}
sample.py
import json
json_file='sample.json'
with open(json_file, 'r') as json_data:
data = json.load(json_data)
jin=data['data']
for emp in jin:
print ("Name :"+emp["Name"])
print ("Phone :"+emp["Phone"])
print ("City :"+emp["Address"][0]["City"])
print ("Country :"+emp["Address"][1]["Country"])
Each record on json file is separated by ','. So, your file should look like:
[{
"Name": "ABC",
"Phone": "123",
"Address": [{
"City": "City-1"
}, {
"Country": "Country-1"
}]
},
{
"Name": "ABC-1",
"Phone": "123-1",
"Address": [{
"City": "City-2"
}, {
"Country": "Country-2"
}]
}
]
You can read the file and output as follows :
import json
my_file='test.json'
with open(my_file, 'r') as my_data:
data = json.load(my_data)
print data
for elm in data:
print elm['Phone']
print elm['Name']
print elm['Address'][0]['City']
print elm['Address'][1]['Country']
First of all this is not valid json format,
You can check here you json format
You should use this try.json file
{
"data":[{
"Name": "ABC",
"Phone":"123",
"Address":[{"City":"City-1"},{"Country":"Country-1"}]
}, {
"Name": "ABC-1",
"Phone":"123-1",
"Address":[{"City":"City-2"},{"Country":"Country-2"}]
}]
}
here is try.py
import json
json_open = open("try.json")
json_data = json.load(json_open)
for i in range(len(json_data['data'])):
print(json_data['data'][i]["Name"])
Note : python code is written in python3
output is look line this
ABC
ABC-1