I'm trying to generate a JSON file with python. But I can't figure out how to append each object correctly and write all of them at once to JSON file. Could you please help me solve this? a, b, and values for x, y, z are calculated in the script.
Thank you so much
This is how the generated JSON file should look like
{
"a": {
"x": 2,
"y": 3,
"z": 4
},
"b": {
"x": 5,
"y": 4,
"z": 4
}
}
This is python script
import json
for i in range(1, 5):
a = geta(i)
x = getx(i)
y = gety(i)
z = getz(i)
data = {
a: {
"x": x,
"y": y,
"z": z
}}
with open('data.json', 'a') as f:
f.write(json.dumps(data, ensure_ascii=False, indent=4))
Just use normal dictionaries in python when constructing the JSON then use the JSON package to export to JSON files.
You can construct them like this (long way):
a_dict = {}
a_dict['id'] = {}
a_dict['id']['a'] = {'properties' : {}}
a_dict['id']['a']['properties']['x'] = '9'
a_dict['id']['a']['properties']['y'] = '3'
a_dict['id']['a']['properties']['z'] = '17'
a_dict['id']['b'] = {'properties' : {}}
a_dict['id']['b']['properties']['x'] = '3'
a_dict['id']['b']['properties']['y'] = '2'
a_dict['id']['b']['properties']['z'] = '1'
or you can use a function:
def dict_construct(id, x, y, z):
new_dic = {id : {'properties': {} } }
values = [{'x': x}, {'y': y}, {'z':z}]
for val in values:
new_dic[id]['properties'].update(val)
return new_dic
return_values = [('a', '9', '3', '17'), ('b', '3', '2', '1')]
a_dict = {'id': {} }
for xx in return_values:
add_dict = dict_construct(*xx)
a_dict['id'].update(add_dict)
print(a_dict)
both give you as a dictionary:
{'id': {'a': {'properties': {'x': '9', 'y': '3', 'z': '17'}}, 'b': {'properties': {'x': '3', 'y': '2', 'z': '1'}}}}
using json.dump:
with open('data.json', 'w') as outfile:
json.dump(a_dict, outfile)
you get as a file:
{
"id": {
"a": {
"properties": {
"x": "9",
"y": "3",
"z": "17"
}
},
"b": {
"properties": {
"x": "3",
"y": "2",
"z": "1"
}
}
}
}
Make sure you have a valid python dictionary (it seems like you already do)
I see you are trying to write your json in a file with
with open('data.json', 'a') as f:
f.write(json.dumps(data, ensure_ascii=False, indent=4))
You are opening data.json on "a" (append) mode, so you are adding your json to the end of the file, that will result on a bad json data.json contains any data already. Do this instead:
with open('data.json', 'w') as f:
# where data is your valid python dictionary
json.dump(data, f)
One way will be to create whole dict at once:
data = {}
for i in range(1, 5):
name = getname(i)
x = getx(i)
y = gety(i)
z = getz(i)
data[name] = {
"x": x,
"y": y,
"z": z
}
And then save
with open('data.json', 'w') as f:
json.dump(data, f, indent=4)
Related
I'm writing some Python code (data extraction from a ConLL-U format file) and I want my data to be stored in a .json file. I'd like to achieve an output's format like the following (x are keys, y are values):
{
"lemma": {"x": "y","x": [{"x":"y"}], "x": "y", "x": [{"x":"y"}], "x": "" },
"lemma1":{"x": "y", "x": [{"x":"y"}], "x": "y", "x": [{"x":"y"}], "x": "y" }...
}
Last section of my code (it's probably quite inefficient, but now I'm just intersted in formatting the json output):
token_info= {}
...
sentences = []
tokens = []
idn_dep_dict = {}
for line in lines:
if line == '\n':
sentences.append(tokens)
tokens = []
else:
fields = line.strip().split('\t')
if len(fields) >= 1:
if fields[0].isdigit():
idn = fields[0]
lemma = fields[1]
upos = fields[3]
xpos = fields[4]
feats = fields[5]
dep = fields[6]
pos_pair = (upos,xpos)
tokens.append((idn, lemma, pos_pair,feats,dep))
idn_dep_dict[idn]=[dep]
else:
continue
for sentence in sentences:
dependencies_dict = {} #dictionary for the dependencies of the current sentence
for token in sentence:
idn, lemma, pos_pair, feats, dep = token
if dep == '0':
dependencies_dict[idn] = 'root'
if dep in idn_dep_dict:
for head_token in sentence:
if head_token[0] == dep:
dependencies_dict[idn] = head_token[2]
# Create a dictionary for the current token's information
current_token = {'x1': [upos], 'x2': [{'0': pos_pair}],'x3': [{'0': dependencies_dict[idn]}],'x4': feats}
token_info[lemma] = current_token
# Write the JSON data to a file
with open('token_info.json', 'w', encoding='utf-8') as f:
json.dump(token_info, f, ensure_ascii=False, indent = 2, separators=(',', ': '))
The current code generates a newline after each [,] or {,} or comma in the json file. I'd like to have each lemma = {corrisponding dictionary} on each line. Is it possible? Thank you all in advance
Serialize one level of the dictionary structure manually like this.
import json
token_info = json.loads('''
{
"lemma": {"x": "y","x2": [{"x":"y"}], "x3": "y", "x4": [{"x":"y"}], "x5": "" },
"lemma1":{"x": "y", "x2": [{"x":"y"}], "x3": "y", "x4": [{"x":"y"}], "x5": "y" }
}
''')
lines = []
for k, v in token_info.items():
ks = json.dumps(k, ensure_ascii=False)
vs = json.dumps(v, ensure_ascii=False, separators=(',', ': '))
lines.append(ks + ': ' + vs)
src = '{\n ' + (',\n '.join(lines)) + '\n}'
print(src)
This will output the following.
{
"lemma": {"x": "y","x2": [{"x": "y"}],"x3": "y","x4": [{"x": "y"}],"x5": ""},
"lemma1": {"x": "y","x2": [{"x": "y"}],"x3": "y","x4": [{"x": "y"}],"x5": "y"}
}
Is it possible to remove all elements expect first one from each sublists which is act as a value in dict without using any loop
let d =
[
{ "x":1640995200000, "y": [2365,2567.300049,2305,2386.600098] },
{ "x":1643673600000, "y": [2408,2456.399902,2243,2359.550049] },
{ "x":1646092800000, "y": [2359.550049,2688,2180,2634.750000] }
]
output =
[
{ "x":1640995200000, "y": 2365 },
{ "x":1643673600000, "y": 2408 },
{ "x":1646092800000, "y": 2359.550049 }
]
Using an explicit loop of some kind will make it clearer to the reader / maintainer what you're trying to do. However, if you really want to obfuscate the process then you could do this:
d = [
{ "x":1640995200000, "y": [2365,2567.300049,2305,2386.600098] },
{ "x":1643673600000, "y": [2408,2456.399902,2243,2359.550049] },
{ "x":1646092800000, "y": [2359.550049,2688,2180,2634.750000] }
]
output = list(map(lambda _d: {'x': _d['x'], 'y': _d['y'][0]}, d))
print(output)
Output:
[{'x': 1640995200000, 'y': 2365}, {'x': 1643673600000, 'y': 2408}, {'x': 1646092800000, 'y': 2359.550049}]
Assume that the lists have at least one element in each.
d = [{"x":item["x"], "y": item["y"][0]} for item in d]
More about list comprehension could be found here https://www.w3schools.com/python/python_lists_comprehension.asp
import pandas as pd
df = pd.DataFrame(d)
df['y'] = df['y'].apply(lambda x: x[0])
df.to_dict('records')
I have file1.txt with following contents;
[
{
"SERIAL": "124584",
"X": "30024.1",
},
{
"SERIAL": "114025",
"X": "14006.2",
}
]
I have file2.txt with following contents;
[
{
"SERIAL": "344588",
"X": "48024.1",
},
{
"SERIAL": "255488",
"X": "56006.2",
}
]
I want to combine the 2 files into single file output.txt that looks like this;
[
{
"SERIAL": "124584",
"X": "30024.1",
},
{
"SERIAL": "114025",
"X": "14006.2",
},
{
"SERIAL": "344588",
"X": "48024.1",
},
{
"SERIAL": "255488",
"X": "56006.2",
},
]
The tricky part is the [] at the end of each individual file.
I am using python v3.7
Firstly to be JSON compliant, you may remove all the trailing commas (ref: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Trailing_commas)
Then you can use the following code:
import json
with open("file1.txt") as f1:
d1 = json.load(f1)
with open("file2.txt") as f2:
d2 = json.load(f2)
d3 = d1 + d2
with open("output.txt", "w") as out:
json.dump(d3, out)
Here is the solution to read content from file and then append them.
from ast import literal_eval
with open("/home/umesh/Documents/text1.txt", "r") as data
first_file_data = data.read()
with open("/home/umesh/Documents/text2.txt", "r") as data:
second_file_data = data.read()
first_file_data = literal_eval(first_file_data)
second_file_data = literal_eval(second_file_data)
for item in second_file_data:
first_file_data.append(item)
print(first_file_data)
OUTPUT
[{'SERIAL': '124584', 'X': '30024.1'},{'SERIAL': '114025', 'X': '14006.2'},{'SERIAL': '344588', 'X': '48024.1'},{'SERIAL': '255488', 'X': '56006.2'}]
text file content
This solves your problem
import ast
import json
with open('file1.txt') as f:
data = ast.literal_eval(f.read())
with open('file2.txt') as f:
data2 = ast.literal_eval(f.read())
data.extend(data2)
print(data)
with open('outputfile', 'w') as fout: # write to a file
json.dump(data, fout)
OUTPUT:
[{'SERIAL': '124584', 'X': '30024.1'}, {'SERIAL': '114025', 'X': '14006.2'}, {'SERIAL': '344588', 'X': '48024.1'}, {'SERIAL': '255488', 'X': '56006.2'}]
Since both of the content of the files are lists you can concatenate them together as following
file1 = [{'SERIAL': '124584', 'X': '30024.1'}, {'SERIAL': '114025', 'X': '14006.2'}]
file2 = [{'SERIAL': '344588', 'X': '48024.1'}, {'SERIAL': '255488', 'X': '56006.2'}]
totals = file1 + file2
Result
[{'SERIAL': '124584', 'X': '30024.1'},
{'SERIAL': '114025', 'X': '14006.2'},
{'SERIAL': '344588', 'X': '48024.1'},
{'SERIAL': '255488', 'X': '56006.2'}]
I am trying to load a string into python object. I am getting below error:
This is error : Expecting value: line 15 column 15 (char 351)
{'allData': None}
When I change the value for cprob to below code, there is no issue:
"cprob": {
"T": 1,
"A": 2,
"C": 3
}
Data in test.txt:
[{
"V": ["Offer", "Interview", "Grades", "Admission", "Experience"],
"E": [["Grades", "Interview"],
["Experience", "Interview"],
["Grades", "Admission"],
["Interview", "Offer"]],
"Vdata": {
"Offer": {
"ord": 4,
"numoutcomes": 2,
"vals": ["0", "1"],
"parents": ["Interview"],
"children": 0,
"cprob": {
"['0']": [.9, .1],
"['1']": [.4, .6],
"['2']": [.01, .99]
}
}
}
}]
Code:
import json
class JsonData:
def __init__(self, path):
self.allData = None
def dictload(self, path):
f = open(path, 'r')
ftext = f.read()
print(ftext)
try:
self.allData = json.loads(ftext)
except Exception as e:
print('This is error : ', e)
path = "test.txt"
jsonData = JsonData(path)
jsonData.dictload(path)
print(vars(jsonData))
It seems python's json module doesn't understand .1 to be 0.1. If you add the 0 in front of the period it will work.
I am working to convert CSV file to structured Json file.
CSV.File
address,type,floor,door
"1","is","an","example"
"2","is","an","example"
"3","is","an","example"
"4","is","an","example"
"5","is","an","example"
"6","is","an","example"
"7","is","an","example"
First, I read the csv file and list all the column's items to lists.
import pandas as pd
with open('data.csv', 'rb') as csvfile:
reader = csv.reader(csvfile)
for row in reader:
df = pd.read_csv(csvfile)
listAddress= df.address
listType= df.type
listFloor= df.floor
listDoor=df.door
In order to get lists like this one :
listAddress=["1","2","3","4","5","6","7"]
Now, I want to automate the process and make a list of dictionaries
Output= []
tempJson = {"1": 1,"2": "is"}
for i in range(len(listAddress)):
tempJson["Address"]= listAddress[i]
tempJson["Type"]= listType[j]
Output.(tempJson)
json.dumps(output)
Here is the problem, I'd like to make a list of dictionaries, I can do this in JS using map. But I am not proficient in Python.
Can I simplify the this last piece of code using Map
Output:
[
{
"1": 1,
"2": "is"
},
{
"1": 2,
"2":"is"
},
{
"1": 3,
"2": "is"
},{
"1": 4,
"2": "is"
},
{
"1": 5,
"2":"is"
},
{
"1": 6,
"2": "is"
},
{
"1": 7,
"2": "is"
}
]
You can use a list comprehension here:
>>> from pprint import pprint
>>> import csv
>>> with open('data.csv') as f:
next(f) #skip the header
reader = csv.reader(f, delimiter=',', quotechar='"')
d = [{'1':int(row[0]), '2': row[1]} for row in reader]
...
>>> pprint(d)
[{'1': 1, '2': 'is'},
{'1': 2, '2': 'is'},
{'1': 3, '2': 'is'},
{'1': 4, '2': 'is'},
{'1': 5, '2': 'is'},
{'1': 6, '2': 'is'},
{'1': 7, '2': 'is'}]
A simple change would be
Output= []
tempJson = {"1": 1,"2": "is"}
for i in range(len(listAddress)):
tempJson["Address"]= listAddress[i]
tempJson["Type"]= listType[i]
Output.append(tempJson.copy())
something probably considered more pythonic would be:
output = [{"1": 1, "2": "is", "Address": a, "Type": t}
for a, t in zip(listAddress, listType)]