I need to append template in place 'here' and I have no idea how to do that.
There's how json look like:
{
"1st_thing":
[{"1st":"1stvalue"},{"2nd":"2ndvalue"}, /*here*/],
"2nd_thing":
[{"xx":"XXvalue"},{"yy":"YYvalue"},/*here*/]
}
I've tried this and it only do this temporary but I need to save it in 'tokens.json'
with open("tokens.json") as f:
new = json.load(f)
new['captcha_tokens'].append(template)
You can use dump to write to a new file:
json.dump(new, open("tokens_new.json", "w"))
Or if you want to prettify a bit:
json.dump(new, open("tokens_new.json", "w"), ensure_ascii=True, indent=4)
Then you can replace tokens.json with shutil:
import shutil
shutil.copy("tokens_new.json", "tokens.json")
Related
I have a json file with the following format:
{
"responses":[
{
"id":"123",
"cid":"01A",
"response":{nested lists and dictionaries}
},
{
"id":"456",
"cid":"54G",
"response":{nested lists and dictionaries}
}
]}
And so on.
And I want to convert it into a json file like this:
{"id":"123", "cid":"01A", "response":{nested lists and dictionaries}},
{"id":"456", "cid":"54G", "response":{nested lists and dictionaries}}
or
{responses:[
{"id":"123", "cid":"01A", "response":{nested lists and dictionaries}},
{"id":"456", "cid":"54G", "response":{nested lists and dictionaries}}
]}
I don't care about the surrounding format as long as I have the information for each ID in just one line.
I have to do this while reading it because things like pd.read_json don't read this kind of file.
Thanks!
Maybe just dump it line wise? But I guess I didn't understand your question right?
import json
input_lines = {"responses": ...}
with open("output.json", "w") as f:
for line in input_lines["responses"]:
f.write(json.dumps(line) + "\n")
You can use the built-in json library to print each response on a separate line. The json.dump() function has an option to indent, if you want that, but its default is to put everything on one line, like what you want.
Here's an example that works for the input you showed in your post.
#!/usr/bin/env python3
import json
import sys
with open(sys.argv[1]) as json_file:
obj = json.load(json_file)
print("{responses:[")
for response in obj['responses']:
print(json.dumps(response))
print("]}")
Usage (assuming you named the program format_json.py):
$ chmod +x format_json.py
$ format_json.py my_json_input.json > my_json_output.json
Or, if you're not in a command-line environment, you can also hardcode the input and output filenames:
#!/usr/bin/env python3
import json
import sys
infile = 'my_json_input.json'
outfile = 'my_json_output.json'
with open(infile) as json_file:
obj = json.load(json_file)
print("{responses:[", file=outfile)
for response in obj['responses']:
print(json.dumps(response), file=outfile)
print("]}", file=outfile)
I have a json file that I am using as a Dictionary in python.
The json file is really long with 10k+ records. I need to replace the $home part in the "iscategorical" with the value of "id". After making the changes, I want to save this file so that I can use it again as a dictionary. Thank you for the help. Here is a sample:
{
"maps": [
{
"id": "xyzp",
"iscategorical": "/u/$home/app/home"
},
{
"id": "trtn",
"iscategorical": "/u/app/$home/user"
}
]}
I am understanding that you are able to load the file successfully, and all you want to do is replace the strings and save the structure to file again.
For this, we can traverse the list of dictionaries in the data, and modify the value of item['iscategorical'] by replacing $home with the value of item['id'].
We can then dump the modified structure back to (a new) json file.
import json
with open('data.json') as f:
data = json.load(f)
for item in data['maps']:
item['iscategorical'] = item['iscategorical'].replace('$home', item['id'])
with open('new_data.json', 'w') as f:
json.dump(data, f)
Your question seems similar to - Parsing values from a JSON file? .
However for your case below snippet should work.
import json
with open('idata.json') as infile:
data = json.load(infile)
for elem in data["maps"]:
elem['iscategorical']=elem['iscategorical'].replace('$home',elem['id'])
with open('odata.json', 'w') as outfile:
json.dump(data, outfile)
If it's a file, one thing you can do is load the file in and read line by line.
for everyline, you can use regex to find and replace. Then you can either overwrite the file or write onto a new file.
For example,
line.replace('$home', 'id')
Alternatively, you can load the json python in and convert it into a string. Then replace the text using the regex. Finally, converts back to Python dictionary using json.load().
However, 10k line is too long. I think reading a file, line-by-line, would be a better solutions.
EDIT:
Here is the code sample.
from tempfile import mkstemp
from shutil import move
from os import fdopen, remove
def replace(file_path, pattern, subst):
#Create temp file
fh, abs_path = mkstemp()
with fdopen(fh,'w') as new_file:
with open(file_path) as old_file:
for line in old_file:
new_file.write(line.replace(pattern, subst))
#Remove original file
remove(file_path)
#Move new file
move(abs_path, file_path)
replace('./text.txt', '$home', 'id')
"The JSON file is really long with 10k+ records" -Try this way it should help for large files.
input.json
{"maps":[{"id":"xyzp","iscategorical":"/u/$home/app/home"},{"id":"trtn","iscategorical":"/u/app/$home/user"}]}
import json
with open('input.json') as f:
data = json.load(f)
my_list = []
def get_some_data():
for item in data['maps']:
yield(item['id'], item['iscategorical'])
for id, iscat in get_some_data():
temp_dict = {}
temp_dict['id'] = id
temp_dict['iscategorical'] = iscat.replace('$home', id)
my_list.append(temp_dict)
maps_dict = {}
maps_dict['maps'] = my_list
with open('output.json', 'w') as f:
json.dump(maps_dict, f)
output.json:
{"maps": [{"id": "xyzp", "iscategorical": "/u/**xyzp**/app/home"}, {"id": "trtn", "iscategorical": "/u/app/**trtn**/user"}]}
I am getting a JSON file with following format :
// 20170407
// http://info.employeeportal.org
{
"EmployeeDataList": [
{
"EmployeeCode": "200005ABH9",
"Skill": CT70,
"Sales": 0.0,
"LostSales": 1010.4
}
]
}
Need to remove the extra comment lines present in the file.
I tried with the following code :
import json
import commentjson
with open('EmployeeDataList.json') as json_data:
employee_data = json.load(json_data)
'''employee_data = json.dump(json.load(json_data))'''
'''employee_data = commentjson.load(json_data)'''
print(employee_data)`
Still not able to remove the comments from the file and bring
the JSON file in correct format.
Not getting where things are going wrong? Any direction in this regard is highly appreciated.Thanks in advance
You're not using commentjson correctly. It has the same interface as the json module:
import commentjson
with open('EmployeeDataList.json', 'r') as handle:
employee_data = commentjson.load(handle)
print(employee_data)
Although in this case, your comments are simple enough that you probably don't need to install an extra module to remove them:
import json
with open('EmployeeDataList.json', 'r') as handle:
fixed_json = ''.join(line for line in handle if not line.startswith('//'))
employee_data = json.loads(fixed_json)
print(employee_data)
Note the difference here between the two code snippets is that json.loads is used instead of json.load, since you're parsing a string instead of a file object.
Try JSON-minify:
JSON-minify minifies blocks of JSON-like content into valid JSON by removing all whitespace and JS-style comments (single-line // and multiline /* .. */).
I usually read the JSON as a normal file, delete the comments and then parse it as a JSON string. It can be done in one line with the following snippet:
with open(path,'r') as f: jsonDict = json.loads('\n'.join(row for row in f if not row.lstrip().startswith("//")))
IMHO it is very convenient because it does not need CommentJSON or any other non standard library.
Well that's not a valid json format so just open it like you would a text document then delete anything from// to \n.
with open("EmployeeDataList.json", "r") as rf:
with open("output.json", "w") as wf:
for line in rf.readlines():
if line[0:2] == "//"
continue
wf.write(line)
Your file is parsable using HOCON.
pip install pyhocon
>>> from pyhocon import ConfigFactory
>>> conf = ConfigFactory.parse_file('data.txt')
>>> conf
ConfigTree([('EmployeeDataList',
[ConfigTree([('EmployeeCode', '200005ABH9'),
('Skill', 'CT70'),
('Sales', 0.0),
('LostSales', 1010.4)])])])
If it is the same number of lines every time you can just do:
fh = open('EmployeeDataList.NOTjson',"r")
rawText = fh.read()
json_data = rawText[rawText.index("\n",3)+1:]
This way json_data is now the string of text without the first 3 lines.
I have the a JSON file which contains {"67790": {"1": {"kwh": 319.4}}}. Now I create a dictionary a_dict which I need to append to the JSON file.
I tried this code:
with open(DATA_FILENAME, 'a') as f:
json_obj = json.dump(a_dict, json.load(f)
f.write(json_obj)
f.close()
What is wrong with the code? How can I fix the problem?
Assuming you have a test.json file with the following content:
{"67790": {"1": {"kwh": 319.4}}}
Then, the code below will load the json file, update the data inside using dict.update() and dump into the test.json file:
import json
a_dict = {'new_key': 'new_value'}
with open('test.json') as f:
data = json.load(f)
data.update(a_dict)
with open('test.json', 'w') as f:
json.dump(data, f)
Then, in test.json, you'll have:
{"new_key": "new_value", "67790": {"1": {"kwh": 319.4}}}
Hope this is what you wanted.
You need to update the output of json.load with a_dict and then dump the result.
And you cannot append to the file but you need to overwrite it.
json_obj=json.dumps(a_dict, ensure_ascii=False)
I have this script which abstract the json objects from the webpage. The json objects are converted into dictionary. Now I need to write those dictionaries in a file. Here's my code:
#!/usr/bin/python
import requests
r = requests.get('https://github.com/timeline.json')
for item in r.json or []:
print item['repository']['name']
There are ten lines in a file. I need to write the dictionary in that file which consist of ten lines..How do I do that? Thanks.
To address the original question, something like:
with open("pathtomyfile", "w") as f:
for item in r.json or []:
try:
f.write(item['repository']['name'] + "\n")
except KeyError: # you might have to adjust what you are writing accordingly
pass # or sth ..
note that not every item will be a repository, there are also gist events (etc?).
Better, would be to just save the json to file.
#!/usr/bin/python
import json
import requests
r = requests.get('https://github.com/timeline.json')
with open("yourfilepath.json", "w") as f:
f.write(json.dumps(r.json))
then, you can open it:
with open("yourfilepath.json", "r") as f:
obj = json.loads(f.read())