I have a 1000 json files, I need to change the value of a specific line with numeric sequence in all files.
An example
the specific line is - "name": "carl 00",
I need it to be like following
File 1
"name": "carl 1",
File 1
"name": "carl 2",
File 3
"name": "carl 3",
What is the right script to achieve the above using python
This should do the trick. But you're not very clear about how the data is stored in the actual json file. I listed two different approaches. The first is to parse the json file into a python dict then manipulate the data and then turn it back into a character string and then save it. The second is what I think you mean by "line". You can split the file's character string into a list then change the line you want, and remake the full string again, then save it.
This also assumes your json files are in the same folder as the python script.
import os
import json
my_files = [name1, name2, name3, ...] # ['file_name.json', ...]
folder_path = os.path.dirname(__file__)
for i, name in enumerate(my_files):
path = f'{folder_path}/{name}'
with open(path, 'r') as f:
json_text = f.read()
# if you know the key(s) in the json file...
json_dict = json.loads(json_text)
json_dict['name'] = json_dict['name'].replace('00', str(i))
new_json_str = json.dumps(json_dict)
# if you know the line number in the file...
line_list = json_text.split('\n')
line_list[line_number - 1] = line_list[line_number - 1].replace('00', str(i))
new_json_str = '\n'.join(line_list)
with open(path, 'w') as f:
f.write(new_json_str)
Based on your edit, this is what you want:
import os
import json
my_files = [f'{i}.json' for i in range(1, 1001)]
folder_path = os.path.dirname(__file__) # put this .py file in same folder as json files
for i, name in enumerate(my_files):
path = f'{folder_path}/{name}'
with open(path, 'r') as f:
json_text = f.read()
json_dict = json.loads(json_text)
json_dict['name'] = f'carl {i}'
# include these lines if you want "symbol" and "subtitle" changed
json_dict['symbol'] = f'carl {i}'
json_dict['subtitle'] = f'carl {i}'
new_json_str = json.dumps(json_dict)
with open(path, 'w') as f:
f.write(new_json_str)
Without knowing more, the below loop will accomplish the posts requirements.
name = 'carl'
for i in range(0,1001):
print(f'name: {name} {i}')
I want to delete everything in the object "name" in the given json file example but keep the the object, in simple words I want to clear the object.
{
"names": [
{
"player": "Player_Name",
"TB:": "12389",
"BW:": "596",
"SW:": "28",
"CQ:": "20"
}
]
}
I used tried this code:
with open('players.json', 'w') as w:
with open('players.json', 'r') as r:
for line in r:
element = json.loads(line.strip())
if 'names' in element:
del element['names']
w.write(json.dumps(element))
but it just clears the whole json file
sorry for my bad english
The problem is that you open the same file twice - for reading and for writing simultaneously. Also a JSON cannot be parsed line by line, only as a whole.
import json
# 1. read
with open('players.json', 'r') as r:
data = json.load(r)
# 2. modify
# (you might want to check if data is a dict)
data['names'] = []
# 3. write
with open('players.json', 'w') as w:
data = json.dump(data, w)
I have a JSON file containing various objects each containing elements. With my python script, I only keep the objects I want, and then put the elements I want in a list. But the element has a prefix, which I'd like to suppress form the list.
The post-script JSON looks like that:
{
"ip_prefix": "184.72.128.0/17",
"region": "us-east-1",
"service": "EC2"
}
The "IP/mask" is what I'd like to keep. The List looks like that:
'"ip_prefix": "23.20.0.0/14",'
So what can I do to only keep "23.20.0.0/14" in the list?
Here is the code:
json_data = open(jsonsourcefile)
data = json.load(json_data)
print (destfile)
d=[]
for objects in (data['prefixes']):
if servicerequired in json.dumps(objects):
#print(json.dumps(objects, sort_keys=True, indent=4))
with open(destfile, 'a') as file:
file.write(json.dumps(objects, sort_keys=True, indent=4 ))
with open(destfile, 'r') as reads:
liste = list()
for strip in reads:
if "ip_prefix" in strip:
strip = strip.strip()
liste.append(strip)
print(liste)
Thanks,
dersoi
Ok so i've went through your JSON object
import json, urllib2
url = 'https://ip-ranges.amazonaws.com/ip-ranges.json'
req = urllib2.Request(url)
res = urllib2.urlopen(req)
j = json.load(res)
print j['prefixes'][0]['ip_prefix']
prefixes = j['prefixes']
for i in prefixes:
print i['ip_prefix']
the result:
>>>
23.20.0.0/14
23.20.0.0/14
27.0.0.0/22
43.250.192.0/24
43.250.193.0/24
46.51.128.0/18
46.51.192.0/20
46.51.216.0/21
46.51.224.0/19
etc...
So now you want all into one txt file right?
So you do this:
import json, urllib2
url = 'https://ip-ranges.amazonaws.com/ip-ranges.json'
req = urllib2.Request(url)
res = urllib2.urlopen(req)
j = json.load(res)
#print j['prefixes'][0]['ip_prefix']
prefixes = j['prefixes']
destfile = 'destfile.txt'
with open('destfile.txt', 'w') as f:
for i in prefixes:
#print i['ip_prefix']
f.write(i['ip_prefix'])
f.write('\n')
f.close
Best regards,
Rizzit
I've refactored your code, try this out:
import json
with open('sample.json', 'r') as data:
json_data = json.loads(data.read())
print json_data.get('ip_prefix')
# Output: "184.72.128.0/17"
You can rewrite the second open block as:
with open(destfile, 'r') as reads:
data = json.load(reads)
liste = [i['ip_prefix'] for i in data]
Although, I don't think you need to write to an intermediate file anyway, you could combine both blocks.
I'm trying to save a json page to a csv but it only stores the last row (although everything is visible on the print statement).
Is there anything wrong with my code?
url = 'http://mapit.mysociety.org/areas/WMC'
response = urllib2.urlopen(url)
html = response.read()
data = json.loads(html)
with open('mycsvfile.csv', 'w') as f: # Just use 'w' mode in 3.x
w = csv.DictWriter(f,['id', 'gss','name','country_name'])
w.writeheader()
for entry in data.values():
record = {
"id": entry["id"],
"gss": entry["codes"]["gss"],
"name": entry["name"],
"country_name": entry["country_name"],
}
print record
w.writerow(record)
Thanks in advance.
Kind regards
Hi I am trying to take the data from a json file and insert and id then perform POST REST.
my file data.json has:
{
'name':'myname'
}
and I would like to add an id so that the json data looks like:
{
'id': 134,
'name': 'myname'
}
So I tried:
import json
f = open("data.json","r")
data = f.read()
jsonObj = json.loads(data)
I can't get to load the json format file.
What should I do so that I can convert the json file into json object and add another id value.
Set item using data['id'] = ....
import json
with open('data.json', 'r+') as f:
data = json.load(f)
data['id'] = 134 # <--- add `id` value.
f.seek(0) # <--- should reset file position to the beginning.
json.dump(data, f, indent=4)
f.truncate() # remove remaining part
falsetru's solution is nice, but has a little bug:
Suppose original 'id' length was larger than 5 characters. When we then dump with the new 'id' (134 with only 3 characters) the length of the string being written from position 0 in file is shorter than the original length. Extra chars (such as '}') left in file from the original content.
I solved that by replacing the original file.
import json
import os
filename = 'data.json'
with open(filename, 'r') as f:
data = json.load(f)
data['id'] = 134 # <--- add `id` value.
os.remove(filename)
with open(filename, 'w') as f:
json.dump(data, f, indent=4)
I would like to present a modified version of Vadim's solution. It helps to deal with asynchronous requests to write/modify json file. I know it wasn't a part of the original question but might be helpful for others.
In case of asynchronous file modification os.remove(filename) will raise FileNotFoundError if requests emerge frequently. To overcome this problem you can create temporary file with modified content and then rename it simultaneously replacing old version. This solution works fine both for synchronous and asynchronous cases.
import os, json, uuid
filename = 'data.json'
with open(filename, 'r') as f:
data = json.load(f)
data['id'] = 134 # <--- add `id` value.
# add, remove, modify content
# create randomly named temporary file to avoid
# interference with other thread/asynchronous request
tempfile = os.path.join(os.path.dirname(filename), str(uuid.uuid4()))
with open(tempfile, 'w') as f:
json.dump(data, f, indent=4)
# rename temporary file replacing old file
os.rename(tempfile, filename)
There is really quite a number of ways to do this and all of the above are in one way or another valid approaches... Let me add a straightforward proposition. So assuming your current existing json file looks is this....
{
"name":"myname"
}
And you want to bring in this new json content (adding key "id")
{
"id": "134",
"name": "myname"
}
My approach has always been to keep the code extremely readable with easily traceable logic. So first, we read the entire existing json file into memory, assuming you are very well aware of your json's existing key(s).
import json
# first, get the absolute path to json file
PATH_TO_JSON = 'data.json' # assuming same directory (but you can work your magic here with os.)
# read existing json to memory. you do this to preserve whatever existing data.
with open(PATH_TO_JSON,'r') as jsonfile:
json_content = json.load(jsonfile) # this is now in memory! you can use it outside 'open'
Next, we use the 'with open()' syntax again, with the 'w' option. 'w' is a write mode which lets us edit and write new information to the file. Here s the catch that works for us ::: any existing json with the same target write name will be erased automatically.
So what we can do now, is simply write to the same filename with the new data
# add the id key-value pair (rmbr that it already has the "name" key value)
json_content["id"] = "134"
with open(PATH_TO_JSON,'w') as jsonfile:
json.dump(json_content, jsonfile, indent=4) # you decide the indentation level
And there you go!
data.json should be good to go for an good old POST request
try this script:
with open("data.json") as f:
data = json.load(f)
data["id"] = 134
json.dump(data, open("data.json", "w"), indent = 4)
the result is:
{
"name":"mynamme",
"id":134
}
Just the arrangement is different, You can solve the problem by converting the "data" type to a list, then arranging it as you wish, then returning it and saving the file, like that:
index_add = 0
with open("data.json") as f:
data = json.load(f)
data_li = [[k, v] for k, v in data.items()]
data_li.insert(index_add, ["id", 134])
data = {data_li[i][0]:data_li[i][1] for i in range(0, len(data_li))}
json.dump(data, open("data.json", "w"), indent = 4)
the result is:
{
"id":134,
"name":"myname"
}
you can add if condition in order not to repeat the key, just change it, like that:
index_add = 0
n_k = "id"
n_v = 134
with open("data.json") as f:
data = json.load(f)
if n_k in data:
data[n_k] = n_v
else:
data_li = [[k, v] for k, v in data.items()]
data_li.insert(index_add, [n_k, n_v])
data = {data_li[i][0]:data_li[i][1] for i in range(0, len(data_li))}
json.dump(data, open("data.json", "w"), indent = 4)
This implementation should suffice:
with open(jsonfile, 'r') as file:
data = json.load(file)
data[id] = value
with open(jsonfile, 'w') as file:
json.dump(data, file)
using context manager for the opening of the jsonfile.
data holds the updated object and dumped into the overwritten jsonfile in 'w' mode.
Not exactly your solution but might help some people solving this issue with keys.
I have list of files in folder, and i need to make Jason out of it with keys.
After many hours of trying the solution is simple.
Solution:
async def return_file_names():
dir_list = os.listdir("./tmp/")
json_dict = {"responseObj":[{"Key": dir_list.index(value),"Value": value} for value in dir_list]}
print(json_dict)
return(json_dict)
Response look like this:
{
"responseObj": [
{
"Key": 0,
"Value": "bottom_mask.GBS"
},
{
"Key": 1,
"Value": "bottom_copper.GBL"
},
{
"Key": 2,
"Value": "copper.GTL"
},
{
"Key": 3,
"Value": "soldermask.GTS"
},
{
"Key": 4,
"Value": "ncdrill.DRD"
},
{
"Key": 5,
"Value": "silkscreen.GTO"
}
]
}