I have 25 json files in a folder, named 0.json through 24.json, and I am trying to batch open and rename a perimeter "image" inside of each, which currently all have a placeholder of "https://" in the "image" field.
The .json currently appears as follows for each json file:
{"image": "https://", "attributes": [{"trait_type": "box color", "value": "blue"}, {"trait_type": "box shape", "value": "square"}]}
but should be
{"image": "https://weburlofnewimage/0", "attributes": [{"trait_type": "box color", "value": "blue"}, {"trait_type": "box shape", "value": "square"}]}
I have a central folder on a site like dropbox, that has a url structure of https://weburlofnewimage/0, /1, /2 etc. And so I would like to open each file, and change the value of the "image" key to be replaced with "https://weburlofnewimage/ + current file number + '.png'".
So far I am able to iterate through the files and change the image perimeter successfully within the json files, however the files seem to iterate in a random order, so on loop 1, I am getting file 20, and as a result file 20 is given file 0's image url.
Code as follows:
import json
import os
folderPath = r'/path/FolderWithJson/'
fileNumber = 0
for filename in os.listdir(folderPath):
print('currently on file ' + str(fileNumber))
if not filename.endswith(".json"): continue
filePath = os.path.join(folderPath, filename)
with open(filePath, 'r+') as f:
data = json.load(f)
data['image'] = str('https://weburlofnewimage/' + str(fileNumber) + '.png')
print('opening file ' + str(filePath))
os.remove(filePath)
with open(filePath, 'w') as f:
json.dump(data, f, indent=4)
print('removing file ' + str(filePath))
fileNumber +=1
Which results in me getting the following printouts:
currently on file 10 (on loops 10)
currently preparing file 2.json (its working on file #2...)
opening file /path/FolderWithJson/2.json
removing file /path/FolderWithJson/2.json
And then when I look in 2.json I see the image is changed to "https://weburlofnewimage/10.png" instead of "https://weburlofnewimage/2.png"
Just pull the number from the file name. Don't use your own count. And please remember you never need to use the str function on a string. Many people seem to be getting that bad habit.
import json
import os
folderPath = '/path/FolderWithJson/'
for filename in os.listdir(folderPath):
if not filename.endswith(".json"):
continue
fileNumber = os.path.splitext(filename)[0]
print('currently on file', fileNumber)
filePath = os.path.join(folderPath, filename)
print('opening file', filePath)
with open(filePath, 'r') as f:
data = json.load(f)
data['image'] = 'https://weburlofnewimage/'+fileNumber +'.png'
print('rewriting file', filePath)
with open(filePath, 'w') as f:
json.dump(data, f, indent=4)
You can open a file with a direct path, instead of iterating through the directory. I would use a for loop to insert the numbers into the path, that way they iterate in order.
for fileNumber in range(0,24):
with open(f'my_file/{fileNumber}.json') as f:
...doMyCode...
Related
I have a 1000 json files, I need to change the value of a specific line with numeric sequence in all files.
An example
the specific line is - "name": "carl 00",
I need it to be like following
File 1
"name": "carl 1",
File 1
"name": "carl 2",
File 3
"name": "carl 3",
What is the right script to achieve the above using python
This should do the trick. But you're not very clear about how the data is stored in the actual json file. I listed two different approaches. The first is to parse the json file into a python dict then manipulate the data and then turn it back into a character string and then save it. The second is what I think you mean by "line". You can split the file's character string into a list then change the line you want, and remake the full string again, then save it.
This also assumes your json files are in the same folder as the python script.
import os
import json
my_files = [name1, name2, name3, ...] # ['file_name.json', ...]
folder_path = os.path.dirname(__file__)
for i, name in enumerate(my_files):
path = f'{folder_path}/{name}'
with open(path, 'r') as f:
json_text = f.read()
# if you know the key(s) in the json file...
json_dict = json.loads(json_text)
json_dict['name'] = json_dict['name'].replace('00', str(i))
new_json_str = json.dumps(json_dict)
# if you know the line number in the file...
line_list = json_text.split('\n')
line_list[line_number - 1] = line_list[line_number - 1].replace('00', str(i))
new_json_str = '\n'.join(line_list)
with open(path, 'w') as f:
f.write(new_json_str)
Based on your edit, this is what you want:
import os
import json
my_files = [f'{i}.json' for i in range(1, 1001)]
folder_path = os.path.dirname(__file__) # put this .py file in same folder as json files
for i, name in enumerate(my_files):
path = f'{folder_path}/{name}'
with open(path, 'r') as f:
json_text = f.read()
json_dict = json.loads(json_text)
json_dict['name'] = f'carl {i}'
# include these lines if you want "symbol" and "subtitle" changed
json_dict['symbol'] = f'carl {i}'
json_dict['subtitle'] = f'carl {i}'
new_json_str = json.dumps(json_dict)
with open(path, 'w') as f:
f.write(new_json_str)
Without knowing more, the below loop will accomplish the posts requirements.
name = 'carl'
for i in range(0,1001):
print(f'name: {name} {i}')
example : S1-1.jpg, S1-2.jpg, S1-3.jpg in static directory
Now I wrote like this but it looks so messy
image = open('./static/S1-1.jpg', 'rb') #open binary file in read mode
image_read = image.read()
image_64_encode = base64.encodestring(image_read)
image.close()
image = open('./static/S1-2.jpg', 'rb') #open binary file in read mode
image_read = image.read()
image_64_encode = base64.encodestring(image_read)
image.close()
image = open('./static/S1-3.jpg', 'rb') #open binary file in read mode
image_read = image.read()
image_64_encode = base64.encodestring(image_read)
image.close()
and I want to write to json file's "URL"
json file's example
{ "intents": [ {
"tag": "S1-1",
"patterns": ["Where is S1-1", "S1-1", "Find S1-1","How to go to S1-1","Where S1-1"],
"responses": ["S1-1 : Blue is Library, Red is destination."],
"URL":[""]
} ] }
Thanks in advance
Try this:
files = ["S1-1", "S1-2", "S2-55" ] # a list of all the file names you want to use
for cur in files:
image = open(f'./static/{cur}.jpg', 'rb') #open binary file in read mode
image_read = image.read()
image_64_encode = base64.encodestring(image_read)
image.close()
Try This:
Create a list of file names. This requires you to enter the file names manually.
filenames = ['file1.txt', 'file2.txt', 'file3.txt']
Create a variable to store the file contents. This variable will store the text of the file for each iteration. "File_in" is an empty list that can store the contents of each file on each iteration.
file_in = list()
Use a "for" loop to cycle through each file name in the file name list. This will ensure each file opens and has a reference variable in the "file_in" list:
x = 0 for item in filenames:
file_in[x] = open(item, 'r')
x += 1
I have a dictionnary that group different pattern :
dico_cluster={'cluster_1': ['CUX2', 'CUX1'], 'cluster_2': ['RFX3', 'RFX2'],'cluster_3': ['REST']}
Then I have files in a folder :
"/path/to/test/files/CUX1.txt"
"/path/to/test/files/CUX2.txt"
"/path/to/test/files/RFX3.txt"
"/path/to/test/files/RFX2.txt"
"/path/to/test/files/REST.txt"
"/path/to/test/files/ZEB.txt"
"/path/to/test/files/TEST.txt"
I'm trying to concatenate the files that are in the same cluster. The output file name should be the name of pattern join by underscore "_"
I tried this :
filenames = glob.glob('/path/to/test/files/*.txt')
for clee in dico_cluster.keys():
fname='_'.join(dico_cluster[clee])
outfilename ='/path/to/test/outfiles/'+ fname + ".txt"
for file in filenames:
tf_file=file.split('/')[-1].split('.')[0]
if tf_file in dico_cluster[clee]:
with open(outfilename, 'wb') as outfile:
for filename in filenames:
if filename == outfilename:
# don't want to copy the output into the output
continue
with open(filename, 'rb') as readfile:
shutil.copyfileobj(readfile, outfile)
But it's not working. I'm just concatenating all the files.
I want to cat the file that are in the same cluster.
I would recommend to use os package, it's easier to use.
If I understood your problem I would try to do this by loading the whole content of your files before writing it.
import os
for clee in dico_cluster.keys():
my_clusters =list(set(dico_cluster[clee]))
fname = "_".join(my_clusters)
data = list()
outfilename = os.path.join("/path/to/test/outfiles", fname + ".txt")
for file in filenames:
tmp_dict = dict()
tf_file = os.path.basename(file).split(".")[0]
if tf_file in my_clusters:
with open(file, 'rb') as f1:
data.extend([elm for elm in f1.readlines()])
with open(outfilename, "wb") as _output_file:
for elm in data:
_output_file.write(elm)
I need to generate data and save it on a file in a directory, both are created at run time. "File Not Found error" occurs
I have some data which is created as below method
log = AnalyzeLog()
then I need to save that data in a file with .csv extension in the directory, both the directory and file supposed to be created at run time using the below code but I am not been able to create both...
plot_data_path = "E:\\Malicious_TLS_Detection-master\\M_TLS_Detection\\dataset\\data_model"
dir_name=dataset0
for dir_name in normal_folder_path:
path_to_single = normal_path + "\\" + dir_name
__PrintManager__.single_folder_header(path_to_single)
log.evaluate_features(path_to_single)
__PrintManager__.succ_single_folder_header()
log.create_plot_data(plot_data_path, dir_name)
def create_plot_data(self, path, filename):
__PrintManager__.evaluate_creating_plot()
self.create_dataset(path, filename)
__PrintManager__.succ_evaluate_data()
def create_dataset(self, path, filename):
index = 0
ssl_flow = 0
all_flow = 0
malicious = 0
normal = 0
# file header: label feature
header = [\
'label',\
'avg_domain_name_length',\
'std_domain_name_length',\
'avg_IPs_in_DNS']
with open(
path + "\\dataset-" + filename + ".csv", 'w+',
newline='') as f:
writer = csv.writer(f)
writer.writerow(header)
for key in self.conn_tuple:
label_feature = [\
str(self.conn_tuple[key].is_malicious()),\
length()),\
str(self.conn_tuple[key].avg_IPs_in_DNS())]
writer.writerow(label_feature)
print("<<< dataset file dataset-%s.csv successfully created !" %
filename)
The code just breaks at
with open(
path + "\\dataset-" + filename + ".csv", 'w+',
newline='') as f:
path=E:\\Malicious_TLS_Detection-master\\M_TLS_Detection\\dataset\\data_model
filename=dataset0
Data in the csv format must be created in a file but the following error arises
"No such file or directory: 'E:\Malicious_TLS_Detection-master\M_TLS_Detection\dataset\data_model\dataset-dataset0.csv'"
I have a folder consisting of 7 files, each having several text files inside. I intend to read through them and write each of those nested text files into a single file called ZebraAllRaw.txt. In the end, there must be only one single file containing all the text files that existed in each of those 7 files.
This is the function I have written:
def CombineFiles(folder):
with open('D:/ZebraAllRaw.txt', 'a', encoding="utf-8") as OutFile:
for root, dirs, files in os.walk(folder, topdown= False):
for filename in files:
file_path = os.path.join(root, filename)
with open(file_path, 'r', encoding="utf-8") as f:
content = f.read()
new_content = content.replace('\n', '')
OutFile.write(new_content + "\n")
However, it seems that all the content is written into the new file 9 times, as if it had read through them more than expected.
make sure you con't append the files from different runs.
I only replaced the file mode append with write at the open
def CombineFiles(folder):
with open('D:/ZebraAllRaw.txt', 'w', encoding="utf-8") as OutFile: # mode "w", not "a"
for root, dirs, files in os.walk(folder, topdown= False):
for filename in files:
file_path = os.path.join(root, filename)
with open(file_path, 'r', encoding="utf-8") as f:
content = f.read()
new_content = content.replace('\n', '')
OutFile.write(new_content + "\n")