I am trying to load all the txt files from the folder and plot them. Which way is the easiest to load all txt files?
If you want to match all the files in a particular folder that are .txt files, why not use the glob module?
import pylab, glob
txt_files = glob.iglob("./*.txt")
for data in txt_files:
data = pylab.loadtxt(data)
pylab.plot(data[:,1], data[:,2])
pylab.show()
Related
I want to access some .jp2 image files inside a zip file and create a list of their paths. The zip file contains a directory folder named S2A_MSIL2A_20170420T103021_N0204_R108_T32UNB_20170420T103454.SAFE and I am currently reading the files using glob, after having extracted the folder.
I don't want to have to extract the contents of the zip file first. I read that I cannot use glob within a zip directory, nor I can use wildcards to access files within it, so I am wondering what my options are, apart from extracting to a temporary directory.
The way I am currently getting the list is this:
dirr = r'C:\path-to-folder\S2A_MSIL2A_20170420T103021_N0204_R108_T32UNB_20170420T103454.SAFE'
jp2_files = glob.glob(dirr + '/**/IMG_DATA/**/R60m/*B??_??m.jp2', recursive=True)
There are additional different .jp2 files in the directory, for which reason I am using the glob wildcards to filter the ones I need.
I am hoping to make this work so that I can automate it for many different zip directories. Any help is highly appreciated.
I made it work with zipfile and fnmatch
from zipfile import ZipFile
import fnmatch
zip = path_to_zip.zip
with ZipFile(zipaki, 'r') as zipObj:
file_list = zipObj.namelist()
pattern = '*/R60m/*B???60m.jp2'
filtered_list = []
for file in file_list:
if fnmatch.fnmatch(file, pattern):
filtered_list.append(file)
I have an issue within Jupyter that I cannot find online anywhere and was hoping I could get some help.
Essentially, I want to open .JSON files from multiple folders with different names. For example.
data/weather/date=2022-11-20/data.JSON
data/weather/date=2022-11-21/data.JSON
data/weather/date=2022-11-22/data.JSON
data/weather/date=2022-11-23/data.JSON
I want to be able to output the info inside the data.JSON onto my Jupyter Notebook, but how do I do that as the folder names are all different.
Thank you in advance.
What I tried so far
for path,dirs,files in os.walk('data/weather'): for file in files: if fnmatch.fnmatch(file,'*.json'): data = os.path.join(path,file) print(data)
OUTPUT:
data/weather/date=2022-11-20/data.JSON
data/weather/date=2022-11-21/data.JSON
data/weather/date=2022-11-22/data.JSON
data/weather/date=2022-11-23/data.JSON
But i dont want it to output the directory, I want to actually open the .JSON and display its content
This solution uses the os library to go thru different directories
import os
import json
for root, dirs, files in os.walk('data/weather'):
for file in files:
if file.endswith('.JSON'):
with open(os.path.join(root, file), 'r') as f:
data = json.load(f)
print(data)
I have about 30 .wav files in a folder C:\Users\Maheswar.reddy\Desktop\NLP\wav_folder. I am trying to write code to read all the .wav files in the folder, but I couldn't do it. How can I read all the files at once, given the name of the folder?
I was able to read a single file giving the path now I want to read all the files at once.
It's unclear what you mean by "read all the files at once". Here's an example using a pathlib glob that you can extend to process the files sequentially:
from pathlib import Path
base_path = Path(r"C:\Users\Maheswar.reddy\Desktop\NLP\wav_folder")
for wav_file_path in base_path.glob("*.wav"):
print(f"WAV File: {wav_file_path}")
# do something, e.g. with open(wav_file_path) as wav_file:
If you want to process all of the files concurrently, you'll need to look at threading or multiprocessing.
I have a folder of 400 individual data files saved on my Mac with pathway Users/path/to/file/data. I am trying to write code that will iterate through each data file and plot the data inside of it, however I am having trouble actually importing this folder of all the data into python. Does anyone have a way for me to import this entire folder so I can just iterate through each file by writing
for file in folder:
read data file
plot data file
Any help is greatly appreciated. Thank you!
EDIT: I am using Spyder for this also
You can use the os module to list all files in a directory by path. os provides a function os.listdir(), that when a path is passed, lists all items located in that path, like this: ['file1.py', 'file2.py']. If no argument is passed, it defaults to the current one.
import os
path_to_files = "Users/path/to/file/data"
file_paths = os.listdir(path_to_files)
for file_path in file_paths:
# reading file
# "r" means that you open the file for *reading*
with open(file_path, "r") as file:
lines = file.readlines()
# plot data....
I have multiple zip folders named zip_folder_1, zip_folder_2, ..., zip_folder_n.
All these zip folders are located in the same directory. Each one of these zip folders contains a csv file named "selected_file.csv".
I need to read each one of the "selected_file.csv" located at each one of the zip folders and concatenate them into a single file
Could someone give me a hint on the required python code to solve this problem? I appreciate your help!
This should produce concatenated_data.csv in your working directory, and assumes that all files in my_data_dir are zip files with data in them.
import os, numpy as np, zipfile
def add_data_to_file(new_data,file_name):
if os.path.isfile(file_name):
mode = 'ab'
else:
mode = 'wb'
with open(file_name,mode) as f:
np.savetxt(f,np.array([new_data]),delimiter=',')
my_data_dir = 'C:/my/zip/data/dir/'
data_files = os.listdir(my_data_dir)
for data_file in data_files:
full_path = os.path.join(my_data_dir,data_file)
with zipfile.ZipFile(full_path,'r',zipfile.ZIP_DEFLATED) as zip_file:
with zip_file.open('selected_file.csv','r') as selected_file:
data = np.loadtxt(selected_file,delimiter=",")
add_data_to_file(data,'concatenated_data.csv')