i am trying to write several .csv file into one specific directory
here is my code
with open(f+'.csv', 'w', newline='') as csvfile:
writer = csv.writer(csvfile)
writer.writerow(["index", "B", "G", "R"])
for row in rows:
writer.writerow(row)
writer.writerow(["Mean", mean_b/total_b, mean_g/total_g, mean_r/total_r])
writer.writerow("STD", np.sqrt(var_b/total_b), np.sqrt(var_g/total_g), np.sqrt(var_r/total_r))
i have created the csv file into the directory which is same as the .py file
however i would like to create a directory and create my csv file in it
i know i need to us os.makedirs() function
but i don't know whether i have to create the directory first and designate the path for the csv file or i simply put the directory name into the open() function
please help me
Instead of using os I recommend using the pathlib module.
You can create a directory with:
path = Path('path/to/dir')
path.mkdir(parents=True)
to create the directory and all its missing parent dirs.
After doing this you can create a file in the new directory with
fpath = (path / 'filename').with_suffix('.csv')
with fpath.open(mode='w+') as csvfile:
# your csv writer code
I would simply create the directory and except directory exists error
try:
os.mkdir("./CSV")
except OSError as e:
print("Directory exists")
with open("./CSV/" + f + ".csv", newline="") as csvfile:
[...]
You can add a check for the directory like this just before open statement
dir_path = 'folder_to_save_csv_file_in'
if not os.path.isdir(dir_path):
os.makedirs(dir_path)
with open('{file_path}.csv'.format(file_path=os.path.join(dir_path, file_name), 'w+') as csv_file:
....
Related
Let's assume I am writing Student details as a list into a CSV file.
Currently, I am working in a directory:
/home/ubuntu/Desktop/Pythontraining
but I want to store those CSV file in "StudentDetails" folder which is present in:
/home/ubuntu/Desktop/StudentDetails
as well as
/home/ubuntu/Documents/StudentDetails
I want to store the CSV file in both the directory during runtime. I am creating a new file (outfile), but I want to store it in a different directory. Say, I need to store the "outfile" into the folder whose name is "StudentDetails". In my case, I have created the "StudentDetails" folder in two different directory.
I want to save the file (outfile) in both directories. How should I do it manually?
try:
f=open(outfile, 'w')
for j in m:
writer = csv.writer(f)
writer.writerow(j)
except OSError:
print "Can't Change the current directory"
As I understood, you need to save the same file in two different directories, and to do it in a convenient way.
There are at least two ways:
You can create a function, that will save your desired data into two different files in a loop. And we need a function in order to have our code readable:
def multiple_save_st_details(m, path1, path2):
for outfile in [path1, path2]:
try:
f=open(outfile, 'w')
for j in m:
writer = csv.writer(f)
writer.writerow(j)
except OSError as o:
print("Can't Change the current directory")
We have a file to the first location and then copy it to a different directory:
def save_w_copy_st_details(m, filepath1, path2):
try:
f=open(filepath1, 'w')
for j in m:
writer = csv.writer(f)
writer.writerow(j)
except OSError as o:
print("Can't Change the current directory")
import shutil
shutil.copy2(filepath1, path2)
In that case, the filename for the target path may be excluded and will be taken the same as in the source
I have a csv file that looks like this:
dc_identifier,aubrey_identifier
AR0776-280206-LT513-01,metadc1084267
AR0776-280206-LT513-02,metadc1083385
AR0776-280206-LT513-03,metadc1084185
AR0776-280206-LT513-04,metadc1083449
AR0776-280206-LT513-05,metadc1084294
AR0776-280206-LT513-06,metadc1083393
AR0776-280206-LT513-07,metadc1083604
AR0776-280206-LT513-08,metadc1083956
AR0776-280206-LT513-09,metadc1083223
AR0776-280206-LT513-10,metadc1084224
I need to create folders with the "metadc#######" names within the directory that the script will live in.
Here's what I have so far:
import os
import fileinput
path = 'C:\Users\gpp0020\Desktop\TestDir'
textFile = 'C:\Users\gpp0020\Desktop\TestDir\kxas_ids.csv'
myList = open(textFile, 'rb+')
for line in myList:
for item in line.strip().split(','):
os.makedirs(os.path.join(path, item))
print 'created', item
However! I also need the program to grab files that are named with the identifiers (AR0776-280206-LT513-01, etc) and put them in the corresponding metadc number, according to the csv. Each file is doubled (one .mkv file, and one .mkv.md5 checksum file) and both need to go into the folder.
What's the best way to go about this?
Use the csv library to help with reading the file in:
import csv
import os
import shutil
path = r'C:\Users\gpp0020\Desktop\TestDir'
with open('kxas_ids.csv', 'r', newline='') as f_input:
csv_input = csv.reader(f_input)
header = next(csv_input)
for dv, aubrey in csv_input:
os.makedirs(os.path.join(path, aubrey), exist_ok=True)
mkv = '{}.mkv'.format(dv)
shutil.copy2(os.path.join(path, mkv), os.path.join(path, aubrey, mkv))
mkv_md5 = '{}.mkv.md5'.format(dv)
shutil.copy2(os.path.join(path, mkv_md5), os.path.join(path, aubrey, mkv_md5))
This would for example:
Create a folder called C:\Users\gpp0020\Desktop\TestDir\metadc108426
Copy a file called AR0776-280206-LT513-01.mkv into it.
Copy a file called AR0776-280206-LT513-01.mkv.md5 into it.
It assumes that all files are found in path
i'm trying to run python script which already ran on test environment.
already checked if the path correct and if the file in it.
checked in shell that the file exist.
current code is :
# Open a file
path = 'C:\\Users\\tzahi.k\\Desktop\\netzer\\'
dirs = os.listdir( path )
fileslst = []
alertsCode = (some data)
# loop over to search the relative file
for file in dirs:
if "ALERTS" in file.upper() :
fileslst.append(file)
fileslst.sort()
#open and modify the latest file
with open(fileslst[-1], 'rb') as csvfile:
csvReader = csv.reader(csvfile)
clean_rows = [row for row in csvReader if not any(alert in row[2] for alert in alertsCode)]
error :
IOError:error 2 no such file or directory:'file name'
when i debug in shell i see the path and files
what am i doing wrong?
os.listdir() lists the files relative to the directory.
You need to add the full directory path to the filename for it to be an absolute path again:
with open(os.path.join(path, fileslst[-1]), 'rb') as csvfile:
I am tring to save my .csv file which is a result of some queries in the same location as the .py file.
import os
with open(os.path.dirname(os.path.abspath(__file__))+'MyCSVFile.csv','wb') as output_file:
dict_writer = csv.DictWriter(output_file, keys)
dict_writer.writeheader()
dict_writer.writerows(myList)
I always seem to get my csv file one directory before. When I print os.path.dirname(os.path.abspath(__file__)) it gives me the proper path but the output MyCSVFile is saved one above. What is the problem here?
You have to use os.path.join to save the csv file in the same directory
import os
dirname = os.path.dirname(os.path.abspath(__file__))
csvfilename = os.path.join(dirname, 'MyCSVFile.csv')
with open(csvfilename, 'wb') as output_file:
dict_writer = csv.DictWriter(output_file, keys)
dict_writer.writeheader()
dict_writer.writerows(myList)
This should work as excepted
Remove the call to os.path.dirname since you are already calling os.path.abspath. Calling dirname returns the directory component thus you are getting the directory up in the hierarchy. BTW: use os.path.join to join parts of a directory.
So I am able to parse through multiple csv/txt files, remove the columns I wanted and saved them in a new file. I now want this to go to a new folder so that they are separate from the original files. I have 40 total original files and when I run my script, it moves 32 of the files over to the new folder, but I get a Permission denied when it starts on file 33. Why would this be happening if it works on so many of the previous files?
import os, sys, csv
path = ('C://Users//nelsonj//Desktop//Master_Project')
trimmed_files = ('C://Users//nelsonj//Desktop//Master_Project//Trimmed_Files')
for filename in os.listdir(path):
pref_cols = [0,1,2,4,6,8,10,12,14,18,20,22,24,26,30,34,36,40]
with open(filename, "rb") as sitefile:
with open(os.path.join(trimmed_files, filename.rsplit('.',1)[0] + "_trim.txt"), 'w') as output_file:
reader = csv.reader(sitefile, delimiter=',')
writer = csv.writer(output_file)
for row in reader:
new_row = list(row[i] for i in pref_cols)
writer.writerow(new_row)