Could someone shed me some lights on the file path matter in Python?
For example my codes need to read a batch of files, the file names are listed and stored in a .txt file, C:\filelist.txt, which content is:
C:\1stfile.txt
C:\2ndfile.txt
C:\3rdfile.txt
C:\4thfile.txt
C:\5thfile.txt
And the codes start with:
list_open = open('c:\\aaa.txt')
read_list = list_open.read()
line_in_list = read_list.split('\n')
all run fine. But if I want to read files in another path, such as:
C:\WorkingFolder\6thfile.txt
C:\WorkingFolder\7thfile.txt
C:\WorkingFolder\8thfile.txt
C:\WorkingFolder\9thfile.txt
C:\WorkingFolder\10thfile.txt
It doesn’t work. I guess the path here C:\WorkingFolder\ is not properly put so Python cannot recognize it.
So in what way I shall put it? Thanks.
hello all,
sorry that maybe i didn't make my self clear.
the problem is, a text file, c:\aaa.txt contains below:
C:\1stfile.txt
C:\WorkingFolder\1stfile.txt
why only C:\1stfile.txt is readable, but the other one not?
The reason your program isn't working is that you're not changing the directory properly. Use os.chdir() to do so, then open the files as normal:
import os
path = "C:\\WorkingFolder\\"
# Check current working directory.
retval = os.getcwd()
print "Current working directory %s" % retval
# Now change the directory
os.chdir( path )
# Check current working directory.
retval = os.getcwd()
print "Directory changed successfully %s" % retval
REFERENCES:
http://www.tutorialspoint.com/python/os_chdir.htm
import os
BASEDIR = "c:\\WorkingFolder"
list_open = open(os.path.join(BASEDIR, 'aaa.txt'))
Simply try using forward slashes instead.
list_open = open("C:/WorkingFolder/6thfile.txt", "rt")
It works for me.
Related
I am trying to write a program in python that loops through data from various csv files within a folder. Right now I just want to see that the program can identify the files in a folder but I am unable to have my code print the file names in my folder. This is what I have so far, and I'm not sure what my problem is. Could it be the periods in the folder names in the file path?
import glob
path = "Users/Sarah/Documents/College/Lab/SEM EDS/1.28.20 CZTS hexane/*.csv"
for fname in glob.glob(path):
print fname
No error messages are popping up but nothing will print. Does anyone know what I'm doing wrong?
Are you on a Linux-base system ? If you're not, switch the / for \\.
Is the directory you're giving the full path, from the root folder ? You might need to
specify a FULL path (drive included).
If that still fails, silly but check there actually are files in there, as your code otherwise seems fine.
This code below worked for me, and listed csv files appropriately (see the C:\\ part, could be what you're missing).
import glob
path = "C:\\Users\\xhattam\\Downloads\\TEST_FOLDER\\*.csv"
for fname in glob.glob(path):
print(fname)
The following code gets a list of files in a folder and if they have csv in them it will print the file name.
import os
path = r"C:\temp"
filesfolders = os.listdir(path)
for file in filesfolders:
if ".csv" in file:
print (file)
Note the indentation in my code. You need to be careful not to mix tabs and spaces as theses are not the same to python.
Alternatively you could use os
import os
files_list = os.listdir(path)
out_list = []
for item in files_list:
if item[-4:] == ".csv":
out_list.append(item)
print(out_list)
Are you sure you are using the correct path?
Try moving the python script in the folder when the CSV files are, and then change it to this:
import glob
path = "./*.csv"
for fname in glob.glob(path):
print fname
I am trying to access a file from a Box folder as I am working on two different computers. So the file path is pretty much the same except for the username.
I am trying to load a numpy array from a .npy file and I could easily change the path each time, but it would be nice if I could make it universal.
Here is what the line of code looks like on my one computer:
y_pred_walking = np.load('C:/Users/Eric/Box/CMU_MBL/Data/Calgary/4_Best_Results/Walking/Knee/bidir_lstm_50_50/predictions/y_pred_test.npy')
And here is what the line of code looks like on the other computer:
y_pred_walking = 'C:/Users/erapp/Box/CMU_MBL/Data/Calgary/4_Best_Results/Walking/Knee/bidir_lstm_50_50/predictions/y_pred_test.npy'
The only difference is that the username on one computer is Eric and the other is erapp, but is there a way where I can make the line universal to all computers where all computers will have the Box folder?
You could either save the file to a path that doesn't depend on the user: e.g. 'C:/Box/CMU_MBL/Data/Calgary/4_Best_Results/Walking/Knee/bidir_lstm_50_50/predictions/y_pred_test.npy'
Or you could do some string formatting. One way would be with an environment or configuration variable that indicates which is the relevant user, and then for your load statement:
import os
current_user = os.environ.get("USERNAME") # assuming you're running on the Windows box as the relevant user
# Now load the formatted string. f-strings are better, but this is more obvious since f-strings are still very new to Python
y_pred_walking = 'C:/Users/{user}/Box/CMU_MBL/Data/Calgary/4_Best_Results/Walking/Knee/bidir_lstm_50_50/predictions/y_pred_test.npy'.format(user=current_user)
Yes, there is a way, at least for the problem as it is right now solution is pretty simple: to use f-strings
user='Eric'
y_pred_walking =np.load(f'C:/Users/{user}/Box/CMU_MBL/Data/Calgary/4_Best_Results/Walking/Knee/bidir_lstm_50_50/predictions/y_pred_test.npy')
or more general
def pred_walking(user):
return np.load(f'C:/Users/{user}/Box/CMU_MBL/Data/Calgary/4_Best_Results/Walking/Knee/bidir_lstm_50_50/predictions/y_pred_test.npy')
so on any machine you just do
y_pred_walking=pred_walking(user)
with defined user before, to receive the result
Simply search the folders recursivly for your file:
filename = 'y_pred_test.npy'
import os
import random
# creates 1000 directories with a 1% chance of having the file as well
for k in range(20):
for i in range(10):
for j in range(5):
os.makedirs(f"./{k}/{i}/{j}")
if random.randint(1,100) == 2:
with open(f"./{k}/{i}/{j}/{filename}","w") as f:
f.write(" ")
# search the directories for your file
found_in = []
# this starts searching in your current folder - you can give it your c:\Users\ instead
for root,dirs,files in os.walk("./"):
if filename in files:
found_in.append(os.path.join(root,filename))
print(*found_in,sep = "\n")
File found in:
./17/3/1/y_pred_test.npy
./3/8/1/y_pred_test.npy
./16/3/4/y_pred_test.npy
./16/5/3/y_pred_test.npy
./14/2/3/y_pred_test.npy
./0/5/4/y_pred_test.npy
./11/9/0/y_pred_test.npy
./9/8/1/y_pred_test.npy
If you get read errors because of missing file/directory permissions you can start directly in the users folder:
# Source: https://stackoverflow.com/a/4028943/7505395
from pathlib import Path
home = str(Path.home())
found_in = []
for root,dirs,files in os.walk(home):
if filename in files:
found_in.append(os.path.join(root,filename))
# use found_in[0] or break as soon as you find first file
You can use the expanduser function in the os.path module to modify a path to start from the home directory of a user
https://docs.python.org/3/library/os.path.html#os.path.expanduser
So I have code that completes an analysis for me on imaging data. This code is working fine except that the data is not being outputted into the correct folder. Instead, it is being outputted one folder too high. This is something I obviously want to fix in the long run, but as I am under certain time constraints I want to input code into my script that will simply move the files to the correct folder/directory that I have created. I have tried the mv and shutil commands but they don't seem to be working. I would be grateful if someone had a suggestion for how to fix/improve my method of moving these files to the correct location. If someone has a suggestion for why the files are being outputted to the wrong directory as well that would be awesome. I am relatively new to coding and no expert so please forgive any obvious mistakes. Thank you.
This is where I set up my directories
subject_dir = os.getcwd()
dti_dir = os.path.abspath( os.path.join(subject_dir, 'dti'))
dti_input_dir = os.path.abspath( os.path.join(dti_dir, 'input'))
This is where I entered a few shortcuts
eddy_prefix = 'eddy'
input_path = dti_input_dir
output_prefix = 'dtifit'
output_path = '../dti/dtifit'
output_basename = os.path.abspath(os.path.join(dti_dir, output_path))
infinite_path = os.path.join(os.getenv('INFINITE_PATH'), 'infinite')
dti30 = 'dti30.nii.gz'
dti30_brain = 'bet.b0.dti30.nii.gz'
dti30_mask = 'bet.b0.dti30_mask.nii.gz'
This is where I ran my test.
The test is being run, but my data is being outpputed in dti_dir and not output_basename (this is my second question)
dti = fsl.DTIFit()
dti.inputs.dwi = os.path.join(input_path, eddy_prefix + '.nii.gz')
dti.inputs.bvecs = os.path.join(input_path, eddy_prefix + '.eddy_rotated_bvecs')
dti.inputs.bvals = os.path.abspath(os.path.join(infinite_path, 'dti30.bval'))
dti.inputs.base_name = output_basename
dti.inputs.mask = os.path.join(input_path, dti30_mask)
dti.cmdline
Creating output directory if doesn't exist.
This is working fine and the directory is created in the proper location.
if not os.path.exists(output_basename):
os.makedirs(output_basename)
print('DTI Command line')
print(dti.cmdline)
res = dti.run()
exit()
print('DTIFIT Complete!')
Here I try to move the files and I get the error: IOError: [Errno 2] No such file or directory: even though I know the files exist
src = dti_dir
dst = output_basename
files = os.listdir(src)
for f in files:
if (f.startswith("dtifit_")):
shutil.move(f, dst)
Your problem is most likely in the very first line. You may want to change that to be more exact, pointing to an exact directory rather than os.getcwd(). os.getcwd() will point to wherever the process is executing from, wherever you kicked off your python run, so it could change. You probably want to hard-code it somehow.
For example, you could use something like this, to point to the directory that the file lives in:
Find current directory and file's directory
Another thing you could consider doing is printing out dst in your last few lines and seeing if it's what you expect. You could also use PDB to inspect that value live:
https://pymotw.com/2/pdb/
Edit:
Your problem is using a relative path with shutil.move(). You should make sure that the path is absolute, please see this answer for more info:
stackoverflow.com/a/22230227/7299836
I´m trying to save a file, which I create with the "open" function.
Well I tried nearly everything to change the directory, but nothing works. The file gets always saved in the folder of my file, which I read in before.
file = open(fname[0] + ft, 'w')
file.write("Test")
file.close()
So this is it simple, but what do I have to add, to change the path of creation?
The File Dialog in a individual Function:
global fname
fname = QFileDialog.getOpenFileName(None, 'Please choose your File.',"C:\\Program Files", "Text-Files(*.txt)")
And the File Typ ( in a individual Function too) I set the file type by ticking a check box and ft will set to .py or .pyw
if self.exec_py.isChecked() == True:
global ft
ft = ".py"
I should have mentioned that I already tried os.path.join and os.chdir, but the file will get printed in the file anyway. Any solutions or approaches how to fix it? Here is how i tried it:
tmppath = "C:/temp"
tmp = os.path.join(tmppath,fname[0]+ft)
file = open(tmp, 'w')
Your question is a little short on details, but I am guessing that fname is the tuple returned by QFileDialog, and so fname[0] is the absolute path of the original file. So if you display fname[0], you will see something like this:
>>> fname[0]
'C:\\myfolder\\file.txt'
Now look what happens when you try to use that with os.path.join:
>>> tmppath = 'C:\\temp'
>>> os.path.join(tmppath, fname[0])
'C:\\myfolder\\file.txt'
Nothing! Conclusion: attempting to join two absolute paths will simply return the original path unchanged. What you need to do instead is take the basename of the original path, and join it to the folder where you want to save it:
>>> basename = os.path.basename(fname[0])
>>> basename
'file.txt'
>>> os.path.join(tmppath, basename)
'C:\\tmp\\file.txt'
Now you can use this new path to save your file in the right place.
You need to provide the full filepath
with open(r'C:\entire\path\to\file.txt', 'w') as f:
f.write('test')
If you just provide a file name without a path, it will use the current working directory, which isn't necessarily the directory where the python script your running is located. It will be the directory where you launched the script from.
C:\Users\admin> python C:\path\to\my_script.py
In this instance, the current working directory is C:\Users\admin, not C:\path\to.
I'm working in Python and I'm confused why I can't open the file I'm trying to. The code is pretty simple. Here it is.
import os
def main():
FILE_NAME = "default_template.csv"
source_path = os.path.join("Documents", FILE_NAME)
file = open(source_path, "r")
At this point I get
IOError: [Errno 2] No such file or directory: 'Documents/FILE_NAME'.
I also decided to try to change directories for whatever reason using os.chdir() and passing just about every high level directory on my computer in seperately and nothing worked. In an attempt to find the fix for opening a file I tried editing the path in a bunch of different ways.
I've tried something like:
os.path.join("/derek/Documents", FILE_NAME)
os.path.join("/Documents", FILE_NAME)
os.path.join("~/derek/Documents", FILE_NAME)
os.path.join("~", FILE_NAME)
If anyone could help me out I would be extremely thankful. I'm still new to using python to navigate and manage files.
Python doesn't expand ~ by default. To make it work try os.path.expanduser:
On Unix and Windows, return the argument with an initial component of ~ or ~user replaced by that user‘s home directory.
source_path = os.path.join("~/Documents", FILE_NAME)
source_path = os.path.expanduser(source_path)