basically, I'm trying to store the full path for a file in a list but for some reason os.path.abspath() doesnt seem to work properly
files = os.listdir("TRACKER/")
for f in files:
original_listpaths.append(os.path.abspath(f))
print(original_listpaths)
but my output seems to output this :
'C:\Users\******\Documents\folder\example'
the problem is that it should be :
'C:\Users\******\Documents\folder\TRACKER\example'
the difference is that the second one (the correct one) has the TRACKER included which is the official full path for that file but for some reason my output doesn't include the TRACKER
and eliminates it, What's the problem?
You could try the following code:
files = os.scandir("TRACKER/")
print(files)
original_listpaths = []
for f in files:
original_listpaths.append(os.path.abspath(f))
print(original_listpaths)
files.close()
You need to change your directory to "TRACKER" first. Just put os.chdir("TRACKER") before the loop starts after files = os.listdir("TRACKER/").
Related
I'll try to describe the problem in a simple way.
I have a .txt file that I can not know the full name of it which located under constant path
[for example: the full name is: Hello_stack.txt, I only can give to function the part: 'Hello_']
the input is: Path_to_file/ + 'Hello_'
the expected output is: Path_to_file/Hello_stack.txt
How can i do that?
I tried to give a path and check recursively if part of my file name is exist and if so, to return it's path.
this is my implementation: [of course I'd like to get another way if it works]
def get_CS_R2M_METRO_CALLBACK_FILE_PATH():
directory = 'path_of_file'
file_name = directory + 'part_of_file_name'
const_path = Path(file_name)
for path in [p for p in const_path.rglob("*")]:
if path.is_file():
return path
Thanks for help.
You might retrieve the file list in your path and then select from the list based upon your partial file name. Here is a snippet of code to perform that type of function on a Linux machine.
import os
dir = '/home/craig/Python_Programs/GetFile'
files = os.listdir(dir)
print('Files--> ', files)
for i in files:
myfile = 'Hello_'
if (myfile[0:4] == i[0:4]):
print('File(s) like \"Hello_\"-->', i)
When I executed this simple program over a directory/folder that had various files in the directory, here was the output to the terminal.
Una:~/Python_Programs/GetFile$ python3 GetFile.py
Files--> ['Hello_Stack.txt', 'Okay.txt', 'Hi_Stack.txt', 'GetFile.py', 'Hello_Stack.bak']
File(s) like "Hello_"--> Hello_Stack.txt
File(s) like "Hello_"--> Hello_Stack.bak
The literal value for your path would be different on a Windows machine. I hope this might provide you with a method to achieve your goal.
Regards.
I had 120 files in my source folder which I need to move to a new directory (destination). The destination is made in the function I wrote, based on the string in the filename. For example, here is the function I used.
path ='/path/to/source'
dropbox='/path/to/dropbox'
files = = [os.path.join(path,i).split('/')[-1] for i in os.listdir(path) if i.startswith("SSE")]
sam_lis =list()
for sam in files:
sam_list =sam.split('_')[5]
sam_lis.append(sam_list)
sam_lis =pd.unique(sam_lis).tolist()
# Using the above list
ID = sam_lis
def filemover(ID,files,dropbox):
"""
Function to move files from the common place to the destination folder
"""
for samples in ID:
for fs in files:
if samples in fs:
desination = dropbox + "/"+ samples + "/raw/"
if not os.path.isdir(desination):
os.makedirs(desination)
for rawfiles in fnmatch.filter(files, pat="*"):
if samples in rawfiles:
shutil.move(os.path.join(path,rawfiles),
os.path.join(desination,rawfiles))
In the function, I am creating the destination folders, based on the ID's derived from the files list. When I tried to run this for the first time it threw me FILE NOT exists error.
However, later when I checked the source all files starting with SSE were missing. In the beginning, the files were there. I want some insights here;
Whether or not os.shutil.move moves the files to somewhere like a temp folder instead of destination folder?
whether or not the os.shutil.move deletes the files from the source in any circumstance?
Is there any way I can test my script to find the potential reasons for missing files?
Any help or suggestions are much appreciated?
It is late but people don't understand the op's question. If you move a file into a non-existing folder, the file seems to become a compressed binary and get lost forever. It has happened to me twice, once in git bash and the other time using shutil.move in Python. I remember the python happens when your shutil.move destination points to a folder instead of to a copy of the full file path.
For example, if you run the code below, a similar situation to what the op described will happen:
src_folder = r'C:/Users/name'
dst_folder = r'C:/Users/name/data_images'
file_names = glob.glob(r'C:/Users/name/*.jpg')
for file in file_names:
file_name = os.path.basename(file)
shutil.move(os.path.join(src_folder, file_name), dst_folder)
Note that dst_folder in the else block is just a folder. It should be dst_folder + file_name. This will cause what the Op described in his question. I find something similar on the link here with a more detailed explanation of what went wrong: File moving mistake with Python
shutil.move does not delete your files, if for any reason your files failed to move to a given location, check the directory where your code is stored, for a '+' folder your files are most likely stored there.
How do I get the data from multiple txt files that placed in a specific folder. I started with this could not fix. It gives an error like 'No such file or directory: '.idea' (??)
(Let's say I have an A folder and in that, there are x.txt, y.txt, z.txt and so on. I am trying to get and print the information from all the files x,y,z)
def find_get(folder):
for file in os.listdir(folder):
f = open(file, 'r')
for data in open(file, 'r'):
print data
find_get('filex')
Thanks.
If you just want to print each line:
import glob
import os
def find_get(path):
for f in glob.glob(os.path.join(path,"*.txt")):
with open(os.path.join(path, f)) as data:
for line in data:
print(line)
glob will find only your .txt files in the specified path.
Your error comes from not joining the path to the filename, unless the file was in the same directory you were running the code from python would not be able to find the file without the full path. Another issue is you seem to have a directory .idea which would also give you an error when trying to open it as a file. This also presumes you actually have permissions to read the files in the directory.
If your files were larger I would avoid reading all into memory and/or storing the full content.
First of all make sure you add the folder name to the file name, so you can find the file relative to where the script is executed.
To do so you want to use os.path.join, which as it's name suggests - joins paths. So, using a generator:
def find_get(folder):
for filename in os.listdir(folder):
relative_file_path = os.path.join(folder, filename)
with open(relative_file_path) as f:
# read() gives the entire data from the file
yield f.read()
# this consumes the generator to a list
files_data = list(find_get('filex'))
See what we got in the list that consumed the generator:
print files_data
It may be more convenient to produce tuples which can be used to construct a dict:
def find_get(folder):
for filename in os.listdir(folder):
relative_file_path = os.path.join(folder, filename)
with open(relative_file_path) as f:
# read() gives the entire data from the file
yield (relative_file_path, f.read(), )
# this consumes the generator to a list
files_data = dict(find_get('filex'))
You will now have a mapping from the file's name to it's content.
Also, take a look at the answer by #Padraic Cunningham . He brought up the glob module which is suitable in this case.
The error you're facing is simple: listdir returns filenames, not full pathnames. To turn them into pathnames you can access from your current working directory, you have to join them to the directory path:
for filename in os.listdir(directory):
pathname = os.path.join(directory, filename)
with open(pathname) as f:
# do stuff
So, in your case, there's a file named .idea in the folder directory, but you're trying to open a file named .idea in the current working directory, and there is no such file.
There are at least four other potential problems with your code that you also need to think about and possibly fix after this one:
You don't handle errors. There are many very common reasons you may not be able to open and read a file--it may be a directory, you may not have read access, it may be exclusively locked, it may have been moved since your listdir, etc. And those aren't logic errors in your code or user errors in specifying the wrong directory, they're part of the normal flow of events, so your code should handle them, not just die. Which means you need a try statement.
You don't do anything with the files but print out every line. Basically, this is like running cat folder/* from the shell. Is that what you want? If not, you have to figure out what you want and write the corresponding code.
You open the same file twice in a row, without closing in between. At best this is wasteful, at worst it will mean your code doesn't run on any system where opens are exclusive by default. (Are there such systems? Unless you know the answer to that is "no", you should assume there are.)
You don't close your files. Sure, the garbage collector will get to them eventually--and if you're using CPython and know how it works, you can even prove the maximum number of open file handles that your code can accumulate is fixed and pretty small. But why rely on that? Just use a with statement, or call close.
However, none of those problems are related to your current error. So, while you have to fix them too, don't expect fixing one of them to make the first problem go away.
Full variant:
import os
def find_get(path):
files = {}
for file in os.listdir(path):
if os.path.isfile(os.path.join(path,file)):
with open(os.path.join(path,file), "r") as data:
files[file] = data.read()
return files
print(find_get("filex"))
Output:
{'1.txt': 'dsad', '2.txt': 'fsdfs'}
After the you could generate one file from that content, etc.
Key-thing:
os.listdir return a list of files without full path, so you need to concatenate initial path with fount item to operate.
there could be ideally used dicts :)
os.listdir return files and folders, so you need to check if list item is really file
You should check if the file is actually file and not a folder, since you can't open folders for reading. Also, you can't just open a relative path file, since it is under a folder, so you should get the correct path with os.path.join. Check below:
import os
def find_get(folder):
for file in os.listdir(folder):
if not os.path.isfile(file):
continue # skip other directories
f = open(os.path.join(folder, file), 'r')
for line in f:
print line
I want to rename a file from say {file1} to {file2}. I read about os.rename(file1,file2) in python and is able to do so.
I succeeded only when the the file is placed in the same folder as python script, so I want to ask how can we rename files of other folders i.e. different folder than the one in which python script is placed.
Just use the full path, instead of the relative path:
oldFile = 'C:\\folder\\subfolder\\inFile.txt'
newFile = 'C:\\foo\\bar\\somewhere\\other\\outFile.txt'
os.rename(oldFile, newFile)
To get the double-slash behavior, you can do the following
import os
oldFile = r'C:\folder\subfolder\inFile.txt' # note the r character for raw string
os.path.normpath(oldFile)
Output
'C:\\folder\\subfolder\\inFile.txt'
As others have noted, you need to use full path.
On the other note, take a look at shutil.move documentation, it can also be used for renaming.
I'm trying to import some files to plot, and all was going well until I moved my program to the directory above where it was before. The relevant piece of code that seems to be problematic is below:
import os
import pandas as pd
path = os.getcwd() + '/spectrum_scan/'
files = os.listdir(path)
dframefiles = pd.DataFrame(files)
up = pd.read_csv(dframefiles.ix[i][0])
If I type directly into the shell os.path.exists(path) it returns True.
The first file in the directory spectrum_scan is foo.csv.
When I type os.path.exists(path + 'foo.csv') it returns True but os.path.isfile('foo.csv') returns False.
Also, asking for files and dframefiles returns everything as it should, but when the code is run I get Exception: File foo.csv does not exist.
Is there something obvious I'm missing?
You are using os.listdir(), which returns filenames without a path. You'll need to add the path to these:
files = [os.path.join(path, f) for f in os.listdir(path)]
otherwise python will try and look for 'foo.csv' in the current directory, and not in the spectrum_scan sub-directory where the files really are located.