I am learning python and one of the operations I am trying to complete is to change directories. I am storing directory names in a dictionary and then trying to access these directories as follows :
Creating the directories :
for i in range(length):
try:
original_umask = os.umask(0)
os.makedirs(item_dict[i], 0755)
finally:
os.umask(original_umask)
for i in range (length):
os.chdir(item_dict[i])
This is working. However when I try to do further processing later by accessing these directories I run into the following error :
osdir = os.getcwd()
print " current working directory is " + osdir //works correctly
for i in range (length):
os.chdir((item_dict[i])
Traceback (most recent call last):
File ".\actions.py", line 40, in <module>
os.chdir((item_dict[i]))
WindowsError: [Error 2] The system cannot find the file specified: u'd-025c49f7-116e-4ad1-909e-13cc59b03dc3/'
Here 025c49f7-116e-4ad1-909e-13cc59b03dc3/ is the directory name . The same code works earlier as indicated. Any pointers ?
os.chdir() changes the current directory for the remainder of the Python script. You are probably better off not changing directories at all, and manipulating files in directories using relative paths.
In other words,
os.mkdir('a')
os.chdir('a')
os.mkdir('b')
os.chdir('b')
ends up creating a/b and leaving you inside this directory; attempting to os.chdir('b') again fails, because there is no b inside this directory - you only just created it, so it is obviously empty.
I'm guessing you wanted to create a and b as subdirectories of the current directory, not the latter inside the former. Not doing os.chdir('a') after creating a obviously fixes this; and more generally, unless you specifically want to recursively create a deep sequence of nested directories, there is rarely a need for your programs to switch their working directory.
Needless to say,
os.chdir('a')
newfile = open('c', 'w')
can be rephrased without changing the working directory;
newfile = open(os.path.join(['a', 'c']), 'w')
where in this trivial case open('a/c', 'w') works as well; but if you do nontrivial processing where directory names and/or file names are in variables, you need to know about the more general os.path.join() syntax.
Related
I'm trying to use a program to read from a file with pi-digits. The program and the text file with the pi-digits are in the same directory, but i still get the error message :
with open('pi_digits.txt') as file_object:
contents = file_object.read()
print(contents.rstrip())
Traceback (most recent call last):
File "C:\Python\Python_Work\python_crash_course\files_and_exceptions\file_reader.py", line 1, in <module>
with open('pi_digits.txt') as file_object:
FileNotFoundError: [Errno 2] No such file or directory: 'pi_digits.txt'
I have looked for a solution but haven't found any.
I found a piece of code which supposedly shows me what the working directory is. I get an output that shows a directory that is 2 steps above the directory i have my programs and text file inside.
import os
cwd = os.getcwd() # Get the current working directory (cwd)
files = os.listdir(cwd) # Get all the files in that directory
print("Files in %r: %s" % (cwd, files))
So when i put the pi text document in the directory that the output is showing (>python_work), the program is working. When it does not work is when the text file is in ">files_and_exceptions" which is the same file the program itself is inside. My directory looks like this when it is not working:
>python_work
>python_crash_course
>files_and_exceptions
file_reader.py
pi_digits.txt
show_working_directory.py
And like this when it is working:
>python_work
pi_digits.txt
>python_crash_course
>files_and_exceptions
file_reader.py
show_working_directory.py
I'm new to python and really appreciate any help.
Thanks!
Relative path (one not starting with a leading /) is relative to some directory. In this case (and generally*), it's relative to the current working directory of the process.
In your case, given the information you've provided, for it would be "python_crash_course/files_and_exceptions/pi_digits.txt" in the first case as you appear to be running the script from python_work directory.
If you know the file to be in the same directory as the script itself, you could also say:
import os.path
...
os.path.join(os.path.dirname(__file__), "pi_digits.txt")
instead of "pi_digits.txt". Or the same using pathlib:
from pathlib import Path
...
Path(__file__).with_name("pi_digits.txt")
Actually unless you have a point to anchor to like the script itself, using relative filename / path (using absolute paths brings its own problems too) in the code directly is rather fragile and in that case getting it as a parameter of a function (and ultimately argument of CLI or script call in general) or alternatively reading it from standard input would be more robust.
* I would not make that an absolute statement, because there are situations and functions that can explicitly provide different anchor point (e.g. os.path.relpath or openat(2); or as another example a symlink target)
I am trying to make a python script that automatically moves files from my internal drive to any usb drive that is plugged in. However this destination path is unpredictable because I am not using the same usb drives everytime. With Raspbian Buster full version, the best I can do so far is automount into /media/pi/xxxxx, where that xxxxxx part is unpredictable. I am trying to make my script account for that. I can get the drive mounting points with
drives = os.listdir("/media/pi/")
but I am worried some will be invalid because of not being unmounted before they're yanked out (I need to run this w/o a monitor or keyboard or VNC or any user input other than replacing USB drives). So I'd think I'd need to do a series of try catch statements perhaps in an if elif elif elif chain, to make sure that the destination is truly valid, but I don't know how to do that w/o hardcoding the names in. The only way I know how to iterate thru a set of names I don't know is
for candidate_drive in drives:
but I don't know how to make it go onto the next candidate drive only if the current one is throwing an exception.
System: Raspberry Pi 4, Raspbian buster full, Python 3.7.
Side note: I am also trying this on Buster lite w/ usbmount, which does have predictable mounting names, but I can't get exfat and ntfs to mount and that is question for another post.
Update: I was thinking about this more and maybe I need a try, except, else statement where the except is pas and the else is break? I am trying it out.
Update2: I rethought my problem and maybe instead of looking for the exception to determine when to try the next possible drive, perhaps I could instead look for a successful transfer and break the loop if so.
import os
import shutil
files_to_send = os.listdir("/home/outgoing_files/")
source_path = "/home/outgoing_files/"
possible_USB_drives = os.listdir("/media/")
for a_possible_drive in possible_USB_drives:
try:
destpath = os.path.join("/media/", a_possible_drive)
for a_file in files_to_send:
shutil.copy(source_path + a_file, destpath)
except:
pass # transfer to possible drive didn't work
else:
break # Stops this entire program bc the transfer worked!
If you have an unsigned number of directories in side of a directory, etc... You cannot use nested for cicles. You need to implement a recursive call function. If you have directories inside a directory, you would like to review all the directories, this is posible iterating over the list of directories using a same function that iterate over them, over and over, until it founds the file.
Lets see an example, you have a path structure like this:
path
dir0
dir2
file3
dir1
file2
file0
file1
You have no way to now how many for cicles are required to iterate over al elements in this structure. You can call an iteration (for cicle) over all elements of a single directory, and do the same to the elements inside that elements. In this structure dirN where N is a number means a directory, and fileN means a file.
You can use os.listdir() function to get the contents of a directory:
print(os.listdir("path/"))
returns:
['dir0', 'dir1', 'file0.txt', 'file1.txt']
Using this function with all the directories you can get all the elements of the structure. You only need to iterate over all the directories. Specificly directories, because if you use a file:
print(os.listdir("path/file0.txt"))
you get an error:
NotADirectoryError: [WinError 267]
But, remember that in Python exists the generator expressions.
String work
If you have a mainpath you need to get access to a a directory inside this with a full string reference: "path/dirN". You cannot access directly to the file that does not is in the scope of the .py script:
print(os.listdir("dir0/"))
gets an error
FileNotFoundError: [WinError 3]
So you need to always format the initial mainpath with the actual path, in this way you can get access to al the elements of the structure.
Filtering
I said that you could use an generator expression to get just the directories of the structure, recursively. Lets take a look to a function:
def searchfile(paths: list,search: str):
for path in paths:
print("Actual path: {}".format(path))
contents = os.listdir(path)
print("Contents: {}".format(contents))
dirs = ["{}{}/".format(path,x) for x in contents if os.path.isdir("{}/{}/".format(path,x)) == True]
print("Directories: {} \n".format(dirs))
searchfile(dirs,search)
In this function we are getting the contents of the actual path, with os.listdir() and then filtering it with a generator expression. Obviusly we use recursive function call with the dirs of the actual path: searchfile(dirs,search)
This algorithm can be applied to any file structure, because the path argument is a list. So you can iterate over directories with directories with directories, and that directories with more directories inside of them.
If you want to get an specific file you could use the second argument, search. Also you can implement a conditional and get the specific path of the file found:
if search in contents:
print("File found! \n")
print("{}".format(os.path.abspath(search)))
sys.exit()
I hope have helped you.
I made a function in Python (WinOS) that ends up creating new directories (where future work is stored). So at some point, I define the directories and after I use os.mkdir(os.pahth.join('current_dir', 'new_dir1', 'new_dir2')). And it works.
The problem is that this same function is not working on Linux OS (Ubuntu). In Linux I can only generate a single directory. I.e.:
os.mkdir(os.path.join('current_dir', 'new_dir1', 'new_dir2')) returns error :
(OSError: [Errno 2] No such file or directory: '/[current_dir]/new_dir1/new_dir2')
but:
os.mkdir(os.path.join('current_dir', 'new_dir1')) - This works, returns the single directory
but I need to create 2 directories in Linux, not one...
I have tried several "easy combos" such as
new_dirs = os.path.join('new_dir1', 'new_dir2')
os.mkdir(os.path.join('current_dir', new_dirs)
returns the same error:
OSError: [Errno 2] No such file or directory: '/[current_dir]/new_dir1/new_dir2'
What I do (and works in Linux) is the next:
#Generate the path for the output files
working_path = os.path.join(outfile_path, 'First_Corregistration', 'Split_Area', '')
#Verify is the path exist, if not, create it.
if not os.path.exists(working_path):
os.mkdir(working_path)
Can somebody tell me how to create 2 or more new directories (one inside the other) with Linux. I highlight again what is more weird: My current solution works with Windows, but not with Linux.
instead of os.mkdir use os.makedirs, should work.
simple example:
os.makedirs("brand/new/directory")
should create the directories: brand, new and directory
from https://docs.python.org/3/library/os.html#os.mkdirs :
Recursive directory creation function. Like mkdir(), but makes all intermediate-level directories needed to contain the leaf directory.
I am getting a FileNotFound error when iterating through a list of files in Python on Windows.
The specific error I get looks like:
FileNotFoundError: File b'fileName.csv' does not exist
In my code, I first ask for input on where the file is located and generate a list using os (though I also tried glob):
directory = input('In what directory are your files located?')
fileList = [s for s in os.listdir(directory) if s.endswith('.csv')]
When I print the list, it does not contain byte b before any strings, as expected (but I still checked). My code seems to break at this step, which generates the error:
for file in fileList:
pd.read_csv(file) # breaks at this line
I have tried everything I could find on Stack Overflow to solve this problem. That includes:
Putting an r or b or rb right before the path string
Using both the relative and absolute file paths
Trying different variations of path separators (/, \, \\, etc.)
I've been dealing with Windows-related issues lately (since I normally work in Mac or Linux) so that was my first suspicion. I'd love another set of eyes to help me figure out where the snag is.
A2A.
Although the list was generated correctly because the full directory path was used, the working directory when the file was being run was .. You can verify this by running os.getcwd(). When later iterating through the list of file names, the program could not find those file names in the . directory, as it shouldn't. That list is just a list of file names; there was no directory tied to it so it used the current directory.
The easiest fix here is to change the directory the file is running through after the input. So,
directory = input('In what directory are your files located?')
os.chdir(directory) # Points os to given directory to avoid byte issue
If you need to access multiple directories, you could either do this right before you switch to each directory or save the full file path into your list instead.
How to remove path related problems in python?
For e.g. I have a module test.py inside a directory TEST
**test.py**
import os
file_path = os.getcwd() + '/../abc.txt'
f = open(file_path)
lines = f.readlines()
f.close
print lines
Now, when I execute the above program outside TEST directory, it gives me error:-
Traceback (most recent call last):
File "TEST/test.py", line 4, in ?
f = open(file_path)
IOError: [Errno 2] No such file or directory: 'abc.txt'
how to resolve this kind of problem. Basically this is just a small example that I have given up.
I am dealing with a huge problem of this kind.
I am using existing packages, which needs to be run only from that directory where it exists, how to resolve such kind of problems, so that I can run the program from anywhere I want.
Or able to deal with the above example either running inside TEST directory or outside TEST directory.
Any help.?
I think the easiest thing is to change the current working directory to the one of the script file:
import os
os.chdir(os.path.dirname(__file__))
This may cause problems, however, if the script is also working with files in the original working directory.
Your code is looking at the current working directory, and uses that as a basis for finding the files it needs. This is almost never a good idea, as you are now finding out.
The solution mentioned in the answer by Emil Vikström is a quickfix solution, but a more correct solution would be to not use current working directory as a startingpoint.
As mentioned in the other answer, __file__ isn't available in the interpreter, but it's an excellent solution for your code.
Rewrite your second line to something like this:
file_path = os.path.join(os.path.dirname(__file__), "..", "abc.txt")
This will take the directory the current file is in, join it first with .. and then with abc.txt, to create the path you want.
You should fix similar usage of os.getcwd() elsewhere in your code in the same way.