I have the following file structure:
test/
test1.py
test2.py
text.txt
Here are the contents of the files
test1.py:
import sys
sys.path.append('../')
import test2
test2.read()
test2.py:
def read():
with open('text.txt', 'rb') as f:
print f.read()
if __name__ == "__main__":
read()
text.txt contains a line of text. When I run test1.py, I get a "File not found" error:
Traceback (most recent call last):
File "test1.py", line 5, in <module>
test2.read()
File "../test2.py", line 2, in read
with open('text.txt', 'rb') as f:
IOError: [Errno 2] No such file or directory: 'text.txt'
I kind of understand why this error is coming up. But how do I deal with these kind of errors. I would like the code in test2.py to be like library code which I can use anywhere.
sys.path used for python path (PYTHONPATH eviroment variable).
i.e. where to look for python libraries when you import some library.
it dose not effect where open() is looking for files.
when you open(filename). the filename is relative to the procees working directory. (the path the code was run from)
so if you want to access a flie that its path is relative to the path of the code file, then you can use the builtin variable __file__ which hold the current file path.
so you can change test2.py to be:
import os
def read():
with open(os.path.join(os.path.dirname(__file__),'text.txt'), 'rb') as f:
print f.read()
The proper way of doing what you're asking for is to use pkg_resources as described here. Basically something like the following would be what you want in test2.py:
import pkg_resources
def read():
with pkg_resources.resource_stream(__name__, 'text.txt') as f:
print f.read()
if __name__ == "__main__":
read()
Don't use relative path, then. Use the full path:
with open(r'C:\Somewhere\someplace\test.txt') as f:
Related
I'm simply trying to alter a CSV file with Python.
When all the files were in the same dir, everything was fine.
Now that the input files are in a different dir than where the output files will be, everything blows up, apparently b/c the files do not exist?
I first found this:
open() in Python does not create a file if it doesn't exist
Then I learned to change to the directory, which helped me loop over the CSVs in the target dir:
Moving up one directory in Python
When I run the command:
python KWRG.py ../Weekly\ Reports\ -\ Inbound/Search\ Activity/ 8/9/2021
I will get:
Traceback (most recent call last): File "KWRG.py", line 15, in <module> with open(args.input, 'r') as in_file, open(args.output, 'w') as out_file: IsADirectoryError: [Errno 21] Is a directory: '../Weekly Reports - Inbound/Search Activity/'
Sorry If I'm missing the obvious here, but why is the file not being created in the directory that I'm pointing the script to (or at all for that matter)?
The code:
import csv
import argparse
import os
# Create a parser to take arguments
#...snip...
cur_dir = os.getcwd()
reports_dir = os.chdir(os.path.join(cur_dir, args.dir))
for csv_file in os.listdir(reports_dir):
# Shorthand the name of the file
#...snip...
# Open the in and out files
with open(csv_file, 'r') as in_file, open(f'{out_name}-Search-Activity-{args.date}.csv', 'w+') as out_file:
# Re-arrange CSV
# EOF
Your problem is with this line:
reports_dir = os.chdir(os.path.join(cur_dir, args.dir))
os.chdir() doesn't return anything, it just performs the operation requested - changing the current working directory. From an interactive session with the REPL:
>>> import os
>>> result = os.chdir("/Users/mattdmo/")
>>> result
>>>
For your purposes, all you need is
reports_dir = os.path.join(cur_dir, args.dir)
and you'll be all set.
Folder Structure
Project_Folder -|
Scripts-|
images -|
file.py
Inside file.py
import requests
import os
import sys
sys.path.append('.')
url = "http://d2np4vr8r37sds.cloudfront.net/800313-kkenlukmjy-1481094947.jpg"
image = requests.get(url)
IMG_DIR = 'images'
IMG_DIR_ABS = os.path.join('.', IMG_DIR)
filename = "800313-kkenlukmjy-1481094947.jpg"
fullfilename = os.path.join(IMG_DIR_ABS, filename)
with open(fullfilename, 'wb') as f:
f.write(image.content)
f.close()
When I am running this code I am getting following error
python scripts/file.py
Traceback (most recent call last):
File "scripts/file.py", line 9, in <module>
with open(fullfilename, 'wb') as f:
IOError: [Errno 2] No such file or directory: './images/800313-kkenlukmjy-1481094947.jpg'
IMG_DIR_ABS = os.path.join('.', IMG_DIR) does not resolve the location of the current script. . is the current directory, not the directory of the current script.
What you want to do is:
IMG_DIR_ABS = os.path.join(os.path.dirname(__file__), IMG_DIR)
__file__ being the full filepath of the currently running script, which matches your expectations given your directory tree.
the advantage of this method is that you can launch your script from any directory.
Aside: no need for f.close() within the with block. with context handles that for you when exiting the block.
You are running your script from Project_Folder, but there is no such path './images/800313-kkenlukmjy-1481094947.jpg' if You look from that directory. Try using './scripts/images/800313-kkenlukmjy-1481094947.jpg' instead.
Say I have a Python project that is structured as follows:
project
/data
test.csv
/package
__init__.py
module.py
main.py
__init__.py:
from .module import test
module.py:
import csv
with open("..data/test.csv") as f:
test = [line for line in csv.reader(f)]
main.py:
import package
print(package.test)
When I run main.py I get the following error:
C:\Users\Patrick\Desktop\project>python main.py
Traceback (most recent call last):
File "main.py", line 1, in <module>
import package
File "C:\Users\Patrick\Desktop\project\package\__init__.py", line 1, in <module>
from .module import test
File "C:\Users\Patrick\Desktop\project\package\module.py", line 3, in <module>
with open("../data/test.csv") as f:
FileNotFoundError: [Errno 2] No such file or directory: '../data/test.csv'
However, if I run module.py from the package directory, I don’t get any errors. So it seems that the relative path used in open(...) is only relative to where the originating file is being run from (i.e __name__ == "__main__")? I don't want to use absolute paths. What are some ways to deal with this?
Relative paths are relative to current working directory.
If you do not want your path to be relative, it must be absolute.
But there is an often used trick to build an absolute path from current script: use its __file__ special attribute:
from pathlib import Path
path = Path(__file__).parent / "../data/test.csv"
with path.open() as f:
test = list(csv.reader(f))
This requires python 3.4+ (for the pathlib module).
If you still need to support older versions, you can get the same result with:
import csv
import os.path
my_path = os.path.abspath(os.path.dirname(__file__))
path = os.path.join(my_path, "../data/test.csv")
with open(path) as f:
test = list(csv.reader(f))
[2020 edit: python3.4+ should now be the norm, so I moved the pathlib version inspired by jpyams' comment first]
For Python 3.4+:
import csv
from pathlib import Path
base_path = Path(__file__).parent
file_path = (base_path / "../data/test.csv").resolve()
with open(file_path) as f:
test = [line for line in csv.reader(f)]
This worked for me.
with open('data/test.csv') as f:
My Python version is Python 3.5.2 and the solution proposed in the accepted answer didn't work for me. I've still were given an error
FileNotFoundError: [Errno 2] No such file or directory
when I was running my_script.py from the terminal. Although it worked fine when I run it through Run/Debug Configurations from the PyCharm IDE (PyCharm 2018.3.2 (Community Edition)).
Solution:
instead of using:
my_path = os.path.abspath(os.path.dirname(__file__)) + some_rel_dir_path
as suggested in the accepted answer, I used:
my_path = os.path.abspath(os.path.dirname(os.path.abspath(__file__))) + some_rel_dir_path
Explanation:
Changing os.path.dirname(__file__) to os.path.dirname(os.path.abspath(__file__))
solves the following problem:
When we run our script like that: python3 my_script.py
the __file__ variable has a just a string value of "my_script.py" without path leading to that particular script. That is why method dirname(__file__) returns an empty string "". That is also the reason why my_path = os.path.abspath(os.path.dirname(__file__)) + some_rel_dir_path is actually the same thing as my_path = some_rel_dir_path. Consequently FileNotFoundError: [Errno 2] No such file or directory is given when trying to use open method because there is no directory like "some_rel_dir_path".
Running script from PyCharm IDE Running/Debug Configurations worked because it runs a command python3 /full/path/to/my_script.py (where "/full/path/to" is specified by us in "Working directory" variable in Run/Debug Configurations) instead of justpython3 my_script.py like it is done when we run it from the terminal.
Try
with open(f"{os.path.dirname(sys.argv[0])}/data/test.csv", newline='') as f:
I was surprised when the following code worked.
import os
for file in os.listdir("../FutureBookList"):
if file.endswith(".adoc"):
filename, file_extension = os.path.splitext(file)
print(filename)
print(file_extension)
continue
else:
continue
So, I checked the documentation and it says:
Changed in version 3.6: Accepts a path-like object.
path-like object:
An object representing a file system path. A path-like object is
either a str or...
I did a little more digging and the following also works:
with open("../FutureBookList/file.txt") as file:
data = file.read()
There are two directories on my desktop, DIR1 and DIR2 which contain the following files:
DIR1:
file1.py
DIR2:
file2.py myfile.txt
The files contain the following:
file1.py
import sys
sys.path.append('.')
sys.path.append('../DIR2')
import file2
file2.py
import sys
sys.path.append( '.' )
sys.path.append( '../DIR2' )
MY_FILE = "myfile.txt"
myfile = open(MY_FILE)
myfile.txt
some text
Now, there are two scenarios. The first works, the second gives an error.
Scenario 1
I cd into DIR2 and run file2.py and it runs no problem.
Scenario 2
I cd into DIR1 and run file1.py and it throws an error:
Traceback (most recent call last):
File "<absolute-path>/DIR1/file1.py", line 6, in <module>
import file2
File "../DIR2/file2.py", line 9, in <module>
myfile = open(MY_FILE)
IOError: [Errno 2] No such file or directory: 'myfile.txt'
However, this makes no sense to me, since I have appended the path to file1.py using the command sys.path.append('../DIR2').
Why does this happen when file1.py, when file2.py is in the same directory as myfile.txt yet it throws an error? Thank you.
You can create a path relative to a module by using a module's __file__ attribute. For example:
myfile = open(os.path.join(
os.path.dirname(__file__),
MY_FILE))
This should do what you want regardless of where you start your script.
Replace
MY_FILE = "myfile.txt"
myfile = open(MY_FILE)
with
MY_FILE = os.path.join("DIR2", "myfile.txt")
myfile = open(MY_FILE)
That's what the comments your question has are referring to as the relative path solution. This assumes that you're running it from the dir one up from myfile.txt... so not ideal.
If you know that my_file.txt is always going to be in the same dir as file2.py then you can try something like this in file2..
from os import path
fname = path.abspath(path.join(path.dirname(__file__), "my_file.txt"))
myfile = open(fname)
My Python script works perfectly if I execute it directly from the directory it's located in. However if I back out of that directory and try to execute it from somewhere else (without changing any code or file locations), all the relative paths break and I get a FileNotFoundError.
The script is located at ./scripts/bin/my_script.py. There is a directory called ./scripts/bin/data/. Like I said, it works absolutely perfectly as long as I execute it from the same directory... so I'm very confused.
Successful Execution (in ./scripts/bin/): python my_script.py
Failed Execution (in ./scripts/): Both python bin/my_script.py and python ./bin/my_script.py
Failure Message:
Traceback (most recent call last):
File "./bin/my_script.py", line 87, in <module>
run()
File "./bin/my_script.py", line 61, in run
load_data()
File "C:\Users\XXXX\Desktop\scripts\bin\tables.py", line 12, in load_data
DATA = read_file("data/my_data.txt")
File "C:\Users\XXXX\Desktop\scripts\bin\fileutil.py", line 5, in read_file
with open(filename, "r") as file:
FileNotFoundError: [Errno 2] No such file or directory: 'data/my_data.txt'
Relevant Python Code:
def read_file(filename):
with open(filename, "r") as file:
lines = [line.strip() for line in file]
return [line for line in lines if len(line) == 0 or line[0] != "#"]
def load_data():
global DATA
DATA = read_file("data/my_data.txt")
Yes, that is logical. The files are relative to your working directory. You change that by running the script from a different directory.
What you could do is take the directory of the script you are running at run time and build from that.
import os
def read_file(filename):
#get the directory of the current running script. "__file__" is its full path
path, fl = os.path.split(os.path.realpath(__file__))
#use path to create the fully classified path to your data
full_path = os.path.join(path, filename)
with open(full_path, "r") as file:
#etc
Your resource files are relative to your script. This is OK, but you need to use
os.path.realpath(__file__)
or
os.path.dirname(sys.argv[0])
to obtain the directory where the script is located. Then use os.path.join() or other function to generate the paths to the resource files.