We Keep Coding

Creating new CSV files from inputs in different directory

I'm simply trying to alter a CSV file with Python.
When all the files were in the same dir, everything was fine.
Now that the input files are in a different dir than where the output files will be, everything blows up, apparently b/c the files do not exist?
I first found this:
open() in Python does not create a file if it doesn't exist
Then I learned to change to the directory, which helped me loop over the CSVs in the target dir:
Moving up one directory in Python
When I run the command:
python KWRG.py ../Weekly\ Reports\ -\ Inbound/Search\ Activity/ 8/9/2021
I will get:
Traceback (most recent call last): File "KWRG.py", line 15, in <module> with open(args.input, 'r') as in_file, open(args.output, 'w') as out_file: IsADirectoryError: [Errno 21] Is a directory: '../Weekly Reports - Inbound/Search Activity/'
Sorry If I'm missing the obvious here, but why is the file not being created in the directory that I'm pointing the script to (or at all for that matter)?
The code:
import csv
import argparse
import os
# Create a parser to take arguments
#...snip...
cur_dir = os.getcwd()
reports_dir = os.chdir(os.path.join(cur_dir, args.dir))
for csv_file in os.listdir(reports_dir):
# Shorthand the name of the file
#...snip...
# Open the in and out files
with open(csv_file, 'r') as in_file, open(f'{out_name}-Search-Activity-{args.date}.csv', 'w+') as out_file:
# Re-arrange CSV
# EOF

Your problem is with this line:
reports_dir = os.chdir(os.path.join(cur_dir, args.dir))
os.chdir() doesn't return anything, it just performs the operation requested - changing the current working directory. From an interactive session with the REPL:
>>> import os
>>> result = os.chdir("/Users/mattdmo/")
>>> result
>>>
For your purposes, all you need is
reports_dir = os.path.join(cur_dir, args.dir)
and you'll be all set.

Reading a file using a relative path in a Python project

Say I have a Python project that is structured as follows:
project
/data
test.csv
/package
__init__.py
module.py
main.py
__init__.py:
from .module import test
module.py:
import csv
with open("..data/test.csv") as f:
test = [line for line in csv.reader(f)]
main.py:
import package
print(package.test)
When I run main.py I get the following error:
C:\Users\Patrick\Desktop\project>python main.py
Traceback (most recent call last):
File "main.py", line 1, in <module>
import package
File "C:\Users\Patrick\Desktop\project\package\__init__.py", line 1, in <module>
from .module import test
File "C:\Users\Patrick\Desktop\project\package\module.py", line 3, in <module>
with open("../data/test.csv") as f:
FileNotFoundError: [Errno 2] No such file or directory: '../data/test.csv'
However, if I run module.py from the package directory, I don’t get any errors. So it seems that the relative path used in open(...) is only relative to where the originating file is being run from (i.e __name__ == "__main__")? I don't want to use absolute paths. What are some ways to deal with this?

Relative paths are relative to current working directory.
If you do not want your path to be relative, it must be absolute.
But there is an often used trick to build an absolute path from current script: use its __file__ special attribute:
from pathlib import Path
path = Path(__file__).parent / "../data/test.csv"
with path.open() as f:
test = list(csv.reader(f))
This requires python 3.4+ (for the pathlib module).
If you still need to support older versions, you can get the same result with:
import csv
import os.path
my_path = os.path.abspath(os.path.dirname(__file__))
path = os.path.join(my_path, "../data/test.csv")
with open(path) as f:
test = list(csv.reader(f))
[2020 edit: python3.4+ should now be the norm, so I moved the pathlib version inspired by jpyams' comment first]

For Python 3.4+:
import csv
from pathlib import Path
base_path = Path(__file__).parent
file_path = (base_path / "../data/test.csv").resolve()
with open(file_path) as f:
test = [line for line in csv.reader(f)]

This worked for me.
with open('data/test.csv') as f:

My Python version is Python 3.5.2 and the solution proposed in the accepted answer didn't work for me. I've still were given an error
FileNotFoundError: [Errno 2] No such file or directory
when I was running my_script.py from the terminal. Although it worked fine when I run it through Run/Debug Configurations from the PyCharm IDE (PyCharm 2018.3.2 (Community Edition)).
Solution:
instead of using:
my_path = os.path.abspath(os.path.dirname(__file__)) + some_rel_dir_path
as suggested in the accepted answer, I used:
my_path = os.path.abspath(os.path.dirname(os.path.abspath(__file__))) + some_rel_dir_path
Explanation:
Changing os.path.dirname(__file__) to os.path.dirname(os.path.abspath(__file__))
solves the following problem:
When we run our script like that: python3 my_script.py
the __file__ variable has a just a string value of "my_script.py" without path leading to that particular script. That is why method dirname(__file__) returns an empty string "". That is also the reason why my_path = os.path.abspath(os.path.dirname(__file__)) + some_rel_dir_path is actually the same thing as my_path = some_rel_dir_path. Consequently FileNotFoundError: [Errno 2] No such file or directory is given when trying to use open method because there is no directory like "some_rel_dir_path".
Running script from PyCharm IDE Running/Debug Configurations worked because it runs a command python3 /full/path/to/my_script.py (where "/full/path/to" is specified by us in "Working directory" variable in Run/Debug Configurations) instead of justpython3 my_script.py like it is done when we run it from the terminal.

Try
with open(f"{os.path.dirname(sys.argv[0])}/data/test.csv", newline='') as f:

I was surprised when the following code worked.
import os
for file in os.listdir("../FutureBookList"):
if file.endswith(".adoc"):
filename, file_extension = os.path.splitext(file)
print(filename)
print(file_extension)
continue
else:
continue
So, I checked the documentation and it says:
Changed in version 3.6: Accepts a path-like object.
path-like object:
An object representing a file system path. A path-like object is
either a str or...
I did a little more digging and the following also works:
with open("../FutureBookList/file.txt") as file:
data = file.read()

OS listdir not reading certain files

I'm trying to access a folder and read the text files in it. Eventually I'm going to build them into a basic dictionary to dump into JSON but something's going on with the files. I'm getting an error message:
FileNotFoundError: [Errno 2] No such file or directory: ___
The thing is, this code IS working on my test files ... so maybe it's somehow a file issue?
import json
import os
for file in os.listdir("filepath"):
with open ('%s.json' % file, 'w') as fp:
file = f.read()
f = open(file, 'r')
print (f)
These are the text files not getting 'found/read':
https://drive.google.com/open?id=0B4zJC6biI6jERDFHSzZvUUJaRE0
Has anyone else run into this problem / found a solution that might work?

Python - Open a file with a wildcard (%) directory path in Windows

In Python, I'm trying to open a file that gets saved to the %TEMP% directory. I've tried:
file = open("%TEMP%\file.txt")
and
file = open("%%TEMP%%\file.txt")
and
file = open("%TEMP%\\file.txt")
and
file = open("%%TEMP%%\\file.txt")
And always get (this one specifically for that last example):
IOError: [Errno 2] No such file or directory: '%%TEMP%%\\file.txt'
For sanity's sake, from Windows command prompt I do a type %TEMP%\file.txt and it prints out the file OK. Any help?

Use os.environ
import os
f = open(os.path.join(os.environ['TEMP'], 'file.txt'))
You can also use os.path.expandvars
import os
f = open(os.path.expandvars(r'%TEMP%\file.txt'))

Python IDLE doesn't find text files unless I specify the path

In my Python books and also the Python Documents this code should be enough to open a file:
f = open("file.txt", "r")
But if I do this I will get an error message telling me file.txt doesn't exist. If I however use the whole path where file.txt is located it opens it:
f = open("C:/Users/Me/Python/file.txt", "r")
Is there an explanation for this?

In short - the immediate search path (the current working directory) is where Python will look... (so on Windows - possibly it'll assume C:\Pythonxy)
Yes, it depends on where Python/IDLE is executed... for it to use its search path:
>>> import os
>>> os.getcwd()
'/home/jon'
>>> open('testing.txt')
<open file 'testing.txt', mode 'r' at 0x7f86e140edb0>
And in a shell - changing directories... then launching Python/IDLE
jon#forseti:~$ cd /srv
jon#forseti:/srv$ idle
>>> import os
>>> os.getcwd()
'/srv'
>>> open('testing.txt')
Traceback (most recent call last):
File "<pyshell#2>", line 1, in <module>
open('testing.txt')
IOError: [Errno 2] No such file or directory: 'testing.txt'