We Keep Coding

How can I get a PyInstaller executable to create a file in the same directory?

I found the answer
So it looks like PyInstaller actually runs in a temp directory, not your own, which explains my issue. This is an explanation for that. I guess I will keep this up incase people in the future have problems.
Original question
I am trying to use PyInstaller to create an executable of a simple python script called test.py that just creates a text file and adds numbers to it, as a test of PyInstaller. This file works correctly when run normally.
from os.path import dirname, abspath
def main():
txtfile = dirname(abspath(__file__)) + '/text.txt'
with open(txtfile, 'w') as f:
for num in range(101):
f.write(str(num) + '\n')
if __name__ == '__main__':
main()
print('script executed')
When I use:
pyinstaller test.py --onefile in the same directory as test.py it successfully creates the dist file with the binary file test inside of it.
when I cd dist and do ./test to execute the file inside dist it successfully prints out main called and script executed but it doesn't actually create the file. So, main is being called and the script is being executed, but the file isn't created at all, and I am quite confused about what I'm doing wrong..I must be getting file paths messed up? But I have specified the exact full path with os.path, so it doesn't make sense to me.
The system exit code is 0, and there are no errors raised when I call ./test

I found this that shows that PyInstaller will save to a temp file. I created this script below to check if the script is being executed directly or via PyInstaller.
import os
import sys
def create_file(path):
with open(path + '/test.txt', 'w') as f:
for num in range(101):
f.write(str(num) + '\n')
def check_using_pyinstaller():
if getattr(sys, 'frozen', False):
application_path = os.path.dirname(sys.executable)
return application_path
return os.path.dirname(os.path.abspath(__file__))
def main():
path = check_using_pyinstaller()
os.chdir(path)
create_file(path)
if __name__ == '__main__':
main()

Reading a file using a relative path in a Python project

Say I have a Python project that is structured as follows:
project
/data
test.csv
/package
__init__.py
module.py
main.py
__init__.py:
from .module import test
module.py:
import csv
with open("..data/test.csv") as f:
test = [line for line in csv.reader(f)]
main.py:
import package
print(package.test)
When I run main.py I get the following error:
C:\Users\Patrick\Desktop\project>python main.py
Traceback (most recent call last):
File "main.py", line 1, in <module>
import package
File "C:\Users\Patrick\Desktop\project\package\__init__.py", line 1, in <module>
from .module import test
File "C:\Users\Patrick\Desktop\project\package\module.py", line 3, in <module>
with open("../data/test.csv") as f:
FileNotFoundError: [Errno 2] No such file or directory: '../data/test.csv'
However, if I run module.py from the package directory, I don’t get any errors. So it seems that the relative path used in open(...) is only relative to where the originating file is being run from (i.e __name__ == "__main__")? I don't want to use absolute paths. What are some ways to deal with this?

Relative paths are relative to current working directory.
If you do not want your path to be relative, it must be absolute.
But there is an often used trick to build an absolute path from current script: use its __file__ special attribute:
from pathlib import Path
path = Path(__file__).parent / "../data/test.csv"
with path.open() as f:
test = list(csv.reader(f))
This requires python 3.4+ (for the pathlib module).
If you still need to support older versions, you can get the same result with:
import csv
import os.path
my_path = os.path.abspath(os.path.dirname(__file__))
path = os.path.join(my_path, "../data/test.csv")
with open(path) as f:
test = list(csv.reader(f))
[2020 edit: python3.4+ should now be the norm, so I moved the pathlib version inspired by jpyams' comment first]

For Python 3.4+:
import csv
from pathlib import Path
base_path = Path(__file__).parent
file_path = (base_path / "../data/test.csv").resolve()
with open(file_path) as f:
test = [line for line in csv.reader(f)]

This worked for me.
with open('data/test.csv') as f:

My Python version is Python 3.5.2 and the solution proposed in the accepted answer didn't work for me. I've still were given an error
FileNotFoundError: [Errno 2] No such file or directory
when I was running my_script.py from the terminal. Although it worked fine when I run it through Run/Debug Configurations from the PyCharm IDE (PyCharm 2018.3.2 (Community Edition)).
Solution:
instead of using:
my_path = os.path.abspath(os.path.dirname(__file__)) + some_rel_dir_path
as suggested in the accepted answer, I used:
my_path = os.path.abspath(os.path.dirname(os.path.abspath(__file__))) + some_rel_dir_path
Explanation:
Changing os.path.dirname(__file__) to os.path.dirname(os.path.abspath(__file__))
solves the following problem:
When we run our script like that: python3 my_script.py
the __file__ variable has a just a string value of "my_script.py" without path leading to that particular script. That is why method dirname(__file__) returns an empty string "". That is also the reason why my_path = os.path.abspath(os.path.dirname(__file__)) + some_rel_dir_path is actually the same thing as my_path = some_rel_dir_path. Consequently FileNotFoundError: [Errno 2] No such file or directory is given when trying to use open method because there is no directory like "some_rel_dir_path".
Running script from PyCharm IDE Running/Debug Configurations worked because it runs a command python3 /full/path/to/my_script.py (where "/full/path/to" is specified by us in "Working directory" variable in Run/Debug Configurations) instead of justpython3 my_script.py like it is done when we run it from the terminal.

Try
with open(f"{os.path.dirname(sys.argv[0])}/data/test.csv", newline='') as f:

I was surprised when the following code worked.
import os
for file in os.listdir("../FutureBookList"):
if file.endswith(".adoc"):
filename, file_extension = os.path.splitext(file)
print(filename)
print(file_extension)
continue
else:
continue
So, I checked the documentation and it says:
Changed in version 3.6: Accepts a path-like object.
path-like object:
An object representing a file system path. A path-like object is
either a str or...
I did a little more digging and the following also works:
with open("../FutureBookList/file.txt") as file:
data = file.read()

Imported code cannot open files in its directory

I have the following file structure:
test/
test1.py
test2.py
text.txt
Here are the contents of the files
test1.py:
import sys
sys.path.append('../')
import test2
test2.read()
test2.py:
def read():
with open('text.txt', 'rb') as f:
print f.read()
if __name__ == "__main__":
read()
text.txt contains a line of text. When I run test1.py, I get a "File not found" error:
Traceback (most recent call last):
File "test1.py", line 5, in <module>
test2.read()
File "../test2.py", line 2, in read
with open('text.txt', 'rb') as f:
IOError: [Errno 2] No such file or directory: 'text.txt'
I kind of understand why this error is coming up. But how do I deal with these kind of errors. I would like the code in test2.py to be like library code which I can use anywhere.

sys.path used for python path (PYTHONPATH eviroment variable).
i.e. where to look for python libraries when you import some library.
it dose not effect where open() is looking for files.
when you open(filename). the filename is relative to the procees working directory. (the path the code was run from)
so if you want to access a flie that its path is relative to the path of the code file, then you can use the builtin variable __file__ which hold the current file path.
so you can change test2.py to be:
import os
def read():
with open(os.path.join(os.path.dirname(__file__),'text.txt'), 'rb') as f:
print f.read()

The proper way of doing what you're asking for is to use pkg_resources as described here. Basically something like the following would be what you want in test2.py:
import pkg_resources
def read():
with pkg_resources.resource_stream(__name__, 'text.txt') as f:
print f.read()
if __name__ == "__main__":
read()

Don't use relative path, then. Use the full path:
with open(r'C:\Somewhere\someplace\test.txt') as f:

Create a file in a directory using python

I am writing a python script in which I read a text file and I want to create a new text file in another directory.
I wrote the following code:
def treatFiles(oldFile, newFile):
with open(oldFile) as old, open(newFile, 'w') as new:
count = 0
for line in old:
count += 1
if count%2 == 0:
pass
else:
new.write(line)
if __name__ == '__main__':
from sys import argv
import os
os.makedirs('NewFiles')
new = '/NewFiles/' + argv[1]
treatFiles(argv[1], new)
I tried running this code with a text file in the same directory than my python script, but got an error like
FileNotFoundError: [Errno 2] No such file or directory: '/NewFiles/testFile'
Apparently, it is not clear that NewFiles is a directory in which it should create the new file... How can I correct this?

The problem is actually that in unix, /NewFiles/ means a folder in the root directory called NewFiles, not in the current directory. Remove the leading / and it should be fine.

Relative paths break when executing Python script from Windows batch?

My Python script works perfectly if I execute it directly from the directory it's located in. However if I back out of that directory and try to execute it from somewhere else (without changing any code or file locations), all the relative paths break and I get a FileNotFoundError.
The script is located at ./scripts/bin/my_script.py. There is a directory called ./scripts/bin/data/. Like I said, it works absolutely perfectly as long as I execute it from the same directory... so I'm very confused.
Successful Execution (in ./scripts/bin/): python my_script.py
Failed Execution (in ./scripts/): Both python bin/my_script.py and python ./bin/my_script.py
Failure Message:
Traceback (most recent call last):
File "./bin/my_script.py", line 87, in <module>
run()
File "./bin/my_script.py", line 61, in run
load_data()
File "C:\Users\XXXX\Desktop\scripts\bin\tables.py", line 12, in load_data
DATA = read_file("data/my_data.txt")
File "C:\Users\XXXX\Desktop\scripts\bin\fileutil.py", line 5, in read_file
with open(filename, "r") as file:
FileNotFoundError: [Errno 2] No such file or directory: 'data/my_data.txt'
Relevant Python Code:
def read_file(filename):
with open(filename, "r") as file:
lines = [line.strip() for line in file]
return [line for line in lines if len(line) == 0 or line[0] != "#"]
def load_data():
global DATA
DATA = read_file("data/my_data.txt")

Yes, that is logical. The files are relative to your working directory. You change that by running the script from a different directory.
What you could do is take the directory of the script you are running at run time and build from that.
import os
def read_file(filename):
#get the directory of the current running script. "__file__" is its full path
path, fl = os.path.split(os.path.realpath(__file__))
#use path to create the fully classified path to your data
full_path = os.path.join(path, filename)
with open(full_path, "r") as file:
#etc

Your resource files are relative to your script. This is OK, but you need to use
os.path.realpath(__file__)
or
os.path.dirname(sys.argv[0])
to obtain the directory where the script is located. Then use os.path.join() or other function to generate the paths to the resource files.