We Keep Coding

Regarding file io in Python

I mistakenly, typed the following code:
f = open('\TestFiles\'sample.txt', 'w')
f.write('I just wrote this line')
f.close()
I ran this code and even though I have mistakenly typed the above code, it is however a valid code because the second backslash ignores the single-quote and what I should get, according to my knowledge is a .txt file named "\TestFiles'sample" in my project folder. However when I navigated to the project folder, I could not find a file there.
However, if I do the same thing with a different filename for example. Like,
f = open('sample1.txt', 'w')
f.write('test')
f.close()
I find the 'sample.txt' file created in my folder. Is there a reason for the file to not being created even though the first code was valid according to my knowledge?
Also is there a way to mention a file relative to my project folder rather than mentioning the absolute path to a file? (For example I want to create a file called 'sample.txt' in a folder called 'TestFiles' inside my project folder. So without mentioning the absolute path to TestFiles folder, is there a way to mention the path to TestFiles folder relative to the project folder in Python when opening files?)
I am a beginner in Python and I hope someone could help me.
Thank you.

What you're looking for are relative paths, long story short, if you want to create a file called 'sample.txt' in a folder 'TestFiles' inside your project folder, you can do:
import os
f = open(os.path.join('TestFiles', 'sample1.txt'), 'w')
f.write('test')
f.close()
Or using the more recent pathlib module:
from pathlib import Path
f = open(Path('TestFiles', 'sample1.txt'), 'w')
f.write('test')
f.close()
But you need to keep in mind that it depends on where you started your Python interpreter (which is probably why you're not able to find "\TestFiles'sample" in your project folder, it's created elsewhere), to make sure everything works fine, you can do something like this instead:
from pathlib import Path
sample_path = Path(Path(__file__).parent, 'TestFiles', 'sample1.txt')
with open(sample_path, "w") as f:
f.write('test')
By using a [context manager]{https://book.pythontips.com/en/latest/context_managers.html} you can avoid using f.close()

When you create a file you can specify either an absolute filename or a relative filename.
If you start the file path with '\' (on Win) or '/' it will be an absolute path. So in your first case you specified an absolute path, which is in fact:
from pathlib import Path
Path('\Testfile\'sample.txt').absolute()
WindowsPath("C:/Testfile'sample.txt")
Whenever you run some code in python, the relative paths that will be generate will be composed by your current folder, which is the folder from which you started the python interpreter, which you can check with:
import os
os.getcwd()
and the relative path that you added afterwards, so if you specify:
Path('Testfiles\sample.txt').absolute()
WindowsPath('C:/Users/user/Testfiles/sample.txt')
In general I suggest you use pathlib to handle paths. That makes it safer and cross platform. For example let's say that your scrip is under:
project
src
script.py
testfiles
and you want to store/read a file in project/testfiles. What you can do is get the path for script.py with __file__ and build the path to project/testfiles
from pathlib import Path
src_path = Path(__file__)
testfiles_path = src_path.parent / 'testfiles'
sample_fname = testfiles_path / 'sample.txt'
with sample_fname.open('w') as f:
f.write('yo')

As I am running the first code example in vscode, I'm getting a warning
Anomalous backslash in string: '\T'. String constant might be missing an r prefix.
And when I am running the file, it is also creating a file with the name \TestFiles'sample.txt. And it is being created in the same directory where the .py file is.
now, if your working tree is like this:
project_folder
-testfiles
-sample.txt
-something.py
then you can just say: open("testfiles//hello.txt")
I hope you find it helpful.

Extracting file directory from a .txt file?

I have a text file known as testConfigFile which is as follow :
inputCsvFile = BIN+"/testing.csv"
description = "testing"
In which BIN is my parent directory of the folder (already declared using os.getcwd in my python script).
The problem I'm facing now is, how to read and extract the BIN+"testing.csv" from the testConfigFile.txt.
Since the name testing.csv might be changed to other names, so it will be a variable. I'm planning to do something like, first the script reads the keyword "inputCsvFile = " then it will automatically extract the words behind it, which is "BIN+"testing.csv".
f = open("testConfigFile","r")
line = f.readlines(f)
if line.startswith("inputCsvFile = ")
inputfile = ...
This is my failed partial code, where I've no idea on how to fix it. Is there anyone willing to help me?

Reading a config off a unstructured txt file is not the best idea. Python actually is able to parse config files that are structured in a certain way. I have restructured your txt file so that it is easier to work with. The config file extension does not really matter, I have changed it to .ini in this case.
app.ini:
[csvfilepath]
inputCsvFile = BIN+"/testing.csv"
description = "testing"
Code:
from configparser import ConfigParser # Available by default, no install needed.
config = ConfigParser() # Create a ConfigParser instance.
config.read('app.ini') # You can input the full path to the config file.
file_path = config.get('csvfilepath', 'inputCsvFile')
file_description = config.get('csvfilepath', 'description')
print(f"CSV File Path: {file_path}\nCSV File Description: {file_description}")
Output:
CSV File Path: BIN+"/testing.csv"
CSV File Description: "testing"
To read more about configparser, you may refer here.
For a simple tutorial on configparser, you may refer here.

FILE (.json) NOT FOUND

This is my directory where i have activated a virtual environment:
I'm working on a flask project to create a rest API, and I have a JSON credential file (google vision file), but when I run the code it says file not found even if it's in the same directory. I've activated a virtualenv for this particular project. mainone.py is the code I'm trying to run.
This is the error I am getting:
"File {} was not found.".format(filename)
google.auth.exceptions.DefaultCredentialsError: File date_scanner.json was not found.
And this is the code block where I am using accessing the particular file:
from flask import Flask,request,jsonify
import os,io,re,glob,base64
from google.cloud import vision
from google.cloud.vision import types
from PIL import Image
os.environ['GOOGLE_APPLICATION_CREDENTIALS']=r'date_scanner.json'
client=vision.ImageAnnotatorClient()

This is likely because the "working folder" of the Python process is not the same as where the file is located. There is a small (albeit convoluted) way to generate filenames which will always work:
Use __file__ to get the filename of the Python file where this is called.
Strip the filename from that path using os.path.dirname
Append the filename that you want to open using os.path.join
from os.path import dirname, join
def get_filename(filename):
here = dirname(__file__)
output = join(here, filename)
return output
Some food for thought
However, in this case there is something you should be careful about: The file contains security credentials and should not live in your source code. So while the above example will solve the immediate problem, the security of this should be addressed.
The best way for this is to externalise the filename of the credentials file into a config file. Say, for example you would use a .ini file as config file, you could have something like this in your code somewhere:
config = configparser.ConfigParser()
config.read('config.ini')
vision_creds_file = config.get('google_vision', 'filename')
and then place the following into config.ini:
[google_vision]
filename = /path/to/filename.json
This still requires you to place the config.ini file somewhere which should be documented in your application, as you still cannot add this to the source code (maybe a sample file with defaults).

.pyc file closes immediately

I am using this code to convert .py file into .pyc
import py_compile
import os
import glob
path = ''
for infile in glob.glob( os.path.join(path, '*.py') ):
print(f"current file is: {infile}")
strDestination = 'compiled/' + infile + 'c'
py_compile.compile(infile,strDestination)
I converted 6 similar scripts, that uses:
pandas, calendar, datetime, gc, numpy ,multiprocessing modules. However, when I try to execute .pyc files, 3 one them work fine and 3 of them closes immediately.
I try to add an input() at the start and the end of the code, but it also closes immediately.
If I execute the same script .py, it works normally.
When the conversion occurs, is it necessary to avoid some specific library ?
#### UPDATE
I noticed if I change the name of the .pyc file to the same name of the one that worked, it works. However, it needs to be on the directory folder. Why ?

The answer: .pyc File name cannot have special characters.
I hope this help someone.

File not found from Python although file exists

I'm trying to load a simple text file with an array of numbers into Python. A MWE is
import numpy as np
BASE_FOLDER = 'C:\\path\\'
BASE_NAME = 'DATA.txt'
fname = BASE_FOLDER + BASE_NAME
data = np.loadtxt(fname)
However, this gives an error while running:
OSError: C:\path\DATA.txt not found.
I'm using VSCode, so in the debug window the link to the path is clickable. And, of course, if I click it the file opens normally, so this tells me that the path is correct.
Also, if I do print(fname), VSCode also gives me a valid path.
Is there anything I'm missing?
EDIT
As per your (very helpful for future reference) comments, I've changed my code using the os module and raw strings:
BASE_FOLDER = r'C:\path_to_folder'
BASE_NAME = r'filename_DATA.txt'
fname = os.path.join(BASE_FOLDER, BASE_NAME)
Still results in error.
Second EDIT
I've tried again with another file. Very basic path and filename
BASE_FOLDER = r'Z:\Data\Enzo\Waste_Code'
BASE_NAME = r'run3b.txt'
And again, I get the same error.
If I try an alternative approach,
os.chdir(BASE_FOLDER)
a = os.listdir()
then select the right file,
fname = a[1]
I still get the error when trying to import it. Even though I'm retrieving it directly from listdir.
>> os.path.isfile(a[1])
False

Using the module os you can check the existence of the file within python by running
import os
os.path.isfile(fname)
If it returns False, that means that your file doesn't exist in the specified fname. If it returns True, it should be read by np.loadtxt().
Extra: good practice working with files and paths
When working with files it is advisable to use the amazing functionality built in the Base Library, specifically the module os. Where os.path.join() will take care of the joins no matter the operating system you are using.
fname = os.path.join(BASE_FOLDER, BASE_NAME)
In addition it is advisable to use raw strings by adding an r to the beginning of the string. This will be less tedious when writing paths, as it allows you to copy-paste from the navigation bar. It will be something like BASE_FOLDER = r'C:\path'. Note that you don't need to add the latest '\' as os.path.join takes care of it.

You may not have the full permission to read the downloaded file. Use
sudo chmod -R a+rwx file_name.txt
in the command prompt to give yourself permission to read if you are using Ubuntu.

For me the problem was that I was using the Linux home symbol in the link (~/path/file). Replacing it with the absolute path /home/user/etc_path/file worked like charm.