I'm trying to understand how logfiles work with the Python 2 logging module.
I know that I can save the log output to a text file using something like:
logging.basicConfig(filename='example.log',level=logging.DEBUG)
It's not clear to me from the documentation on:
Whether or not absolute filename paths are valid
The proper syntax for specifying relative paths (assuming that ../example.log is valid).
If I execute this script from /home/bob, how do I specify that I want the logfile saved to the /tmp directory instead - using both absolute and relative paths?
Is logging.basicConfig(filename='../../tmp/example.log') valid?
Similarly, is logging.basicConfig(filename='/tmp/example.log') valid?
When stating just the filename, it would be written to the current directory.
Using Python IDLE you can check that as follows
>>> import logging
>>> logging.basicConfig(filename='relative.log')
>>> logging.info('test')
C:\Python27\relative.log
My working dir is Python27, and I have a file there named relative.log with my logged message.
When you change the file location to ../relative.log, I get a new file at the parent directory of Python27. So relative path does work for logging:
>>> import logging
>>> logging.basicConfig(filename='../relative.log')
>>> logging.info('test123')
C:\relative.log
And logging module also supports absolute path:
>>> logging.basicConfig(filename=r'c:\abs_path.log')
>>> logging.info('test')
C:\abs_path.log
It is always better to use absolute path, as explicit is better than implicit.
They are both valid. But relative paths (with ..) will select different files depending on the directory you are in when you run it.
So when you run logger.py in /home/user/projects/python and the filename is ../log.txt the file will be saved in /home/user/projects. On the other hand when you run the script in /home/user then log.txt will be saved in /home/.
Absolute paths always work and are more reliable. If you want a file in the current directory I recommend this approach:
basedir = os.path.abspath(os.path.dirname(__file__))
filename = os.path.join(basedir, 'file.txt')
Related
I mistakenly, typed the following code:
f = open('\TestFiles\'sample.txt', 'w')
f.write('I just wrote this line')
f.close()
I ran this code and even though I have mistakenly typed the above code, it is however a valid code because the second backslash ignores the single-quote and what I should get, according to my knowledge is a .txt file named "\TestFiles'sample" in my project folder. However when I navigated to the project folder, I could not find a file there.
However, if I do the same thing with a different filename for example. Like,
f = open('sample1.txt', 'w')
f.write('test')
f.close()
I find the 'sample.txt' file created in my folder. Is there a reason for the file to not being created even though the first code was valid according to my knowledge?
Also is there a way to mention a file relative to my project folder rather than mentioning the absolute path to a file? (For example I want to create a file called 'sample.txt' in a folder called 'TestFiles' inside my project folder. So without mentioning the absolute path to TestFiles folder, is there a way to mention the path to TestFiles folder relative to the project folder in Python when opening files?)
I am a beginner in Python and I hope someone could help me.
Thank you.
What you're looking for are relative paths, long story short, if you want to create a file called 'sample.txt' in a folder 'TestFiles' inside your project folder, you can do:
import os
f = open(os.path.join('TestFiles', 'sample1.txt'), 'w')
f.write('test')
f.close()
Or using the more recent pathlib module:
from pathlib import Path
f = open(Path('TestFiles', 'sample1.txt'), 'w')
f.write('test')
f.close()
But you need to keep in mind that it depends on where you started your Python interpreter (which is probably why you're not able to find "\TestFiles'sample" in your project folder, it's created elsewhere), to make sure everything works fine, you can do something like this instead:
from pathlib import Path
sample_path = Path(Path(__file__).parent, 'TestFiles', 'sample1.txt')
with open(sample_path, "w") as f:
f.write('test')
By using a [context manager]{https://book.pythontips.com/en/latest/context_managers.html} you can avoid using f.close()
When you create a file you can specify either an absolute filename or a relative filename.
If you start the file path with '\' (on Win) or '/' it will be an absolute path. So in your first case you specified an absolute path, which is in fact:
from pathlib import Path
Path('\Testfile\'sample.txt').absolute()
WindowsPath("C:/Testfile'sample.txt")
Whenever you run some code in python, the relative paths that will be generate will be composed by your current folder, which is the folder from which you started the python interpreter, which you can check with:
import os
os.getcwd()
and the relative path that you added afterwards, so if you specify:
Path('Testfiles\sample.txt').absolute()
WindowsPath('C:/Users/user/Testfiles/sample.txt')
In general I suggest you use pathlib to handle paths. That makes it safer and cross platform. For example let's say that your scrip is under:
project
src
script.py
testfiles
and you want to store/read a file in project/testfiles. What you can do is get the path for script.py with __file__ and build the path to project/testfiles
from pathlib import Path
src_path = Path(__file__)
testfiles_path = src_path.parent / 'testfiles'
sample_fname = testfiles_path / 'sample.txt'
with sample_fname.open('w') as f:
f.write('yo')
As I am running the first code example in vscode, I'm getting a warning
Anomalous backslash in string: '\T'. String constant might be missing an r prefix.
And when I am running the file, it is also creating a file with the name \TestFiles'sample.txt. And it is being created in the same directory where the .py file is.
now, if your working tree is like this:
project_folder
-testfiles
-sample.txt
-something.py
then you can just say: open("testfiles//hello.txt")
I hope you find it helpful.
I am so new with python and pycharm and i got confuse!!
When I run my project in pycharm it gives me an error about not finding the path of my file. The physical file path is:
'../Project/BC/RequiredFiles/resources/a_reqs.csv'
My project working directory is "Project/BC" and the project running file (startApp.sh) is there too. but the .py file that wants to work with a_req.csv is inside the "RequiredFiles" folder. There is the following code in the .py file:
reqsfile = os.getcwd() + "/resources/a_reqs.csv"
it returns: '../Project/BC/resources/a_reqs.csv'
instead of: '../Project/BC/resources/RequiredFiles/a_reqs.csv'
while the .py file is in "RequiredFiles" the os.getcwd() must include it too. but it does not.
The problem is that i can not change the addressing code. because this code works in another IDE and other people who work with the code in other platform or OS do not have any problem. I am working in mac OS and if i am not mistaken the code works with windows!!
So, how can i tell Pycharm (in mac) to see and load "RequiredFiles" folder as the subfolder of my working directory!!!
os.getcwd returns the current working directory of the process (which may be the directory where startApp.sh is located or another one, depending on the PyCharm's run configuration setting, or, if you start the program from the command line, the directory in which you execute the command).
To make a path independent on the current working directory, you can take the directory where your Python file is located and build the path from it:
os.path.dirname(__file__) + "/resources/a_reqs.csv"
From your question what I see is:
reqsfile = os.getcwd() + "/resources/a_reqs.csv"
Which produces: "../Project/BC/resources/a_reqs.csv", whereas your desired output is
"../Project/BC/resources/RequiredFiles/a_reqs.csv". Since we know os.getcwd is returning "/Project/BC/", then to get your desired result you should be doing:
reqsfile = os.getcwd() + "/resources/RequiredFiles/a_reqs.csv"
But since you want the solution to work with or without the RequiredFiles subdirectory you could apply a conditional solution, ie something like:
import os.path
if os.path.exists(os.getcwd() + "/resources/RequiredFiles/a_reqs.csv"):
reqsfile = os.getcwd() + "/resources/RequiredFiles/a_reqs.csv"
else:
reqsfile = os.getcwd() + "/resources/a_reqs.csv"
This solution will set the reqsfile to the csv in the RequiredFiles directory if the directory exists, and thus will work for you. On the other-hand, if the RequiredFiles directory doesn't exist, it will default to the csv in /resources/.
Typically when groups collaborate on projects, the maintain the same file hierarchy so that these types of issues are avoided, so you might want to consider moving the csv from /RequiredFiles/ to /resources/.
How can I set the current path of my python file "myproject.py" to the file itself?
I do not want something like this:
path = "the path of myproject.py"
In mathematica I can set:
SetDirectory[NotebookDirectory[]]
The advantage with the code in Mathematica is that if I change the path of my Mathematica file, for example if I give it to someone else or I put it in another folder, I do not need to do anything extra. Each time Mathematica automatically set the directory to the current folder.
I want something similar to this in Python.
The right solution is not to change the current working directory, but to get the full path to the directory containing your script or module then use os.path.join to build your files path:
import os
ROOT_PATH = os.path.dirname(os.path.abspath(__file__))
# then:
myfile_path = os.path.join(ROOT_PATH, "myfile.txt")
This is safer than messing with current working directory (hint : what would happen if another module changes the current working directory after you did but before you access your files ?)
I want to set the directory in which the python file is, as working directory
There are two step:
Find out path to the python file
Set its parent directory as the working directory
The 2nd is simple:
import os
os.chdir(module_dir) # set working directory
The 1st might be complex if you want to support a general case (python file that is run as a script directly, python file that is imported in another module, python file that is symlinked, etc). Here's one possible solution:
import inspect
import os
module_path = inspect.getfile(inspect.currentframe())
module_dir = os.path.realpath(os.path.dirname(module_path))
Use the os.getcwd() function from the built in os module also there's os.getcwdu() which returns a unicode object of the current working directory
Example usage:
import os
path = os.getcwd()
print path
#C:\Users\KDawG\Desktop\Python
Let's consider the below code:
fp=open('PR1.txt','r')
ch=fp.readlines()
print "%s" % (' '.join(ch))
print "\n"
fp.close()
Above code gives an error:
IOError: [Errno 2] No such file or directory: 'PR1.txt'
But when i am provididng its full location i.e;
fp=open('D:/PR1.txt','r')
then it is working properly...
IS it necessary to provide full location of file or there is some other way too?
No, it is not necessary, but you need to be certain you are running your script with the right working directory. Your script working directory is evidently not D:/.
In practice, it is better to only use relative paths if you are in full control of the working directory. You can get the current working directory with os.getcwd() and set it with os.chdir() but using absolute paths is usually better.
For paths relative to the current module or script, use the __file__ global to produce a directory name:
import os.path
here = os.path.dirname(os.path.absolute(__file__))
then use os.path.join() to make relative paths absolute in reference to here.
This question already has answers here:
How do you properly determine the current script directory?
(16 answers)
How to know/change current directory in Python shell?
(7 answers)
Closed 5 years ago.
How do I determine:
the current directory (where I was in the shell when I ran the Python script), and
where the Python file I am executing is?
To get the full path to the directory a Python file is contained in, write this in that file:
import os
dir_path = os.path.dirname(os.path.realpath(__file__))
(Note that the incantation above won't work if you've already used os.chdir() to change your current working directory, since the value of the __file__ constant is relative to the current working directory and is not changed by an os.chdir() call.)
To get the current working directory use
import os
cwd = os.getcwd()
Documentation references for the modules, constants and functions used above:
The os and os.path modules.
The __file__ constant
os.path.realpath(path) (returns "the canonical path of the specified filename, eliminating any symbolic links encountered in the path")
os.path.dirname(path) (returns "the directory name of pathname path")
os.getcwd() (returns "a string representing the current working directory")
os.chdir(path) ("change the current working directory to path")
Current working directory: os.getcwd()
And the __file__ attribute can help you find out where the file you are executing is located. This Stack Overflow post explains everything: How do I get the path of the current executed file in Python?
You may find this useful as a reference:
import os
print("Path at terminal when executing this file")
print(os.getcwd() + "\n")
print("This file path, relative to os.getcwd()")
print(__file__ + "\n")
print("This file full path (following symlinks)")
full_path = os.path.realpath(__file__)
print(full_path + "\n")
print("This file directory and name")
path, filename = os.path.split(full_path)
print(path + ' --> ' + filename + "\n")
print("This file directory only")
print(os.path.dirname(full_path))
The pathlib module, introduced in Python 3.4 (PEP 428 — The pathlib module — object-oriented filesystem paths), makes the path-related experience much much better.
pwd
/home/skovorodkin/stack
tree
.
└── scripts
├── 1.py
└── 2.py
In order to get the current working directory, use Path.cwd():
from pathlib import Path
print(Path.cwd()) # /home/skovorodkin/stack
To get an absolute path to your script file, use the Path.resolve() method:
print(Path(__file__).resolve()) # /home/skovorodkin/stack/scripts/1.py
And to get the path of a directory where your script is located, access .parent (it is recommended to call .resolve() before .parent):
print(Path(__file__).resolve().parent) # /home/skovorodkin/stack/scripts
Remember that __file__ is not reliable in some situations: How do I get the path of the current executed file in Python?.
Please note, that Path.cwd(), Path.resolve() and other Path methods return path objects (PosixPath in my case), not strings. In Python 3.4 and 3.5 that caused some pain, because open built-in function could only work with string or bytes objects, and did not support Path objects, so you had to convert Path objects to strings or use the Path.open() method, but the latter option required you to change old code:
File scripts/2.py
from pathlib import Path
p = Path(__file__).resolve()
with p.open() as f: pass
with open(str(p)) as f: pass
with open(p) as f: pass
print('OK')
Output
python3.5 scripts/2.py
Traceback (most recent call last):
File "scripts/2.py", line 11, in <module>
with open(p) as f:
TypeError: invalid file: PosixPath('/home/skovorodkin/stack/scripts/2.py')
As you can see, open(p) does not work with Python 3.5.
PEP 519 — Adding a file system path protocol, implemented in Python 3.6, adds support of PathLike objects to the open function, so now you can pass Path objects to the open function directly:
python3.6 scripts/2.py
OK
To get the current directory full path
>>import os
>>print os.getcwd()
Output: "C :\Users\admin\myfolder"
To get the current directory folder name alone
>>import os
>>str1=os.getcwd()
>>str2=str1.split('\\')
>>n=len(str2)
>>print str2[n-1]
Output: "myfolder"
Pathlib can be used this way to get the directory containing the current script:
import pathlib
filepath = pathlib.Path(__file__).resolve().parent
If you are trying to find the current directory of the file you are currently in:
OS agnostic way:
dirname, filename = os.path.split(os.path.abspath(__file__))
If you're using Python 3.4, there is the brand new higher-level pathlib module which allows you to conveniently call pathlib.Path.cwd() to get a Path object representing your current working directory, along with many other new features.
More info on this new API can be found here.
To get the current directory full path:
os.path.realpath('.')
Answer to #1:
If you want the current directory, do this:
import os
os.getcwd()
If you want just any folder name and you have the path to that folder, do this:
def get_folder_name(folder):
'''
Returns the folder name, given a full folder path
'''
return folder.split(os.sep)[-1]
Answer to #2:
import os
print os.path.abspath(__file__)
I think the most succinct way to find just the name of your current execution context would be:
current_folder_path, current_folder_name = os.path.split(os.getcwd())
If you're searching for the location of the currently executed script, you can use sys.argv[0] to get the full path.
For question 1, use os.getcwd() # Get working directory and os.chdir(r'D:\Steam\steamapps\common') # Set working directory
I recommend using sys.argv[0] for question 2 because sys.argv is immutable and therefore always returns the current file (module object path) and not affected by os.chdir(). Also you can do like this:
import os
this_py_file = os.path.realpath(__file__)
# vvv Below comes your code vvv #
But that snippet and sys.argv[0] will not work or will work weird when compiled by PyInstaller, because magic properties are not set in __main__ level and sys.argv[0] is the way your executable was called (it means that it becomes affected by the working directory).