I am working on a program that will edit all local files ending in a csv extension. When I call the location of the directory and then change directory I get an error. The error is due to extra \'s being added to the path. How can I call the path without these extra \'s?
I've looked around and there are similar issues but every example I see is for a hard written location and not a movable one.
import os
import glob
import sys
path = os.path.abspath(__file__)
extension = '.csv'
os.chdir(os.path.abspath(__file__))
result = glob.glob('*'.format(extension))
print(path)
print(result)
os.chdir() needs a directory not a file which is what you are giving it. try changing os.chdir(os.path.abspath(__file__)) to os.chdir(os.path.dirname(path))
import os
import glob
import sys
__file__ = 'test.txt'
path = os.path.abspath(__file__)
print(path)
extension = '.csv'
os.chdir(os.path.dirname(path))
result = glob.glob('*'.format(extension))
print(path)
print(result)
How can i apply a script for file modification in all subfolder (in python)?
I created a little script for rename some pictures but my program only change pictures in the script folder and not in subfolder.
from PIL import Image
from os import *
import sys
from os.path import basename
import os
#from PIL.ExifTags import TAGS
from datetime import datetime
extension = ''
#path='/home/pi/Desktop/testrename'
folder_path = "/home/pi/Desktop/testrename/"
l=[]
import PIL.Image
from os import walk
#from path import path
import glob
EXIF_DATETIME = 36867
def renamefinaljpeg() :
glob.glob ('*/.jpeg')
if len(fname) < 20 :
try :
old = PIL.Image.open(fname)._getexif()[EXIF_DATETIME]
old2 = old.split(' ')
os.rename (fname, "yes" + old2[0]+' '+fname)
print('fait')
except :
pass
print('pas jpeg')
def renamefinaljpg() :
glob.glob ('*/.jpg')
if len(fname) < 20 :
try :
old = PIL.Image.open(fname)._getexif()[EXIF_DATETIME]
old2 = old.split(' ')
os.rename (fname, "yes" + old2[0]+' '+fname)
print('fait')
except :
pass
print('pas jpg')
rootDir = "/home/pi/Desktop/testrename/"
for dirName, subdirlist, fileList in os.walk(rootDir) :
for fname in fileList :
print(fname)
try :
renamefinaljpg() or renamefinaljpeg()
except :
pass
print('passe')
The image are renamed in the main directory but not in directory tree (but they are read)
Thank you for your help.
It looks like you've already got a lot of ideas for how to do it in your script. Let's use os.walk since that's probably the most straightforward. os.walk will iterate over all of our directories recursively and give us the list of filenames contained in each one. To filter the files to only .jpg we can use fnmatch.fnmatch.
import fnmatch
import os
import sys
from PIL import Image
folder_path = '/home/pi/Desktop/testrename/old'
# Give special numbers a specific name so it's easier to remember what it
# actually means.
EXIF_DATETIME = 36867
def renamefinal(dir_path, filename):
try:
old = Image.open(file_path)._getexif()[EXIF_DATETIME]
date, time = old.split(' ', maxsplit=1)
new_filename = date + ' ' + filename
os.rename(
# Source file name, including directory and filename
os.path.join(dir_path, filename),
# Destination file name, including date
os.path.join(dir_path, new_filename))
print(
'jpg renommé ({}): {} to {}'.format(
dir_path, filename, new_filename))
except:
# Including filename in our output so we know what to check if
# something goes wrong.
print('pas jpg ({}): {}'.format(dir_path, filename))
for path, dirs, files in os.walk(folder_path):
for filename in files:
if not fnmatch.fnmatch(filename, '*.jpg'):
# Go to the next file, skipping the rest of the loop for this file.
continue
if 30 <= len(filename):
continue
renamefinal(path, filename)
Some more that might help:
If you want to take arguments to your script check out the argparse module.
Try to catch exceptions more specifically, for example using except KeyError: instead of except: when you know that EXIF_DATETIME may not be in the exif data - also try putting your try: ... except: ... block only around the lines that might actually fail.
Check out the logging module instead of using print to show information about what your script is doing.
I've something like this in my program:
A main script main.py inside a folder named 'OpenFileinaModule'. There's a folder called 'sub' inside it with a script called subScript.py and a file xlFile.xlsx, which is opened by subScript.py.
OpenFileinaModule/
main.py
sub/
__init__.py (empty)
subScript.py
xlFile.xlsx
Here is the code:
sub.Script.py:
import os, openpyxl
class Oop:
def __init__(self):
__file__='xlFile.xlsx'
__location__ = os.path.realpath(
os.path.join(os.getcwd(), os.path.dirname(__file__)))
print os.path.join(__location__, __file__)
self.wrkb = openpyxl.load_workbook(os.path.join(__location__,
__file__),read_only=True)
main.py:
import sub.subScript
objt=sub.subScript.Oop()
When I execute main.py, I get the error:
IOError: [Errno 2] No such file or directory: 'C:\\Users\\...\\OpenFileInaModule\\xlFile.xlsx'
It jumps the sub folder...
I've tried
__file__='sub/xlFile.xlsx'
But then the "sub" folder is duplicated:
IOError: [Errno 2] No such file or directory: 'C:\\Users\\...\\OpenFileInaModule\\sub\\sub/xlFile.xlsx'
How to open xlFile.xlsx with subScript.py from main.py?
you're overriding __file__ with __file='xlFile.xlsx', do you mean to do this?
I think you want something like
import os
fname = 'xlFile.xlsx'
this_file = os.path.abspath(__file__)
this_dir = os.path.dirname(this_file)
wanted_file = os.path.join(this_dir, fname)
I'd suggest always using the absolute path for a file, especially if you're on windows when a relative path might not make sense if the file is on a different drive (I actually have no idea what it would do if you asked it for a relative path between devices).
Please avoid using __file__ and __location__ to name your variables, these are more like builtin variables which might cause a confusion.
Note something here:
__location__ = os.path.realpath(
os.path.join(os.getcwd(), os.path.dirname(__file__)))
You have not included sub directory and the above joins only the CWD + os.path.dirname(__file__). This doesn't get you to the file. Please read the documentation of os.path.dirname: os.path.dirname(__file__) returns an empty string here.
def __init__(self):
file = 'xlFile.xlsx'
location = os.path.join('sub', file)
location = os.path.abspath(location) # absolute path to file
location = os.path.realpath(location) # rm symbolic links in path
self.wrkb = openpyxl.load_workbook(location)
for example, a.boo method calls b.foo method. In b.foo method, how can I get a's file name (I don't want to pass __file__ to b.foo method)...
You can use the inspect module to achieve this:
frame = inspect.stack()[1]
module = inspect.getmodule(frame[0])
filename = module.__file__
Python 3.5+
One-liner
To get the full filename (with path and file extension), use in the callee:
import inspect
filename = inspect.stack()[1].filename
Full filename vs filename only
To retrieve the caller's filename use inspect.stack(). Additionally, the following code also trims the path at the beginning and the file extension at the end of the full filename:
# Callee.py
import inspect
import os.path
def get_caller_info():
# first get the full filename (including path and file extension)
caller_frame = inspect.stack()[1]
caller_filename_full = caller_frame.filename
# now get rid of the directory (via basename)
# then split filename and extension (via splitext)
caller_filename_only = os.path.splitext(os.path.basename(caller_filename_full))[0]
# return both filename versions as tuple
return caller_filename_full, caller_filename_only
It can then be used like so:
# Caller.py
import callee
filename_full, filename_only = callee.get_caller_info()
print(f"> Filename full: {filename_full}")
print(f"> Filename only: {filename_only}")
# Output
# > Filename full: /workspaces/python/caller_filename/caller.py
# > Filename only: caller
Official docs
os.path.basename(): to remove the path from the filename (still includes the extension)
os.path.splitext(): to split the the filename and the file extension
Inspired by ThiefMaster's answer but works also if inspect.getmodule() returns None:
frame = inspect.stack()[1]
filename = frame[0].f_code.co_filename
This can be done with the inspect module, specifically inspect.stack:
import inspect
import os.path
def get_caller_filepath():
# get the caller's stack frame and extract its file path
frame_info = inspect.stack()[1]
filepath = frame_info[1] # in python 3.5+, you can use frame_info.filename
del frame_info # drop the reference to the stack frame to avoid reference cycles
# make the path absolute (optional)
filepath = os.path.abspath(filepath)
return filepath
Demonstration:
import b
print(b.get_caller_filepath())
# output: D:\Users\Aran-Fey\a.py
you can use the traceback module:
import traceback
and you can print the back trace like this:
print traceback.format_stack()
I haven't used this in years, but this should be enough to get you started.
Reading all these solutions, it seems like this works as well?
import inspect
print inspect.stack()[1][1]
The second item in the frame already is the file name of the caller, or is this not robust?
Suppose my python code is executed a directory called main and the application needs to access main/2091/data.txt.
how should I use open(location)? what should the parameter location be?
I found that below simple code will work.. does it have any disadvantages?
file = "\2091\sample.txt"
path = os.getcwd()+file
fp = open(path, 'r+');
With this type of thing you need to be careful what your actual working directory is. For example, you may not run the script from the directory the file is in. In this case, you can't just use a relative path by itself.
If you are sure the file you want is in a subdirectory beneath where the script is actually located, you can use __file__ to help you out here. __file__ is the full path to where the script you are running is located.
So you can fiddle with something like this:
import os
script_dir = os.path.dirname(__file__) #<-- absolute dir the script is in
rel_path = "2091/data.txt"
abs_file_path = os.path.join(script_dir, rel_path)
This code works fine:
import os
def read_file(file_name):
file_handle = open(file_name)
print file_handle.read()
file_handle.close()
file_dir = os.path.dirname(os.path.realpath('__file__'))
print file_dir
#For accessing the file in the same folder
file_name = "same.txt"
read_file(file_name)
#For accessing the file in a folder contained in the current folder
file_name = os.path.join(file_dir, 'Folder1.1/same.txt')
read_file(file_name)
#For accessing the file in the parent folder of the current folder
file_name = os.path.join(file_dir, '../same.txt')
read_file(file_name)
#For accessing the file inside a sibling folder.
file_name = os.path.join(file_dir, '../Folder2/same.txt')
file_name = os.path.abspath(os.path.realpath(file_name))
print file_name
read_file(file_name)
I created an account just so I could clarify a discrepancy I think I found in Russ's original response.
For reference, his original answer was:
import os
script_dir = os.path.dirname(__file__)
rel_path = "2091/data.txt"
abs_file_path = os.path.join(script_dir, rel_path)
This is a great answer because it is trying to dynamically creates an absolute system path to the desired file.
Cory Mawhorter noticed that __file__ is a relative path (it is as well on my system) and suggested using os.path.abspath(__file__). os.path.abspath, however, returns the absolute path of your current script (i.e. /path/to/dir/foobar.py)
To use this method (and how I eventually got it working) you have to remove the script name from the end of the path:
import os
script_path = os.path.abspath(__file__) # i.e. /path/to/dir/foobar.py
script_dir = os.path.split(script_path)[0] #i.e. /path/to/dir/
rel_path = "2091/data.txt"
abs_file_path = os.path.join(script_dir, rel_path)
The resulting abs_file_path (in this example) becomes: /path/to/dir/2091/data.txt
It depends on what operating system you're using. If you want a solution that is compatible with both Windows and *nix something like:
from os import path
file_path = path.relpath("2091/data.txt")
with open(file_path) as f:
<do stuff>
should work fine.
The path module is able to format a path for whatever operating system it's running on. Also, python handles relative paths just fine, so long as you have correct permissions.
Edit:
As mentioned by kindall in the comments, python can convert between unix-style and windows-style paths anyway, so even simpler code will work:
with open("2091/data/txt") as f:
<do stuff>
That being said, the path module still has some useful functions.
I spend a lot time to discover why my code could not find my file running Python 3 on the Windows system. So I added . before / and everything worked fine:
import os
script_dir = os.path.dirname(__file__)
file_path = os.path.join(script_dir, './output03.txt')
print(file_path)
fptr = open(file_path, 'w')
Try this:
from pathlib import Path
data_folder = Path("/relative/path")
file_to_open = data_folder / "file.pdf"
f = open(file_to_open)
print(f.read())
Python 3.4 introduced a new standard library for dealing with files and paths called pathlib. It works for me!
Code:
import os
script_path = os.path.abspath(__file__)
path_list = script_path.split(os.sep)
script_directory = path_list[0:len(path_list)-1]
rel_path = "main/2091/data.txt"
path = "/".join(script_directory) + "/" + rel_path
Explanation:
Import library:
import os
Use __file__ to attain the current script's path:
script_path = os.path.abspath(__file__)
Separates the script path into multiple items:
path_list = script_path.split(os.sep)
Remove the last item in the list (the actual script file):
script_directory = path_list[0:len(path_list)-1]
Add the relative file's path:
rel_path = "main/2091/data.txt
Join the list items, and addition the relative path's file:
path = "/".join(script_directory) + "/" + rel_path
Now you are set to do whatever you want with the file, such as, for example:
file = open(path)
import os
def file_path(relative_path):
dir = os.path.dirname(os.path.abspath(__file__))
split_path = relative_path.split("/")
new_path = os.path.join(dir, *split_path)
return new_path
with open(file_path("2091/data.txt"), "w") as f:
f.write("Powerful you have become.")
If the file is in your parent folder, eg. follower.txt, you can simply use open('../follower.txt', 'r').read()
Get the path of the parent folder, then os.join your relative files to the end.
# get parent folder with `os.path`
import os.path
BASE_DIR = os.path.dirname(os.path.abspath(__file__))
# now use BASE_DIR to get a file relative to the current script
os.path.join(BASE_DIR, "config.yaml")
The same thing with pathlib:
# get parent folder with `pathlib`'s Path
from pathlib import Path
BASE_DIR = Path(__file__).absolute().parent
# now use BASE_DIR to get a file relative to the current script
BASE_DIR / "config.yaml"
Python just passes the filename you give it to the operating system, which opens it. If your operating system supports relative paths like main/2091/data.txt (hint: it does), then that will work fine.
You may find that the easiest way to answer a question like this is to try it and see what happens.
Not sure if this work everywhere.
I'm using ipython in ubuntu.
If you want to read file in current folder's sub-directory:
/current-folder/sub-directory/data.csv
your script is in current-folder
simply try this:
import pandas as pd
path = './sub-directory/data.csv'
pd.read_csv(path)
When I was a beginner I found these descriptions a bit intimidating. As at first I would try
For Windows
f= open('C:\Users\chidu\Desktop\Skipper New\Special_Note.txt','w+')
print(f)
and this would raise an syntax error. I used get confused alot. Then after some surfing across google. found why the error occurred. Writing this for beginners
It's because for path to be read in Unicode you simple add a \ when starting file path
f= open('C:\\Users\chidu\Desktop\Skipper New\Special_Note.txt','w+')
print(f)
And now it works just add \ before starting the directory.
In Python 3.4 (PEP 428) the pathlib was introduced, allowing you to work with files in an object oriented fashion:
from pathlib import Path
working_directory = Path(os.getcwd())
path = working_directory / "2091" / "sample.txt"
with path.open('r+') as fp:
# do magic
The with keyword will also ensure that your resources get closed properly, even if you get something goes wrong (like an unhandled Exception, sigint or similar)