My Python script can't resolve the relative path on a Linux server in the following script:
import boto3
import os
conn = boto3.client('s3', region_name="eu-west-1", endpoint_url="https://example.com", config=Config(signature_version="s3", s3={'addressing_style': 'path'}))
conn.download_file('mytestbucket22', 'file.csv', os.path.join(os.getcwd(), 'static', 'filecache', 'file.csv'))
Error:
[Errno 2] No such file or directory: '/home/vcap/app/static/filecache/file.csv.D3e3D7aF'
However when I do it like this it works and it saves the file to the path of my script.
conn.download_file('mytestbucket22', 'file.csv', 'file.csv')
My folder and file structure looks like this:
--script.py
--static
----filecache
How can I save the file to the folder filecache? Thanks
conn.download_file('mytestbucket22', 'file.csv', os.path.join(os.getcwd(), 'static', 'filecache', 'file.csv'))
Documentation references for the modules, constants and functions used above:
The os and os.path modules.
The __file__ constant
os.path.realpath(path) (returns "the canonical path of the specified filename, eliminating any symbolic links encountered in the path")
os.path.dirname(path) (returns "the directory name of pathname path")
os.getcwd() (returns "a string representing the current working directory")
os.chdir(path) ("change the current working directory to path")
I would use
conn.download_file('mytestbucket22', 'file.csv', os.path.join(os.path.dirname(sys.argv[0]), 'static', 'filecache', 'file.csv'))
Related
How do I get the current file's directory path?
I tried:
>>> os.path.abspath(__file__)
'C:\\python27\\test.py'
But I want:
'C:\\python27\\'
The special variable __file__ contains the path to the current file. From that we can get the directory using either pathlib or the os.path module.
Python 3
For the directory of the script being run:
import pathlib
pathlib.Path(__file__).parent.resolve()
For the current working directory:
import pathlib
pathlib.Path().resolve()
Python 2 and 3
For the directory of the script being run:
import os
os.path.dirname(os.path.abspath(__file__))
If you mean the current working directory:
import os
os.path.abspath(os.getcwd())
Note that before and after file is two underscores, not just one.
Also note that if you are running interactively or have loaded code from something other than a file (eg: a database or online resource), __file__ may not be set since there is no notion of "current file". The above answer assumes the most common scenario of running a python script that is in a file.
References
pathlib in the python documentation.
os.path - Python 2.7, os.path - Python 3
os.getcwd - Python 2.7, os.getcwd - Python 3
what does the __file__ variable mean/do?
Using Path from pathlib is the recommended way since Python 3:
from pathlib import Path
print("File Path:", Path(__file__).absolute())
print("Directory Path:", Path().absolute()) # Directory of current working directory, not __file__
Note: If using Jupyter Notebook, __file__ doesn't return expected value, so Path().absolute() has to be used.
In Python 3.x I do:
from pathlib import Path
path = Path(__file__).parent.absolute()
Explanation:
Path(__file__) is the path to the current file.
.parent gives you the directory the file is in.
.absolute() gives you the full absolute path to it.
Using pathlib is the modern way to work with paths. If you need it as a string later for some reason, just do str(path).
Try this:
import os
dir_path = os.path.dirname(os.path.realpath(__file__))
import os
print(os.path.dirname(__file__))
I found the following commands return the full path of the parent directory of a Python 3 script.
Python 3 Script:
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
from pathlib import Path
#Get the absolute path of a Python3.6 and above script.
dir1 = Path().resolve() #Make the path absolute, resolving any symlinks.
dir2 = Path().absolute() #See #RonKalian answer
dir3 = Path(__file__).parent.absolute() #See #Arminius answer
dir4 = Path(__file__).parent
print(f'dir1={dir1}\ndir2={dir2}\ndir3={dir3}\ndir4={dir4}')
REMARKS !!!!
dir1 and dir2 works only when running a script located in the current working directory, but will break in any other case.
Given that Path(__file__).is_absolute() is True, the use of the .absolute() method in dir3 appears redundant.
The shortest command that works is dir4.
Explanation links: .resolve(), .absolute(), Path(file).parent().absolute()
USEFUL PATH PROPERTIES IN PYTHON:
from pathlib import Path
#Returns the path of the current directory
mypath = Path().absolute()
print('Absolute path : {}'.format(mypath))
#if you want to go to any other file inside the subdirectories of the directory path got from above method
filePath = mypath/'data'/'fuel_econ.csv'
print('File path : {}'.format(filePath))
#To check if file present in that directory or Not
isfileExist = filePath.exists()
print('isfileExist : {}'.format(isfileExist))
#To check if the path is a directory or a File
isadirectory = filePath.is_dir()
print('isadirectory : {}'.format(isadirectory))
#To get the extension of the file
fileExtension = mypath/'data'/'fuel_econ.csv'
print('File extension : {}'.format(filePath.suffix))
OUTPUT:
ABSOLUTE PATH IS THE PATH WHERE YOUR PYTHON FILE IS PLACED
Absolute path : D:\Study\Machine Learning\Jupitor Notebook\JupytorNotebookTest2\Udacity_Scripts\Matplotlib and seaborn Part2
File path : D:\Study\Machine Learning\Jupitor Notebook\JupytorNotebookTest2\Udacity_Scripts\Matplotlib and seaborn Part2\data\fuel_econ.csv
isfileExist : True
isadirectory : False
File extension : .csv
works also if __file__ is not available (jupyter notebooks)
import sys
from pathlib import Path
path_file = Path(sys.path[0])
print(path_file)
Also uses pathlib, which is the object oriented way of handling paths in python 3.
IPython has a magic command %pwd to get the present working directory. It can be used in following way:
from IPython.terminal.embed import InteractiveShellEmbed
ip_shell = InteractiveShellEmbed()
present_working_directory = ip_shell.magic("%pwd")
On IPython Jupyter Notebook %pwd can be used directly as following:
present_working_directory = %pwd
I have made a function to use when running python under IIS in CGI in order to get the current folder:
import os
def getLocalFolder():
path=str(os.path.dirname(os.path.abspath(__file__))).split(os.sep)
return path[len(path)-1]
Python 2 and 3
You can simply also do:
from os import sep
print(__file__.rsplit(sep, 1)[0] + sep)
Which outputs something like:
C:\my_folder\sub_folder\
This can be done without a module.
def get_path():
return (__file__.replace(f"<your script name>.py", ""))
print(get_path())
When I run this code:
script_path = os.path.realpath(__file__)
new_abs_path = os.path.join(script_path, 'Users')
if not os.path.exists(new_abs_path):
os.makedirs(new_abs_path)
I get this error:
FileNotFoundError: [WinError 3] The system cannot find the path specified '<script drive>\\<script path>\\<script filename>\\User'
Since you use Python 3.9, you can use Path.mkdir:
from pathlib import Path
path = Path(__file__).parent / 'Users'
path.mkdir(exist_ok=True)
I strongly advise you to use pathlib.Path, it provide very useful tools to manipulate path, files and directories.
Try this:
import os
path = os.path.join(os.path.dirname(__file__), 'Users')
if not os.path.exists(path):
os.makedirs(path)
You forget to take the directory of your python script. You can t create a folder inside a file...
I have an annual file located on our netwrok that contains sub-folders based on each week of the year. I am trying to write a script that will copy a "Template" document saved on my desktop, then iterate through the annual file, and paste the Template into each weekly sub-folder.
I have a script that is generating a "PermissionError: [Errno 13] Permission denied:" though I am not sure why, and therefore not sure what to do about it.
import os
import shutil
myPath = r"""K:\PROCESS\YEAR"""
myDocument = r"""C:\Users\me.domain\OneDrive - Co Name\Desktop\Template.xlsx"""
for filename in os.listdir(myPath):
with open(os.path.join(myPath,filename)) as myFile:
copyfile(myDocument, myFile)
Error is,
PermissionError: [Errno 13] Permission denied: ‘K:\PROCESS\YEAR\WEEK 1’
# line 'with open(os.path.join(myPath,filename)) as myFile:'
I tried moving the target document to the program root and modified the script based on some sugestions as seen below.
import os
from shutil import copy2
my_doc_path = "Template.xlsx"
myPath = "K:\PROCESS\YEAR"
for directory in os.listdir(myPath):
copy2(my_doc_path, directory)
I am no longer receiving any errors; however, the target document is not being coppied to the target directory. Instead, a nondescript "file" with the target directory name is being created in the program root as seen in the image below.
What ended up working:
import os
from shutil import copy
my_doc_path = "Template.xlsx"
myPath = "K:\PROCESS\YEAR"
for directory in os.listdir(myPath):
target_directory = myPath + '\\' + directory
check_file = target_directory + '\\' + my_doc_path
if not os.path.exists(check_file):
copy(my_doc_path, target_directory)
Instead of opening a file, try to copy your file in the target folders.
from shutil import copy
my_doc_path = "/here/is/my/doc/path.xlsx"
myPath = "K:\PROCESS\YEAR"
for directory in os.listdir(myPath):
copy(my_doc_path, directory)
Or if you need also the metadata:
from shutil import copy2
my_doc_path = "/here/is/my/doc/path.xlsx"
myPath = "K:\PROCESS\YEAR"
for directory in os.listdir(myPath):
copy2(my_doc_path, directory)
Why copy/copy2 instead of copyfile? take a look at this comment
The document name needed to be appended to myPath variable. I incorrectly assumed the copy2() function was looking for the path to save the document to, I didnt realize it required the full destination path including the document itself.
Thanks for the feedback everyone, it got me pointed in the right direction.
i want a python script that do these three tasks:
check if the path contains word file copied to specific destination
check if the path contains pdf files copied to specific destination
check if the path contains directory and copy the hole folder to
specific destination.
for this reason i am using the os.walk() to list the dirs and files of the path
and i am using shutil library to copy files and dirs.
code
import os
from distutils.dir_util import copy_tree
import shutil
from os import path
import datetime
def main():
src = "C:/Users/LT GM/Desktop/Python_files/"
dst2 = "C:/Users/LT GM/Desktop/"
for root,dirs,files in os.walk(src):
for name in files:
print("files: ",os.path.join(root,name))
for name in dirs:
copieddst = copy_tree(src,dst2)
print("directory: ",os.path.join(root,name))
print(" coppied directory :{0}".format(copieddst) )
# make a duplicate of an existing file
if path.exists(src):
# get the path to the file in the current directory
print("****")
src = path.realpath("pandas.pdf")
#seperate the path from the filter
head, tail = path.split(src)
print("path:" +head)
print("file:" +tail)
dst =str(datetime.date.today()) + tail
# nowuse the shell to make a copy of the file
shutil.copy(src, dst)
if __name__=="__main__":
main()
the problem is that i can copy the files or the content of the directory. not the hole directory and how to check if its pdf or doc files?
If you want to copy directory rather than file, then use shutil.copytree. In usage it is similiar to shutil.copy2, that is:
import shutil
shutil.copytree('mydir', 'mydircopy')
Note that by default dirs_exist_ok is False meaning that destination should not exist, when shutil.copytree is launched.
I've something like this in my program:
A main script main.py inside a folder named 'OpenFileinaModule'. There's a folder called 'sub' inside it with a script called subScript.py and a file xlFile.xlsx, which is opened by subScript.py.
OpenFileinaModule/
main.py
sub/
__init__.py (empty)
subScript.py
xlFile.xlsx
Here is the code:
sub.Script.py:
import os, openpyxl
class Oop:
def __init__(self):
__file__='xlFile.xlsx'
__location__ = os.path.realpath(
os.path.join(os.getcwd(), os.path.dirname(__file__)))
print os.path.join(__location__, __file__)
self.wrkb = openpyxl.load_workbook(os.path.join(__location__,
__file__),read_only=True)
main.py:
import sub.subScript
objt=sub.subScript.Oop()
When I execute main.py, I get the error:
IOError: [Errno 2] No such file or directory: 'C:\\Users\\...\\OpenFileInaModule\\xlFile.xlsx'
It jumps the sub folder...
I've tried
__file__='sub/xlFile.xlsx'
But then the "sub" folder is duplicated:
IOError: [Errno 2] No such file or directory: 'C:\\Users\\...\\OpenFileInaModule\\sub\\sub/xlFile.xlsx'
How to open xlFile.xlsx with subScript.py from main.py?
you're overriding __file__ with __file='xlFile.xlsx', do you mean to do this?
I think you want something like
import os
fname = 'xlFile.xlsx'
this_file = os.path.abspath(__file__)
this_dir = os.path.dirname(this_file)
wanted_file = os.path.join(this_dir, fname)
I'd suggest always using the absolute path for a file, especially if you're on windows when a relative path might not make sense if the file is on a different drive (I actually have no idea what it would do if you asked it for a relative path between devices).
Please avoid using __file__ and __location__ to name your variables, these are more like builtin variables which might cause a confusion.
Note something here:
__location__ = os.path.realpath(
os.path.join(os.getcwd(), os.path.dirname(__file__)))
You have not included sub directory and the above joins only the CWD + os.path.dirname(__file__). This doesn't get you to the file. Please read the documentation of os.path.dirname: os.path.dirname(__file__) returns an empty string here.
def __init__(self):
file = 'xlFile.xlsx'
location = os.path.join('sub', file)
location = os.path.abspath(location) # absolute path to file
location = os.path.realpath(location) # rm symbolic links in path
self.wrkb = openpyxl.load_workbook(location)