I want to read the properties file in Python without giving the entire path. Because if my code is deployed somewhere else then my package would fail if I pass the hardcore value. So below is my code to read the properties file by giving the entire path:
import configparser
config = configparser.RawConfigParser()
config.read('C:\\Users\\s\\IdeaProjects\\PysparkETL\\src\\main\\resources\\configs.properties')
dbname = config['DB']['dbname']
username=config['DB']['user']
password=config['DB']['password']
table=config['DB']['tablename']
driver=config['DB']['driver']
print(dbname)
and below is my configs.properties file:
[DB]
dbname=ourdb
user=root
password=root
tablename=loadtable
driver=com.mysql.cj.jdbc.Driver
I tried different ways like ConfigParser instead of RawConfigParser but didn't work. Is there any way to load files from the classpath?
Also, I tried different ways from this link but it didn't help. All I need is a path to pass it to config.read method but it should not be hardcoded as I did it in the code.
Below is my project structure:
Also, as suggested I tried below code and passed the URL to the config.read method but it's not working.
props = os.path.join(
os.path.dirname("resources"), # folder where the python file is
'src/main/resources/configs.properties'
)
config.read(props)
I get below error:
raise KeyError(key)
KeyError: 'DB'
if the file will always be in the same place relative to your python file, you can do the following:
props = os.path.join(
os.path.dirname(__name__), # folder where the python file is
'relative/path/to/configs.properties'
)
I'm trying to open a file that is in the same directory as the app from views.
-app
--views.py
--about.txt
--...
My code to open the file is..
def home(request):
with open('about.txt','r') as f:
about = f
about = about.split('\n')
about = '<br/>'.join(about)
return render(request, 'app/home.html', {'about':about})
But I keep getting an error of `
FileNotFoundError at /
[Errno 2] No such file or directory: 'about.txt'
After thinking about this, I thought of putting it in a static dir but it would still give the same error.
I am still quite a beginner at django.
Django 2.2.3
Python 3.7.3
Edit:
I don't know if this is the reason... but when pressing enter for new line, it makes it on a new indentation.
TL;DR
You have to prepend your path with something like app to create app/about.txt. I have insufficient information to tell you exactly what, but here is how to find out:
When you run your app, the working directory is probably not in app. You can figure out what path it is running in by using os.getcwd(). For example:
import os
# ...
def home(request):
print(os.getcwd())
with open('about.txt','r') as f:
about = f
about = about.split('\n')
about = '<br/>'.join(about)
return render(request, 'app/home.html', {'about':about})
As #KlausD. mentioned, your path is relative. Whenever code is being run, it is run in a "working directory". For example, if I ran python views.py in the app directory, the current working directory (cwd for short) would be app. Then, when a relative path is given, like about.txt (which really means ./about.txt, where . represents the cwd), it looks in the cwd for about.txt.
In Python, I am trying to create a tar with two empty directories in it, and then add a list of files to each empty directory within the tar. I have tried doing it this way below, but It does not work.
def ISIP_tar_files():
with tarfile.open("eeg_files.tar", "w") as f:
ep_dir = tarfile.TarInfo("Eplilepsy Reports")
not_ep_dir = tarfile.TarInfo("Non Epilepsy Reports")
ep_dir.type = not_ep_dir.type = tarfile.DIRTYPE
f.addfile(ep_dir)
f.addfile(not_ep_dir)
with ep_dir.open():
for name in ep_list:
f.tarfile.add(name)
I honestly did not believe it would work, but it was worth a try because I couldn't find any other solutions on Google. This is just one module of the code, and it does not include the main program or imports. ep_list is a list of files with paths, it looks similar to:
ep_list = [/data/foo/bar/file.txt, /data/foo/bar2/file2.txt, ...]
Any Sugegstions?
import tarfile
import os
ep_list = ['./foo/bar/file.txt', './foo/bar/file2.txt']
def ISIP_tar_files():
with tarfile.open("eeg_files.tar", "w") as f:
ep_dir = tarfile.TarInfo("Eplilepsy Reports")
not_ep_dir = tarfile.TarInfo("Non Epilepsy Reports")
ep_dir.type = not_ep_dir.type = tarfile.DIRTYPE
ep_dir.mode = not_ep_dir.mode = 0o777
f.addfile(ep_dir)
f.addfile(not_ep_dir)
for name in ep_list:
f.add(name, arcname="Eplilepsy Reports/" + os.path.basename(name), recursive=False)
The directory file permission mode should be made executable at least for the owner. Otherwise it cannot be extracted.
arcname is the alternative name for the file in the archive.
recursive means whether or not keep the original directories added recursively, its default value is True.
Suppose my python code is executed a directory called main and the application needs to access main/2091/data.txt.
how should I use open(location)? what should the parameter location be?
I found that below simple code will work.. does it have any disadvantages?
file = "\2091\sample.txt"
path = os.getcwd()+file
fp = open(path, 'r+');
With this type of thing you need to be careful what your actual working directory is. For example, you may not run the script from the directory the file is in. In this case, you can't just use a relative path by itself.
If you are sure the file you want is in a subdirectory beneath where the script is actually located, you can use __file__ to help you out here. __file__ is the full path to where the script you are running is located.
So you can fiddle with something like this:
import os
script_dir = os.path.dirname(__file__) #<-- absolute dir the script is in
rel_path = "2091/data.txt"
abs_file_path = os.path.join(script_dir, rel_path)
This code works fine:
import os
def read_file(file_name):
file_handle = open(file_name)
print file_handle.read()
file_handle.close()
file_dir = os.path.dirname(os.path.realpath('__file__'))
print file_dir
#For accessing the file in the same folder
file_name = "same.txt"
read_file(file_name)
#For accessing the file in a folder contained in the current folder
file_name = os.path.join(file_dir, 'Folder1.1/same.txt')
read_file(file_name)
#For accessing the file in the parent folder of the current folder
file_name = os.path.join(file_dir, '../same.txt')
read_file(file_name)
#For accessing the file inside a sibling folder.
file_name = os.path.join(file_dir, '../Folder2/same.txt')
file_name = os.path.abspath(os.path.realpath(file_name))
print file_name
read_file(file_name)
I created an account just so I could clarify a discrepancy I think I found in Russ's original response.
For reference, his original answer was:
import os
script_dir = os.path.dirname(__file__)
rel_path = "2091/data.txt"
abs_file_path = os.path.join(script_dir, rel_path)
This is a great answer because it is trying to dynamically creates an absolute system path to the desired file.
Cory Mawhorter noticed that __file__ is a relative path (it is as well on my system) and suggested using os.path.abspath(__file__). os.path.abspath, however, returns the absolute path of your current script (i.e. /path/to/dir/foobar.py)
To use this method (and how I eventually got it working) you have to remove the script name from the end of the path:
import os
script_path = os.path.abspath(__file__) # i.e. /path/to/dir/foobar.py
script_dir = os.path.split(script_path)[0] #i.e. /path/to/dir/
rel_path = "2091/data.txt"
abs_file_path = os.path.join(script_dir, rel_path)
The resulting abs_file_path (in this example) becomes: /path/to/dir/2091/data.txt
It depends on what operating system you're using. If you want a solution that is compatible with both Windows and *nix something like:
from os import path
file_path = path.relpath("2091/data.txt")
with open(file_path) as f:
<do stuff>
should work fine.
The path module is able to format a path for whatever operating system it's running on. Also, python handles relative paths just fine, so long as you have correct permissions.
Edit:
As mentioned by kindall in the comments, python can convert between unix-style and windows-style paths anyway, so even simpler code will work:
with open("2091/data/txt") as f:
<do stuff>
That being said, the path module still has some useful functions.
I spend a lot time to discover why my code could not find my file running Python 3 on the Windows system. So I added . before / and everything worked fine:
import os
script_dir = os.path.dirname(__file__)
file_path = os.path.join(script_dir, './output03.txt')
print(file_path)
fptr = open(file_path, 'w')
Try this:
from pathlib import Path
data_folder = Path("/relative/path")
file_to_open = data_folder / "file.pdf"
f = open(file_to_open)
print(f.read())
Python 3.4 introduced a new standard library for dealing with files and paths called pathlib. It works for me!
Code:
import os
script_path = os.path.abspath(__file__)
path_list = script_path.split(os.sep)
script_directory = path_list[0:len(path_list)-1]
rel_path = "main/2091/data.txt"
path = "/".join(script_directory) + "/" + rel_path
Explanation:
Import library:
import os
Use __file__ to attain the current script's path:
script_path = os.path.abspath(__file__)
Separates the script path into multiple items:
path_list = script_path.split(os.sep)
Remove the last item in the list (the actual script file):
script_directory = path_list[0:len(path_list)-1]
Add the relative file's path:
rel_path = "main/2091/data.txt
Join the list items, and addition the relative path's file:
path = "/".join(script_directory) + "/" + rel_path
Now you are set to do whatever you want with the file, such as, for example:
file = open(path)
import os
def file_path(relative_path):
dir = os.path.dirname(os.path.abspath(__file__))
split_path = relative_path.split("/")
new_path = os.path.join(dir, *split_path)
return new_path
with open(file_path("2091/data.txt"), "w") as f:
f.write("Powerful you have become.")
If the file is in your parent folder, eg. follower.txt, you can simply use open('../follower.txt', 'r').read()
Get the path of the parent folder, then os.join your relative files to the end.
# get parent folder with `os.path`
import os.path
BASE_DIR = os.path.dirname(os.path.abspath(__file__))
# now use BASE_DIR to get a file relative to the current script
os.path.join(BASE_DIR, "config.yaml")
The same thing with pathlib:
# get parent folder with `pathlib`'s Path
from pathlib import Path
BASE_DIR = Path(__file__).absolute().parent
# now use BASE_DIR to get a file relative to the current script
BASE_DIR / "config.yaml"
Python just passes the filename you give it to the operating system, which opens it. If your operating system supports relative paths like main/2091/data.txt (hint: it does), then that will work fine.
You may find that the easiest way to answer a question like this is to try it and see what happens.
Not sure if this work everywhere.
I'm using ipython in ubuntu.
If you want to read file in current folder's sub-directory:
/current-folder/sub-directory/data.csv
your script is in current-folder
simply try this:
import pandas as pd
path = './sub-directory/data.csv'
pd.read_csv(path)
When I was a beginner I found these descriptions a bit intimidating. As at first I would try
For Windows
f= open('C:\Users\chidu\Desktop\Skipper New\Special_Note.txt','w+')
print(f)
and this would raise an syntax error. I used get confused alot. Then after some surfing across google. found why the error occurred. Writing this for beginners
It's because for path to be read in Unicode you simple add a \ when starting file path
f= open('C:\\Users\chidu\Desktop\Skipper New\Special_Note.txt','w+')
print(f)
And now it works just add \ before starting the directory.
In Python 3.4 (PEP 428) the pathlib was introduced, allowing you to work with files in an object oriented fashion:
from pathlib import Path
working_directory = Path(os.getcwd())
path = working_directory / "2091" / "sample.txt"
with path.open('r+') as fp:
# do magic
The with keyword will also ensure that your resources get closed properly, even if you get something goes wrong (like an unhandled Exception, sigint or similar)
I am getting a file posting from a file:
file = request.post['ufile']
I want to get the path. How can I get it?
You have to use the request.FILES dictionary.
Check out the official documentation about the UploadedFile object, you can use the UploadedFile.temporary_file_path attribute, but beware that only files uploaded to disk expose it (that is, normally, when using the TemporaryFileUploadHandler uploads handler).
upload = request.FILES['ufile']
path = upload.temporary_file_path
In the normal case, though, you would like to use the file handler directly:
upload = request.FILES['ufile']
content = upload.read() # For small files
# ... or ...
for chunk in upload.chunks():
do_somthing_with_chunk(chunk) # For bigger files
You should use request.FILES['ufile'].file.name
you will get like this /var/folders/v7/1dtcydw51_s1ydkmypx1fggh0000gn/T/tmpKGp4mX.upload
and use file.name, your upload file have to bigger than 2.5M.
if you want to change this , see File Upload Settings
We cannot get the file path from the post request, only the filename, because flask doesn't has the file system access. If you need to get the file and perform some operations on it then you can try creating a temp directory save the file there, you can also get the path.
import tempfile
import shutil
dirpath = tempfile.mkdtemp()
# perform some operations if needed
shutil.rmtree(dirpath) # remove the temp directory