If I want to read files named "ABCbook.txt" under directors A/B/C, which means the path A/B/C depends on the files' name and the folders A/B/C are hierarchical. How can I achieve it in Python?
We will use os.path.join to make the file path in a platform-independent way, then open the file using the normal with open... technique.
import os
my_file = os.path.join('A', 'B', 'C', 'ABCbook.txt')
with open(my_file) as f:
# your code to work on the file goes here
for line in f:
print(line)
import os
filename = "ABCbook.txt"
path = list(filename[:3]) + [filename]
syspath = os.path.join(*path)
print(syspath)
output (on windows):
A\B\C\ABCbook.txt
on linux or mac it will return
A/B/C/ABCbook.txt
Related
I'm wondering How it should be possible to open a file with python without taking into account the extension.
For example, I have this line (it doesn't work I know because it doesn't exist a path with changelog.* file obviously):
with open(f"/home/user/Desktop/Projects/{app}/CHANGELOG.*", 'r') as changelog:
I would like to open changelog from a specific app (app1, app2, ...). In app1, changelog is changelog.rst but in app2, it's changelog.md.
How it will be possible to write in the open() function something like /path/to/file/changelog.* ?
I don't find something which could help me.
One possible solution is to check if the file with the extension exists.
Ex:
import os
exts = [".rst", ".md"]
for ext in exts:
if os.path.isfile(filepath + ext):
with open(f"/home/user/Desktop/Projects/{app}/CHANGELOG" + ext, 'r') as changelog:
.....
or if you know the extension in each app you can use a dict
Ex:
exts = {"app1": "rst", "app2":"md"}
for app, ext in exts.items():
with open(f"/home/user/Desktop/Projects/{app}/CHANGELOG.{ext}", 'r') as changelog:
.....
You could run through all files in the folder with os.listdir() or on a specific folder with os.listdir(path) and open a file with a certain name. Like this:
import os
path_to_file = "your/path/to/file"
for file in os.listdir(path_to_file):
if (file.split('.')[0] == "changelog"):
with open(os.path.join(path_to_file,file), 'r') as changelog:
# Do something with file
The code above should work with any extension, not only .rst and .md. If you'd only want it to work with these two extensions, I think #Rakesh's answer is an easier approach.
I'd recommend using glob to allow you to find the full filename of the file before you open it.
import glob
import os
os.chdir(FILE_LOCATION) #if needed
files = glob.glob("CHANGELOG.*")
with open(files[0], 'r') as changelog:
....
You should be using python glob library for this requirement
here is the code:
import glob
files = glob.glob('CHANGELOG.*');
for changelog in files:
with open(changelog,'r') as f:
# read f
def app(ext):
changelog = f"/path/.../changelog.{ext}"
return open(changelog, 'r')
app1 = app(rst)
app2 = app(md)
I'm trying to make the migration from Windows to Linux, some of my Python code has not survived the move.
this is the Windows version:
path = r'C:\\Users\\x\\PythonTestFiles\\TestFile.csv'
file = open(path, newline='')
TestFileRaw = csv.reader(file)
header = next(TestFileRaw)
dataPC = [row for row in TestFileRaw]
etc....
I'm trying to address a csv file in a certain directory but I can't figure out the Linux addressing.
Can anyone Help?
here is some info for linux directory structure
press Alt + Enter over your file then in that properties dialog copy the path to root its something like /home/<user_name>/Documents then append that path with file name of csv file.
At Final your csv file's path becomes something like
/home/<user_name>/Documents/filename.csv
You could use separator from os module:
import os
path = os.path.join('your_folder_name', 'your_file_name')
That way it could be Windows/Linux independent.
Here is official documentation to os.path module.
Use whatever path is your csv file in linux!
path = r'/home/x/python_test_files/test_file.csv'
file = open(path, newline='')
TestFileRaw = csv.reader(file)
header = next(TestFileRaw)
dataPC = [row for row in TestFileRaw]
I am trying to open the file from folder and read it but it's not locating it. I am using Python3
Here is my code:
import os
import glob
prefix_path = "C:/Users/mpotd/Documents/GitHub/Python-Sample-
codes/Mayur_Python_code/Question/wx_data/"
target_path = open('MissingPrcpData.txt', 'w')
file_array = [os.path.abspath(f) for f in os.listdir(prefix_path) if
f.endswith('.txt')]
file_array.sort() # file is sorted list
for f_obj in range(len(file_array)):
file = os.path.abspath(file_array[f_obj])
join_file = os.path.join(prefix_path, file) #whole file path
for filename in file_array:
log = open(filename, 'r')#<---- Error is here
Error: FileNotFoundError: [Errno 2] No such file or directory: 'USC00110072.txt'
You are not giving the full path to a file to the open(), just its name - a relative path.
Non-absolute paths specify locations in relation to current working directory (CWD, see os.getcwd).
You would have to either os.path.join() correct directory path to it, or os.chdir() to the directory that the files reside in.
Also, remember that os.path.abspath() can't deduce the full path to a file just by it's name. It will only prefix its input with the path of the current working directory, if the given path is relative.
Looks like you are forgetting to modify the the file_array list. To fix this, change the first loop to this:
file_array = [os.path.join(prefix_path, name) for name in file_array]
Let me reiterate.
This line in your code:
file_array = [os.path.abspath(f) for f in os.listdir(prefix_path) if f.endswith('.txt')]
is wrong. It will not give you a list with correct absolute paths. What you should've done is:
import os
import glob
prefix_path = ("C:/Users/mpotd/Documents/GitHub/Python-Sample-"
"codes/Mayur_Python_code/Question/wx_data/")
target_path = open('MissingPrcpData.txt', 'w')
file_array = [f for f in os.listdir(prefix_path) if f.endswith('.txt')]
file_array.sort() # file is sorted list
file_array = [os.path.join(prefix_path, name) for name in file_array]
for filename in file_array:
log = open(filename, 'r')
You are using relative path where you should be using an absolute one. It's a good idea to use os.path to work with file paths. Easy fix for your code is:
prefix = os.path.abspath(prefix_path)
file_list = [os.path.join(prefix, f) for f in os.listdir(prefix) if f.endswith('.txt')]
Note that there are some other issues with your code:
In python you can do for thing in things. You did for thing in range(len(things)) it's much less readable and unnecessary.
You should use context managers when you open a file. Read more here.
I am having a hard time looping through files in a directory that is different from the directory where the script was written. I also ideally would want my script through go to through all files that start with sasa. There are a couple of files in the folder such as sasa.1, sasa.2 etc... as well as other files such as doc1.pdf, doc2.pdf
I use Python Version 2.7 with windows Powershell
Locations of Everything
1) Python Script Location ex: C:Users\user\python_project
2) Main_Directory ex: C:Users\user\Desktop\Data
3) Current_Working_Directory ex: C:Users\user\python_project
Main directory contains 100 folders (folder A, B, C, D etc..)
Each of these folders contains many files including the sasa files of interest.
Attempts at running script
For 1 file the following works:
Script is run the following way: python script1.py
file_path = 'C:Users\user\Desktop\Data\A\sasa.1
def writing_function(file_path):
with open(file_path) as file_object:
lines = file_object.readlines()
for line in lines:
print(lines)
writing_function(file_path)
However, the following does not work
Script is run the following way: python script1.py A sasa.1
import os
import sys
from os.path import join
dr = sys.argv[1]
file_name = sys.argv[2]
file_path = 'C:Users\user\Desktop\Data'
new_file_path = os.path.join(file_path, dr)
new_file_path2 = os.path.join(new_file_path, file_name)
def writing_function(paths):
with open(paths) as file_object:
lines = file_object.readlines()
for line in lines:
print(line)
writing_function(new_file_path2)
I get the following error:
with open(paths) as file_object:
IO Error: [Errno 2] No such file or directory:
'C:Users\\user\\Desktop\\A\\sasa.1'
Please note right now I am just working on one file, I want to be able to loop through all of the sasa files in the folder.
It can be something in the line of:
import os
from os.path import join
def function_exec(file):
code to execute on each file
for root, dirs, files in os.walk('path/to/your/files'): # from your argv[1]
for f in files:
filename = join(root, f)
function_exec(filename)
Avoid using the variable dir. it is a python keyword. Try print(dir(os))
dir_ = argv[1] # is preferable
No one mentioned glob so far, so:
https://docs.python.org/3/library/glob.html
I think you can solve your problem using its ** magic:
If recursive is true, the pattern “**” will match any files and zero
or more directories and subdirectories. If the pattern is followed by
an os.sep, only directories and subdirectories match.
Also note you can change directory location using
os.chdir(path)
I use setuptools to distribute a python package. My directory structure is like following.
Mypackage
--setup.py
--my_package.py
Data
--some_name.log
I want users to put data files in this folder, and name can be anything with the extension .log.
log_list = []
for file in glob.glob('/home/ginger/Mypackage/Data/*.log'):
with open(file,'r') as f:
for line in f:
try:
data = p.parse(line)
except:
pass
log_list.append(data)
This code works fine. But when I try to get the absolute path from the relative path it does not read the data file. Why is this?
path = os.path.abspath("Data/*.log")
log = []
for file in glob.glob(path):
with open(file,'r') as f:
for line in f:
log.append(line)
Found the solution. Path has to be defined as following with the immediate directory.
path = os.path.abspath("Mypackage/Data/*.log")