We Keep Coding

Python to rename files in a directory/folder to csv

I have written a small script to hopefully iterate through my directory/folder and replace act with csv. Essentially, I have 11 years worth of files that have a .act extension and I just want to replace it with .csv
import os
files = os.listdir("S:\\folder\\folder1\\folder2\\folder3")
path = "S:\\folder\\folder1\\folder2\\folder3\\"
#print(files)
for x in files:
new_name = x.replace("act","csv")
os.rename(path+x,path+new_name)
print(new_name)
When I execute this, it worked for the first five files and then failed on the sixth with the following error:
FileNotFoundError: [WinError 2] The system cannot find the file specified: 'S:\\folder\\folder1\\folder2\\folder3\\file_2011_06.act' -> 'S:\\folder\\folder1\\folder2\\folder3\\file_2011_06.csv'
When I searched for "S:\folder\folder1\folder2\folder3\file_2011_06.act" in file explorer, the file opens. Are there any tips on what additional steps I can take to debug this issue?
Admittedly, this is my first programming script. I'm trying to do small/minor things to start learning. So, I likely missed something... Thank you!

In your solution, you use string's replace to replace "act" by "csv". This could lead to problems if your path contains "act" somewhere else, e.g., S:\\facts\\file_2011_01.act would become S:\\fcsvs\\file_2011_01.act and rename will throw a FileNotFoundError because rename cannot create folders.
When dealing with file names (e.g., concatenating path fragments, extracting file extensions, ...), I recommend using os.path or pathlib instead of direct string manipulation.
I would like to propose another solution using os.walk. In contrast to os.listdir, it recursively traverses all sub-directories in a single loop.
import os
def act_to_csv(directory):
for root, folders, files in os.walk(directory):
for file in files:
filename, extension = os.path.splitext(file)
if extension == '.act':
original_filepath = os.path.join(root, file)
new_filepath = os.path.join(root, filename + '.csv')
print(f"{original_filepath} --> {new_filepath}")
os.rename(original_filepath, new_filepath)
Also, I'd recommend to first backup your files before manipulating them with scripts. Would be annoying to loose data or see it becoming a mess because of a bug in a script.

import os
folder="S:\\folder\\folder1\\folder2\\folder3\\"
count=1
for file_name in os.listdir(folder):
source = folder + file_name
destination = folder + str(count) + ".csv"
os.rename(source, destination)
count += 1
print('All Files Renamed')
print('New Names are')
res = os.listdir(folder)
print(res)

Comparison of file list with files in folder

I have a list of filenames but in the directory they are named a little different. I wanna print filenames that are not in directory. Example of files:
FOO_BAR_524B_023D9B01_2021-157T05-34-31__00001_2021-08-30T124702.130.tgz
import os
missing = ['FOO_BAR_524B_023D9B01_2021-157T05-34-31__00001', 'dfiknvbdjfhnv']
for fileName in missing:
for fileNames in next(os.walk('C:\\Users\\foo\\bar'))[2]:
if fileName not in fileNames:
print(fileName)
I cannot get what I'm doing wrong...

The problem is that you iterate over every file in the directory (for fileNames in next(os.walk(...))[2]) and check if fileName is in each of those file names. For every file in the folder where fileName not in fileNames, fileName is printed, resulting in it being printed many times.
This can be fixed by doing a single check to see if all files in the folder do not contain the target file name.
import os
missing = ['FOO_BAR_524B_023D9B01_2021-157T05-34-31__00001', 'dfiknvbdjfhnv']
fileNames = next(os.walk('C:\\Users\\foo\\bar'))[2]
for missingfileName in missing:
if all(missingfileName not in fileName for fileName in fileNames):
print(missingfileName)
If you want it to be more efficient and you are only looking for file names that are prefixes of other names, then you can use a data structure called a trie. For example if missing equals ['bcd'], and there is a file called abcde and these are not considered a match, then a trie is appropriate here.

read two files from two paths at within same loop_python

I have two paths each path contains many files, each file contains data for one day. I need to read the file from the first path, and the other file from the other path that corresponds to the same day. [in the same python loop I want to read the file of the first day in each path]. The files are with the same name and sequence in each path.
I'm trying to use os.listdir(path) in for loop instead of with open (file) as file because I want to read the file as a data frame using pandas. Then use pandas to do data aggregation for each file.

I assume, you are sure, that a file always exists in both directories.
Is this what you are asking?
path1 = "path1"
path2 = "path2"
for fname in os.listdir(path1):
fname1 = os.path.join(path1, fname)
fname2 = os.path.join(path2, fname)
# do your processing here
If it can happen, that a file exists in path1 but not in path 2 you have to check with os.path.isfile() the presence of the other file before opening and skip if absent

Search for multiple files by name and copy to a new folder

I have been trying to write some python code in order to get each line from a .txt file and search for a file with that name in a folder and its subfolders. After this I want to copy that file in a preset destination folder.
The thing is when I test this code I can read all the files in the .txt and I can display all files in a directory and its subdirectories. The problem rises when I have to compare the filename I read from the .txt (line by line as I said) with all the filenames within the directory folder and then copy the file there.
Any ideas what am I doing wrong?
import os, shutil
def main():
dst = '/Users/jorjis/Desktop/new'
f = open('/Users/jorjis/Desktop/articles.txt', 'rb')
lines = [line[:-1] for line in f]
for files in os.walk("/Users/jorjis/Desktop/folder/"):
for line in lines:
if line == files:
shutil.copy('/dir/file.ext', '/new/dir')

You are comparing the file names from the text file with a tuple with three elements: the root path of the currently visited folder, a list of all subdirectory names in that path, and a list of all file names in that path. Comparing a string with a tuple will never be true. You have to compare each file name with the set of file names to copy. The data type set comes in handy here.
Opening a file together with the with statement ensures that it is closed when the control flow leaves the with block.
The code might look like this:
import os
import shutil
def main():
destination = '/Users/jorjis/Desktop/new'
with open('/Users/jorjis/Desktop/articles.txt', 'r') as lines:
filenames_to_copy = set(line.rstrip() for line in lines)
for root, _, filenames in os.walk('/Users/jorjis/Desktop/folder/'):
for filename in filenames:
if filename in filenames_to_copy:
shutil.copy(os.path.join(root, filename), destination)

If I had to guess, I would say that the files in the .txt contain the entire path. You'd need to add a little more to os.walk to match up completely.
for root, _, files in os.walk("/Users/jorjis/Desktop/folder/"):
for f in files:
new_path = f + root
if new_path in lines:
shutil.copy(new_path, `/some_new_dir')
Then again, I'm not sure what the .txt file looks like so it might be that your original way works. If that's the case, take a closer look at the lines = ... line.

Walking sub directories in Python and saving to same sub directory

First of all thanks for reading this. I am a little stuck with sub directory walking (then saving) in Python. My code below is able to walk through each sub directory in turn and process a file to search for certain strings, I then generate an xlsx file (using xlsxwriter) and post my search data to an Excel.
I have two problems...
The first problem I have is that I want to process a text file in each directory, but the text file name varies per sub directory, so rather than specifying 'Textfile.txt' I'd like to do something like *.txt (would I use glob here?)
The second problem is that when I open/create an Excel I would like to save the file to the same sub directory where the .txt file has been found and processed. Currently my Excel is saving to the python script directory, and consequently gets overwritten each time a new sub directory is opened and processed. Would it be wiser to save the Excel at the end to the sub directory or can it be created with the current sub directory path from the start?
Here's my partially working code...
for root, subFolders, files in os.walk(dir_path):
if 'Textfile.txt' in files:
with open(os.path.join(root, 'Textfile.txt'), 'r') as f:
#f = open(file, "r")
searchlines = f.readlines()
searchstringsFilter1 = ['Filter Used :']
searchstringsFilter0 = ['Filter Used : 0']
timestampline = None
timestamp = None
f.close()
# Create a workbook and add a worksheet.
workbook = xlsxwriter.Workbook('Excel.xlsx', {'strings_to_numbers': True})
worksheetFilter = workbook.add_worksheet("Filter")
Thanks again for looking at this problem.
MikG

I will not solve your code completely, but here are hints:
the text file name varies per sub directory, so rather than specifying 'Textfile.txt' I'd like to do something like *.txt
you can list all files in directory, then check file extension
for filename in files:
if filename.endswith('.txt'):
# do stuff
Also when creating woorkbook, can you enter path? You have root, right? Why not use it?

You don't want glob because you already have a list of files in the files variable. So, filter it to find all the text files:
import fnmatch
txt_files = filter(lambda fn: fnmatch.fnmatch(fn, '*.txt'), files)
To save the file in the same subdirectory:
outfile = os.path.join(root, 'someoutfile.txt')