We Keep Coding

I need to find all text files in all subfolders created on a certain date, open them, and copy the contents into a single text file

My goal is to take the contents of all text files in subfolders created today and move them to a single existing report.txt but I can't seem to find a good way to go about it. I'm not very experienced in coding so any help would be much appreciated. Here is what I have so far (I know it's rubbish):
if getmtime == today:
with open(glob.iglob(drive + "://CADIQ//CADIQ_JOBS//?????????????????????")) as f:
for line in f:
content += line
with open(reportFile, "a") as f:
f.write(content)

Try this, based on How do I list all files of a directory?
import os, time
def last_mod_today(path):
'''
return True if getmtime and time have year, mon, day coincinding in their localtime struct, False else
'''
t_s = time.localtime(os.path.getmtime(path))
today = time.localtime(time.time())
return t_s.tm_mday==today.tm_mday and t_s.tm_year == today.tm_year and t_s.tm_mon == today.tm_mon
name_to_path = lambda d,x:os.path.normpath(os.path.join(os.path.join(os.getcwd(), d),x))
def log_files(d):
'''
walking through the files in d
log the content of f when last modif time for f is today
WARNING : what happens when the file is a JPEG ?
'''
scand_dir = os.path.join(os.getcwd(), d)
print(f"scanning {scand_dir}...")
(_, _, filenames) = next(os.walk(scand_dir))
log = open("log.txt", 'a')
for f in filenames:
if last_mod_today(name_to_path(d,f)):
with open(name_to_path(d,f), 'r') as todays_file:
log.write('##############################\n')
log.write(f"file : {name_to_path(d,f)}\n")
log.write(todays_file.read())
log.write('\n')
log.write('##############################\n')
log.close()
#first scanning files in the current directory
(_, dirnames, _) = next(os.walk('./'))
log_files('./')
#then crawling through the subdirs (one level)
for d in dirnames:
log_files(d)

I would start by creating a desired_date object, which is a datetime.date. You can then format that date into a string, which makes up the pattern you want to look for in your glob. The glob pattern doesn't care about the time, just the date.
from pathlib import Path
import datetime
desired_date = datetime.date(year=2020, month=12, day=22)
pattern = "13.2.1_" + desired_date.strftime("%y_%m_%d") + "_*"
for path in Path("path/to/folders").glob(pattern):
if not path.is_dir():
continue
print(path)
From there, you can visit each path, glob all text files in the current path, and accumulate the lines in each text file. Finally, write everything to one file.

import glob
contents = b''
for file in glob.glob('./*/*.txt'): # u can change as per your directory
fname = file.split(r'\\')[-1]
with open(fname, 'rb') as f1:
contents += f1.read()
with open('report.txt','wb') as rep:
rep.write(contents)
Hope this helps so :)
Better try to read or write files in terms of bytes because sometimes there may be a chance of corrupting data.

How to get the filename in directory with the max number in the filename python?

I have some xml files in a folder as example 'assests/2020/2010.xml', 'assests/2020/20005.xml', 'assests/2020/20999.xml' etc. I want to get the filename with max value in the '2020' folder. For above three files output should be 20999.xml
I am trying as following:
import glob
import os
list_of_files = glob.glob('assets/2020/*')
# latest_file = max(list_of_files, key=os.path.getctime)
# print (latest_file)
I couldn't be able to find logic to get the required file.
Here is the resource that have best answer to my query but I couldn't build my logic.

You can use pathlib to glob for the xml files and access the Path object attributes like .name and .stem:
from pathlib import Path
list_of_files = Path('assets/2020/').glob('*.xml')
print(max((Path(fn).name for fn in list_of_files), key=lambda fn: int(Path(fn).stem)))
Output:
20999.xml

I can't test it out right now, but you may try this:
files = []
for filename in list_of_files:
filename = str(filename)
filename = filename.replace('.xml','') #Assuming it's not printing your complete directory path
filename = int(filename)
files += [filename]
print(files)
This should get you your filenames in integer format and now you should be able to sort them in descending order and get the first item of the sorted list.

Use re to search for the appropriate endings in your file paths. If found use re again to extract the nr.
import re
list_of_files = [
'assests/2020/2010.xml',
'assests/2020/20005.xml',
'assests/2020/20999.xml'
]
highest_nr = -1
highest_nr_file = ''
for f in list_of_files:
re_result = re.findall(r'\d+\.xml$', f)
if re_result:
nr = int(re.findall(r'\d+', re_result[0])[0])
if nr > highest_nr:
highest_nr = nr
highest_nr_file = f
print(highest_nr_file)
Result
assests/2020/20999.xml

You can also try this way.
import os, re
path = "assests/2020/"
files =[
"assests/2020/2010.xml",
"assests/2020/20005.xml",
"assests/2020/20999.xml"
]
n = [int(re.findall(r'\d+\.xml$',file)[0].split('.')[0]) for file in files]
output = str(max(n))+".xml"
print("Biggest max file name of .xml file is ",os.path.join(path,output))
Output:
Biggest max file name of .xml file is assests/2020/20999.xml

import glob
xmlFiles = []
# this will store all the xml files in your directory
for file in glob.glob("*.xml"):
xmlFiles.append(file[:4])
# this will print the maximum one
print(max(xmlFiles))

How to sort a list by creation date? (Python - OS library)

I'd like to rename a list of video cuts in a folder (in my case /Desktop/Cuts/)
from:
xxx.mp4, yyy.mp4, ..., zzz.mp4
to:
1.mp4, 2.mp4, ..., 33.mp4.
1.mp4 will be the video that was first created in the folder. 2.mp4 will be the second, etc.
What I am doing now (please see code below) renames the video cuts in /Cuts/, but according to an order that I don't understand, and that is definitely NOT creation date. (it does not seem alphabetical, either)
import os
path = '/Users/ntelmaut/Desktop/Cuts/'
i = 1
for file in os.listdir(path):
new_file_name = "{}.mp4".format(i)
os.rename(path + file, path + new_file_name)
#print path + file
i = i+1
I have tried make a variable out of os.listdir(path) and to apply sorted() function, but to no avail yet (I am a newbie to Python - sorry).
Would love a pointer, here.
Thank you!

Solution
import glob
import os
dirpath = '/Users/ntelmaut/Desktop/Cuts/'
files = glob.glob(os.path.join(path, '*.mp4'))
files.sort(key=os.path.getmtime)
for i, filename in enumerate(files, 1):
try:
os.rename(filename, "{0}.mp4".format(i))
except OSError as e:
print("OSError: ", e)

If I understand correctly this is what you want.
import os
path = '/Users/ntelmaut/Desktop/Cuts/'
def sort_key(file):
return os.stat(file).st_ctime
files = os.listdir(path)
sorted_files = sorted((os.path.join(path, f) for f in files), key=sort_key)
i = 1
for file in sorted_files:
new_file_name = f"{i}.mp4"
os.rename(file, new_file_name)
i += 1
A cleaner way would be to use the pathlib module.
from pathlib import Path
p = Path('/Users/ntelmaut/Desktop/Cuts/')
def sort_key(path):
return path.stat().st_ctime
sorted_files = sorted(p.iterdir(), key=sort_key)
for idx, file in enumerate(sorted_files, 1):
new_name = f"{idx}.mp4"
file.rename(p / new_name)

Python file sort

So i wrote this to help me sort files in a bunch of different folders, it works by taking the first file from each folder and creating a folder for it then the 2nd file from each folder and does the same and so on, but when ever I run the code nothing happens can someone help.
import os, sys
path = "\Users\mac\Desktop\soliddd sort\dir"
fdir = os.listdir(path)
f = len(fdir[0])
array = [[] for i in xrange(f)]
def setArray(c, i, j):
array[j][i] = c[j]
def chooseFile(j):
for i in fdir:
setArray(fdir[i], i, j)
def makedir(f, fdir):
for i in f:
folder = r"\Users\mac\Desktop\soliddd sort\dir"+str(i)
if not os.path.exists(folder):
os.makedirs(folder)
for j in fdir:
with open(os.path.join(folder, array[i][j], 'wb')) as temp:
temp.write(buff)
folder.close()
def main():
for j in f:
chooseFile(j)
makedir(f, fdir)

A tiny example on how you can go about sorting files into folders, as I couldn't understand why you were randomly selecting files from different locations. This example gets all the file names, and then lets you select a common filename to sort into a new folder
def sort_files(base_dir="C:\sorted_files")
import os, shutil
folder_list = [
'folder_path_a',
'folder_path_b',
]
file_list = []
for folder in folder_list:
file_list.extend([os.path.join(folder, file_name) for file_name in os.listdir(folder)])
key = raw_input("Group files with <text> in the filename: ")
matching_files = [ file_name for file_name in file_list if key in file_name ]
for file_name in matching_files:
shutil.move(file_name, os.path.join(os.path.join(base_dir,key), os.path.basename(file_name)))

Count how many text files in a directory using python 2.7 and edit them

I need help with counting the amount of files in a directory/folder (Windows 7) in python so then if there is more than 20 .txt files I can delete some there is always 20 files that are .txt. If you create another one, using python, it deletes the oldest one.
Any answers will be helpful, thank you.
import os
(_, _, my_files) = os.walk("C:\\Users\\guest.user\\task2").next()
amount = len([f for f in my_files if f.endswith('.txt')])
print amount
if amount > 20:
#This is my next problem - needs to truncate the file once there is more than 20

Count the txt files
import os
(_, _, my_files) = os.walk('some_directory').next()
print len([f for f in my_files if f.endswith('.txt')])
Count the txt files and delete the ones that're beyond the first 10
import os
(_, _, my_files) = os.walk('some_directory').next()
if len([f for f in my_files if f.endswith('.txt')]) > 10:
for f in my_files[9:]:
os.remove(os.path.join('some_directory', f))
Count the txt files and delete anything after x-number of files
import os
my_directory = 'some_directory'
max_files = 20
(_, _, my_files) = os.walk(my_directory).next()
if len([f for f in my_files if f.endswith('.txt')]) > max_files:
for f in my_files[max_files-1:]:
os.remove(os.path.join(my_directory, f))

For the size, use the following:
import os
filesInDir = os.listdir(myPath) # returns a list with all the file names in that directory
numOfFilesInDir = len(filesInDir)
To delete a file, use the following:
import os
os.delete(pathToFile)
To check the time of the creation of the file:
import os
(mode, ino, dev, nlink, uid, gid, size, atime, mtime, ctime) = os.stat(pathToFile)
# ctime is the creation time of the file.
To create a file:
open(filePath, 'w')