We Keep Coding

While obtaining hash files, some folders and files from the directory are not showing up

My code was working just fine before adding the hash function. I was getting the list of all folders and files in my directory in the Pretty Table. Once I added the hash function, I got maybe 5 of the files in that directory with hashes in the table. I am not sure where I have gone wrong. Please forgive me, I am new to this. We are not learning to code from scratch, but have to modify existing codes to function the way we need it to.
# Python Standard Libaries
import os #file system methode
import hashlib #hashing function
import sys #system methods
import time #time conversions
# Python 3rd Party Libraries
from prettytable import PrettyTable # pip install prettytable
# Local Functions
def GetFileMetaData(fileName):
#obtain file system metadata
try:
metaData = os.stat(fileName) # Use the stat method to obtain meta data
fileSize = metaData.st_size # Extract fileSize and MAC Times
timeLastAccess = metaData.st_atime
timeLastModified = metaData.st_mtime
timeCreated = metaData.st_ctime
macTimeList = [timeLastModified, timeCreated, timeLastAccess] # Group the MAC Times in a List
return True, None, fileSize, macTimeList
except Exception as err:
return False, str(err), None, None
# Psuedo Constants
# Start of the Script
tbl = PrettyTable(['FilePath','FileSize','UTC-Modified', 'UTC-Accessed', 'UTC-Created', 'SHA-256 HASH'])
#file check
while True:
targetFolder = input("Enter Target Folder: ")
if os.path.isdir(targetFolder):
break
else:
print("\nInvalid Folder ... Please Try Again")
print("Walking: ", targetFolder, "\n")
print()
for currentRoot, dirList, fileList in os.walk(targetFolder):
for nextFile in fileList:
fullPath = os.path.join(currentRoot, nextFile)
absPath = os.path.abspath(fullPath)
fileSize = os.path.getsize(absPath)
success, errInfo, fileSize, macList = GetFileMetaData(absPath)
if success:
#convert to readable Greenich Time
modTime = time.strftime("%Y-%m-%d %H:%M:%S", time.gmtime(macList[0]))
accTime = time.strftime("%Y-%m-%d %H:%M:%S", time.gmtime(macList[1]))
creTime = time.strftime("%Y-%m-%d %H:%M:%S", time.gmtime(macList[2]))
#hashing function
with open(absPath, 'rb') as target:
fileContents = target.read()
sha256Obj = hashlib.sha256()
sha256Obj.update(fileContents)
hexDigest = sha256Obj.hexdigest()
tbl.add_row( [ absPath, fileSize,modTime, accTime, creTime, hexDigest] )
tbl.align = "l" # align the columns left justified
# display the table
print (tbl.get_string(sortby="FileSize", reversesort=True))
print("\nScript-End\n")

Delete list of files in folder which is older than x days, python

I am trying to delete files after a 90 day retention period, and folders after a 15 day retention period. I have stored the paths in a config.txt file that looks like this:
[My Config]
path1 = C:\Users\mywork\pa\config\*.xml
path2 = C:\Users\mywork\data\unittest\*.csv
path3 = C:\Users\mywork\data\inttest\*.csv
path4 = C:\Users\mywork\logs
I am trying to convert the number of days into seconds, get the ctime from the os.stat(path) method using the attribute st_ctime. Then I compare the ctime with the time previously calculated. If the result is greater than the desired days of the user, it checks whether it is a file or folder. If it is a file, use the os.remove(path) and otherwise use the shutil.rmtree() method:
import os
import shutil
import time
import glob
import configparser
def main():
# initializing the count
deleted_folders_count = 0
deleted_files_count = 0
configParser = configparser.RawConfigParser()
config = configparser.ConfigParser()
config.read_file(open(r'C:\Users\PycharmProjects\untitled\config.txt'))
# specify the path
path = [config.get('My Config', 'path1'), config.get('My Config', 'path2'), config.get('My Config', 'path3')]
print(path)
folder_retn_days = 15
log_retn_days = 90
# converting days to seconds
# time.time() returns current time in seconds
folder_seconds = time.time() - (folder_retn_days * 24 * 60 * 60)
log_seconds = time.time() - (log_retn_days * 24 * 60 * 60)
for idx, word in enumerate(path):
print(word)
files = glob.glob(word)
def remove_file(files):
# removing the file
if not os.remove(files):
# success message
print(f"{files} is removed successfully")
else:
# failure message
print(f"Unable to delete the {files}"
def remove_folder(files):
# removing the folder
if not shutil.rmtree(files):
# success message
print(f"{files} is removed successfully")
else:
# failure message
print(f"Unable to delete the {files}")
def get_file_or_folder_age(files):
# getting ctime of the file/folder
# time will be in seconds
ctime = os.stat(files).st_ctime
# returning the time
return ctime
if folder_seconds >= get_file_or_folder_age(files):
print(files)
#remove_files(files)
#deleted_folders_count += 1
I am stuck here. While printing files, it is giving this error:
ctime = os.stat(files).st_ctime
TypeError: stat: path should be string, bytes, os.PathLike or integer, not list
I tried many things, but am getting the same error.

You're close, but as the error indicates stat() takes a single path, not a list of paths. So you have to call it for each path:
def get_file_or_folder_age(file):
# getting ctime of the file/folder
# time will be in seconds
ctime = os.stat(file).st_ctime
# returning the time
return ctime
for path in files:
if folder_seconds >= get_file_or_folder_age(file):
print(files) # Or other action you'd like to take.
I think you will run into other problems, but that is your proximal one.

You pass files into os.stats, which is a list of files retrieved from using glob.glob. os.stats requires a path, not a list (you can refer to this document here: https://docs.python.org/3/library/os.html#os.stat).
As your requirements, I figure that you are trying to get the latest modified time of a list of files. In that case, try the following code snippet:
import os
def get_last_modified_time(files: list):
return max([os.stat(file).st_ctime for file in files])

you can also use os.scandir and fnmatch to get the same result
import datetime, os, fnmatch
path = "C:\\temp\\*.csv"
cutoff_date = (datetime.datetime.today() - datetime.timedelta(days=90)).timestamp()
dirname = os.path.dirname(path)
delete_list = [ f.name for f in os.scandir(dirname) if f.is_file()
and f.stat().st_mtime > cutoff_date
and fnmatch.fnmatch(dirname + "\\" + f.name, path) ]

Drag folder to .exe program

I have a python code. I managed to make my .py to .exe but when I want to drag and drop a folder with my files to compare them with other files, it won't work. Just creating my output folder and exits the program. Here is the python code and where should I modify the code when I drag and drop the folder to execute the program? Thank you
import difflib
import os
from pathlib import Path
import shutil
import time
ts = time.time()
fold1_path = input("First folder:")
path1 = []
path1.append(fold1_path)
list1 = list(Path(path1[0]).glob("*.htm"))
## Create empty folder for the result
x = os.path.expanduser("~")+ r"\Desktop\rezultat_script"
path = x
if not os.path.exists(path):
os.makedirs(path)
# Copy files into different folder for processing
files = list1.copy()
for f in files:
shutil.copy(f, path)
fold2_path = input("Second folder:")
list2 = list(Path(fold2_path).glob("*.htm"))
def compare(folder1, folder2):
for num in range(len(list1)):
show_pre = "Before editing:"
show_post = "After editing:"
show_pre_lines = open(folder1[num]).readlines()
show_post_lines = open(folder2[num]).readlines()
difference = difflib.HtmlDiff(wrapcolumn=100).make_file(show_pre_lines, show_post_lines, show_pre, show_post)
for file in range(len(folder1)):
difference_report = open(folder1[file], "w+")
difference_report.write(difference)
difference_report.close()
fold3_path = list(Path(x).glob("*.htm"))
compare(fold3_path, list2)
print("Script done!")
te = time.time()
total_time = te-ts
print("Total time used: {:10.2f}".format(total_time))

Python: How do I add files to a new directory with a time-stamp?

Im new to python. I am trying to create a script that will add imported files to a new directory with a timestamp, as a daily backup. How do I point to the new directory as it changes name every day? Here is my script:
gis = GIS("https://arcgis.com", "xxx", "xxx")
items = gis.content.search(query="type:Feature Service, owner:xxx", max_items=5000,)
import datetime
import shutil
import os
now = datetime.datetime.today()
nTime = now.strftime("%d-%m-%Y")
source = r"C:\Users\Bruger\xxx\xxx\xxx\Backup\Backup"
dest = os.path.join(source+nTime)
if not os.path.exists(dest):
os.makedirs(dest) #creat dest dir
source_files = os.listdir(source)
for f in source_files:
source_file = os.path.join(source,f)
if os.path.isfile(source_file): #check if source file is a file not dir
shutil.move(source_file,dest) #move all only files (not include dir) to dest dir
for item in items:
service_title = item.title
if service_title == "Bøjninger_09_06":
try:
service_title = item.title
version = "1"
fgdb_title = service_title+version
result = item.export(fgdb_title, "File Geodatabase")
result.download(r"C:\Users\Bruger\xxx\xxx\xxx\Backup\?????) **what shal I write here in order to point to the new folder?**
result.delete()
except:
print("An error occurred downloading"+" "+service_title)```

You could try an f-string instead of a regular string, like this:
rf"C:\Users\Bruger\xxx\xxx\xxx\Backup\{dest}"

You can use something like that:
result.download("C:\Users\Bruger\xxx\xxx\xxx\Backup\{}".format(dest))

Python Sub process call with filename variable

I've got a small script with monitors when files are added or removed to a directory. The next step is for me to get the script to execute the files (windows batch files) once they’ve been added to the directory. I’m struggling to understand how to use a variable with subprocess call (if this is the best way this can be acheived). Could anyone help me please? Many thanks. Code looks like this so far ;
import sys
import time
import os
inputdir = 'c:\\test\\'
os.chdir(inputdir)
contents = os.listdir(inputdir)
count = len(inputdir)
dirmtime = os.stat(inputdir).st_mtime
while True:
newmtime = os.stat(inputdir).st_mtime
if newmtime != dirmtime:
dirmtime = newmtime
newcontents = os.listdir(inputdir)
added = set(newcontents).difference(contents)
if added:
print "These files added: %s" %(" ".join(added))
import subprocess
subprocess.call(%,shell=True)
removed = set(contents).difference(newcontents)
if removed:
print "These files removed: %s" %(" ".join(removed))
contents = newcontents
time.sleep(15)

This should do what you wanted, cleaned it up a little.
import sys
import time
import os
import subprocess
def monitor_execute(directory):
dir_contents = os.listdir(directory)
last_modified = os.stat(directory).st_mtime
while True:
time.sleep(15)
modified = os.stat(directory).st_mtime
if last_modified == modified:
continue
last_modified = modified
current_contents = os.listdir(directory)
new_files = set(current_contents).difference(dir_contents)
if new_files:
print 'Found new files: %s' % ' '.join(new_files)
for new_file in new_files:
subprocess.call(new_file, shell=True)
lost_files = set(dir_contents).difference(current_contents)
if lost_files:
print 'Lost these files: %s' % ' '.join(lost_files)
dir_contents = current_contents