I am writing a Python (2.5) script that will search through a folder, grab a specific file, and then pass the file and a command to a CMD shell in Windows. This needs to be done with the command program.exe /rptcsv <file being created> <file being used>. Additionally, the .exe HAS to be executed from the C:\Program Files (x86)\Reporter folder, hence the working directory change.
Here's my problem: I need Python to search for one specific file (fnmatch), and then pass the entire command to a shell. When this is done correctly, the process runs in the background without launching the GUI.
I've seen the posts about stdin. stdout, etc, and I don't need anything piped back- I just need Python to pass the whole thing. Right now what happens is Python launches the GUI but fails to pass the command complete with variables. Is there a way to do this?
I'm still a Python n00b, so please forgive any obvious mistakes.
MC01 = 'WKST01*.bat'
MC02 = 'WKST02*.bat'
files = os.listdir(FOLDER)
MC01_CMD = fnmatch.filter(files, MC01)
MC01_CSV = "MC01.csv"
exe = ("reporter.exe /rptcsv", MC01_CSV, MC01_CMD)
os.chdir("C:\Program Files (x86)\Reporter")
os.system("exe")
Edit: Earlier in my code, I used os.walk in the FOLDER:
print "Walking directory..."
for root, dirs, files in os.walk(FOLDER):
for file in files:
pathname = os.path.join(root, file)
Because I switched working directories, it's searching for the MC01_CMD file in C:\Program Files (x86)\Reporter and (of course) it's not there. Is there a way to join pathname and MC01_CMD without creating a new variable so it's got the correct location of MC01_CMD?
os.system takes a single string as command. In your case, that is the string "exe". You need to concatenate the filenames returnd by fnmatch.filter, using " ".join(exe) and then call os.system(command). Note the missing " in os.system(command).
For finding the file in a tree, just concatenate the (absolute path of your) base folder of your os.walk call with the basedir and the filename. You can filter on the filenames during os.walk, too.
MC01 = 'WKST01*.bat'
MC02 = 'WKST02*.bat'
def collect_files(folder, pattern):
for basedir, dirs, files in os.walk(folder):
for file in fnmatch.filter(files, pattern):
yield os.path.join(folder, basedir, file)
MC01_CMD = collect_files(FOLDER, MC01)
MC01_CSV = "MC01.csv"
command = "reporter.exe /rptcsv "+ MC01_CSV + " " + " ".join(MC01_CMD)
os.chdir("C:\Program Files (x86)\Reporter")
os.system(command)
Variables aren't be expanded in string literals. So os.system("exe") is the same as typing exe in cmd and pressing enter.
So my guess for the correct code would be:
MC01_CSV = MC01 + ".csv"
os.chdir("C:\Program Files (x86)\Reporter")
os.system("reporter.exe /rptcsv " + MC01_CSV + " " + MC01_CMD)
Related
I am working on a program that compares entries in cataloguing software (Rucio) with the files in storage. From the cataloguing, I get a path to what it believes the storage location for the file is. I then search that location for the file to see if it exists there or not. I have successfully created a bash script that performs this, but it would be a lot better if it could be redone in python.
The problem I have encountered is that python will not find the files, even when I know they exist there. I have tried stuff like
if path.exists(fulladdress):
does stuff
And providing a file I know exists it still does not find it. I suspect it has to do with the fact that the folder is huge, over 100 TB and over 287000 files, so it does not search the whole folder and therefore does not find the file.
Does there exist a python solution that works for folders that big?
Best regards
Piotr
the bash script that works is:
os.system("cd; cd directory_with_files; test -e file_in_directory _exist && echo filename >> found.txt || echo filename >> not_found "
tried running this:
def findfile(name, path):
for dirpath, dirname, filename in os.walk(path):
if name in filename:
return os.path.join(dirpath, name)
def compere_checksum(not_missing_files):
not_missing_files_file = open(not_missing_files, 'r')
lines_not_missing_files_file = not_missing_files_file.readlines()
#Extract a list of fiels i know exist
for line in lines_not_missing_files_file:
line.replace(' ','')
line_list=line.split(",")
address=line_list[0].replace("LUND: file://", "")
#address= path to the folder
fille=address[address.rindex('/')+1:]
#fille the mane of the file
address=address.replace(fille,"")
#search for the file using bash
os.system("test -e {} && echo Found {}".format(line_list[0],fille))
#search for the file using python function abovea
filepath=findfile(address,fille)
print(filepath)
address is something along the lines of "/projects/dir/dir/dir/dir/dir/mc20/v12/4.0GeV/v2.2.1-3e/"
and fille is looks like this "mc_v12-4GeV-3e-inclusive_run1310195_t1601591250.root"
The script returns:
Found mc_v12-4GeV-3e-inclusive_run1310220_t1601591602.root
None
Found mc_v12-4GeV-3e-inclusive_run1310246_t1601592829.root
None
Found mc_v12-4GeV-3e-inclusive_run1310247_t1601591229.root
None
Found mc_v12-4GeV-3e-inclusive_run1310248_t1601591216.root
None
Found mc_v12-4GeV-3e-inclusive_run1310249_t1601591416.root
None
Found mc_v12-4GeV-3e-inclusive_run1310250_t1601591472.root
None
so the bash script finds it but the python does not
I can use:
while open(file) as f:
do stuff
Dont know why this works and not
path.exists
or
def findfile(name, path):
for dirpath, dirname, filename in os.walk(path):
if name in filename:
return os.path.join(dirpath, name)
but whatever, as long as it works it is fine.
import os
def findfile(name, path):
for dirpath, dirname, filename in os.walk(path):
if name in filename:
return os.path.join(dirpath, name)
filepath = findfile("file2.txt", "/")
print(filepath)
Attempting to write a function that walks a file system and returns the absolute path and filename for use in another function.
Example "/testdir/folderA/222/filename.ext".
Having tried multiple versions of this I cannot seem to get it to work properly.
filesCheck=[]
def findFiles(filepath):
files=[]
for root, dirs, files in os.walk(filepath):
for file in files:
currentFile = os.path.realpath(file)
print (currentFile)
if os.path.exists(currentFile):
files.append(currentFile)
return files
filesCheck = findFiles(/testdir)
This returns
"filename.ext" (only one).
Substitute in currentFile = os.path.join(root, file) for os.path.realpath(file) and it goes into a loop in the first directory. Tried os.path.join(dir, file) and it fails as one of my folders is named 222.
I have gone round in circles and get somewhat close but haven't been able to get it to work.
Running on Linux with Python 3.6
There's a several things wrong with your code.
There are multiple values are being assigned to the variable name files.
You're not adding the root directory to each filename os.walk() returns which can be done with os.path.join().
You're not passing a string to the findFiles() function.
If you fix those things there's no longer a need to call os.path.exists() because you can be sure it does.
Here's a working version:
import os
def findFiles(filepath):
found = []
for root, dirs, files in os.walk(filepath):
for file in files:
currentFile = os.path.realpath(os.path.join(root, file))
found.append(currentFile)
return found
filesCheck = findFiles('/testdir')
print(filesCheck)
Hi I think this is what you need. Perhaps you could give it a try :)
from os import walk
path = "C:/Users/SK/Desktop/New folder"
files = []
for (directoryPath, directoryNames, allFiles) in walk(path):
for file in allFiles:
files.append([file, f"{directoryPath}/{file}"])
print(files)
Output:
[ ['index.html', 'C:/Users/SK/Desktop/New folder/index.html'], ['test.py', 'C:/Users/SK/Desktop/New folder/test.py'] ]
I've tried to write some code which will rename some files in a folder - essentially, they're listed as xxx_(a).bmp whereas they need to be xxx_a.bmp, where a runs from 1 to 2000.
I've used the inbuilt os.rename function to essentially swap them inside of a loop to get the right numbers, but this gives me FileNotFoundError [WinError2] the system cannot find the file specified Z:/AAA/BBB/xxx_(1).bmp' -> 'Z:/AAA/BBB/xxx_1.bmp'.
I've included the code I've written below if anyone could point me in the right direction. I've checked that I'm working in the right directory and it gives me the directory I'm expecting so I'm not sure why it can't find the files.
import os
n = 2000
folder = r"Z:/AAA/BBB/"
os.chdir(folder)
saved_path = os.getcwd()
print("CWD is" + saved_path)
for i in range(1,n):
old_file = os.path.join(folder, "xxx_(" + str(i) + ").bmp")
new_file = os.path.join(folder, "xxx_" +str(i)+ ".bmp")
os.rename(old_file, new_file)
print('renamed files')
The problem is os.rename doesn't create a new directory if the new name is a filename in a directory that does not currently exist.
In order to create the directory first, you can do the following in Python3:
os.makedirs(dirname, exist_ok=True)
In this case dirname can contain created or not-yet-created subdirectories.
As an alternative, one may use os.renames, which handles new and intermediate directories.
Try iterating files inside the directory and processing the files that meet your criteria.
from pathlib import Path
import re
folder = Path("Z:/AAA/BBB/")
for f in folder.iterdir():
if '(' in f.name:
new_name = f.stem.replace('(', '').replace(')', '')
# using regex
# new_name = re.sub('\(([^)]+)\)', r'\1', f.stem)
extension = f.suffix
new_path = f.with_name(new_name + extension)
f.rename(new_path)
I've created a simple script to rename my media files that have lots of weird periods and stuff in them that I have obtained and want to organize further. My script kinda works, and I will be editing it to edit the filenames further but my os.rename line throws this error:
[Windows Error: Error 2: The system cannot find the file specified.]
import os
for filename in os.listdir(directory):
fcount = filename.count('.') - 1 #to keep the period for the file extension
newname = filename.replace('.', ' ', fcount)
os.rename(filename, newname)
Does anyone know why this might be? I have a feeling that it doesn't like me trying to rename the file without including the file path?
try
os.rename(filename, directory + '/' + newname);
Triton Man has already answered your question. If his answer doesn't work I would try using absolute paths instead of relative paths.
I've done something similar before, but in order to keep any name clashes from happening I temporarily moved all the files to a subfolder. The entire process happened so fast that in Windows Explorer I never saw the subfolder get created.
Anyhow if you're interested in looking at my script It's shown below. You run the script on the command line and you should pass in as a command-line argument the directory of the jpg files you want renamed.
Here's a script I used to rename .jpg files to multiples of 10. It might be useful to look at.
'''renames pictures to multiples of ten'''
import sys, os
debug=False
try:
path = sys.argv[1]
except IndexError:
path = os.getcwd()
def toint(string):
'''changes a string to a numerical representation
string must only characters with an ordianal value between 0 and 899'''
string = str(string)
ret=''
for i in string:
ret += str(ord(i)+100) #we add 101 to make all the numbers 3 digits making it easy to seperate the numbers back out when we need to undo this operation
assert len(ret) == 3 * len(string), 'recieved an invalid character. Characters must have a ordinal value between 0-899'
return int(ret)
def compare_key(file):
file = file.lower().replace('.jpg', '').replace('dscf', '')
try:
return int(file)
except ValueError:
return toint(file)
#files are temporarily placed in a folder
#to prevent clashing filenames
i = 0
files = os.listdir(path)
files = (f for f in files if f.lower().endswith('.jpg'))
files = sorted(files, key=compare_key)
for file in files:
i += 10
if debug: print('renaming %s to %s.jpg' % (file, i))
os.renames(file, 'renaming/%s.jpg' % i)
for root, __, files in os.walk(path + '/renaming'):
for file in files:
if debug: print('moving %s to %s' % (root+'/'+file, path+'/'+file))
os.renames(root+'/'+file, path+'/'+file)
Edit: I got rid of all the jpg fluff. You could use this code to rename your files. Just change the rename_file function to get rid of the extra dots. I haven't tested this code so there is a possibility that it might not work.
import sys, os
path = sys.argv[1]
def rename_file(file):
return file
#files are temporarily placed in a folder
#to prevent clashing filenames
files = os.listdir(path)
for file in files:
os.renames(file, 'renaming/' + rename_file(file))
for root, __, files in os.walk(path + '/renaming'):
for file in files:
os.renames(root+'/'+file, path+'/'+file)
Looks like I just needed to set the default directory and it worked just fine.
folder = r"blah\blah\blah"
os.chdir(folder)
for filename in os.listdir(folder):
fcount = filename.count('.') - 1
newname = filename.replace('.', ' ', fcount)
os.rename(filename, newname)
I have written a script to move video files from one directory to another, it will also search sub directories using os.walk. however if the script finds a video file it will only move the file and not the containing folder. i have added an if statement to check if the containing folder is different to the original search folder.
i cant find the code to actually move(or rename?) the folder and file to a different directory. i have read/watch a lot on moving files and there is a lot of information on that, but i cant find anything for moving folders.
i have tried using shutil.move and os.rename and i get an error both times. when i try and search for the problem i get a lot of results about how to move files, or how to change the current working directory of python.
any advice(even how to phrase the google search to accuratly describe how to find a tutorial on the subject) would be really appreciated. it's my first real world python program and ive learnt a lot but this last step is wearing me down!
EDIT: when trying to use os.rename(src_file, dst_file) i get the error WindowsError: error 3 The system cannot find the path specified.
when trying shutil.move(src_file, dst_file) i get ioerror errno 2 no such file or directory "H:\\Moviesfrom download...\OneOfTheVideoFilesNotInParentFolder ie the folder and the file needs to move.
thanks.
ps like i said it's my first script outside of code academy so any random suggestions would also be appreciated.
import os
import shutil
import time
movietypes = ('.3gp', '.wmv', '.asf', '.avi', '.flv', '.mov', '.mp4', '.ogm', '.mkv',
'. mpg', '.mpg', '.nsc', '.nsv', '.nut', '.a52', '.tta', '.wav', '.ram', '.asf',
'.wmv', '. ogg', '.mka', '.vid', '.lac', '.aac', '.dts', '.tac',
'.dts', '.mbv')
filewrite = open('H:\\Movies from download folder\\Logs\\logstest.txt', 'w')
dir_src = "C:\\Users\\Jeremy\\Downloads\\"
dir_dst = "H:\\Movies from download folder\\"
for root, dirs, files in os.walk(dir_src):
for file in files:
if file.endswith(movietypes) == True:
filestr = str(file)
locationoffoundfile = os.path.realpath(os.path.join(root,filestr))
folderitwasin = locationoffoundfile.replace(dir_src,'')
folderitwasin = folderitwasin.replace(filestr,'')
pathofdir = os.path.realpath(root) + "\\"
if pathofdir != dir_src:
src_file = locationoffoundfile
dst_file = dir_dst + folderitwasin + filestr
os.rename(src_file, dst_file) #****This line is the line im having issues with***
print src_file
print dst_file
filewrite.write(file + " " + "needs to have dir and file moved Moved!" + '\n')
else:
src_file = os.path.join(dir_src, file)
dst_file = os.path.join(dir_dst, file)
print src_file
print dst_file
shutil.move(src_file, dst_file)
filewrite.write(os.path.dirname(file) + '\n')
filewrite.write(file + " " + "needs to have file moved Moved!" + '\n')
filewrite.close()
Looks like you're only moving files, without doing anything about the folders. So if you try to move
C:\Users\Jeremy\Downloads\anime\pokemon.avi
to
H:\Movies from download folder\anime\pokemon.avi
it will fail because there's no anime directory on H:\ yet.
Before iterating through files, iterate through dirs to ensure that the directory exists at your destination, creating it if necessary.
for root, dirs, files in os.walk(dir_src):
for dir in dirs:
dest_dir = os.path.join(dir_dst, dir)
if not os.path.isdir(dest_dir):
os.mkdir(dest_dir)
for file in files:
#rest of code goes here as usual...
As these are MS Windows paths use forward slashes instead and declare path as a string literal; e.g.
dir_dst = r"H:/Movies from download folder/"