I have a .jar archive that loads a file and then does some things with it and writes it to the disk again.
If I call this .jar directly from the command prompt, everything works. But when I try to do it from within python, I get the following error:
Input file ("C:\xxx.txt") was not found or was not readable.
This is my python code:
import sys, os, subprocess
if os.path.isdir(sys.argv[1]):
for file in os.listdir("."):
print (" ".join(['java', '-jar', sys.argv[2], 'd', "\"" + os.path.abspath(file) + "\"", "\""+os.path.join(os.path.join(os.path.abspath(os.path.dirname(file)), "output"), file) + "\""]))
subprocess.call(['java', '-jar', sys.argv[2], 'd', "\"" + os.path.abspath(file) + "\"", "\""+os.path.join(os.path.join(os.path.abspath(os.path.dirname(file)), "output"), file) + "\""])
When I copy the printed statement into the commandline, the jar executes perfectly; everything works. I tried running cmd as an admin, but that didn't help.
The problem is the extra quotes you're adding. When you pass subprocess a list of args, it already quotes them appropriately; if you quote them yourself, it'll end up quoting your quotes, so instead of passing an argument that, when unquoted, means C:\xxx.txt, you'll be passing an argument that, when unquoted, means "C:\xxx.txt", which is not a valid pathname.
The rule of thumb for Windows* is: If you know exactly what each argument should be, pass them as a list, and don't try to quote them yourself; if you know exactly what the final command-line string should be, pass it as a string, and don't try to break it into a list of separate arguments yourself.
* Note that this is only for Windows. On POSIX, unless you're using shell=True, you should basically never use a string.
Related
I want to use metaflac (https://linux.die.net/man/1/metaflac) command from within a python script.
from subprocess import run
flac_files = "/home/fricadelle/Artist - Album (2008)/*.flac"
run(['metaflac', '--add-replay-gain', flac_files])
I get
The FLAC file could not be opened. Most likely the file does not exist
or is not readable.
if I add shell = True to the run function I'd get:
ERROR: you must specify at least one FLAC file;
metaflac cannot be used as a pipe
So what do I do wrong? Thanks!
PS: of course the command works fine in a shell:
metaflac --add-replay-gain /home/fricadelle/Artist\ -\ Album \(2008\)/*.flac
Unless you specify shell=True (and as a first approximation, you should never specify shell=True), the arguments you provide are passed as is, with no shell expansions, word-splitting or dequoting. So the filename you pass as an argument is precisely /home/fricadelle/Artist - Album (2008)/*.flac, which is not the name of any file. (That's why you don't need to add backslashes before the spaces and parentheses. If you specified shell=True -- and I repeat, you really should avoid that -- then you would need to include backslashes so that the shell doesn't split the name into several different words.)
When you type
flac_files = "/home/fricadelle/Artist - Album (2008)/*.flac unquoted in a shell, the shell will try to expand that to a list of all the files whose names match then pattern, and will then pass that list as separate arguments. Since subprocess.run doesn't do this, you will have to do it yourself, which you would normally do with glob.glob. For example,
from subprocess import run
from glob import glob
flac_files = "/home/fricadelle/Artist - Album (2008)/*.flac"
run(['metaflac', '--add-replay-gain'] + glob(flac_files))
Note: unlike the shell, glob.glob will return an empty list if the pattern matches no files. You really should check for this error rather than invoke metaflac with no filename options.
See the answer here for a better explanation.
Globbing doesn't work the way you're expecting it to here, you need to specify shell=True, but then you'll need to drop the list.
run('metaflac --add-replay-gain ' + flac_files, shell=True)
Should do the trick.
the string that contains a file looks like this in the console:
>>> target_file
'src//data//annual_filings//ABB Ltd//ABB_ar_2015.pdf'
I got the target_file from a call to os.walk
The goal is to build a command to run in subprocess.call
Something like:
from subprocess import call
cmd_ = r'qpdf-7.0.0/bin/qpdf --password=%s --decrypt %s %s' %('', target_file, target_file)
call([cmd_])
I tried different variations, setting shell to either True or False.
Replacing the // with /,\ etc.
The issue seems to be with the space in the folder (I can not change the folder name).
The python code needs to run on Windows
you have to define cmd_ as a list of arguments not a list with a sole string in it, or subprocess interprets the string as the command (doesn't even try to split the args):
cmd_ = ['qpdf-7.0.0/bin/qpdf','--password=%s'%'','--decrypt',target_file, target_file]
call(cmd_)
and leave the quoting to subprocess
As a side note, no need to double the slashes. It works, but that's unnecessary.
I am trying to split a file into a number of parts via a python script:
Here is my snippet:
def bashCommandFunc(commandToRun):
process = subprocess.Popen(commandToRun.split(), stdout=subprocess.PIPE)
output = process.communicate()
return output
filepath = "/Users/user/Desktop/TempDel/part-00000"
numParts = "5"
splitCommand = "split -l$((`wc -l < " + filepath + "/" + numParts + ")) " + filepath
splitCommand:
'split -l$((`wc -l < /Users/user/Desktop/TempDel/part-00000`/5)) /Users/user/Desktop/TempDel/part-00000'
If I run this command on a terminal, it splits the file as it's supposed to, but it fails for the above defined subprocess function.
I have tested the function for other generic commands and it works fine.
I believe the character " ` " (tilde) might be an issue,
What is the work around to getting this command to work?
Are there some better ways to split a file from python into "n" parts.
Thanks
You'll have to let Python run this line via a full shell, rather than trying to run it as a command. You can do that by adding shell=True option and not splitting your command. But you really shouldn't do that if any part of the command may be influenced by users (huge security risk).
You could do this in a safer way by first calling wc, getting the result and then calling split. Or even implement the whole thing in pure Python instead of calling out to other commands.
What happens now is that you're calling split with first parameter -l$((``wc, second parameter -l, etc.
I am working on a program that will find some files and provide the file information to a NSIS script. The NSIS script accepts the command line as follows
makensis.exe /DON="This is one" /DOD="c:\path1\path2 to dir\path 3" scriptfile.nsi
The values of the switches will change on each execution of the program. I have tried to get this to execute using subprocess.call and subprocess.Popen. The issue I am having has to do with quoting.
First of all the subprocess calls seem to put the entire argument statement between double quotes making NSIS see them as one argument. Second I am having some difficulty getting the individual switches properly quoted on the command line. Here is a snippet of what my program currently looks like.
subprocess.Popen([setup.profile['NSISExe'], ' /DON="' + setup.profile['DESC'] + '" /DOD="' + setup.profile['InstallDir'] + \
'" /DMT="' + app.machine_type.get() + '" /DSD="' + os.path.join(WinShellVar.LOCAL_APPDATA, 'MLC CAD', appname) + \
'" /DXV=X6 ' + setup.profile['NSISScript']])
And here is the output from NSIS
Can't open script " /DON="Mastercam X6 Standard" /DOD="C:\Users\John\Desktop" /D
MT="mill" /DSD="C:\Users\John\AppData\Local\MLC CAD\mcdeftool" /DXV=X6 bin\packa
ge.002.nsi"
As you can see I am using a mixed bag of data, getting some bits for dicts and some from class calls (be easy on me if my terms are somewhat incorrect, I have been learning python for about 4 days now, correct me please just nicely). If using this data like this is "unpythonic" let me know.
Looking forward to your input
disclaimer -- I don't use windows
I think you probably want something like:
subprocess.Popen([setup.profile['NSISExe'], '/DON=' + setup.profile['DESC'],
'/DOD=' + setup.profile['InstallDir'],
'/DMT=' + app.machine_type.get(),
'/DSD=' + os.path.join(WinShellVar.LOCAL_APPDATA, 'MLC CAD', appname),
'/DXV=X6',
setup.profile['NSISScript']])
When the shell reads the commandline, it splits on non-quoted, non-escaped whitespace. When you pass a list to Popen, it expects the list elements to be the way it would look after the shell split the arguments. The other option is to pass a string (instead of a list) exactly as you would put it into the windows shell and pass shell=True to Popen. But that method isn't preferred as it is much more vulnerable to shell-injection insecurities.
I'm trying to call 'sed' from Python and having troubles passing the command line via either subprocess.check_call() or os.system().
I'm on Windows 7, but using the 'sed' from Cygwin (it's in the path).
If I do this from the Cygwin shell, it works fine:
$ sed 's/ /\ /g' <"C:foobar" >"C:foobar.temp"
In Python, I've got the full pathname I'm working with in "name". I tried:
command = r"sed 's/ /\ /g' " + "<" '\"' + name + '\" >' '\"' + name + '.temp' + '\"'
subprocess.check_call(command, shell=True)
All the concatenation is there to make sure I have double quotes around the input and output filenames (in case there are blank spaces in the Windows file path).
I also tried it replacing the last line with:
os.system(command)
Either way, I get this error:
sed: -e expression #1, char 2: unterminated `s' command
'amp' is not recognized as an internal or external command,
operable program or batch file.
'nbsp' is not recognized as an internal or external command,
operable program or batch file.
Yet, as I said, it works OK from the console. What am I doing wrong?
The shell used by subprocess is probably not the shell you want. You can specify the shell with executable='path/to/executable'. Different shells have different quoting rules.
Even better might be to skip subprocess altogether, and write this as pure Python:
with open("c:foobar") as f_in:
with open("c:foobar.temp", "w") as f_out:
for line in f_in:
f_out.write(line.replace(' ', ' '))
I agree with Ned Batchelder's assessment, but think what you might want to consider using the following code because it likely does what you ultimately want to accomplish which can be done easily with the help of Python's fileinput module:
import fileinput
f = fileinput.input('C:foobar', inplace=1)
for line in f:
line = line.replace(' ', ' ')
print line,
f.close()
print 'done'
This will effectively update the given file in place as use of the keyword suggests. There's also an optional backup= keyword -- not used above -- which will save a copy of the original file if desired.
BTW, a word of caution about using something like C:foobar to specify the file name because on Windows it means a file of that name in whatever the current directory is on drive C:, which might not be what you want.
I think you'll find that, in Windows Python, it's not actually using the CygWin shell to run your command, it's instead using cmd.exe.
And, cmd doesn't play well with single quotes the way bash does.
You only have to do the following to confirm that:
c:\pax> echo hello >hello.txt
c:\pax> type "hello.txt"
hello
c:\pax> type 'hello.txt'
The system cannot find the file specified.
I think the best idea would be to use Python itself to process the file. The Python language is a cross-platform one which is meant to remove all those platform-specific inconsistencies, such as the one you've just found.