I know a very similar question has already been asked but since none of the solutions posted there worked for my problem I try to make it replicable:
So I'm calling this script to merge some shapefiles (all files in one folder) like this:
shpfiles = 'shapefile_a.shp shapefile_b.shp'
subprocess.call(['python', 'shapemerger.py', '%s' % shpfiles])
I only get the Usage Instructions from the script so I cant determine what goes wrong. If i call the script directly in the terminal it works.
Any help is appreciated.
Every time a program is started, it receives a list of arguments it was invoked with. This is often called argv (v stands for vector, i.e. one-dimensional array). The program parses this list, extracts options, parameters, filenames, etc. depending on its own invocation syntax.
When working at the command line, the shell takes care of parsing the input line, starting new program or programs and passing them their argument list.
When a program is called from another program, the caller is responsible to provide the correct arguments. It could delegate this work to shell. The price for it is very high. There is substantial overhead and possibly a security risk! Avoid this approach whenever possible.
Finally to the question itself:
shpfiles = 'shapefile_a.shp shapefile_b.shp'
subprocess.call(['python', 'shapemerger.py', '%s' % shpfiles])
This will invoke python to run the script shapemerger.py with one argument shapefile_a.shp shapefile_b.shp. The script expects filenames and receives this one name. The file "shapefile_a.shp shapefile_b.shp" does not exist, but the script probably stops before attempting to access that file, because it expect 2 or more files to process.
The correct way is to pass every filename as one argument. Assuming shpfiles is a whitespace separated list:
subprocess.call(['python', 'shapemerger.py'] + shpfiles.split())
will generate a list with 4 items. It is important to understand that this approach will fail if there is a space in a filename.
Related
I want to start my python script from a batch file. It requires four arguments, that are processed in the script through sys.argv[n].
These arguments are paths, that are varying from programme call to programme call. So how do I create a batch file, where you can transfer new arguments for the programme call everytime you run the code? I have learned Python for just one semester and am stuck with this problem.
Initially I have created a batchfile to test whether the code is even trying to be executed. Of course it ran an error due to the arguments missing. Then I tried to pass the arguments through the batchfile, which didn't work and gave me the error like before 'list index out of range':
set 1= path1
set 2= path2
set 3= path3
set 4= path4
"C:\Users\Administrator\AppData\Local\Programs\Python\Python36\python.exe" "C:\repo\PythonProgramme.py" %1 %2 %3 %4
pause
I expected the code to run, but as already said, the aguments where not being passed. Can somebody help me fix this issue and maybe give me a hint, what to use for varying arguments (paths)?
You need to surround your variable name with % for it to work. You should read the documentation for set and maybe also this article about batch variables.
In the meantime the following should work.
set var1=path1
set var2=path2
set var3=path3
set var4=path4
"C:\[...]\python.exe" "C:\repo\PythonProgramme.py" %var1% %var2% %var3% %var4%
pause
The error you get is from your (unknown) python code.
While unusual you can set variable names with just numbers
BUT to reference these vars you need to enclose them in percent signs on both sides %1%
With %1..%4 you reference the command line args passed to the batch itself,
if nothing was passed, there are no args for python to process
spaces around the equal sign in a set become part of the variable name/content.
While variables names starting with a letter are preferable (as Jacques Gaudin suggests +1), you can try:
"C:\Users\Administrator\AppData\Local\Programs\Python\Python36\python.exe" ^
"C:\repo\PythonProgramme.py" %1% %2% %3% %4%
or, if the Path'es set may contain spaces:
"C:\Users\Administrator\AppData\Local\Programs\Python\Python36\python.exe" ^
"C:\repo\PythonProgramme.py" "%1%" "%2%" "%3%" "%4%"
I completely understand that I should have written the script right the first time, but the fact is I have a script that generates a data file based upon two values passed to it from the command line- like this:
[sinux1~]: ./sim_gen.py 100 .3
I need to call this script from within another script, iterating over a range of values. I searched around and after navigating through all of the "you shouldn't," I tried :
exec(open("./sim_gen.py 100 .3").read())
And this doesn't seem to work.
Help?
Let's break this down into pieces:
exec(open("./sim_gen.py 100 .3").read())
This is equivalent to:
f = open("./sim_gen.py 100 .3")
contents = f.read()
exec(contents)
That open is the same open you use for, say, reading a text file or a CSV. You're asking for a file named "sim_gen.py 100 .3" in the current directory. Do you have one? Of course not. So the open fails.
The best solution is, as you already know, to rewrite sim_gen.py so that you can import it and call a function and pass the arguments to it.
Failing that, the cleanest answer is probably to just run the Python script as a subprocess:
import subprocess
import sys
subprocess.run([sys.executable, "./sim_gen.py", "100", ".3"])
Notice that this is effectively the same thing you're doing when you run the script from your shell, so if it was OK there, it's almost surely OK here.
If you really need to exec for some reason, you will need to do something really hacky, and temporarily change argv for that script's code:
import sys
_argv = sys.argv
try:
sys.argv = ["./sim_gen.py", "100", ".3"]
with open("./sim_gen.py 100 .3"):
exec(f.read())
finally:
sys.argv = _argv
Although really, unless the point of running this is to silently modify your own module's globals or the like, you (a) almost certainly don't really need exec, and (b) want to pass an explicit globals argument even if you do really need it.
I am trying to use Python to run an executable (Windows 7) with parameters. I have been able to make the program run, but the amount of parameters I can use that will prove the Python script worked with parameters is limited. The best one is formatted like so:
-debugoutput debug.txt
I have tested this using a windows shortcut with an edited target and it works, it creates a debug output in the program directory.
Here is the code I am using:
import subprocess
args = [r"C:\Users\MyName\LevelEditor\LevelEditor.exe", "-debugoutput debug.txt"]
subprocess.call(args)
This does run the program, but the debug output is not created. I have tried putting an "r" in front of the parameter but this made no difference. I assume it is a simple formatting error but I can't find any examples to learn from that are doing the same thing.
UPDATE:
Thanks for the answers everyone, all the same, simple formatting error indeed.
In-code definition results in invocation of shell command line:
C:\Users\MyName\LevelEditor\LevelEditor.exe "-debugoutput debug.txt"
As you can see, by merging -debugoutput debug.txt to single list element, you explicitly stated that space between them shouldn't be parsed as command line argument separator.
To achieve expected behavior put file name string as separate element to argument list.
[r"C:\Users\MyName\LevelEditor\LevelEditor.exe", "-debugoutput", "debug.txt"]
As far as I know you need to split the arguments by the space, so your args would look like:
args = [r"C:\Users\MyName\LevelEditor\LevelEditor.exe", "-debugoutput", "debug.txt"]
Does that work?
I do not know if it works, but
import subprocess
args = [r"C:\Users\MyName\LevelEditor\LevelEditor.exe", "-debugoutput", "debug.txt"]
subprocess.run(args)
Following the docs
Is there a difference between os.execl() and os.execv() in python? I was using
os.execl(python, python, *sys.argv)
to restart my script (from here). But it seems to start from where the previous script left.
I want the script to start from the beginning when it restarts. Will this
os.execv(__file__,sys.argv)
do the job? command and idea from here. I couldn't find difference between them from the python help/documentation. Is there a way do clean restart?
For a little more background on what I am trying to do please see my other question
At the low level they do the same thing: they replace the running process image with a new process.
The only difference between execv and execl is the way they take arguments. execv expects a single list of arguments (the first of which should be the name of the executable), while execl expects a variable list of arguments.
Thus, in essence, execv(file, args) is exactly equivalent to execl(file, *args).
Note that sys.argv[0] is already the script name. However, this is the script name as passed into Python, and may not be the actual script name that the program is running under. To be correct and safe, your argument list passed to exec* should be
['python', __file__] + sys.argv[1:]
I have just tested a restart script with the following:
os.execl(sys.executable, 'python', __file__, *sys.argv[1:])
and this works fine. Be sure you're not ignoring or silently catching any errors from execl - if it fails to execute, you'll end up "continuing where you left off".
According to the Python documentation there's no real functional difference between execv and execl:
The “l” and “v” variants of the exec* functions differ in how command-line arguments are passed. The “l” variants are perhaps the easiest to work with if the number of parameters is fixed when the code is written; the individual parameters simply become additional parameters to the execl*() functions. The “v” variants are good when the number of parameters is variable, with the arguments being passed in a list or tuple as the args parameter. In either case, the arguments to the child process should start with the name of the command being run, but this is not enforced.
No idea why one seems to restart the script where it left off but I'd guess that that is unrelated.
I have looked at the suggested similar questions and whilst useful, do not actual match my problem and I'm still struggling.
I am using a batch file to run a series of python files, and one of these python scripts returns a variable to be used as an argument in a later script (it has to be exported to console because it is also used as a parameter in an executable.)
By batch file looks like this:
#echo off
#title AutoStats
set raw_dir ='raw_directory'
set today_dir ='today_directory'
set archive_dir='archive_directory'
set error_file='error_directory'
set DateEstate='dE'
set OTQ_File='OTQ_File'
cd C:\dev\OTQtxt
for /f "delims=" %%a in ('get_date.py') do set $date=%%a
python create_csv.py %$date% %raw_dir% %archive_dir% %error_file%
pause
The python script looks like this:
from sys import argv
date, raw_dir, today_dir, archive_dir, error_file = argv[1:]
print date
print raw_dir
print today_dir
print archive_dir
print error_file
The reason for using argv[1:] is because I don't want to use the script name as an argument
In the future it will obviously do more than this, this is just for testing whether I can get the arguments in.
The error is as the title states. This only occurs when running it from the batch file, if I run it from powershell and type in the arguments myself then it works.
I find it odd that when typing the arguments myself in powershell the script works fine, when using the parameters set in the .bat it returns an error.
Can anybody shed some light on why this might be. I've never used batch files until now so it might just be a simple mistake.
Whilst the problem pointed out by Ffisegydd was correct, the real mistake causing the problem to happen with a different number of argument was with with setting of parameters in the batch file.
for the first 2 set lines I added a space after the parameter name:
set today_dir ='today_directory'
should have been:
set today_dir='today_directory'