I have a PowerShell script, which has two parameters, the first one is a string, the second one is an array of string.
I would like to call this PowerShell script from my python code. How to pass the array type parameter to PowerShell?
If I write something like this:
subprocess.run(['powershell.exe', 'script.ps1', 'arg1', '#("str1", "str2")'])
Powershell think '#("str1", "str2")' is a string, not an array.
Edit
I found a workaround
subprocess.run(['powershell.exe', 'script.ps1 arg1 #("str1", "str2")'])
It doesn't look beautiful, but works. and in this way, I can't use -File after powershell.exe
Your original command does work as written (except that you must use .\script.ps1 rather than script.ps1, unless the script is in the system's path), as does the second one you added later, because it implicitly uses the PowerShell CLI's -Command parameter rather than its
-File parameter.
In short:
Passing arrays is fundamentally only supported with -Command, which interprets the subsequent arguments as PowerShell code, where the usual PowerShell syntax applies.
With -File, by contrast, all arguments after the target-script argument are passed verbatim, as strings, so there is no concept of an array.
I suggest using the following approach, for increased robustness and conceptual clarity:
subprocess.run(['powershell.exe', '-noprofile', '-c', '.\script.ps1 arg1 #("str1", "str2")'])
Note: You can omit #(...) around the array elements - #() is never needed for array literals in PowerShell.
Note:
-noprofile ensures that PowerShell doesn't load the $PROFILE file(s), which avoids potential slow-downs and side effects.
-c (-Command) makes it explicit that you're passing PowerShell code rather than a script file with literal arguments (-File)
Do note that -Command arguments are subject to additional interpretation by PowerShell, so if you pass, say, a token $foo$ you intend to be a literal, PowerShell will expand it to just $ (if no $foo variable is defined), because it expands $foo as a variable reference; passing `$foo`$ (backtick-escaping) prevents that.
Note the .\ before script.ps1: Since you're using -Command you cannot execute a script by file name only (unless the script happens to be located in a directory listed in $env:PATH); as from inside PowerShell, executing scripts from the current directory requires .\ for security reasons; by contrast, file-name-only invocation does work with -File.
The script file as well as its arguments are passed as a single argument, which reflects how PowerShell will process the command.
-Command is the default in Windows PowerShell, but no longer in PowerShell Core (pwsh.exe), which defaults to -File; it is generally a good idea to explicitly use -Command (-c) or -File (-f) to make it obvious how PowerShell will interpret the arguments.
How subprocess.run() builds the command line and how PowerShell parses it:
Your original Python command passes #("str1", "str2") as an individual argument to subprocess.run():
subprocess.run(['powershell.exe', '.\script.ps1', 'arg1', '#("str1", "str2")'])
This results in the following command line executed behind the scenes:
powershell.exe .\script.ps1 arg1 "#(\"str1\", \"str2\")"
Note how only #("str1", "str2") is double-quoted, and how the embedded " chars. are escaped as \".
As an aside: PowerShell's CLI (arguments passed to powershell.exe) uses the customary \-escaping of literal " chars.; inside PowerShell, however, it is ` (backtick) that serves as the escape character.
Your second command combines the script.ps1 and #("str1", "str2") into a single argument:
subprocess.run(['powershell.exe', '.\script.ps1 arg1 #("str1", "str2")'])
This results in the following command line:
powershell.exe ".\script.ps1 arg1 #(\"str1\", \"str2\")"
Note how the single argument passed is double-quoted as a whole.
Generally, subprocess.run() automatically encloses a given argument in "..." (double quotes) if it contains spaces.
Independently, it escapes embedded (literal) " chars. as \".
Even though these command lines are obviously different, PowerShell's (implied) -Command logic processes them the same, because it uses the following algorithm:
First, enclosing double quotes around each argument, if present, are removed.
The resulting strings, if there are multiple, are concatenated with spaces.
The resulting single string is then executed as PowerShell code.
If you apply this algorithm to either of the above command lines, PowerShell ends up executing the same code, namely:
.\script.ps1 arg1 #("str1", "str2")
Lets say your python array is arr
try to do this:
subprocess.run(['powershell.exe', 'script.ps1', 'arg1', '\"{}\"'.format(','.join(arr))])
To send array in powershell script you can send it as "item1,item2,item3"
and the function str.join allow you to get this format easly
If this doesn't work, i would try to edit the script to use the $args argument in the powershell script to change the way you using your arguments
You can use single quotes on the command line - e.g. #('str1', 'str2') or escape the double quotes with backslashes - e.g. #(\"str1\", \"str2\")
For example with this script:
script.ps1
param( [string[]] $s )
write-host $s.GetType().FullName
write-host $s.Length
write-host ($s | fl * | out-string)
You can call it from a command prompt like this:
C:\> powershell.exe .\script.ps1 #('str1', 'str2')
System.String[]
2
str1
str2
or like this:
C:> powershell.exe .\script.ps1 #(\"str1\", \"str2\")
System.String[]
2
str1
str2
You might need to apply some python escape characters to get the desired result in your code though.
rI would like to know if it is possible to capture an entered full command line with pipe or semicolon as below:
$> python foo.py arg arg | arg arg
OR
$> python foo.py arg arg ; arg arg
Today in my attempts, sys.argv is returning only what is typed in the left side of the pipe/semicolon and the second part runs as an independent command (what is understandable, but not desired :) ).
I tried the code:
if not '\'' in sys.argv or not '"' in sys.argv:
print 'foo failed'
exit
to force the commands be quoted (and maybe to force the system to see everything as a single command line), but did not work and the second part keeps being executed after the break.
Python is not given access to those parts. Those are not part of the command arguments for Python, those are input for the shell. Pipes, quoting and semicolons are part of the shell syntax, not a command line for subprocesses that the shell starts.
The shell splits out syntax you give it, then calls Python with just the arguments addressed to the python binary. You can't retrieve the whole shell commands from subprocesses, that'd be a potential security issue.
If you want to pass on information to the Python script, you must do so in the command arguments. That means that if you must include quotes in your arguments, you must first escape them at the shell level, so they are not interpreted as shell syntax, e.g.
python foo.py arg1 '|' arg2
is then available in sys.argv as
['foo.py', 'arg1', '|', 'arg2']
where the single quotes around the | tell the shell to treat that character as argument text.
You need to consult the documentation for your specific shell environment for the details on how quoting works. For example, if you use bash, read the Bash manual section on quoting.
My Shell script is executed like this from the command line
./pro.sh "Argument1"
I am calling it from my python script currently like this
subprocess.call(shlex.split('bash pro.sh "Argument1"'))
How do I pass the value of Argument1 as a variable. My argument to the script can be any string. How do I achieve this?
You can use
subprocess.Popen(["bash", "pro.sh", "Argument1"])
If your string argument is multiple words, it should work fine.
subprocess.Popen(["bash", "pro.sh", "Argument with multiple words"])
As long as the multiple words are in one string in the list passed to subprocess.Popen(), it is considered one argument in the argument list for the command.
You should not use shell=True unless you have a good reason. It can be a security problem if you aren't very careful how it is used.
use subprocess to call your shell script
subprocess.Popen(['run.sh %s %s' % (var1, var2)], shell = True)
I'm trying to create a basic function that will pass a filename and arguments to a program using call() from the subprocess module. The filename and arguments are variables. When I use call() it takes the variables but the called program reads their strings with " included.
Here's the code in question:
from subprocess import call
def mednafen():
print "Loading "+romname+"..."
call(["mednafen", args, romname])
print "Mednafen closed."
romname="kirby.zip"
args="-fs 1"
mednafen()
I expected this would execute
mednafen -fs 1 kirby.zip
but instead it appears to interpret the variable's strings as this:
mednafen "-fs 1" "kirby.zip"
Because of this, mednafen isn't able to run because it can't parse an argument that starts with ".
It works as expected if I use shell=True but that feature is apparently strongly discouraged because it's easy to exploit?
call("mednafen "+ args +" "+romname+"; exit", shell=True)
Is there a way to do this without using the shell=True format?
Well, yes. That's exactly what the documentation says it does. Create and pass a list containing the command and all arguments instead.
EDIT: The solution suggested by Jonas Wielicki is to make sure every single string that would normally be separated by spaces in shell syntax is listed as a separate item; That way call() will read them properly. shlex is unnecessary.
args = ["-fs", "1"]
call(['mednafen']+args+[rom])
My initial (less concise) solution:
shlex.split() takes the variables/strings I feed it and converts them into a list of string literals, which in turn causes the called command to parse them correctly rather than interpreting the variables as strings within quotes.
So instead of the argument being treated like "-fs 0" I'm getting -fs 0 like I originally wanted.
import shlex
call(shlex.split("mednafen "+args+" "+romname))
What are the advantages of using list over string in subprocess methods? The ones I understand so far:
Security if input comes from external sources
Portability over different operating systems
Are there any others?
In my particular case, I'm using subprocess library to run tests on a software. Input does not come from external source. Tests are run only on Linux. Therefore, I see no benefit of lists over strings.
On POSIX, list and string arguments have different meaning and are used in different contexts.
You use a string argument and shell=True to run a shell command e.g.:
from subprocess import check_output
output = check_output("dmesg | grep hda", shell=True)
A list argument is used to run a command without the shell e.g.:
from subprocess import check_call
check_call(["ls", "-l"])
One exception is that call("ls") is equivalent to call(["ls"]) (a command with no arguments).
You should use a list argument with shell=False (default) except in those cases when you need the shell so the string argument is used.
It is almost always an error to use a list argument and shell=True (the arguments are interpreted as arguments to the shell itself instead of the command in this case). Don't use it.
If your question: what are the advantages of shell=False and hence the list argument over a string argument:
you don't need to escape the arguments, no shell interpolation such as word splitting, parameter expansion, command substitution occurs: what you see is what you get
support for arguments with spaces
support for arguments with special characters such as quotes, dollar sign, etc
it is clear where arguments boundaries are. They are explicitely separated.
it is clear what program is executed: it is the first item in the list
an argument that is populated from an untrusted source won't be able to execute arbitrary commands
why run a superfluous shell process unless you need it
Sometimes, it might be more convenient/readable to specify an argument as a string in the source code; shlex.split() could be used to convert it to a list:
import shlex
from subprocess import check_call
cmd = shlex.split('/bin/vikings -input eggs.txt -output "spam spam.txt" '
'''-cmd "echo '$MONEY'"''')
check_call(cmd)
See the docs.
On Windows, the arguments are interpreted differently. The native format is a string and the passed list is converted to a string using subprocess.list2cmdline() function that may not work for all Windows programs. shell=True is only necessary to run builtin shell commands.
If list2cmdline() creates a correct command line for your executable (different programs may use different rules for interpreting the command line) then a list argument could be used for portability and to avoid escaping separate arguments manually.