I have been working on an issue that requires a Python script to run via the PowerShell command line. The script should pass the command to the command line and save the output. However, I'm running into an issue where some command line arguments are not recognized.
import subprocess
try:
output = subprocess.check_output\
(["Write-Output 'Hello world'"], shell=True)
# (["dir"], shell=True)
except subprocess.CalledProcessError as e:
print(e.output)
print('^Error Output^')
If I use the current command with the check_output command, I get an error stating that:
'"Write-Output 'Hello world'"' is not recognized as an internal or external command,
operable program or batch file.
If I just use the "dir" line, the script runs just fine. I'm at odds here as to why this would be happening. This is not the exact script that I'm running, but it produces the same problem on my machine. If I just type the problem command into the command line, it would output "Hello world" onto the new line just as expected.
Any insight as to why this would be happening would be greatly appreciated. If it's of relevance, I would like to not use any sort of admin privilege workaround.
I believe this is because in Windows your default Shell is not PowerShell, you could Execute a Powershell command, calling the executable by executing Powershell with the arguments you need.
For Example
POWERSHELL_COMMAND = r'C:\WINDOWS\system32\WindowsPowerShell\v1.0\powershell.exe'
subprocess.Popen([POWERSHELL_COMMAND,
'-ExecutionPolicy', 'Unrestricted',
'Write-Output', 'Hello World'],
stdout = subprocess.PIPE,
stderr = subprocess.PIPE)
if powershell is not in path you could use the full path for the executable
or if it's in path you could use just POWERSHELL_COMMAND = "powershell" as command, becareful, with the backslashed windows paths, to avoid errors you could use raw strings.
To verify that you have powershell in path, you could go to the configurations and check, or you could just open a cmd and type powershell and if It works, then you could assume that powershell is in path.
From the docs:
On Windows with shell=True, the COMSPEC environment variable specifies the default shell.
So set COMSPEC=powershell allows to make shell=True use powershell as default instead of cmd
Related
I am using an anaconda environment both for the python code and the terminal.
When I want to execute a program in the shell (Windows CMD) with the environment activated. The program ogr2ogr returns the correct output with the given parameter. The tool ogr2ogr has been installed via a conda package.
But when I execute the my python code, the ogr2ogr returns an errors output. I thought it might be to different installations used due to usage of different environments (without my knowledge), but this is ownly a guess.
The python code goes as follows:
from pathlib import Path
from subprocess import check_call, STDOUT
...
file_path = Path(file_name)
destination = str(file_path.with_suffix(".gpkg"))
command = f"ogr2ogr -f GPKG -s_srs EPSG:25833 -t_srs EPSG:25833 {destination} GMLAS:{file_name} -oo REMOVE_UNUSED_LAYERS=YES"
check_call(command, stderr=STDOUT, shell=True)
ogr2ogr translates a file into another format. Which is also done, but when I open the file, I see, it's not done 100 % correctly.
When I copy the value of the string command and copy it to the shell and execute the command the execution is done correctly!
How can I correct the behaviour of using subprocess.check_call
I have a python script which calls the Window's nbtstat command, using subprocess, in order to get the hostname of a computer from its IP address. This is done in cmd using:
nbtstat -A 172.16.137.2
Running the below script results in a WindowsError: [Error 2] The system cannot find the file specified:
import subprocess
p = subprocess.Popen(['nbtstat', '-A', '172.16.137.2'], std=subprocess.PIPE)
I also tried running nbtstat using cmd but got the error message: 'nbtstat' is not recognized as an internal or external command, operable program or batch file.:
import subprocess
p = subprocess.Popen(['cmd', '/c', 'nbtstat', '-A', '172.16.137.2'], stdout=subprocess.PIPE)
I don't understand why the nbtstat command works in the command prompt, but not within the script.
I recently suffered from the same issue. I could call 'nbtstat' from command prompt in Windows, but not through subprocess in python. I was able to check with others and found that they successfully returned information where my same code failed, telling me that it was a problem with my local installtion. I first tried the "Repair" option in the installer to no effect. Only after a complete uninstall and re-install was I able to successfully return information from subprocess calls.
I am trying to use Python's Popen to change my working directory and execute a command.
pg = subprocess.Popen("cd c:/mydirectory ; ./runExecutable.exe --help", stdout=subprocess.PIPE, stderr=subprocess.STDOUT, shell=True)
buff,buffErr = pg.communicate()
However, powershell returns "The system cannot find the path specified." The path does exist.
If I run
pg = subprocess.Popen("cd c:/mydirectory ;", stdout=subprocess.PIPE, stderr=subprocess.STDOUT, shell=True)
it returns the same thing.
However, if i run this: (without the semicolon)
pg = subprocess.Popen("cd c:/mydirectory",stdout=subprocess.PIPE, stderr=subprocess.STDOUT, shell=True)
The command returns without an error. This leads me to believe that the semicolon is issue. What is the cause for this behavior and how can I get around it?
I know I can just do c:/mydirectory/runExecutable.exe --help, but I would like to know why this is happening.
UPDATE :
I have tested passing the path to powershell as the argument for Popen's executable parameter. Just powershell.exe may not be enough. To find the true absolute path of powershell, execute where.exe powershell. Then you can pass it into Popen. Note that shell is still true. It will use the default shell but pass the command to powershell.exe
powershell = C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe
pg = subprocess.Popen("cd c:/mydirectory ; ./runExecutable.exe", stdout=subprocess.PIPE, stderr=subprocess.STDOUT, shell=True, executable=powershell)
buff,buffErr = pg.communicate()
//It works!
In your subprocess.Popen() call, shell=True means that the platform's default shell should be used.
While the Windows world is - commendably - moving from CMD (cmd.exe) to PowerShell, Python determines what shell to invoke based on the COMSPEC environment variable, which still points to cmd.exe, even in the latest W10 update that has moved toward PowerShell in terms of what the GUI offers as the default shell.
For backward compatibility, this will not change anytime soon, and will possibly never change.
Therefore, your choices are:
Use cmd syntax, as suggested in Maurice Meyer's answer.
Do not use shell = True and invoke powershell.exe explicitly - see below.
Windows only: Redefine environment variable COMSPEC before using shell = True - see below.
A simple Python example of how to invoke the powershell binary directly, with command-line switches followed by a single string containing the PowerShell source code to execute:
import subprocess
args = 'powershell', '-noprofile', '-command', 'set-location /; $pwd'
subprocess.Popen(args)
Note that I've deliberately used powershell instead of powershell.exe, because that opens up the possibility of the command working on Unix platforms too, once PowerShell Core is released.
Windows only: An example with shell = True, after redefining environment variable COMSPEC to point to PowerShell first:
import os, subprocess
os.environ["COMSPEC"] = 'powershell'
subprocess.Popen('Set-Location /; $pwd', shell=True)
Note:
COMSPEC is only consulted on Windows; on Unix platforms, the shell executable is invariably /bin/sh
As of Windows PowerShell v5.1 / PowerShell Core v6-beta.3, invoking powershell with just -c (interpreted as -Command) still loads the profiles files by default, which can have unexpected side effects (with the explicit invocation of powershell used above, -noprofile suppresses that).
Changing the default behavior to not loading the profiles is the subject of this GitHub issue, in an effort to align PowerShell's CLI with that of POSIX-like shells.
You can concat multiple commands using '&' character instead of a semicolon. Try this:
pg = subprocess.Popen("cd c:/mydirectory & ./runExecutable.exe --help", stdout=subprocess.PIPE, stderr=subprocess.STDOUT, shell=True)
buff,buffErr = pg.communicate()
I am attempting to scrape a terminal window of the list of fonts installed on the curent hosting server. I have written the following code:
import subprocess
cmd = 'fc-list'
output = subprocess.Popen(cmd, stdout=subprocess.PIPE ).communicate()[0]
but when i call this code, an exception is raised:
[Errno 2] No such file or directory
I can open a terminal window, and this works fine. What am i doing wrong?
You need to provide the absolute path to the executable. When you open a terminal window you then have a shell running which will search in $PATH to find the program. When you run the program directly, via subprocess, you do not have a shell to search $PATH. (note: it is possible to tell subprocess that you do want a shell, but usually this leads to security vulnerabilities)
Here is what you would want to use:
import subprocess
cmd = '/usr/local/bin/fc-list'
output = subprocess.Popen(cmd, stdout=subprocess.PIPE ).communicate()[0]
Running on Windows, the following Python program produces the above output (the content of the output variable)
import commands
cmd = "dir"
(output) = commands.getoutput(cmd)
print output
Very interesting and frustrating. Can someone point me to the explanation please?
See the documentation:
cmd is actually run as { cmd ; } 2>&1
Which is where the { are coming from. You should use subprocess instead.
import subprocess
p = subprocess.Popen(['dir'],stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
stdout,stderr = p.communicate()
The shell=True is required in this specific case as dir is an internal DOS command. Normally one should provide full path to the executable and keep shell disabled.