I am using an anaconda environment both for the python code and the terminal.
When I want to execute a program in the shell (Windows CMD) with the environment activated. The program ogr2ogr returns the correct output with the given parameter. The tool ogr2ogr has been installed via a conda package.
But when I execute the my python code, the ogr2ogr returns an errors output. I thought it might be to different installations used due to usage of different environments (without my knowledge), but this is ownly a guess.
The python code goes as follows:
from pathlib import Path
from subprocess import check_call, STDOUT
...
file_path = Path(file_name)
destination = str(file_path.with_suffix(".gpkg"))
command = f"ogr2ogr -f GPKG -s_srs EPSG:25833 -t_srs EPSG:25833 {destination} GMLAS:{file_name} -oo REMOVE_UNUSED_LAYERS=YES"
check_call(command, stderr=STDOUT, shell=True)
ogr2ogr translates a file into another format. Which is also done, but when I open the file, I see, it's not done 100 % correctly.
When I copy the value of the string command and copy it to the shell and execute the command the execution is done correctly!
How can I correct the behaviour of using subprocess.check_call
I have to run a few commands of WinSCP from a Python class using subprocess.
The goal is to connect a local Windows machine and a Windows server with no FTP installed and download some files. This is what I tried
python
proc = subprocess.Popen(['WinSCP.exe', '/console', '/WAIT', user:password#ip:folder , '/WAIT','get' ,'*.txt'], shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
With this I get it to open the WinSCP console and connect to the server, but it doesn't execute the get command. Is the problem because the get is executed on the Windows console and not in the WinSCP console?
I also tried replacing winscp.exe /console for winscp.com /command.
Is there any way to do this?
If you want do without generating a script file, you can use a code like this:
import subprocess
process = subprocess.Popen(
['WinSCP.com', '/ini=nul', '/command',
'open ftp://user:password#example.com', 'get *.txt', 'exit'],
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
for line in iter(process.stdout.readline, b''): # replace b'' with '' for Python 2
print(line.decode().rstrip())
The code uses:
/command switch to specify commands on WinSCP command-line;
winscp.com instead of winscp.exe, as winscp.com is a console application, so its output can be read by Python.
Though using the array for the arguments won't work, if there are spaces in command arguments (like file names). Then you will have to format the complete command-line yourself. See Python double quotes in subprocess.Popen aren't working when executing WinSCP scripting.
So when using the /script option you should specify a file containing the batch commands.
To specify all the commands on the command line as you're doing, use the /command option instead of /script.
I am attempting to scrape a terminal window of the list of fonts installed on the curent hosting server. I have written the following code:
import subprocess
cmd = 'fc-list'
output = subprocess.Popen(cmd, stdout=subprocess.PIPE ).communicate()[0]
but when i call this code, an exception is raised:
[Errno 2] No such file or directory
I can open a terminal window, and this works fine. What am i doing wrong?
You need to provide the absolute path to the executable. When you open a terminal window you then have a shell running which will search in $PATH to find the program. When you run the program directly, via subprocess, you do not have a shell to search $PATH. (note: it is possible to tell subprocess that you do want a shell, but usually this leads to security vulnerabilities)
Here is what you would want to use:
import subprocess
cmd = '/usr/local/bin/fc-list'
output = subprocess.Popen(cmd, stdout=subprocess.PIPE ).communicate()[0]
I would like to run an executable that performs some processing on a dataset located on a remote filer. As part of the design, I'd like the location of the filer to be flexible and something that's passed to my python program at runtime.
I've put together the following bit of code that illustrates my problem, but using the python command, so anyone can run this:
#!/usr/bin/env python
import os
import subprocess
def runMySubProcess(cmdstr, iwd):
p = subprocess.Popen(cmdstr,
shell=True,
cwd=iwd,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE)
(stdout, stderr) = p.communicate()
if stderr:
raise IOError, stderr
return stdout
if __name__ == '__main__':
print runMySubProcess('python -h', 'C:\\')
print runMySubProcess('python -h', '\\\\htpc\\nas')
This works great as long as iwd is on a share that's be mapped to a drive letter on the machine. But if iwd is a UNC path the subprocess.Popen() call ends up with stderr output, which in turn throws the IOError exception:
Traceback (most recent call last):
File "test.py", line 19, in <module>
print runMySubProcess('dir', '\\\\htpc\\nas')
File "test.py", line 14, in runMySubProcess
raise IOError, stderr
IOError: '\\htpc\nas'
CMD.EXE was started with the above path as the current directory.
UNC paths are not supported. Defaulting to Windows directory.
Is there a way to make this subprocess call work without resorting to parsing iwd and making a temporary drive mount on the machine that exists while the subprocess command executes? I want to avoid having to manage the creation and cleanup of drive mounts. And of course, I'd rather not have to deal with (albeit unlikely) case where all drive letters are currently in use on the machine.
The problem is not with Popen,, but with cmd.exe, which does not allow the working directory to be a UNC path. It just does not; try it. You may have better luck specifying shell=False on your Popen() call, assuming that whatever executable you're running can handle a UNC path, but of course if what you're trying to run is a command that's built in to cmd.exe you don't have a choice.
Is there a simple way to run a Python script on Windows/Linux/OS X?
On the latter two, subprocess.Popen("/the/script.py") works, but on Windows I get the following error:
Traceback (most recent call last):
File "test_functional.py", line 91, in test_functional
log = tvnamerifiy(tmp)
File "test_functional.py", line 49, in tvnamerifiy
stdout = PIPE
File "C:\Python26\lib\subprocess.py", line 595, in __init__
errread, errwrite)
File "C:\Python26\lib\subprocess.py", line 804, in _execute_child
startupinfo)
WindowsError: [Error 193] %1 is not a valid Win32 application
monkut's comment: The use case isn't clear. Why use subprocess to run a python script? Is there something preventing you from importing the script and calling the necessary function?
I was writing a quick script to test the overall functionality of a Python-command-line tool (to test it on various platforms). Basically it had to create a bunch of files in a temp folder, run the script on this and check the files were renamed correctly.
I could have imported the script and called the function, but since it relies on sys.argv and uses sys.exit(), I would have needed to do something like..
import sys
import tvnamer
sys.argv.append("-b", "/the/folder")
try:
tvnamer.main()
except BaseException, errormsg:
print type(errormsg)
Also, I wanted to capture the stdout and stderr for debugging incase something went wrong.
Of course a better way would be to write the script in more unit-testable way, but the script is basically "done" and I'm doing a final batch of testing before doing a "1.0" release (after which I'm going to do a rewrite/restructure, which will be far tidier and more testable)
Basically, it was much easier to simply run the script as a process, after finding the sys.executable variable. I would have written it as a shell-script, but that wouldn't have been cross-platform. The final script can be found here
Just found sys.executable - the full path to the current Python executable, which can be used to run the script (instead of relying on the shbang, which obviously doesn't work on Windows)
import sys
import subprocess
theproc = subprocess.Popen([sys.executable, "myscript.py"])
theproc.communicate()
How about this:
import sys
import subprocess
theproc = subprocess.Popen("myscript.py", shell = True)
theproc.communicate() # ^^^^^^^^^^^^
This tells subprocess to use the OS shell to open your script, and works on anything that you can just run in cmd.exe.
Additionally, this will search the PATH for "myscript.py" - which could be desirable.
Yes subprocess.Popen(cmd, ..., shell=True) works like a charm. On Windows the .py file extension is recognized, so Python is invoked to process it (on *NIX just the usual shebang). The path environment controls whether things are seen. So the first arg to Popen is just the name of the script.
subprocess.Popen(['myscript.py', 'arg1', ...], ..., shell=True)
It looks like windows tries to run the script using its own EXE framework rather than call it like
python /the/script.py
Try,
subprocess.Popen(["python", "/the/script.py"])
Edit: "python" would need to be on your path.
For example, to execute following with command prompt or BATCH file we can use this:
C:\Python27\python.exe "C:\Program files(x86)\dev_appserver.py" --host 0.0.0.0 --post 8080 "C:\blabla\"
Same thing to do with Python, we can do this:
subprocess.Popen(['C:/Python27/python.exe', 'C:\\Program files(x86)\\dev_appserver.py', '--host', '0.0.0.0', '--port', '8080', 'C:\\blabla'], shell=True)
or
subprocess.Popen(['C:/Python27/python.exe', 'C:/Program files(x86)/dev_appserver.py', '--host', '0.0.0.0', '--port', '8080', 'C:/blabla'], shell=True)
You are using a pathname separator which is platform dependent. Windows uses "\" and Unix uses "/".
When you are running a python script on windows in subprocess you should use python in front of the script name. Try:
process = subprocess.Popen("python /the/script.py")
Supplemental info: It is worth noting that the documentation states that you need to use shell=True if you are using a dos shell command like dir
without it you get something like this.
>>> import subprocess
>>> subprocess.run(['dir'])
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\foo\AppData\Local\Programs\Python\Python38\lib\subprocess.py", line 489, in run
with Popen(*popenargs, **kwargs) as process:
File "C:\Users\foo\AppData\Local\Programs\Python\Python38\lib\subprocess.py", line 854, in __ini
t__
self._execute_child(args, executable, preexec_fn, close_fds,
File "C:\Users\foo\AppData\Local\Programs\Python\Python38\lib\subprocess.py", line 1307, in _exe
cute_child
hp, ht, pid, tid = _winapi.CreateProcess(executable, args,
FileNotFoundError: [WinError 2] The system cannot find the file specified
>>> subprocess.run(['dir'], shell=True)
Volume in drive J is garbage
Volume Serial Number is 5EE7-B084
Also you can use path like objects for the args which is recent addition.
from pathlib import Path
subprocess.run(Path('c:/proj/myfile.bat'))
Also worth noting there is a whole set of windows specific controls that allow you to control how a process is spawned which concurrent operations can use.
So controlling subprocesses on windows is not as simple as posix style.