This question already has answers here:
Why subprocess.Popen doesn't work when args is sequence?
(3 answers)
Closed 6 years ago.
In my terminal if I run: echo $(pwd), I got /home/abr/workspace, but when I tried to run this script in python like this:
>>> import subprocess
>>> cmd = ['echo', '$(pwd)']
>>> subprocess.check_output(cmd, shell=True)
I get '\n'. How to fix this?
Use os package:
import os
print os.environ.get('PWD', '')
From the documentation on the subprocess module:
If args is a sequence, the first item specifies the command string,
and any additional items will be treated as additional arguments to
the shell itself.
You want:
subprocess.check_output("echo $(pwd)", shell=True)
Try this:
cmd = 'echo $(pwd)'
subprocess.check_output(cmd, shell=True)
In subprocess doc it specified that cmd should be a string when shell=True.
From the documentation:
The shell argument (which defaults to False) specifies whether to use
the shell as the program to execute. If shell is True, it is
recommended to pass args as a string rather than as a sequence.
A better way to achieve this is probably to use the os module from the python standard library, like this:
import os
print os.getcwd()
>> "/home/abr/workspace"
The getcwd() function returns a string representing the current working directory.
The command subpreocess.check_output will return the output of the command you are calling:
Example:
#echo 2
2
from python
>>>subprocess.check_output(['echo', '2'], shell=True)
>>>'2\n'
the '\n' is included because that is what the command does it prints the output sting and then puts the current on a new line.
now back to your problem; assuming you want the output of 'PWD', first of all you have to get rid of the shell. If you provide the shell argument, the command will be run in a shell environment and you won't see the returned string.
subprocess.check_output(['pwd'])
Will return the current directory + '\n'
On a personal note, I have a hard time understanding what you are trying to do, but I hope this helps solve it.
Related
So i have a code where i use the subprocess.popen method, i want to pass a string path on settings but i dont know how to do it.
there is my code :
screenCommand = rep_reference+"\osgedit\osgviewer.exe "
iterationsPath = str(path + "\sreplace.osg")
command = "{0} --screen 0 --window 0 0 1920 1080 path+\sreplace.osg {1} " .format(screenCommand, listFix[e])
process = subprocess.Popen(command)
time.sleep(3.0)
process.kill()
how to path the iterationsPath variable on my command settings ?
You pass the various arguments and options in a list along with the command.
For example, to open VSCode in the current directory, in the shell you would do:
$ code .
Using subprocess.Popen() (or subprocess.call()):
>>> import subprocess
>>> subprocess.Popen(['code', '.'])
>>>
Essentially you split the shell command by spaces.
From the docs
On POSIX, if args is a string, the string is interpreted as the name
or path of the program to execute. However, this can only be done if
not passing arguments to the program.
https://docs.python.org/3/library/subprocess.html#subprocess.Popen
My question is related to this earlier question - Python subprocess usage
I am trying to run this command using python
nccopy -k 4 "http://www.esrl.noaa.gov/psd/thredds/dodsC/Datasets/ncep.reanalysis2/pressure/air.2014.nc?air[408:603][2][20:34][26:40]" foo.nc
When I run the above command I should be able to see a file called foo.nc on my disk or a network error stating unable to access that URL or remote URL not found.
Currently the ESRL NOAA server is down - so when I run the above command I get
syntax error, unexpected $end, expecting SCAN_ATTR or SCAN_DATASET or SCAN_ERROR
context: ^
NetCDF: Access failure
Location: file nccopy.c; line 1348
I should get the same error when I run the python script
This is the code I have and I am unable to figure out exactly how to proceed further -
I tried splitting up "-k 4" into two arguments and removing the quotes and I still get this error nccopy : invalid format : 4
Results of print(sys.argv) data.py
['data.py', '-k', '4', 'http://www.esrl.noaa.gov/psd/thredds/dodsC/Datasets/ncep.reanalysis2/pressure/air.2014.nc?air[480:603][20:34][26:40]', 'foo.nc']
import numpy as np
import subprocess
import sys
url = '"http://www.esrl.noaa.gov/psd/thredds/dodsC/Datasets/ncep.reanalysis2/pressure/air.2014.nc?air[408:603][2][20:34][26:40]"'
outputFile = 'foo.nc'
arg1 = "-k 4"
arg3 = url
arg4 = outputFile
print (input)
subprocess.check_call(["nccopy",arg1,arg3,arg4])
There's two dilemmas here.
One being that subprocess processes your arguments and tries to use 4 as a separate argument.
The other being that system calls still goes under normal shell rules, meaning that parameters and commands will be parsed for metacharacters aka special characters. In this case you're wrapping [ and ].
There for you need to separate each parameters and it's value into separate objects in the parameter-list, for instance -k 4 should be ['-k', '4'] and you need to wrap parameters/values in '...' instead of "...".
Try this, shlex.split() does the grunt work for you, and i swapped the encapsulation characters around the URL:
import numpy as np
import subprocess
import sys
import shlex
url = "'http://www.esrl.noaa.gov/psd/thredds/dodsC/Datasets/ncep.reanalysis2/pressure/air.2014.nc?air[408:603][2][20:34][26:40]'"
outputFile = 'foo.nc'
command_list = shlex.split('nccopy -k 4 ' + url + ' ' + outpufFile)
print(command_list)
subprocess.check_call(command_list)
Instead of arg1 = "-k 4", use two arguments instead.
import subprocess
url = 'http://www.esrl.noaa.gov/psd/thredds/dodsC/Datasets/ncep.reanalysis2/pressure/air.2014.nc?air[408:603][2][20:34][26:40]'
outputFile = 'foo.nc'
arg1 = "-k"
arg2 = "4"
arg3 = url
arg4 = outputFile
subprocess.check_call(["nccopy", arg1, arg2, arg3, arg4])
See also here Python subprocess arguments
If you have a working shell command that runs a single program with multiple arguments and you want to parameterized it e.g., to use a variable filename instead of the hardcoded value then you could use shlex.split() to create a list of command-line arguments that you could pass to subprocess module and replace the desired argument with a variable e.g.:
>>> shell_command = "python -c 'import sys; print(sys.argv)' 1 't w o'"
>>> import shlex
>>> shlex.split(shell_command)
['python', '-c', 'import sys; print(sys.argv)', '1', 't w o']
To run the command using the same Python interpreter as the parent script, sys.executable could be used and we can pass a variable instead of '1':
#!/usr/bin/env python
import random
import sys
import subprocess
variable = random.choice('ab')
subprocess.check_call([sys.executable, '-c', 'import sys; print(sys.argv)',
variable, 't w o'])
Note:
one command-line argument per list item
no shlex.split() in the final code
there are no quotes inside 't w o' i.e., 't w o' is used instead of '"t w o"' or "'t w o'"
subprocess module does not run the shell by default and therefore you don't need to escape shell meta-characters such as a space inside the command-line arguments. And in reverse, if your command uses some shell functionality (e.g., file patterns) then either reimplement the corresponding features in Python (e.g., using glob module) or use shell=True and pass the command as a string as is. You might need pipes.quote(), to escape variable arguments in this case. Wildcard not working in subprocess call using shlex
This question already has answers here:
How do I execute a program or call a system command?
(65 answers)
Convert POSIX->WIN path, in Cygwin Python, w/o calling cygpath
(4 answers)
Closed 7 years ago.
In Perl, if I want to execute a shell command such as foo, I'll do this:
#!/usr/bin/perl
$stdout = `foo`
In Python I found this very complex solution:
#!/usr/bin/python
import subprocess
p = subprocess.Popen('foo', shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
stdout = p.stdout.readlines()
retval = p.wait()
Is there any better solution ?
Notice that I don't want to use call or os.system. I would like to place stdout on a variable
An easy way is to use sh package.
some examples:
import sh
print(sh.ls("/"))
# same thing as above
from sh import ls
print(ls("/"))
Read more of the subprocess docs. It has a lot of simplifying helper functions:
output = subprocess.check_output('foo', shell=True, stderr=subprocess.STDOUT)
You can try this
import os
print os.popen('ipconfig').read()
#'ipconfig' is an example of command
This question already has answers here:
Shell command fails from python, ok from shell
(2 answers)
Closed 9 years ago.
I'm trying to execute a shell command from within Python (2.6.4) to evaluate a simple formula before passing it as an argument to another program. My input is something simple like this:
$[2*2]
I want to evaluate that expression and get the result from within my Python script so I can use it later. Currently, I'm doing this (where token is $[2*2]):
token = subprocess.Popen(["echo", token], stdout=subprocess.PIPE).communicate()[0].strip()
I expect the output to be 4, but instead it's just giving me back my original token ($[2*2]). Obviously when I jump to the shell and run this command by hand (echo $[2*2]), I get 4 as expected.
Is there something special about how Python executes this command that I'm missing?
When you run echo $[2*2] in your shell, the shell evaluates $[2*2] and passes the results of that evaluation to the echo command. In your Python code, you are passing the $[2*2] to echo directly and hence, it is returning just that.
You can invoke a shell to evaluate your command using shell=True in subprocess.Popen:
token = subprocess.Popen(["echo " + token],
stdout=subprocess.PIPE, shell=True).communicate()[0].strip()
subprocess.Popen will automatically escape any arguments passed in so that the command is run with those literal strings, or in other words the command that is run is equivalent to echo '$[2*2]'. There are two ways to get around this:
Use shell=True:
>>> import subprocess
>>> token = '$[2*2]'
>>> subprocess.Popen("echo " + token, stdout=subprocess.PIPE, shell=True).communicate()[0].strip()
'4'
Run bash or another shell as your command:
>>> subprocess.Popen(["bash", "-c", "echo " + token], stdout=subprocess.PIPE).communicate()[0].strip()
'4'
This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
Running shell command from python and capturing the output
I want to capture the output of a command into a variable, so later that variable can be used again. I need to change this script so it does that:
#!/usr/bin/python
import os
command = raw_input("Enter command: ")
os.system(command)
If I enter "ls" when I run this script, I get this output:
Documents Downloads Music Pictures Public Templates Videos
I want to capture that string (the output of the ls command) into a variable so I can use it again later. How do I do this?
import subprocess
command = raw_input("Enter command: ")
p = subprocess.Popen(command, stdout=subprocess.PIPE)
output, error = p.communicate()
The output of the command can be captured with the subprocess module, specifically, the check_output function..
output = subprocess.check_output("ls")
See also the documentation for subprocess.Popen for the argument list that check_output takes.
This is the way I've done it in the past.
>>> import popen2
__main__:1: DeprecationWarning: The popen2 module is deprecated. Use the subprocess module.
>>> exec_cmd = popen2.popen4("echo shell test")
>>> output = exec_cmd[0].read()
>>> output
'shell test\n'