This question already has answers here:
Shell command fails from python, ok from shell
(2 answers)
Closed 9 years ago.
I'm trying to execute a shell command from within Python (2.6.4) to evaluate a simple formula before passing it as an argument to another program. My input is something simple like this:
$[2*2]
I want to evaluate that expression and get the result from within my Python script so I can use it later. Currently, I'm doing this (where token is $[2*2]):
token = subprocess.Popen(["echo", token], stdout=subprocess.PIPE).communicate()[0].strip()
I expect the output to be 4, but instead it's just giving me back my original token ($[2*2]). Obviously when I jump to the shell and run this command by hand (echo $[2*2]), I get 4 as expected.
Is there something special about how Python executes this command that I'm missing?
When you run echo $[2*2] in your shell, the shell evaluates $[2*2] and passes the results of that evaluation to the echo command. In your Python code, you are passing the $[2*2] to echo directly and hence, it is returning just that.
You can invoke a shell to evaluate your command using shell=True in subprocess.Popen:
token = subprocess.Popen(["echo " + token],
stdout=subprocess.PIPE, shell=True).communicate()[0].strip()
subprocess.Popen will automatically escape any arguments passed in so that the command is run with those literal strings, or in other words the command that is run is equivalent to echo '$[2*2]'. There are two ways to get around this:
Use shell=True:
>>> import subprocess
>>> token = '$[2*2]'
>>> subprocess.Popen("echo " + token, stdout=subprocess.PIPE, shell=True).communicate()[0].strip()
'4'
Run bash or another shell as your command:
>>> subprocess.Popen(["bash", "-c", "echo " + token], stdout=subprocess.PIPE).communicate()[0].strip()
'4'
Related
This question already has answers here:
Passing multiple variables from python script to shell script
(3 answers)
Closed 2 years ago.
I am using Python 3.6.3.
I have created a python script 'create.py'. This python script calls and runs a bash script 'verify.sh'.
The 'verify.sh' script sends an email:
#!/bin/sh
emailGroup="dummy#email.com"
echo "The variable of interest is x(insert here): $1 " | mail -s "The variable of interest is x(insert here)" ${emailGroup}
So in my python script 'x' is determined. I want to insert x into the 'verify.sh' script above so that it goes out with the email.
If that's really the full extent of your verify.sh script, you can remove it entirely.
import subprocess
sent = subprocess.run(
['mail', '-s', 'The variable of interest is x(insert here)', 'dummy#email.com'],
input='The variable of interest is x(insert here): {0}'.format(x),
text=True, check=True)
If you are on an earlier version of Python than 3.7, you will want to use universal_newlines=True instead of text=True.
You don't really need mail -s either; see e.g. How to send an email with Python?
Here's an example of how to do that.
First the Python script:
#!/usr/bin/python3
import subprocess
recipient = "me#stackoverflow.com"
subject = "Python Script Calls Bash Script"
body = "This is a test, yes it is, yes it is."
notify_script = "/home/me/Scripts/notify"
subprocess.run([notify_script, recipient, subject, body])
Second the Bash script:
#!/bin/bash
recipient="$1"
subject="$2"
body="$3"
dummy_mailer -r "$recipient" -s "$subject" -b "$body"
Any number of args (see ARG_MAX) can be sent from the Python script using subprocess.run(), simply add or remove them from the list given to run(). e.g.
subprocess.run([path_to_script, arg_1])
subprocess.run([path_to_script, arg_1, arg_2])
subprocess.run([path_to_script, arg_1, arg_2, arg_3, arg_4])
I am currently trying to utilize strace to automatically trace a programm 's system calls. To then parse and process the data obtained, I want to use a Python script.
I now wonder, how would I go about calling strace from Python?
Strace is usually called via command line and I don't know of any C library compiled from strace which I could utilize.
What is the general way to simulate an access via command line via Python?
alternatively: are there any tools similar to strace written natively in Python?
I'm thankful for any kind of help.
Nothing, as I'm clueless
You need to use the subprocess module.
It has check_output to read the output and put it in a variable, and check_call to just check the exit code.
If you want to run a shell script you can write it all in a string and set shell=True, otherwise just put the parameters as strings in a list.
import subprocess
# Single process
subprocess.check_output(['fortune', '-m', 'ciao'])
# Run it in a shell
subprocess.check_output('fortune | grep a', shell=True)
Remember that if you run stuff in a shell, if you don't escape properly and allow user data to go in your string, it's easy to make security holes. It is better to not use shell=True.
You can use commands as the following:
import commands
cmd = "strace command"
result = commands.getstatusoutput(cmd)
if result[0] == 0:
print result[1]
else:
print "Something went wrong executing your command"
result[0] contains the return code, and result[1] contains the output.
Python 2 and Python 3 (prior 3.5)
Simply execute:
subprocess.call(["strace", "command"])
Execute and return the output for processing:
output = subprocess.check_output(["strace", "command"])
Reference: https://docs.python.org/2/library/subprocess.html
Python 3.5+
output = subprocess.run(["strace", "command"], caputure_output=True)
Reference: https://docs.python.org/3.7/library/subprocess.html#subprocess.run
I have written a python code to check if file exists in Hadoop file system or not. Python function receives location passed from another function and bash code within checks if location exists.
def check_file_exists_in_hadoop(loc):
yourdir = "/somedirectory/inhadoop/"+loc
cmd = '''
hadoop fs -test -d ${yourdir};
if [ $? -eq 0 ]
then
echo "Directory exists!"
else
echo "Directory does not exists!"
fi
'''
res = subprocess.check_output(cmd, shell=True)
output = (str(res, "utf-8").strip())
print(output)
if output == "Directory exists!":
print("Yay!!!!")
else:
print("Oh no!!!!")
How to pass 'yourdir' variable inside bash portion of code.
All that playing around in shells looks awkward, why not just do:
def check_file_exists_in_hadoop(loc):
path = "/somedirectory/inhadoop/" + loc
res = subprocess.run(["hadoop", "fs", "-test", "-d", path])
return res.returncode == 0
You can execute as:
if check_file_exists_in_hadoop('foo.txt'):
print("Yay!!!!")
else:
print("Oh noes!!!!")
When you execute/run a process/program in a Unix-like system, it receives an array of arguments (exposed as e.g., sys.argv in Python). you can construct these in various ways but passing them to run gives you the most direct control. You can of course use a shell to do this, but starting up a shell just to do this seems unnecessary. Given that this argument list is just a list of strings in Python you can use normal list/string manipulations to construct whatever you need.
Using a shell can be useful, but as Gilles says you need to be careful to sanitise/escape your input — not everybody loves little bobby tables!
Pass the string as an argument to the shell. Instead of using shell=True, which runs ['sh', '-c', cmd] under the hood, invoke a shell explicitly. After the shell code, the first argument is the shell or script name (which is unused here), then the next argument is available as "$1" in the shell snippet, the next argument as "$2", etc.
cmd = '''
hadoop fs -test -d "$1";
…
'''
res = subprocess.check_output(['sh', '-c', cmd, 'sh', yourdir])
Alternatively, pass the string as an environment variable.
cmd = '''
hadoop fs -test -d "$yourdir";
…
'''
env = os.environ.copy()
env['yourdir'] = yourdir
res = subprocess.check_output(cmd, shell=True, env=env)
In the shell snippet, note the double quotes around $1 or $yourdir.
Do not interpolate the string into the shell command directly, i.e. don't use things like 'test -d {}'.format(yourdir). That doesn't work if the string contains shell special characters: it's a gaping security hole. For example if yourdir is a; rm -rf ~ then you've just kissed your data goodbye.
This question already has answers here:
Why subprocess.Popen doesn't work when args is sequence?
(3 answers)
Closed 6 years ago.
In my terminal if I run: echo $(pwd), I got /home/abr/workspace, but when I tried to run this script in python like this:
>>> import subprocess
>>> cmd = ['echo', '$(pwd)']
>>> subprocess.check_output(cmd, shell=True)
I get '\n'. How to fix this?
Use os package:
import os
print os.environ.get('PWD', '')
From the documentation on the subprocess module:
If args is a sequence, the first item specifies the command string,
and any additional items will be treated as additional arguments to
the shell itself.
You want:
subprocess.check_output("echo $(pwd)", shell=True)
Try this:
cmd = 'echo $(pwd)'
subprocess.check_output(cmd, shell=True)
In subprocess doc it specified that cmd should be a string when shell=True.
From the documentation:
The shell argument (which defaults to False) specifies whether to use
the shell as the program to execute. If shell is True, it is
recommended to pass args as a string rather than as a sequence.
A better way to achieve this is probably to use the os module from the python standard library, like this:
import os
print os.getcwd()
>> "/home/abr/workspace"
The getcwd() function returns a string representing the current working directory.
The command subpreocess.check_output will return the output of the command you are calling:
Example:
#echo 2
2
from python
>>>subprocess.check_output(['echo', '2'], shell=True)
>>>'2\n'
the '\n' is included because that is what the command does it prints the output sting and then puts the current on a new line.
now back to your problem; assuming you want the output of 'PWD', first of all you have to get rid of the shell. If you provide the shell argument, the command will be run in a shell environment and you won't see the returned string.
subprocess.check_output(['pwd'])
Will return the current directory + '\n'
On a personal note, I have a hard time understanding what you are trying to do, but I hope this helps solve it.
This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
Running shell command from python and capturing the output
I want to capture the output of a command into a variable, so later that variable can be used again. I need to change this script so it does that:
#!/usr/bin/python
import os
command = raw_input("Enter command: ")
os.system(command)
If I enter "ls" when I run this script, I get this output:
Documents Downloads Music Pictures Public Templates Videos
I want to capture that string (the output of the ls command) into a variable so I can use it again later. How do I do this?
import subprocess
command = raw_input("Enter command: ")
p = subprocess.Popen(command, stdout=subprocess.PIPE)
output, error = p.communicate()
The output of the command can be captured with the subprocess module, specifically, the check_output function..
output = subprocess.check_output("ls")
See also the documentation for subprocess.Popen for the argument list that check_output takes.
This is the way I've done it in the past.
>>> import popen2
__main__:1: DeprecationWarning: The popen2 module is deprecated. Use the subprocess module.
>>> exec_cmd = popen2.popen4("echo shell test")
>>> output = exec_cmd[0].read()
>>> output
'shell test\n'