For one functionality, I need bash commands (12 lines of bash code). How can I put these 12 lines in between my Python code? At the moment I was using:
import subprocess
command = 'bash 1-line code'
subprocess.call(command, shell=True)
This worked but I was only using one line of code, now I have 12 and the '' seems not to work well...
Any suggestions?
Just extend what you were doing. Put all your bash codes in a file named, say script.sh and call it using python. You can call it as you were calling normal commands, i.e using subprocess module:
import subprocess
subprocess.call(['./script.sh'])
Related
This question already has answers here:
subprocess.Popen() error (No such file or directory) when calling command with arguments as a string
(2 answers)
Closed 2 years ago.
I am trying to write a program that opens a gnome-terminal window and executes a python file in it.
When I call the gnome-terminal subprocess with the subprocess module like this:
import subprocess
subprocess.call(['gnome-terminal', '-x', 'python3 '+filename])
I get the following error:
Failed to execute child process "python3 /home/user/Documents/test.py” (No such file or directory)
I have tried to cd to the directory /home/user/Documents/test.py first and then run the file, but it didn't work.
You're trying to execute the literal command python3 /home/user/Documents/test.py which obviously doesn't exist on your system.
When you type that line in a shell, the shell will split it on spaces and in the end it will call python3 with /home/user/Documents/test.py as argument.
When using subprocess.call, you have to do the splitting yourself.
I believe you need to pass your filename as another element in the array. I don't have gnome-terminal, but I replicated your issue with plain sh.
import subprocess
subprocess.call(['gnome-terminal', '-x', 'python3', filename])
Try this:
from os import system
system("gnome-terminal -e 'bash -c \"python3 %s\"'"%filename)
Add other commands using a semicolon:
system("gnome-terminal -e 'bash -c \"python3 %s; [second command]\"'")
Try this (i assume that python3 is set in PATH)
from subprocess import Popen
command="gnome-terminal -x python3"+filename
proc=Popen(command)
if this not works
then try to run your python file first , and see if it works or not
python filename
I executed this code in python: (test.py)
from subprocess import Popen
p = Popen("file.bat").wait()
Here is file.bat:
#echo off
start c:\python27\python.exe C:\Test\p1.py %*
start c:\python27\python.exe C:\Test\p2.py %*
pause
Here is p1.py:
This line is error
print "Hello word"
p2.py is not interesting
I want to know the exception(not only compiling error) in p1.py by running test.py?
How can I do this?
Thanks!
Here's how I got it working:
test.py
from subprocess import Popen
p = Popen(["./file.sh"]).wait()
Make sure to add the [] around file, as well as the ./. You can also add arguments, like so:
["./file.sh", "someArg"]
Note that I am not on Windows, but this fixed it on Ubuntu. Please comment if you are still having issues
EDIT:
I think the real solution is: Using subprocess to run Python script on Windows
This way you can run a python script from python, while still using Popen
The command:
make foo -f $*
Has different functionality when called from the command line versus when it is called from a python script as follows:
import os
os.system(make foo -f $*)
As stated here: http://tldp.org/LDP/abs/html/internalvariables.html#APPREF
$* in a bat file is basically all the positional parameters seen as a single word.
Python seems to be parsing it as simply "$*". Is there anyway to get around this and replicate the same functionality?
I realise I can write a .bat script and call that with python but I was hoping for something more eloquent.
As you point out, $* has no special meaning in python. The comprehension is done entirely by whatever shell you are using. If you want to pass all positional parameters passed to your script to some command, then you can try the following
import os, sys
os.system("make foo -f {}".format(" ".join(sys.argv[1:])))
Please note however that os.system is deprecated. You should probably use
import subprocess, sys
subprocess.check_call("make foo -f {}".format(" ".join(sys.argv[1:])), shell=True)
instead.
Edit
As suggested in the comments, one should avoid using shell=True whenever the command is built from "untrusted" input, such as the command line of the program. Therefore a much better alternative is to use
import subprocess, sys
subprocess.check_call(['make', 'foo', '-f'] + sys.argv[1:])
I need to run an OpenFOAM command by automatized python script.
My python code contains the lines
subprocess.Popen(['OF23'], shell=True)
subprocess.Popen(['for i in *; do surfaceConvert $i file_path/$i.stlb; done', shell=True)
where OF23 is a shell command is defined in alias as
alias OF23='export PATH=/usr/lib64/openmpi/bin/:$PATH;export LD_LIBRARY_PATH=/usr/lib64/openmpi/lib/:$LD_LIBRARY_PATH;source /opt/OpenFOAM/OpenFOAM-2.3.x/etc/bashrc'
This script runs the OpenFOAM command in terminal and the file_path defines the stl files which are converted to binary format
But when I run the script, I am getting 'OF23' is not defined.
How do I make my script to run the alias command and also perform the next OpenFOAM file conversion command
That's not going to work, even once you've resolved the alias problem. Each Python subprocess.Popen is run in a separate subshell, so the effects of executing OF23 won't persist to the second subprocess.Popen.
Here's a brief demo:
import subprocess
subprocess.Popen('export ATEST="Hello";echo "1 $ATEST"', shell=True)
subprocess.Popen('echo "2 $ATEST"', shell=True)
output
1 Hello
2
So whether you use the alias, or just execute the aliased commands directly, you'll need to combine your commands into one subprocess.Popen call.
Eg:
subprocess.Popen('''export PATH=/usr/lib64/openmpi/bin/:$PATH;
export LD_LIBRARY_PATH=/usr/lib64/openmpi/lib/:$LD_LIBRARY_PATH;
source /opt/OpenFOAM/OpenFOAM-2.3.x/etc/bashrc;
for i in *;
do surfaceConvert $i file_path/$i.stlb;
done''', shell=True)
I've used a triple-quoted string so I can insert linebreaks, to make the shell commands easier to read.
Obviously, I can't test that exact command sequence on my machine, but it should work.
You need to issue shopt -s expand_aliases to activate alias expansion. From bash(1):
Aliases are not expanded when the shell is not interactive, unless the expand_aliases shell option is set using shopt [...]
If that does not help, check if the shell executed from your Python program is actually Bash (e.g. by echoing $BASH).
If your command may use bash-isms then you could pass executable parameter otherwise /bin/sh is used. To expand aliases, you could use #Michael Jaros' suggestion:
#!/usr/bin/env python
import subprocess
subprocess.check_call("""
shopt -s expand_aliases
OF23
for i in *; do surfaceConvert $i file_path/$i.stlb; done
"""], shell=True, executable='/bin/bash')
If you already have a working bash-script then just call it as is.
Though to make it more robust and maintainable, you could convert to Python parts that provide the most benefit e.g., here's how you could emulate the for-loop:
#!/usr/bin/env python
import subprocess
for entry in os.listdir():
subprocess.check_call(['/path/to/surfaceConvert', entry,
'file_path/{entry}.stlb'.format(entry)])
It allows filenames to contain shell meta-characters such as spaces.
To configure the environment for a child process, you could use Popen's env parameter e.g., env=dict(os.environ, ENVVAR='value').
It is possible to emulate source bash command in Python but you should probably leave the parts that depend on it in bash-script.
Is there any way to call a shell script and use the functions/variable defined in the script from python?
The script is unix_shell.sh
#!/bin/bash
function foo
{
...
}
Is it possible to call this function foo from python?
Solution:
For functions: Convert Shell functions to python functions
For shell local variables(non-exported), run this command in shell, just before calling python script:
export $(set | tr '\n' ' ')
For shell global variables(exported from shell), in python, you can:
import os
print os.environ["VAR1"]
Yes, in a similar way to how you would call it from another bash script:
import subprocess
subprocess.check_output(['bash', '-c', 'source unix_shell.sh && foo'])
This can be done with subprocess. (At least this was what I was trying to do when I searched for this)
Like so:
output = subprocess.check_output(['bash', '-c', 'source utility_functions.sh; get_new_value 5'])
where utility_functions.sh looks like this:
#!/bin/bash
function get_new_value
{
let "new_value=$1 * $1"
echo $new_value
}
Here's how it looks in action...
>>> import subprocess
>>> output = subprocess.check_output(['bash', '-c', 'source utility_functions.sh; get_new_value 5'])
>>> print(output)
b'25\n'
No, that's not possible. You can execute a shell script, pass parameters on the command line, and it could print data out, which you could parse from Python.
But that's not really calling the function. That's still executing bash with options and getting a string back on stdio.
That might do what you want. But it's probably not the right way to do it. Bash can not do that many things that Python can not. Implement the function in Python instead.
With the help of above answer and this answer, I come up with this:
import subprocess
command = 'bash -c "source ~/.fileContainingTheFunction && theFunction"'
stdout = subprocess.getoutput(command)
print(stdout)
I'm using Python 3.6.5 in Ubuntu 18.04 LTS.
I do not know to much about python, but if You use export -f foo after the shell script function definition, then if You start a sub bash, the function could be called. Without export, You need to run the shell script as . script.sh inside the sub bash started in python, but it will run everything in it and will define all the functions and all variables.
You could separate each function into their own bash file. Then use Python to pass the right parameters to each separate bash file.
This may be easier than just re-writing the bash functions in Python yourself.
You can then call these functions using
import subprocess
subprocess.call(['bash', 'function1.sh'])
subprocess.call(['bash', 'function2.sh'])
# etc. etc.
You can use subprocess to pass parameters too.