I have a Python3 script that needs to call a shell script with some parameters. When I call this shell script directly form the terminal - it works. The shell script call from terminal:
source $HW/scripts/gen.sh -top $TOP -proj opy_fem -clean
But when I try to call the shell script exactly the same way from Python 3 using os.system (or os.popen - same result), the shell script fails to run. Python call to the shell script:
os.system("source $HW/scripts/gen.sh -top $TOP -proj opy_fem -clean")
Get the next errors:
/project/users/alona/top_fabric_verif_env/logic/hw/scripts/gen.sh: line 18: syntax error near unexpected token `('
/project/users/alona/top_fabric_verif_env/logic/hw/scripts/gen.sh: line 18: `foreach i ( $* )'
Could you please shed light on why the same shell script fails to run from Python?
Thank you for any help
foreach is a C-shell command. csh (and derivates like tcsh) are not standard system shells in Unix/Linux.
If you need to use a specific shell, for instance the C-shell:
os.system('/bin/csh -c "put the command here"')
This will execute the /bin/csh in the standard shell, but starting two shells instead of one creates an additional overhead. A better solution is:
subprocess.run(['/bin/csh', '-c', 'put the command here'])
Note that using the shell's source ... command does not make much sense when the shell exits after the command.
I am trying to run the Perl script rtpc.pl from a Python script from the command line with two input arguments. Normally, I run the Perl script from the command line with two arguments:
perl rtpc.pl [ARG1] [ARG2]
What I want is a separate python script that I can call from the command line that I can input [ARG1] and [ARG2] into while compiling the Perl script rtpc.pl.
Something like:
python3 pyscript.py
So far, in my python script, I am using the subprocess module, but I am a little unsure how to pass arguments for my perl script.
pipe = subprocess.Popen(["perl", "rtpc.pl"], stdout=subprocess.PIPE)
How would I input the arguments needed to equivalently run the Perl script terminal command? I should also mention that the shell that I am using is tcsh.
Add them to the list you pass to subprocess.Popen().
arg1 = 'foo'
arg2 = 'bar'
pipe = subprocess.Popen(["perl", "rtpc.pl", arg1, arg2], stdout=subprocess.PIPE)
I have a shell script that calls a function from a python file that I created. That function needs two arguments, one string and one list of strings. I am trynig to figure out how can i call that script form the console. My scirpt is:
#!/bin/sh
cd $1
python -c "from my_file import my_function; my_function( \"$2\" )"
The first argument is the path of my_file.py while the second one is a python list of strings. How can I call the script from the terminal?
My script is working if I call it from a python file like (and have input two strings for example):
arguments = ["./script.sh", path, args1]
ret_val = subprocess.Popen(arguments, stdout=subprocess.PIPE)
Perhaps you can pass it as command line argument to python.
#!/bin/sh
cd $1
python -c "from my_file import my_function; import sys; my_function(sys.argv[1])" $2
Is there any way to call a shell script and use the functions/variable defined in the script from python?
The script is unix_shell.sh
#!/bin/bash
function foo
{
...
}
Is it possible to call this function foo from python?
Solution:
For functions: Convert Shell functions to python functions
For shell local variables(non-exported), run this command in shell, just before calling python script:
export $(set | tr '\n' ' ')
For shell global variables(exported from shell), in python, you can:
import os
print os.environ["VAR1"]
Yes, in a similar way to how you would call it from another bash script:
import subprocess
subprocess.check_output(['bash', '-c', 'source unix_shell.sh && foo'])
This can be done with subprocess. (At least this was what I was trying to do when I searched for this)
Like so:
output = subprocess.check_output(['bash', '-c', 'source utility_functions.sh; get_new_value 5'])
where utility_functions.sh looks like this:
#!/bin/bash
function get_new_value
{
let "new_value=$1 * $1"
echo $new_value
}
Here's how it looks in action...
>>> import subprocess
>>> output = subprocess.check_output(['bash', '-c', 'source utility_functions.sh; get_new_value 5'])
>>> print(output)
b'25\n'
No, that's not possible. You can execute a shell script, pass parameters on the command line, and it could print data out, which you could parse from Python.
But that's not really calling the function. That's still executing bash with options and getting a string back on stdio.
That might do what you want. But it's probably not the right way to do it. Bash can not do that many things that Python can not. Implement the function in Python instead.
With the help of above answer and this answer, I come up with this:
import subprocess
command = 'bash -c "source ~/.fileContainingTheFunction && theFunction"'
stdout = subprocess.getoutput(command)
print(stdout)
I'm using Python 3.6.5 in Ubuntu 18.04 LTS.
I do not know to much about python, but if You use export -f foo after the shell script function definition, then if You start a sub bash, the function could be called. Without export, You need to run the shell script as . script.sh inside the sub bash started in python, but it will run everything in it and will define all the functions and all variables.
You could separate each function into their own bash file. Then use Python to pass the right parameters to each separate bash file.
This may be easier than just re-writing the bash functions in Python yourself.
You can then call these functions using
import subprocess
subprocess.call(['bash', 'function1.sh'])
subprocess.call(['bash', 'function2.sh'])
# etc. etc.
You can use subprocess to pass parameters too.
I have a Python script script.py that has been defined as executable and which begins with the following sha-bang:
#!/usr/bin/env python -W all
But when I call it from the shell, this is what I get:
$ ./script.py
/usr/bin/env: python -W all: No such file or directory
Calling it directly works though:
$ env python -W all script.py
... some good stuff happens here
What am I doing wrong here?
On a shebang line, you only get one argument. So python -W all is being passed to env as one argument. On the command line, the shell correctly parses the arguments before invoking env.