When I'm working on a bash script and need to write a particularly complex logic I usually fall back on using python, like this:
#!/bin/bash
function foo() {
python << END
if 1:
print "hello"
END
}
foo
How can I do the same thing from within a Makefile?
You may write a bash script containing your functions, say myscript.sh:
#!/bin/bash
foo() {
python << END
if 1:
print "hello $1"
END
}
Now here is a Makefile:
SHELL = /bin/bash
mytarget ::
#source myscript.sh ;\
foo world
Finally type in your terminal:
$ make mytarget
hello world
Some explanations on the Makefile: defining SHELL let make know which shell to run. The :: stands for phony target (and a little more); you can replace it with : for an actual target.
The key point is to run source and call the function in the same shell, that is, in the same line (since make run a different shell for each line); this is achieved by ;\ at the end of each line.
Related
I am invoking the bash script from python script.
I want the bash script to add an element to dictionary "d" in the python script
abc3.sh:
#!/bin/bash
rank=1
echo "plugin"
function reg()
{
if [ "$1" == "what" ]; then
python -c 'from framework import data;data(rank)'
echo "iamin"
else
plugin
fi
}
plugin()
{
echo "i am plugin one"
}
reg $1
python file:
import sys,os,subprocess
from collections import *
subprocess.call(["./abc3.sh what"],shell=True,executable='/bin/bash')
def data(rank,check):
d[rank]["CHECK"]=check
print d[1]["CHECK"]
If I understand correctly, you have a python script that runs a shell script, that in turn runs a new python script. And you'd want the second Python script to update a dictionnary in the first script. That will not work like that.
When you run your first python script, it will create a new python process, which will interpret each instruction from your source script.
When it reaches the instruction subprocess.call(["./abc3.sh what"],shell=True,executable='/bin/bash'), it will spawn a new shell (bash) process which will in turn interpret your shell script.
When the shell script reaches python -c <commands>, it invokes a new python process. This process is independant from the initial python process (even if you run the same script file).
Because each of theses scripts will run in a different process, they don't have access to each other data (the OS makes sure that each process is independant from each other, excepted for specific inter-process communications methods).
What you need to do: use some kind of interprocess mechanism, so that the initial python script gets data from the shell script. You may for example read data from the shell standard output, using https://docs.python.org/3/library/subprocess.html#subprocess.check_output
Let's suppose that you have a shell plugin that echoes the value:
echo $1 12
The mockup python script looks like (I'm on windows/MSYS2 BTW, hence the strange paths for a Linux user):
import subprocess
p = subprocess.Popen(args=[r'C:\msys64\usr\bin\sh.exe',"-c","C:/users/jotd/myplugin.sh myarg"],stdout=subprocess.PIPE,stderr=subprocess.PIPE)
o,e= p.communicate()
p.wait()
if len(e):
print("Warning: error found: "+e.decode())
result = o.strip()
d=dict()
d["TEST"] = result
print(d)
it prints the dictionary, proving that argument has been passed to the shell, and went back processed.
Note that stderr has been filtered out to avoid been mixed up with the results, but is printed to the console if occurs.
{'TEST': b'myarg 12'}
#!/bin/bash
#=================
function func1 {
#=================
echo "I have chickens"
func2 100
}
#================
function func2 {
#================
number=$1
python << END
print "I have", number, "chickens"
END
}
func1
In bash I want to pass func2 an argument. In func2 I want to print the argument using python.
Desired output:
I have chickens
I have 100 chickens
Regular bash string variable substitution works:
$ export n=3
$ python << END
> print "I have ${n} chickens"
> END
Output:
I have 3 chickens
Because of this, you have to be very careful if you wish to continue your (rather unusual) approach. A much more common way would be to have the Bash and Python programs in separate files and run the Python script from the Bash script with some command line arguments. You can then access them from Python as sys.argv, or, if you want to do anything more sophisticated, argparse.
This thread discusses a way of running Python code from within a Bash script.
Is there any way to do something similar from within a Perl script? i.e. is there any way to run Python code typed on a Perl script? Note that I am not asking about running a Python file from a Perl script. I am asking about running Python code directly typed within the same file that has the Perl script (in the same way that the other thread discussed how to run Perl code from which a Bash script).
Example:
# /bin/perl
use 5.010
my $some_perl_variable = 'hello';
# ... BEGIN PYTHON BLOCK ...
# We are still in the same file. But we are now running Python code
import sys;
print some_perl_variable # Notice that this is a perl variable
for r in range(3):
print r
# ... END PYTHON BLOCK ...
say "We are done with the Perl script!"
say "The output of the Python block is:"
print $output"
1;
Should print:
We are done with the Perl script!
The output of the Python block is:
hello
1
2
3
We are done with the perl script
It sounds like you would be interested in the Inline module. It allows Perl to call code in many other languages, and relies on support modules for each language.
You don't say what you want to do, but you mention Python and there is an Inline::Python.
Yes, the same technique (here-docs) can be used for Perl.
Perl in Bash:
perl <<'END' # note single quotes to avoid $variable interpolation
use 5.010;
say "hello world";
END
or
perl -E'say "hello from perl"'
Bash in Perl:
use autodie; # less error handling
open my $bash, "|-", "bash";
print $bash <<'END'; # single quotes again
echo hello from bash
END
Perl in Bash in Perl:
use autodie; # less error handling
open my $bash, "|-", "bash";
print $bash <<'END'; # single quotes again
perl <<'INNER_END'
use 5.010;
say "hello inception";
INNER_END
END
(which I ironically tested on the commandline, in another heredoc)
I've seen some shell scripts in which they pass a file by writing the contents of the file in the same shell script. For instance:
if [[ $uponly -eq 1 ]]; then
mysql $tabular -h database1 -u usertrack -pabulafia usertrack << END
select host from host_status where host like 'ld%' and status = 'up';
END
exit 0
fi
I've been able to do something similar in which I do:
python << END
print 'hello world'
END
If the name of the script is say myscript.sh then I can run it by executing sh myscript.sh. and I obtain my desired output.
Question is, can we do something similar in a Makefile? I've been looking around and they all say that I have do something like:
target:
python #<<
print 'hello world'
<<
But that doesn't work.
Here are the links where I've been looking:
http://www.opussoftware.com/tutorial/TutMakefile.htm#Response%20Files
http://www.scribd.com/doc/2369245/Makefile-Memo
You can do something like this:
define TMP_PYTHON_PROG
print 'hello'
print 'world'
endef
export TMP_PYTHON_PROG
target:
#python -c "$$TMP_PYTHON_PROG"
First, you're defining a multi line variable with define and endef. Then you need to export it to the shell otherwise it will treat each new line as a new command. Then you reinsert the shell variable using $$.
The reason your #<< thing didn't work is that it appears to be a feature of a non-standard make variant. Similarly, the define command that mVChr mentions is specific (as far as I'm aware) to GNU Make. While GNU Make is very widely distributed, this trick won't work in a BSD make, nor in a POSIX-only make.
I feel it's good, as a general principle, to keep makefiles as portable as possible; and if you're writing a Makefile in the context of an autoconf-ed system, it's more important still.
A fully portable technique for doing what you're looking for is:
target:
{ echo "print 'First line'"; echo "print 'second'"; } | python
or equivalently, if you want to lay things out a bit more tidily:
target:
{ echo "print 'First line'"; \
echo "print 'second'"; \
} | python
I updated my python interpreter, but I think the old one is still called. When I check for the version I get:
$ python -V
Python 3.0.1
But I believe the old interpreter is still being called. When I run the command:
python myProg.py
The script runs properly. But when I invoke it with the command
./myProg.py
I get the error message:
AttributeError: 'str' object has no attribute 'format'
Which apparently is due to the old interpreter being called. How can I fix this? I run Mac OS X 10.5. Has it something to do with the first line:
#!/usr/bin/python
I just started out with python and am not very familiar with interpreted languages, so I am not too sure what is going on.
According to the first line of the script, #!/usr/bin/python, you are calling the Python interpreter at /usr/bin/python (which is most likely the one that ships with Mac OS X). You have to change that path to the path where you installed your Python 3 interpreter (likely /usr/local/bin/python or /opt/local/bin/python); or you can just change that line to read #!/usr/bin/env python, which will call the python listed first in your PATH variable (which seems to be the newer version you installed).
Firstly, the recommended shebang line is:
#!/usr/bin/env python
This will make sure the python interpreter that is invoked when you ./foo.py is the same interpreter that is invoked when you invoke python from the command line.
From your description, I suspect that if you did:
which python
It would not give you /usr/bin/python. It would give you something else, which is where the python 3 interpreter lives. You can either modify your shebang line to the above, or replace the path to the python interpreter with the path returned by which.
Try which python. I will tell you which python interpreter is used in your environment.
If it is not /usr/bin/python like in the script, then your suspicion is confirmed.
It's very possibly what you suspect, that the shebang line is calling the older version. Two things you might want to check:
1) what version is the interpreter at /usr/bin/python:
/usr/bin/python -V
2) where is the python 3 interpreter you installed:
which python
If you get the correct one from the command line, then replace your shebang line with this:
#!/usr/bin/env python
Addendum: You could also replace the older version of python with a symlink to python 3, but beware that any major OS X updates (ie: 10.5.6 to 10.5.7) will likely break this:
sudo mv /usr/bin/python /usr/bin/python25
sudo ln -s /path/to/python/3/python /usr/bin/python
run 'which python' - if this gives a different answer than /usr/bin/python, change #!/usr/bin/python to have that path instead.
It may be a bit odd providing a Perl script to answer a Python question, but it works for Python just as well as it does for Perl. This is a script called 'fixin', meaning 'fix interpreter'. It changes the shebang line to the correct string for your current PATH.
#!/Users/jleffler/perl/v5.10.0/bin/perl
#
# #(#)$Id: fixin.pl,v 1.3 2003/03/11 21:20:08 jleffler Exp $
#
# FIXIN: from Programming Perl
# Usage: fixin [-s] [file ...]
# Configuration
$does_hashbang = 1; # Kernel recognises #!
$verbose = 1; # Verbose by default
# Construct list of directories to search.
#absdirs = reverse grep(m!^/!, split(/:/, $ENV{'PATH'}, 999));
# Process command line arguments
if ($ARGV[0] eq '-s')
{
shift;
$verbose = 0;
}
die "Usage: $0 [-s] [file ...]\n" unless #ARGV || !-t;
#ARGV = '-' unless #ARGV;
# Process each file.
FILE: foreach $filename (#ARGV)
{
open(IN, $filename) || ((warn "Can't process $filename: $!\n"), next);
$_ = <IN>;
next FILE unless /^#!/; # Not a hash/bang file
chop($cmd = $_);
$cmd =~ s/^#! *//;
($cmd, $arg) = split(' ', $cmd, 2);
$cmd =~ s!^.*/!!;
# Now look (in reverse) for interpreter in absolute path
$found = '';
foreach $dir (#absdirs)
{
if (-x "$dir/$cmd")
{
warn "Ignoring $found\n" if $verbose && $found;
$found = "$dir/$cmd";
}
}
# Figure out how to invoke interpreter on this machine
if ($found)
{
warn "Changing $filename to $found\n" if $verbose;
if ($does_hashbang)
{
$_ = "#!$found";
$_ .= ' ' . $arg if $arg ne '';
$_ .= "\n";
}
else
{
$_ = <<EOF;
:
eval 'exec $found $arg -S \$0 \${1+"\$#"}'
if \$running_under_some_shell;
EOF
}
}
else
{
warn "Can't find $cmd in PATH, $filename unchanged\n" if $verbose;
next FILE;
}
# Make new file if necessary
if ($filename eq '-') { select(STDOUT); }
else
{
rename($filename, "$filename.bak") ||
((warn "Can't modify $filename"), next FILE);
open(OUT, ">$filename") ||
die "Can't create new $filename: $!\n";
($def, $ino, $mode) = stat IN;
$mode = 0755 unless $dev;
chmod $mode, $filename;
select(OUT);
}
# Print the new #! line (or the equivalent) and copy the rest of the file.
print;
while (<IN>)
{
print;
}
close IN;
close OUT;
}
The code is derived from a script of the same name in the original Camel Book ('Programming Perl', first edition). This copy has been hacked a bit since then - and should be hacked some more. But I use it routinely -- indeed, I just copied it from one Mac to another, and since I've not installed Perl 5.10.0 on the second, I ran:
$ perl fixin fixin
Changing fixin to /usr/bin/perl
$
Thereby changing from the private install Perl to the standard one.
Exercise for the reader - rewrite the script in Python.