I'm trying to write a bash function named myrun, such that doing
myrun script.py
with a Python file:
#MYRUN:nohup python -u script.py &
import time
print 'Hello world'
time.sleep(2)
print 'Once again'
will run the script with the command specified in the first line of the file, just after #MYRUN:.
What should I insert in .bashrc to allow this? Here is what I have now:
myrun () {
[[ "$1" = "" ]] && echo "usage: myrun python_script.py" && return 0
<something with awk here or something else?>
}
A minimalist version:
$ function myrun {
[[ "$1" = "" ]] && echo "usage: myrun python_script.py" && return
local cmd=$(head -n 1 < "$1" | sed s'/# *MYRUN://')
$cmd
}
$ myrun script.py
appending output to nohup.out
$ cat nohup.out
Hello world
Once again
$
(It's not clear to me whether you're better off using eval "$cmd" or simply $cmd in the last line of the function, but if you want to include the "&" in the MYCMD directive, then $cmd is simpler.)
With some basic checking:
function myrun {
[[ "$1" = "" ]] && echo "usage: myrun python_script.py" && return
local cmd=$(head -n 1 <"$1")
if [[ $cmd =~ ^#MYRUN: ]] ; then cmd=${cmd#'#MYRUN:'}
else echo "myrun: #MYRUN: header not found" >&2 ; false; return ; fi
if [[ -z $cmd ]] ; then echo "myrun: no command specified" >&2 ; false; return; fi
$cmd # or eval "$cmd" if you prefer
}
This is unrelated to Bash. Unfortunately, the shebang line cannot portably contain more than a single argument or option group.
If your goal is to specify options to Python, the simplest thing is probably a simple sh wrapper:
#!/bin/sh
nohup python -u <<'____HERE' &
.... Your Python script here ...
____HERE
Related
I like to start a python program as a daemon. I checked a bit the internet and came up with two solutions.
The first one is to use start-stop-daemon on something like this
#!/bin/sh
start() {
start-stop-daemon -S -q -m -b -p /var/run/sample.pid --exec /usr/bin/python /bin/sample.py && echo "OK" || echo "Failed"
}
stop() {
start-stop-daemon -K -q -p /var/run/sample.pid
}
case "${1}" in
start)
start
echo "sample service started"
;;
stop)
stop
echo "sample service stopped"
;;
restart)
stop
start
echo "sample service restarted"
;;
*)
echo "usage: $0 {start|stop|restart}"
;;
esac
The second one is to use python-daemon
so I guess we have an extra python file - lets name it "sampleDaemon.py"
import daemon
import sample
with daemon.DaemonContext():
sample()
And a bash script (I am not sure about that - I guess this way):
#!/bin/sh
/usr/bin/python /bin/sampleDeamon.py
What is the best way to do it? What are the limitations? The python-deamon documentation is not that big(or I didn't found it).
When trying to activate a venv within the terminal embedded in PyCharm, I recieve the following errors:
$ . venv/Scripts/activate
bash: cygpath: command not found
bash: basename: command not found
Furthermore, after this command fails all Unix-like commands are no longer recognized:
$ ls
bash: ls: command not found
()
I have tried using both Cygwin and Git-Bash as Unix-like terminals in PyCharm. I have also added C:\cygwin64\bin to my PATH variable. Virtualenv activates fine in both the standalone windows of Cygwin and Git-Bash.
Here are my settings in PyCharm for both Cygwin and Git-Bash:
"C:\Program Files\Git\bin\sh" -lic "cd ${OLDPWD-.}; bash"
"C:\cygwin64\bin\sh" -lic "cd ${OLDPWD-.}; bash"
Not sure if this is relevant, but I noticed that the cwd of the embedded terminal is a weird path:
$ pwd
/cygdrive/c/Users/account_name/Documents/python-projects/project
I have tried cd'ing into the 'normal' path but I have the same results.
I don't know enough about shell scripting to look at the activate file and really know what's going on. But I am 99% sure the issue is coming from these lines (mainly because cypath and basename are only found on these lines)
VIRTUAL_ENV="$(if [ "$OSTYPE" "==" "cygwin" ]; then cygpath -u 'C:\Users\account_name\Documents\python-projects\project\venv'; else echo '/C/Users/account_name/Documents/python-projects/project/venv'; fi;)"
export VIRTUAL_ENV
and
if [ -z "${VIRTUAL_ENV_DISABLE_PROMPT-}" ] ; then
_OLD_VIRTUAL_PS1="$PS1"
if [ "x" != x ] ; then
PS1="$PS1"
else
PS1="(`basename \"$VIRTUAL_ENV\"`) $PS1"
fi
export PS1
fi
Then again, these are commands that come with cygwin (I think) so the Unix-like commands might be crashing out before this line.
All other commands seem to be working as intended, but everything stops working as soon as the failed activation happens. When I swap back to the Windows command prompt using cmd, my venv is immediately activated by PyCharm like normal.
Any input would be greatly helpful because I am at a loss!
Update:
Was able to get the . venv/Scripts/activate command to work. I decided to try and run the activate script piece by piece by pasting it into the terminal. Oddly enough, nothing crashed here...
Turns out for some reason if I swap the order of execution in the script, everything works.
If anyone could tell me how to avoid this, it would be greatly beneficial - at the moment I am going to have to manually adjust each activate scripts in all my virtual environments.
Working activate script:
VIRTUAL_ENV="$(if [ "$OSTYPE" "==" "cygwin" ]; then cygpath -u 'C:\Users\account_name\Documents\python-projects\project\venv'; else echo '/C/Users/account_name/Documents/python-projects/project/venv'; fi;)"
export VIRTUAL_ENV
_OLD_VIRTUAL_PATH="$PATH"
PATH="$VIRTUAL_ENV/Scripts:$PATH"
export PATH
# unset PYTHONHOME if set
if ! [ -z "${PYTHONHOME+_}" ] ; then
_OLD_VIRTUAL_PYTHONHOME="$PYTHONHOME"
unset PYTHONHOME
fi
if [ -z "${VIRTUAL_ENV_DISABLE_PROMPT-}" ] ; then
_OLD_VIRTUAL_PS1="$PS1"
if [ "x" != x ] ; then
PS1="$PS1"
else
PS1="(`basename \"$VIRTUAL_ENV\"`) $PS1"
fi
export PS1
fi
# Make sure to unalias pydoc if it's already there
alias pydoc 2>/dev/null >/dev/null && unalias pydoc
pydoc () {
python -m pydoc "$#"
}
# This should detect bash and zsh, which have a hash command that must
# be called to get it to forget past commands. Without forgetting
# past commands the $PATH changes we made may not be respected
if [ -n "${BASH-}" ] || [ -n "${ZSH_VERSION-}" ] ; then
hash -r 2>/dev/null
fi
# This file must be used with "source bin/activate" *from bash*
# you cannot run it directly
deactivate () {
unset -f pydoc >/dev/null 2>&1
# reset old environment variables
# ! [ -z ${VAR+_} ] returns true if VAR is declared at all
if ! [ -z "${_OLD_VIRTUAL_PATH+_}" ] ; then
PATH="$_OLD_VIRTUAL_PATH"
export PATH
unset _OLD_VIRTUAL_PATH
fi
if ! [ -z "${_OLD_VIRTUAL_PYTHONHOME+_}" ] ; then
PYTHONHOME="$_OLD_VIRTUAL_PYTHONHOME"
export PYTHONHOME
unset _OLD_VIRTUAL_PYTHONHOME
fi
# This should detect bash and zsh, which have a hash command that must
# be called to get it to forget past commands. Without forgetting
# past commands the $PATH changes we made may not be respected
if [ -n "${BASH-}" ] || [ -n "${ZSH_VERSION-}" ] ; then
hash -r 2>/dev/null
fi
if ! [ -z "${_OLD_VIRTUAL_PS1+_}" ] ; then
PS1="$_OLD_VIRTUAL_PS1"
export PS1
unset _OLD_VIRTUAL_PS1
fi
unset VIRTUAL_ENV
if [ ! "${1-}" = "nondestructive" ] ; then
# Self destruct!
unset -f deactivate
fi
}
# unset irrelevant variables
# deactivate nondestructive
Old activate script:
# This file must be used with "source bin/activate" *from bash*
# you cannot run it directly
deactivate () {
unset -f pydoc >/dev/null 2>&1
# reset old environment variables
# ! [ -z ${VAR+_} ] returns true if VAR is declared at all
if ! [ -z "${_OLD_VIRTUAL_PATH+_}" ] ; then
PATH="$_OLD_VIRTUAL_PATH"
export PATH
unset _OLD_VIRTUAL_PATH
fi
if ! [ -z "${_OLD_VIRTUAL_PYTHONHOME+_}" ] ; then
PYTHONHOME="$_OLD_VIRTUAL_PYTHONHOME"
export PYTHONHOME
unset _OLD_VIRTUAL_PYTHONHOME
fi
# This should detect bash and zsh, which have a hash command that must
# be called to get it to forget past commands. Without forgetting
# past commands the $PATH changes we made may not be respected
if [ -n "${BASH-}" ] || [ -n "${ZSH_VERSION-}" ] ; then
hash -r 2>/dev/null
fi
if ! [ -z "${_OLD_VIRTUAL_PS1+_}" ] ; then
PS1="$_OLD_VIRTUAL_PS1"
export PS1
unset _OLD_VIRTUAL_PS1
fi
unset VIRTUAL_ENV
if [ ! "${1-}" = "nondestructive" ] ; then
# Self destruct!
unset -f deactivate
fi
}
# unset irrelevant variables
deactivate nondestructive
VIRTUAL_ENV="$(if [ "$OSTYPE" "==" "cygwin" ]; then cygpath -u 'C:\Users\account_name\Documents\python-projects\project\venv'; else echo '/C/Users/account_name/Documents/python-projects/project/venv'; fi;)"
export VIRTUAL_ENV
_OLD_VIRTUAL_PATH="$PATH"
PATH="$VIRTUAL_ENV/Scripts:$PATH"
export PATH
# unset PYTHONHOME if set
if ! [ -z "${PYTHONHOME+_}" ] ; then
_OLD_VIRTUAL_PYTHONHOME="$PYTHONHOME"
unset PYTHONHOME
fi
if [ -z "${VIRTUAL_ENV_DISABLE_PROMPT-}" ] ; then
_OLD_VIRTUAL_PS1="$PS1"
if [ "x" != x ] ; then
PS1="$PS1"
else
PS1="(`basename \"$VIRTUAL_ENV\"`) $PS1"
fi
export PS1
fi
# Make sure to unalias pydoc if it's already there
alias pydoc 2>/dev/null >/dev/null && unalias pydoc
pydoc () {
python -m pydoc "$#"
}
# This should detect bash and zsh, which have a hash command that must
# be called to get it to forget past commands. Without forgetting
# past commands the $PATH changes we made may not be respected
if [ -n "${BASH-}" ] || [ -n "${ZSH_VERSION-}" ] ; then
hash -r 2>/dev/null
fi
VIRTUAL_ENV="$(if [ "$OSTYPE" "==" "cygwin" ]; then cygpath -u 'C:\Users\account_name\Documents\python-projects\project\venv'; else echo '/C/Users/account_name/Documents/python-projects/project/venv'; fi;)"
export VIRTUAL_ENV
_OLD_VIRTUAL_PATH="$PATH"
PATH="$VIRTUAL_ENV/Scripts:$PATH"
export PATH
# unset PYTHONHOME if set
if ! [ -z "${PYTHONHOME+_}" ] ; then
_OLD_VIRTUAL_PYTHONHOME="$PYTHONHOME"
unset PYTHONHOME
fi
if [ -z "${VIRTUAL_ENV_DISABLE_PROMPT-}" ] ; then
_OLD_VIRTUAL_PS1="$PS1"
if [ "x" != x ] ; then
PS1="$PS1"
else
PS1="(`basename \"$VIRTUAL_ENV\"`) $PS1"
fi
export PS1
fi
# Make sure to unalias pydoc if it's already there
alias pydoc 2>/dev/null >/dev/null && unalias pydoc
pydoc () {
python -m pydoc "$#"
}
# This should detect bash and zsh, which have a hash command that must
# be called to get it to forget past commands. Without forgetting
# past commands the $PATH changes we made may not be respected
if [ -n "${BASH-}" ] || [ -n "${ZSH_VERSION-}" ] ; then
hash -r 2>/dev/null
fi
# This file must be used with "source bin/activate" *from bash*
# you cannot run it directly
deactivate () {
unset -f pydoc >/dev/null 2>&1
# reset old environment variables
# ! [ -z ${VAR+_} ] returns true if VAR is declared at all
if ! [ -z "${_OLD_VIRTUAL_PATH+_}" ] ; then
PATH="$_OLD_VIRTUAL_PATH"
export PATH
unset _OLD_VIRTUAL_PATH
fi
if ! [ -z "${_OLD_VIRTUAL_PYTHONHOME+_}" ] ; then
PYTHONHOME="$_OLD_VIRTUAL_PYTHONHOME"
export PYTHONHOME
unset _OLD_VIRTUAL_PYTHONHOME
fi
# This should detect bash and zsh, which have a hash command that must
# be called to get it to forget past commands. Without forgetting
# past commands the $PATH changes we made may not be respected
if [ -n "${BASH-}" ] || [ -n "${ZSH_VERSION-}" ] ; then
hash -r 2>/dev/null
fi
if ! [ -z "${_OLD_VIRTUAL_PS1+_}" ] ; then
PS1="$_OLD_VIRTUAL_PS1"
export PS1
unset _OLD_VIRTUAL_PS1
fi
unset VIRTUAL_ENV
if [ ! "${1-}" = "nondestructive" ] ; then
# Self destruct!
unset -f deactivate
fi
}
# unset irrelevant variables
# deactivate nondestructive
I have installed odoo 8.0 (formerly openerp) source in my ubuntu 14.04
When i try this command (to run odoo server) from terminal
python odoo.py
it was run normally.
But, how can i know the return value of python command?
i want to see if the command running normally, or
it is raise an exception?
So i can make *.sh file to run that command.
I have tried this code
if [ -f odoo.py ]; then
python odoo.py
rc=$?;
if [ $rc !=0 ]; then
echo "Error when starting odoo-server,"
echo "Odoo-server not started."
else
sleep 1
echo "Odoo-server started."
fi
else
echo "Cannot found odoo.py"
echo "Odoo-server not started."
fi
but, when the command fail
it is always give me this output
Odoo-server started
Any one can help me?
Your spacing is wrong:
if [ $rc !=0 ]; then
Should be:
if [ $rc != 0 ]; then
But that is a text test, an arithmetic test would be:
if [ $rc -ne 0 ]; then
or preferably:
if (( rc != 0 )); then
However, a better way would be:
if python odoo.py
then
sleep 1
echo "Odoo-server started."
else
echo "Error when starting odoo-server," >&2
echo "Odoo-server not started." >&2
fi
I have written a bash script that consists of multiple Unix commands and Python scripts. The goal is to make a pipeline for detecting long non coding RNA from a certain input. Ultimately I would like to turn this into an 'app' and host it on some bioinformatics website. One problem I am facing is using getopt tools in bash. I couldn't find a good tutorial that I understand clearly. In addition any other comments related to the code is appreciated.
#!/bin/bash
if [ "$1" == "-h" ]
then
echo "Usage: sh $0 cuffcompare_output reference_genome blast_file"
exit
else
wget https://github.com/TransDecoder/TransDecoder/archive/2.0.1.tar.gz && tar xvf 2.0.1 && rm -r 2.0.1
makeblastdb -in $3 -dbtype nucl -out $3.blast.out
grep '"u"' $1 | \
gffread -w transcripts_u.fa -g $2 - && \
python2.7 get_gene_length_filter.py transcripts_u.fa transcripts_u_filter.fa && \
TransDecoder-2.0.1/TransDecoder.LongOrfs -t transcripts_u_filter.fa
sed 's/ .*//' transcripts_u_filter.fa | grep ">" | sed 's/>//' > transcripts_u_filter.fa.genes
cd transcripts_u_filter.fa.transdecoder_dir
sed 's/|.*//' longest_orfs.cds | grep ">" | sed 's/>//' | uniq > longest_orfs.cds.genes
grep -v -f longest_orfs.cds.genes ../transcripts_u_filter.fa.genes > longest_orfs.cds.genes.not.genes
sed 's/^/>/' longest_orfs.cds.genes.not.genes > temp && mv temp longest_orfs.cds.genes.not.genes
python ../extract_sequences.py longest_orfs.cds.genes.not.genes ../transcripts_u_filter.fa longest_orfs.cds.genes.not.genes.fa
blastn -query longest_orfs.cds.genes.not.genes.fa -db ../$3.blast.out -out longest_orfs.cds.genes.not.genes.fa.blast.out -outfmt 6
python ../filter_sequences.py longest_orfs.cds.genes.not.genes.fa.blast.out longest_orfs.cds.genes.not.genes.fa.blast.out.filtered
grep -v -f longest_orfs.cds.genes.not.genes.fa.blast.out.filtered longest_orfs.cds.genes.not.genes.fa > lincRNA_final.fa
fi
Here is how I run it:
sh test.sh cuffcompare_out_annot_no_annot.combined.gtf /mydata/db/Brapa_sequence_v1.2.fa TE_RNA_transcripts.fa
If you wanted the call to be :
test -c cuffcompare_output -r reference_genome -b blast_file
You would have something like :
#!/bin/bash
while getopts ":b:c:hr:" opt; do
case $opt in
b)
blastfile=$OPTARG
;;
c)
comparefilefile=$OPTARG
;;
h)
echo "USAGE : test -c cuffcompare_output -r reference_genome -b blast_file"
;;
r)
referencegenome=$OPTARG
;;
\?)
echo "Invalid option: -$OPTARG" >&2
exit 1
;;
:)
echo "Option -$OPTARG requires an argument." >&2
exit 1
;;
esac
done
In the string ":b:c:hr:",
- the first ":" tells getopts that we'll handle any errors,
- subsequent letters are the allowable flags. If the letter is followed by a ':', then getopts will expect that flag to take an argument, and supply that argument as $OPTARG
I have a simple python script i need to start and stop and i need to use a start.sh and stop.sh script to do it.
I have start.sh:
#!/bin/sh
script='/path/to/my/script.py'
echo 'starting $script with nohup'
nohup /usr/bin/python $script &
and stop.sh
#!/bin/sh
PID=$(ps aux | grep "/path/to/my/script.py" | awk '{print $2}')
echo "killing $PID"
kill -15 $PID
I'm mainly concerned with the stop.sh script. I think that's an appropriate way to find the pid but i wouldn't bet much on it. start.sh successfully starts it. when i run stop.sh, i can no longer find the process by "ps aux | grep 'myscript.py'" but the console outputs:
killing 25052
25058
./stop.sh: 5: kill: No such process
so it seems like it works AND gives an error of sorts with "No such process".
Is this actually an error? Am I approaching this in a sane way? Are there other things I should be paying attention to?
EDIT - I actually ended up with something like this:
start.sh
#!/bin/bash
ENVT=$1
COMPONENTS=$2
TARGETS=("/home/user/project/modules/script1.py" "/home/user/project/modules/script2.py")
for target in "${TARGETS[#]}"
do
PID=$(ps aux | grep -v grep | grep $target | awk '{print $2}')
echo $PID
if [[ -z "$PID" ]]
then
echo "starting $target with nohup for env't: $ENVT"
nohup python $target $ENVT $COMPONENTS &
fi
done
stop.sh
#!/bin/bash
ENVT=$1
TARGETS=("/home/user/project/modules/script1.py" "/home/user/project/modules/script2.py")
for target in "${TARGETS[#]}"
do
pkill -f $target
echo "killing process $target"
done
It is because ps aux |grep SOMETHING also finds the grep SOMETHING process, because SOMETHING matches. After the execution the grep is finished, so it cannot find it.
Add a line: ps aux | grep -v grep | grep YOURSCRIPT
Where -v means exclude. More in man grep.
The "correct" approach would probably be to have your script write its pid to a file in /var/run, and clear it out when you kill the script. If changing the script is not an option, have a look at start-stop-daemon.
If you want to continue with the grep-like approach, have a look at proctools. They're built in on most GNU/Linux machines and readily available on BSD including OS X:
pkill -f /path/to/my/script.py
init-type scripts are useful for this. This is very similar to one I use. You store the pid in a file, and when you want to check if it's running, look into the /proc filesystem.
#!/bin/bash
script_home=/path/to/my
script_name="$script_home/script.py"
pid_file="$script_home/script.pid"
# returns a boolean and optionally the pid
running() {
local status=false
if [[ -f $pid_file ]]; then
# check to see it corresponds to the running script
local pid=$(< "$pid_file")
local cmdline=/proc/$pid/cmdline
# you may need to adjust the regexp in the grep command
if [[ -f $cmdline ]] && grep -q "$script_name" $cmdline; then
status="true $pid"
fi
fi
echo $status
}
start() {
echo "starting $script_name"
nohup "$script_name" &
echo $! > "$pid_file"
}
stop() {
# `kill -0 pid` returns successfully if the pid is running, but does not
# actually kill it.
kill -0 $1 && kill $1
rm "$pid_file"
echo "stopped"
}
read running pid < <(running)
case $1 in
start)
if $running; then
echo "$script_name is already running with PID $pid"
else
start
fi
;;
stop)
stop $pid
;;
restart)
stop $pid
start
;;
status)
if $running; then
echo "$script_name is running with PID $pid"
else
echo "$script_name is not running"
fi
;;
*) echo "usage: $0 <start|stop|restart|status>"
exit
;;
esac
ps aux | grep "/path/to/my/script.py"
will return both the pid for the instance of script.py and also for this instance of grep. That'll probably be why you're getting a no such process: by the time you get around to killing the grep, it's already dead.
I don't have a unix box on at the moment, so i can't test this, but it should be fairly simple to get the idea.
start.sh:
if [ -e ./temp ]
then
pid=`cat temp`
echo "Process already exists; $pid"
else
script='/path/to/my/script.py'
echo 'starting $script with nohup'
nohup /usr/bin/python $script &
echo $! > temp
fi
stop.sh:
if [ -e ./temp ]
then
pid=`cat temp`
echo "killing $pid"
kill -15 $PID
rm temp
else
echo "Process not started"
fi
Try this out.