We Keep Coding

How to filter shell output to only number with decimal?

I have CI/CD Config which required python version to be set to default by pyenv. I want to python2 -V output showed up with only, example, 2.7.18. But, rather than showing 2.7.18, it showing full text Python 2.7.18 .
But, when I use it in python3 python -V, it showed the correct & current python3 version (3.9.0).
I use this code to try showing numbers only : $(python -V | grep -Eo '[0-9]\.[0-9]\.[10-19]').
And to set default with pyenv : pyenv global $(python3 -V | grep -Eo '[0-9]\.[0-9]\.[10-19]') $(python -V | grep -Eo '[0-9]\.[0-9]\.[10-19]')
So pyenv $(python3 version) $(python2 version)
Here is the image :
Image of wrong output
Thanks!

A simple way would be to just replace the string Python with the emtpy string, if it exists.
Here a quick one-liner
python -V 2>&1| sed -e "s/Python//g" | xargs
That would print the python version, redirects stderr to stdout, replaces "Python" with "". Xargs without parameters returns the trimmed input string.

Here are a few more ways to get the version number:
# Print 1 word per line, the select the last word:
python -V 2>&1 | xargs -n1 | tail -n1
# Print the last word:
python -V 2>&1 | perl -lane 'print $F[-1];'
# Print the first stretch of 1 or more { digits or periods }:
python -V 2>&1 | grep -Po '[\d.]+'

Subprocess bash script: command not found [duplicate]

This question already has answers here:
Getting "command not found" error in bash script
(6 answers)
Closed 1 year ago.
The following python script:
def run_build(path):
cmd = path + '/build.sh'
p = subprocess.call(cmd)
The following bash script exec two another scripts:
#!/bin/bash
cd "${0%/*}"
echo $(./create_env.sh)
echo $(./set_webhook.sh)
echo $(docker-compose up -d --build)
create_env.sh:
#!/bin/bash
PORT=$(comm -23 <(seq 7000 8000 | sort) <(ss -tan | awk '{print $4}' | cut -d':' -f2 | grep "[0-9]\{1,5\}" | sort -u) | head -n 1)
MONGODB_PORT=$(comm -23 <(seq 27017 27100 | sort) <(ss -tan | awk '{print $4}' | cut -d':' -f2 | grep "[0-9]\{1,5\}" | sort -u) | head -n 1)
destdir=$PWD/.env
echo >> "$destdir"
echo "APP_PORT=$PORT" >> "$destdir"
echo "MONGODB_PORT=$MONGODB_PORT" >> "$destdir"
The output is:
Path: /home/navka/Environments/teststartupservicebot/build.sh
./create_env.sh: line 2: head: command not found
./create_env.sh: line 2: comm: command not found
./create_env.sh: line 2: seq: command not found
...
Where is my problem? Thanks!

I would say your first step would be to place:
echo $PATH
as the first line following the #!/bin/bash shebang line in create_env.sh, to ensure the path is set up.
Make sure it contains the directory for those executables (probably /usr/bin), which you can probably find out by executing (for example) which comm or where comm from a command line.
If it doesn't contain the relevant directory, that explains why it cannot find the executables. In that case, you will need to discover why they're not there.
Perhaps the simplest fix would be to just add something like:
PATH="${PATH}:/usr/bin"
to your environment setup script. This will ensure the path does have the relevant entry.
And, as an aside, if those lines in build.sh are meant to be cumulative (so, for example, set_workbook requires the environment changes made by create_env, you should be aware that these are currently run in sub-shells, meaning changes from one will not persist after the sub-shell exits.
That's not necessarily the case as you persist them to a file, which may be read by the subsequent steps.
If you do need the changes in the environment for subsequent steps (as opposed to a file), you will need to source them in the context of the current shell, such as with:
. ./create_env.sh
As I said, this may not be necessary but you may want to look into it, just in case.

No such file or directory in find running .sh

Running this on osx...
cd ${BUILD_DIR}/mydir && for DIR in $(find ./ '.*[^_].py' | sed 's/\/\//\//g' | awk -F "/" '{print $2}' | sort |uniq | grep -v .py); do
if [ -f $i/requirements.txt ]; then
pip install -r $i/requirements.txt -t $i/
fi
cd ${DIR} && zip -r ${DIR}.zip * > /dev/null && mv ${DIR}.zip ../../ && cd ../
done
cd ../
error:
(env) ➜ sh package_lambdas.sh find: .*[^_].py: No such file or directory
why?

find takes as an argument a list of directories to search. You provided what appears to be regular expression. Because there is no directory named (literally) .*[^_].py, find returns an error.
Below I have revised your script to correct that mistake (if I understand your intention). Because I see so many ill-written shell scripts these days, I've taken the liberty of "traditionalizing" it. Please see if you don't also find it more readable.
Changes:
use #!/bin/sh, guaranteed to be on an Unix-like system. Faster than bash, unless (like OS X) it is bash.
use lower case for variable names to distinguish from system variables (and not hide them).
eschew braces for variables (${var}); they're not needed in the simple case
do not pipe output to /usr/bin/true; route it to dev/null if that's what you mean
rm -f by definition cannot fail; if you meant || true, it's superfluous
put then and do on separate lines, easier to read, and that's how the Bourne shell language was meant to be used
Let && and || serve as line-continuation, so you can see what's happening step by step
Other changes I would suggest:
Use a subshell when changing the working directory temporarily. When it terminates, the working directory is restored automatically (retained by the parent), saving you the cd .. step, and errors.
Use set -e to cause the script to terminate on error. For expected errors, use || true explicitly.
Change grep .py to grep '\.py$', just for good measure.
To avoid Tilting Matchstick Syndrome, use something other than / as a sed substitute delimiter, e.g., sed 's://:/:g'. But sed could be avoided altogether with awk -F '/+' '{print $2}'.
Revised version:
#! /bin/sh
src_dir=lambdas
build_dir=bin
mkdir -p $build_dir/lambdas
rm -rf $build_dir/*.zip
cp -r $src_dir/* $build_dir/lambdas
#
# The sed is a bit complicated to be osx / linux cross compatible :
# ( .//run.sh vs ./run.sh
#
cd $build_dir/lambdas &&
for L in $(find . -exec grep -l '.*[^_].py' {} + |
sed 's/\/\//\//g' |
awk -F "/" '{print $2}' |
sort |
uniq |
grep -v .py)
do
if [ -f $i/requirements.txt ]
then
echo "Installing requirements"
pip install -r $i/requirements.txt -t $i/
fi
cd $L &&
zip -r $L.zip * > /dev/null &&
mv $L.zip ../../ &&
cd ../
done
cd ../

The find(1) manpage says its args are [path ...] [expression], where "expression" consists of "primaries" and "operands" (-flags). '.*[^-].py' doesn't look like any expression, so it's being interpreted as a path, and it's reporting that there is no file named '.*[^-].py' in the working directory.
Perhaps you meant:
find ./ -regex '.*[^-].py'

Bash script running python scripts

I new to bash script, and found one of the code from stackoverflow. I merge it with my script, it doesn't run python. When I try to echo, it is always going to "Good". When I tried to run ps -ef | grep runserver* this always came out and causing the python script not running.
root 1133 0.0 0.4 11988 2112 pts/0 S+ 02:58 0:00 grep --color=auto runserver.py
Here is my code:-
#!/bin/sh
SERVICE='runserver*'
if ps ax | grep -v grep | grep $SERVICE > /dev/null
then
python /var/www/html/rest/runserver.py
python /var/www/html/rest2/runserver.py
else
echo "Good"
fi

If you are more familiar with python, try this instead:
#!/usr/bin/python
import os
import sys
process = os.popen("ps aux | grep -v grep | grep WHATEVER").read().splitlines()
if len(process) == 2:
print "WHATEVER is running - nothing to do"
else:
os.system("WHATEVER &")

Is the following code what you want?
#!/bin/sh
SERVER='runserver*'
CC=`ps ax|grep -v grep|grep "$SERVER"`
if [ "$CC" = "" ]; then
python /var/www/html/rest/runserver.py
python /var/www/html/rest2/runserver.py
else
echo "good"
fi

#NickyMan, the problem is related to the logic in the shell script. Your program don't find runserver and always says "Good".
In this code, if you don't find the server, then exec runserver.
#!/bin/sh
SERVICE='runserver*'
if ps ax | grep -v grep | grep $SERVICE > /dev/null
then
echo "Good"
else
python /var/www/html/rest/runserver.py
python /var/www/html/rest2/runserver.py
fi

Open Stack endpoints API request OS X

I've got the following bash script to parse endpoints JSON:
echo curl -s -H "X-Auth-Token: my_access_token" -X GET "https://api.selvpc.ru/identity/v3/endpoints?interface=public" | python -mjson.tool | grep -Pi '^\s*"url":\s*".*",?$' | awk '{print $2}' | tr -d '"' | sed "s/[%\\\$](tenant_id)s/my_project_id/g")
But bash says:
-bash: syntax error near unexpected token `)'
My hoster says, this script works well on Linux-based OS, but no guarantee to work on OS X. What can be the syntax issue?
EDIT:
If i use the following:
curl -s -H "X-Auth-Token: my_access_token" -X GET "https://api.selvpc.ru/identity/v3/endpoints?interface=public" | python -mjson.tool
JSON parses as expected. But with grep -Pi '^\s*"url":\s*".*",?$' I guess it causes grep warning
usage: grep [-abcDEFGHhIiJLlmnOoqRSsUVvwxZ] [-A num] [-B num] [-C[num]]
[-e pattern] [-f file] [--binary-files=value] [--color=when]
[--context[=num]] [--directories=action] [--label] [--line-buffered]
[--null] [pattern] [file ...]
I guess the first problem is grep error?

As #4ae1e1 suggested, please use a JSON processor for the job. jq is great and it's worthwhile investing your time to learn it.
wget https://github.com/stedolan/jq/releases/download/jq-1.5/jq-osx-amd64
mv jq-osx-amd64 jq
chmod u+x jq
curl -s -H "X-Auth-Token: $TOKEN" https://api.selvpc.ru/identity/v3/endpoints?interface=public | \
./jq -r .endpoints[].url
That will get you a list of OpenStack API endpoints.

I think a python script using python-keystoneclient can be easier to understand and maintain