We Keep Coding

How to filter shell output to only number with decimal?

I have CI/CD Config which required python version to be set to default by pyenv. I want to python2 -V output showed up with only, example, 2.7.18. But, rather than showing 2.7.18, it showing full text Python 2.7.18 .
But, when I use it in python3 python -V, it showed the correct & current python3 version (3.9.0).
I use this code to try showing numbers only : $(python -V | grep -Eo '[0-9]\.[0-9]\.[10-19]').
And to set default with pyenv : pyenv global $(python3 -V | grep -Eo '[0-9]\.[0-9]\.[10-19]') $(python -V | grep -Eo '[0-9]\.[0-9]\.[10-19]')
So pyenv $(python3 version) $(python2 version)
Here is the image :
Image of wrong output
Thanks!

A simple way would be to just replace the string Python with the emtpy string, if it exists.
Here a quick one-liner
python -V 2>&1| sed -e "s/Python//g" | xargs
That would print the python version, redirects stderr to stdout, replaces "Python" with "". Xargs without parameters returns the trimmed input string.

Here are a few more ways to get the version number:
# Print 1 word per line, the select the last word:
python -V 2>&1 | xargs -n1 | tail -n1
# Print the last word:
python -V 2>&1 | perl -lane 'print $F[-1];'
# Print the first stretch of 1 or more { digits or periods }:
python -V 2>&1 | grep -Po '[\d.]+'

Subprocess bash script: command not found [duplicate]

This question already has answers here:
Getting "command not found" error in bash script
(6 answers)
Closed 1 year ago.
The following python script:
def run_build(path):
cmd = path + '/build.sh'
p = subprocess.call(cmd)
The following bash script exec two another scripts:
#!/bin/bash
cd "${0%/*}"
echo $(./create_env.sh)
echo $(./set_webhook.sh)
echo $(docker-compose up -d --build)
create_env.sh:
#!/bin/bash
PORT=$(comm -23 <(seq 7000 8000 | sort) <(ss -tan | awk '{print $4}' | cut -d':' -f2 | grep "[0-9]\{1,5\}" | sort -u) | head -n 1)
MONGODB_PORT=$(comm -23 <(seq 27017 27100 | sort) <(ss -tan | awk '{print $4}' | cut -d':' -f2 | grep "[0-9]\{1,5\}" | sort -u) | head -n 1)
destdir=$PWD/.env
echo >> "$destdir"
echo "APP_PORT=$PORT" >> "$destdir"
echo "MONGODB_PORT=$MONGODB_PORT" >> "$destdir"
The output is:
Path: /home/navka/Environments/teststartupservicebot/build.sh
./create_env.sh: line 2: head: command not found
./create_env.sh: line 2: comm: command not found
./create_env.sh: line 2: seq: command not found
...
Where is my problem? Thanks!

I would say your first step would be to place:
echo $PATH
as the first line following the #!/bin/bash shebang line in create_env.sh, to ensure the path is set up.
Make sure it contains the directory for those executables (probably /usr/bin), which you can probably find out by executing (for example) which comm or where comm from a command line.
If it doesn't contain the relevant directory, that explains why it cannot find the executables. In that case, you will need to discover why they're not there.
Perhaps the simplest fix would be to just add something like:
PATH="${PATH}:/usr/bin"
to your environment setup script. This will ensure the path does have the relevant entry.
And, as an aside, if those lines in build.sh are meant to be cumulative (so, for example, set_workbook requires the environment changes made by create_env, you should be aware that these are currently run in sub-shells, meaning changes from one will not persist after the sub-shell exits.
That's not necessarily the case as you persist them to a file, which may be read by the subsequent steps.
If you do need the changes in the environment for subsequent steps (as opposed to a file), you will need to source them in the context of the current shell, such as with:
. ./create_env.sh
As I said, this may not be necessary but you may want to look into it, just in case.

Killing a process within a python script running on linux

I have Raspbian as the linux distro running on my RPI. I've setup a small socket server using twisted and it receives certain commands from an iOS app. These commands are strings. I started a process when I received "st" and now I want to kill it when i get "sp". This is the way I tried:
Imported OS
Used os.system("...") //to start process
os.system("...") // to kill process
Lets say the service is named xyz.
This is the exact way I tried to kill it:
os.system('ps axf | grep xyz | grep -v grep | awk '{print "kill " $1 }' | sh')
But I got a syntax error. That line runs perfectly when I try it in terminal separately. Is this a wrong way to do this in a python script? How do I fix it?

You will need to escape the quotes in your string:
os.system('ps axf | grep xyz | grep -v grep | awk \'{print "kill " $1 }\' | sh')
Or use a triple quote:
os.system('''ps axf | grep xyz | grep -v grep | awk '{print "kill " $1 }' | sh''')

Alternatively, open the process with Popen(...).pid and then use os.kill()
my_pid = Popen('/home/rolf/test1.sh',).pid
os.kill(int(my_pid), signal.SIGKILL)
Remember to include a shebang in your script (#!/bin/sh)
Edit:
On second thoughts, perhaps
os.kill(int(my_pid), signal.SIGTERM)
is probably a better way to end the process, it at least gives the process the chance to close down gracefully.

Bash/Python processes matching

I can use
pgrep -f 'keyword1 | keyword2'
to run a pgrep and return all processes that match either keyword.
How can I use & to do this instead? I just want processes that contain both keywords
The following patterns failed:
pgrep -f 'keyword1 & keyword2'
pgrep -f 'keyword2 && keyword2'
MAN pgrep(1)
OPTIONS
-f The pattern is normally only matched against the process
name. When -f is set, the full command line is used.
.
Side question:
Is there a built in Python library for running these commands? I couldnt seem to find one and everyone suggested using subprocess.Popen(), which is how I'm running the 'pgrep' command, however I'd prefer a pure Python solution if it's available

I'm not sure you can do that with pgrep you can however use awk:
ps ax -o pid,cmd | awk '{pid = $1; $1=""}/[k]eyword1/ && /keyword2/ {print pid}'
The reason i use [k]eyword1 is to avoid matching the awk process.
If PCRE is supported with pgrep something like this would work:
pgrep -f '(?=.*keyword1)(?=.*keyword2)'

You can use or with a wildcard reversing the pattern to get either keyword in any order.
pgrep -f 'keyword1.*keyword2|keyword2.*keyword1'

The typical way to do a grep 'and' is to grep multiple times. Since pgrep returns pids you have to filter the list of processes directly and then extract the PID:
ps ax -o pid,cmd | grep 'keyword1' | grep 'keyword2' | awk '{print $2}'

Execute complex bash script within Python

I have a bash script which I run on my .csv file and then I run python script on the output of bash script. I would like to make everything into a single script, but bash scrip is quite complex and I couldn't find a way to use it in a Python..
grep "$(grep -E "tcp|udp" results.csv | grep -E "Critical|High|Medium" | awk -F "\"*,\"*" '{print $8}')" results.csv | sort -t',' -k4,4 -k8,8 | awk -F "\"*,\"*" '{print $5,"port",$7"/"$6,$8}' | sed '/tcp\|udp/!d' | awk '!a[$0]++' | sed '/,port,\/,/d' > out
I tried this both as a string, and as a parametrized command with subprocess, however it's just seems way too many complex characters for everything to work.
Is there a way simpler way to run this command in Python?
P.S. I know there are multiple questions & answers regarding this same topic, but none of them worked for me.

Could you please escape all the " double quotes" with \ please try it out and let us know if it worked:
os.system(" grep \"$(grep -E \"tcp|udp\" results.csv | grep -E \"Critical|High|Medium\" | awk -F \"\\\"*,\\\"*\" '{print $8}')\" results.csv | sort -t',' -k4,4 -k8,8 | awk -F \"\\\"*,\\\"*\" '{print $5,\"port\",$7\"/\"$6,$8}' | sed '/tcp\|udp/!d' | awk '!a[$0]++' | sed '/,port,\/,/d' > out ")
The whole command can be put into " your_command_with\"escaped\"double quotes ".
Have a nice day